lestes::lang::cplus::lex::pp_lex

lestes::lang::cplus::lex::pp_lex Class Reference

Integration with flexer. More...

#include <pp_lex.hh>

Inheritance diagram for lestes::lang::cplus::lex::pp_lex:

lestes::std::object lestes::std::mem::keystone List of all members.

Public Types

typedef char char_type
 Type of character in lex.
typedef int size_type
 Type of size in lex.

Public Member Functions

virtual ~pp_lex (void)
 Finalizes object.
void activate (void)
 Prepares pp_lex_guts to communicate with this object.
size_type yy_input (char_type *cbuf, size_type max)
 Stores next characters for YY_INPUT.
void location_save (void)
 Saves location of current input token.
ptr< source_location > location_create (void) const
 Creates location for current token.
ptr< line_control > lines_get (void) const
 Returns the line control.
ptr< pp_token > read (bool read_include)
 Reads token from flex.
void interactive_set (bool a_interactive)
 Sets interactive flag.
bool interactive_get (void) const
 Returns interactive flag.

Static Public Member Functions

static ptr< pp_lex > create (const ptr< pre_lex > &a_input, const ptr< line_control > &a_lines)
 Returns new instance.

Protected Member Functions

 pp_lex (const ptr< pre_lex > &a_input, const ptr< line_control > &a_lines)
 Creates the object.
virtual void gc_mark (void)
 Marks the object.

Private Types

typedef pp_lex_guts::yy_buffer_state * yy_buffer_type
 Type of flex buffer.

Private Member Functions

char_type read_char (void)
 Reads next character from input.
 pp_lex (const pp_lex &copy)
 Hides copy constructor.
pp_lex & operator= (const pp_lex &rhs)
 Hides assignment operator.

Private Attributes

bool interactive
 Flag for interactiveness.
srp< pre_lex > input
 Input stream of tokens.
srp< simple_location > physical
 Physical location of currently processed token.
srp< line_control > lines
 Line control for transforming locations.
srp< ucn_token_buffer > buffer
 Buffer with ucn tokens.
srp< ucn_token_buffer > errors
 Error token buffer.
yy_buffer_type yy_buffer
 Flex buffer.

Detailed Description

Integration with flexer.

Handles integration with pp_lex_guts flexer. Passes special character classes to flexer, storing actual characters in own buffer. Each instance is connected to flexer by calling activate() prior to reading tokens (after file change etc.).

Definition at line 73 of file pp_lex.hh.

Member Typedef Documentation

typedef char lestes::lang::cplus::lex::pp_lex::char_type

Type of character in lex.

Definition at line 76 of file pp_lex.hh.

typedef int lestes::lang::cplus::lex::pp_lex::size_type

Type of size in lex.

Definition at line 78 of file pp_lex.hh.

typedef struct pp_lex_guts::yy_buffer_state* lestes::lang::cplus::lex::pp_lex::yy_buffer_type [read, private]

Type of flex buffer.

Definition at line 106 of file pp_lex.hh.

Constructor & Destructor Documentation

lestes::lang::cplus::lex::pp_lex::~pp_lex ( void   )  [virtual]

Finalizes object.

Finalizes the object, releases flex buffer.

Definition at line 87 of file pp_lex.cc.

References yy_buffer, and yy_delete_buffer().

00088 {
00089         pp_lex_guts::yy_delete_buffer(yy_buffer);
00090 }

lestes::lang::cplus::lex::pp_lex::pp_lex ( const ptr< pre_lex > &  a_input,
const ptr< line_control > &  a_lines 
) [protected]

Creates the object.

Creates the object, allocates flex buffer.

Precondition:
a_input != NULL

a_lines != NULL

Postcondition:
interactive == true
Parameters:
a_input The input stream of tokens.
a_lines The line control.

Definition at line 73 of file pp_lex.cc.

Referenced by create().

00073                                                                            :
00074         interactive(true),
00075         input(checked(a_input)),
00076         physical(NULL),
00077         lines(checked(a_lines)),
00078         buffer(ucn_token_buffer::create(lines)),
00079         errors(ucn_token_buffer::create(lines)),
00080         yy_buffer(pp_lex_guts::yy_new_buffer(NULL,YY_BUF_SIZE))
00081 {
00082 }

lestes::lang::cplus::lex::pp_lex::pp_lex ( const pp_lex &  copy  )  [private]

Hides copy constructor.

Member Function Documentation

void lestes::lang::cplus::lex::pp_lex::activate ( void   ) 

Prepares pp_lex_guts to communicate with this object.

Prepares pp_lex_guts to work with this pp_lex. Sets pp_lex_guts shortcut variables current_utb and current_ppl.

Definition at line 96 of file pp_lex.cc.

References buffer, current_ppl, current_utb, yy_buffer, and yy_switch_to_buffer().

Referenced by read().

00097 {
00098         pp_lex_guts::current_utb = buffer.pointer_get();
00099         pp_lex_guts::current_ppl = this;
00100         yy_switch_to_buffer(yy_buffer);
00101 }

pp_lex::size_type lestes::lang::cplus::lex::pp_lex::yy_input ( char_type *  cbuf,
size_type  max 
)

Stores next characters for YY_INPUT.

Fills buffer with characters from current source file. Called through YY_INPUT flex macro.

Parameters:
cbuf The buffer to fill.
max Maximum count of characters to fill.
Returns:
The actual count of filled characters.

Definition at line 206 of file pp_lex.cc.

References lestes::msg::eolog, interactive_get(), read_char(), lestes::lang::cplus::lex::ucn_token::TOK_EOF, and lestes::lang::cplus::lex::ucn_token::TOK_NOT_EOF.

00207 {
00208         pp_lex_logger << "pp_lex::yy_input()\n" << msg::eolog;
00209 
00210         // force the parser interactive
00211         if (interactive_get()) max = 1;
00212 
00213         // initialize not to interfere with the condition
00214         char_type c = ucn_token::TOK_NOT_EOF;
00215         size_type cnt = 0;
00216         
00217         // put all tokens up to max (including TOK_EOF) into buffer
00218         for (cnt = 0; cnt < max && c != ucn_token::TOK_EOF; cnt++) {
00219                 *cbuf++ = c = read_char();
00220         }
00221 
00222         pp_lex_logger << "pp_lex::yy_input() end\n" << msg::eolog;
00223 
00224         return cnt;
00225 }

void lestes::lang::cplus::lex::pp_lex::location_save ( void   ) 

Saves location of current input token.

Saves location of current input token.

Definition at line 106 of file pp_lex.cc.

References buffer, and physical.

00107 {
00108         physical = buffer->peek_front()->location_get();
00109 }

ptr< source_location > lestes::lang::cplus::lex::pp_lex::location_create ( void   )  const

Creates location for current token.

Creates location from saved token, file information and line number correction.

Precondition:
location_save() was already called.
Returns:
New location with correct logical line.

Definition at line 116 of file pp_lex.cc.

References lassert2, lines, and physical.

00117 {
00118         lassert2(physical,"location_save() was not called yet");
00119         return lines->translate_location(physical);
00120 }

ptr<line_control> lestes::lang::cplus::lex::pp_lex::lines_get ( void   )  const

Returns the line control.

ptr< pp_token > lestes::lang::cplus::lex::pp_lex::read ( bool  line_start  ) 

Reads token from flex.

Reads next token from flex.

Parameters:
line_start Whether reading at start of line.
Returns:
The token read from flex.

Definition at line 127 of file pp_lex.cc.

References activate(), current_ppl, lestes::msg::eolog, errors, lexer_parse(), lines, and lestes::report.

00128 {
00129         pp_lex_logger << "pp_lex::read()\n" << msg::eolog;
00130 
00131         pp_lex_logger << "errors length == " << errors->length() << '\n' << msg::eolog;
00132         while (errors->length()) {
00133                 ptr<ucn_token> tok = errors->peek_front();
00134                 errors->advance(1);
00135 
00136                 pp_lex_logger << "handling error from pre_lex\n" << msg::eolog;
00137                 pp_lex_logger << "error == " << tok->error_get() << msg::eolog;
00138 
00139                 report->report(tok->error_get(),lines->translate_location(tok->location_get()));
00140         }
00141 
00142         // this allows concurrent use of pp_lex_guts by different preprocessor instances
00143         if (pp_lex_guts::current_ppl != this) activate();
00144 
00145         ptr<pp_token> t = pp_lex_guts::lexer_parse(line_start);
00146 
00147         pp_lex_logger << "returning " << t->description_get() << "\n" << msg::eolog;
00148         pp_lex_logger << "pp_lex::read() end\n" << msg::eolog;
00149 
00150         return t;
00151 }

void lestes::lang::cplus::lex::pp_lex::interactive_set ( bool  a_interactive  ) 

Sets interactive flag.

Sets interactive flag.

Parameters:
a_interactive The new flag.

Definition at line 231 of file pp_lex.cc.

References interactive.

00232 {
00233         interactive = a_interactive;
00234 }

bool lestes::lang::cplus::lex::pp_lex::interactive_get ( void   )  const

Returns interactive flag.

Returns interactive flag.

Returns:
The flag.

Definition at line 240 of file pp_lex.cc.

References interactive.

Referenced by yy_input().

00241 {
00242         return interactive;
00243 }

ptr< pp_lex > lestes::lang::cplus::lex::pp_lex::create ( const ptr< pre_lex > &  a_input,
const ptr< line_control > &  a_lines 
) [static]

Returns new instance.

Returns new instance of the object.

Precondition:
a_input != NULL

a_lines != NULL

Parameters:
a_input The input stream of tokens.
a_lines The line control.
Returns:
The new instance with own flexer buffer.

Definition at line 266 of file pp_lex.cc.

References pp_lex().

00267 {
00268         return new pp_lex(a_input,a_lines);
00269 }

void lestes::lang::cplus::lex::pp_lex::gc_mark ( void   )  [protected, virtual]

Marks the object.

Marks the object.

Reimplemented from lestes::std::mem::keystone.

Definition at line 248 of file pp_lex.cc.

References buffer, errors, lestes::std::mem::keystone::gc_mark(), input, lines, and physical.

00249 {
00250         input.gc_mark();
00251         physical.gc_mark();
00252         lines.gc_mark();
00253         buffer.gc_mark();
00254         errors.gc_mark();
00255 	::lestes::std::object::gc_mark();
00256 }

pp_lex::char_type lestes::lang::cplus::lex::pp_lex::read_char ( void   )  [private]

Reads next character from input.

Reads token from input stream, stores it into buffer.

Returns:
The character representing the class of the token on input.

Definition at line 157 of file pp_lex.cc.

References buffer, lestes::msg::eolog, errors, input, lassert2, lestes::lang::cplus::lex::ucn_token::TOK_BASIC, lestes::lang::cplus::lex::ucn_token::TOK_EOF, lestes::lang::cplus::lex::ucn_token::TOK_ERROR, and lestes::lang::cplus::lex::ucn_token::TOK_TRANSLATED.

Referenced by yy_input().

00158 {
00159         pp_lex_logger << "pp_lex::read_char()\n" << msg::eolog;
00160 
00161         ptr<ucn_token> tok = input->read();
00162         ucn_token_type utt = tok->type_get();
00163         
00164         while (utt == ucn_token::TOK_ERROR) {
00165                 pp_lex_logger << "cummulating error\n" << msg::eolog;
00166 
00167                 // cummulate errors
00168                 errors->add_back(tok);
00169 
00170                 tok = input->read();
00171                 utt = tok->type_get();
00172         }
00173                 
00174         // add the regular token
00175         buffer->add_back(tok);
00176 
00177         char_type c;
00178         switch (utt) {
00179                 case ucn_token::TOK_BASIC:
00180                         c = static_cast<char_type>(character::extract_value(tok->value_get()));
00181                         break;
00182                 case ucn_token::TOK_EOF:
00183                         // special character class
00184                         c = static_cast<char_type>(ucn_token::TOK_EOF);
00185                         break;
00186                 case ucn_token::TOK_TRANSLATED:
00187                         // special character class
00188                         c = static_cast<char_type>(ucn_token::TOK_TRANSLATED);
00189                         break;
00190                 default:
00191                         lassert2(false,"You should never get here");
00192         }
00193 
00194         pp_lex_logger << "returning " << static_cast<ulint>(c) << "\n" << msg::eolog;
00195         pp_lex_logger << "pp_lex::read_char() end\n" << msg::eolog;
00196         return c;
00197 }

pp_lex& lestes::lang::cplus::lex::pp_lex::operator= ( const pp_lex &  rhs  )  [private]

Hides assignment operator.

Member Data Documentation

bool lestes::lang::cplus::lex::pp_lex::interactive [private]

Flag for interactiveness.

Definition at line 110 of file pp_lex.hh.

Referenced by interactive_get(), and interactive_set().

srp<pre_lex> lestes::lang::cplus::lex::pp_lex::input [private]

Input stream of tokens.

Definition at line 112 of file pp_lex.hh.

Referenced by gc_mark(), and read_char().

srp<simple_location> lestes::lang::cplus::lex::pp_lex::physical [private]

Physical location of currently processed token.

Definition at line 114 of file pp_lex.hh.

Referenced by gc_mark(), location_create(), and location_save().

srp<line_control> lestes::lang::cplus::lex::pp_lex::lines [private]

Line control for transforming locations.

Definition at line 116 of file pp_lex.hh.

Referenced by gc_mark(), location_create(), and read().

srp<ucn_token_buffer> lestes::lang::cplus::lex::pp_lex::buffer [private]

Buffer with ucn tokens.

Definition at line 118 of file pp_lex.hh.

Referenced by activate(), gc_mark(), location_save(), and read_char().

srp<ucn_token_buffer> lestes::lang::cplus::lex::pp_lex::errors [private]

Error token buffer.

Definition at line 120 of file pp_lex.hh.

Referenced by gc_mark(), read(), and read_char().

yy_buffer_type lestes::lang::cplus::lex::pp_lex::yy_buffer [private]

Flex buffer.

Definition at line 122 of file pp_lex.hh.

Referenced by activate(), and ~pp_lex().

The documentation for this class was generated from the following files:
Generated on Mon Feb 12 18:24:18 2007 for lestes by doxygen 1.5.1-20070107