lestes::lang::cplus::lex::string_joiner

lestes::lang::cplus::lex::string_joiner Class Reference

String literal joining filter. More...

#include <string_joiner.hh>

Inheritance diagram for lestes::lang::cplus::lex::string_joiner:

lestes::lang::cplus::lex::pp_filter lestes::std::object lestes::std::mem::keystone List of all members.

Public Member Functions

ptr< pp_token > read (void)
 Reads next token.

Static Public Member Functions

static ptr< string_joiner > create (const ptr< pp_filter > &a_input)
 Returns new instance.

Protected Member Functions

 string_joiner (const ptr< pp_filter > &a_input)
 Creates the object.
virtual void gc_mark (void)
 Marks the object.

Private Types

enum  state_type {
  START = 0, FIRST_STRING = 1, NEXT_STRING = 2, FIRST_WIDE_STRING = 3,
  NEXT_WIDE_STRING = 4, NEXT_BAD_STRING = 5, SAVED = 6
}
 States of the processor. More...
typedef ::lestes::std::vector<
srp< pp_token > > 		sequence_type
 Type of sequence of adjacent literals.

Private Member Functions

ptr< pp_token > join_sequence (pp_token_type ptt) const
 Returns joined string literal.
 string_joiner (const string_joiner &)
 Hides copy constructor.
string_joiner & operator= (const string_joiner &)
 Hides assignment operator.

Private Attributes

state_type state
 Current state of the processor.
srp< pp_token > saved
 Buffered token.
srp< sequence_type > sequence
 Current sequence of adjacent literals.

Detailed Description

String literal joining filter.

Performs joining of adjacent string literals and adjacent wide string literals.

Definition at line 52 of file string_joiner.hh.

Member Typedef Documentation

typedef ::lestes::std::vector< srp<pp_token> > lestes::lang::cplus::lex::string_joiner::sequence_type [private]

Type of sequence of adjacent literals.

Definition at line 82 of file string_joiner.hh.

Member Enumeration Documentation

enum lestes::lang::cplus::lex::string_joiner::state_type [private]

States of the processor.

Enumerator:
START  No tokens in buffer.
FIRST_STRING  Read first string.
NEXT_STRING  Reading string sequence.
FIRST_WIDE_STRING  Read first wide string.
NEXT_WIDE_STRING  Reading wide string sequence.
NEXT_BAD_STRING  Reading mixed string and wide string sequence.
SAVED  Ordinary token saved in buffer.

Definition at line 65 of file string_joiner.hh.

00065                      {
00066                 //! No tokens in buffer.
00067                 START = 0,
00068                 //! Read first string.
00069                 FIRST_STRING = 1,
00070                 //! Reading string sequence.
00071                 NEXT_STRING = 2,
00072                 //! Read first wide string.
00073                 FIRST_WIDE_STRING = 3,
00074                 //! Reading wide string sequence.
00075                 NEXT_WIDE_STRING = 4,
00076                 //! Reading mixed string and wide string sequence.
00077                 NEXT_BAD_STRING = 5,
00078                 //! Ordinary token saved in buffer.
00079                 SAVED = 6,
00080         } state_type;

Constructor & Destructor Documentation

lestes::lang::cplus::lex::string_joiner::string_joiner ( const ptr< pp_filter > &  a_input  )  [protected]

Creates the object.

Creates the object.

Precondition:
a_input != NULL
Parameters:
a_input The input for the filter.
Postcondition:
state == START

Definition at line 55 of file string_joiner.cc.

Referenced by create().

00055                                                          :
00056         pp_filter(checked(a_input)),
00057         state(START),
00058         saved(NULL),
00059         sequence(sequence_type::create())
00060 {
00061 }

lestes::lang::cplus::lex::string_joiner::string_joiner ( const string_joiner &   )  [private]

Hides copy constructor.

Member Function Documentation

ptr< pp_token > lestes::lang::cplus::lex::string_joiner::read ( void   )  [virtual]

Reads next token.

Reads next token from input. Joins all adjacent string literal tokens into single string literal token. Joins all adjacent normal and wide string literal tokens into single wide string literal token.

Implements lestes::lang::cplus::lex::pp_filter.

Definition at line 68 of file string_joiner.cc.

References FIRST_STRING, FIRST_WIDE_STRING, lestes::lang::cplus::lex::pp_filter::input_read(), join_sequence(), lassert2, lestes::lang::cplus::lex::mixed_strings, NEXT_BAD_STRING, NEXT_STRING, NEXT_WIDE_STRING, lestes::report, saved, SAVED, sequence, START, state, lestes::lang::cplus::lex::pp_token::TOK_STRING_LIT, and lestes::lang::cplus::lex::pp_token::TOK_WSTRING_LIT.

00069 {
00070         if (state == SAVED) {
00071                 ptr<pp_token> tmp = saved;
00072                 saved = NULL;
00073                 state = START;
00074                 return tmp;
00075         }
00076                 
00077         ptr<pp_token> t;
00078         pp_token_type ptt;
00079         // first literal
00080         ptr<pp_token> first;
00081 
00082         while (true) {
00083                 t = input_read();
00084                 ptt = t->type_get();
00085                 
00086                 switch (state) {
00087                         case START:
00088                                 switch (ptt) {
00089                                         case pp_token::TOK_STRING_LIT:
00090                                                 first = t;
00091                                                 state = FIRST_STRING;
00092                                                 break;
00093                                         case pp_token::TOK_WSTRING_LIT:
00094                                                 first = t;
00095                                                 state = FIRST_WIDE_STRING;
00096                                                 break;
00097                                         default:
00098                                                 // fast path
00099                                                 return t;
00100                                 }
00101                                 break;
00102                         case FIRST_STRING:
00103                                 switch (ptt) {
00104                                         case pp_token::TOK_STRING_LIT:
00105                                                 sequence->clear();
00106                                                 sequence->push_back(first);
00107                                                 sequence->push_back(t);
00108                                                 state = NEXT_STRING;
00109                                                 break;
00110                                         case pp_token::TOK_WSTRING_LIT:
00111                                                 report << mixed_strings << t->location_get();
00112                                                 sequence->clear();
00113                                                 sequence->push_back(first);
00114                                                 sequence->push_back(t);
00115                                                 state = NEXT_BAD_STRING;
00116                                                 break;
00117                                         default:
00118                                                 saved = t;
00119                                                 state = SAVED;
00120                                                 return first;
00121                                 }
00122                                 break;
00123                         case NEXT_STRING:
00124                                 switch (ptt) {
00125                                         case pp_token::TOK_WSTRING_LIT:
00126                                                 state = NEXT_BAD_STRING;
00127                                                 report << mixed_strings << t->location_get();
00128                                                 // fall through
00129                                         case pp_token::TOK_STRING_LIT:
00130                                                 sequence->push_back(t);
00131                                                 break;
00132                                         default:
00133                                                 saved = t;
00134                                                 state = SAVED;
00135                                                 return join_sequence(pp_token::TOK_STRING_LIT);
00136                                 }
00137                                 break;
00138                         case FIRST_WIDE_STRING:
00139                                 switch (ptt) {
00140                                         case pp_token::TOK_STRING_LIT:
00141                                                 report << mixed_strings << t->location_get();
00142                                                 sequence->clear();
00143                                                 sequence->push_back(first);
00144                                                 sequence->push_back(t);
00145                                                 state = NEXT_BAD_STRING;
00146                                                 break;
00147                                         case pp_token::TOK_WSTRING_LIT:
00148                                                 sequence->clear();
00149                                                 sequence->push_back(first);
00150                                                 sequence->push_back(t);
00151                                                 state = NEXT_WIDE_STRING;
00152                                                 break;
00153                                         default:
00154                                                 saved = t;
00155                                                 state = SAVED;
00156                                                 return first;
00157                                 }
00158                                 break;
00159                         case NEXT_WIDE_STRING:
00160                                 switch (ptt) {
00161                                         case pp_token::TOK_STRING_LIT:
00162                                                 report << mixed_strings << t->location_get();
00163                                                 sequence->push_back(t);
00164                                                 state = NEXT_BAD_STRING;
00165                                                 break;
00166                                         case pp_token::TOK_WSTRING_LIT:
00167                                                 sequence->push_back(t);
00168                                                 break;
00169                                         default:
00170                                                 saved = t;
00171                                                 state = SAVED;
00172                                                 return join_sequence(pp_token::TOK_WSTRING_LIT);
00173                                 }
00174                                 break;
00175                         case NEXT_BAD_STRING:
00176                                 switch (ptt) {
00177                                         case pp_token::TOK_STRING_LIT:
00178                                         case pp_token::TOK_WSTRING_LIT:
00179                                                 sequence->push_back(t);
00180                                                 break;
00181                                         default:
00182                                                 saved = t;
00183                                                 state = SAVED;
00184                                                 return join_sequence(pp_token::TOK_WSTRING_LIT);
00185                                 }
00186                                 break;
00187                         default:
00188                                 lassert2(false,"You should never get here");
00189                 }
00190         }
00191 }

ptr< string_joiner > lestes::lang::cplus::lex::string_joiner::create ( const ptr< pp_filter > &  a_input  )  [static]

Returns new instance.

Returns new instance.

Precondition:
a_input != NULL
Parameters:
a_input The input for the filter.
Returns:
New instance of the class.

Definition at line 239 of file string_joiner.cc.

References string_joiner().

00240 {
00241         return new string_joiner(a_input);
00242 }

void lestes::lang::cplus::lex::string_joiner::gc_mark ( void   )  [protected, virtual]

Marks the object.

Marks the object.

Reimplemented from lestes::lang::cplus::lex::pp_filter.

Definition at line 226 of file string_joiner.cc.

References lestes::lang::cplus::lex::pp_filter::gc_mark(), saved, and sequence.

00227 {
00228         sequence.gc_mark();
00229         saved.gc_mark();
00230         pp_filter::gc_mark();
00231 }

ptr< pp_token > lestes::lang::cplus::lex::string_joiner::join_sequence ( pp_token_type  ptt  )  const [private]

Returns joined string literal.

Returns token containing joined string literals. Does not affect the contents of the sequence.

Precondition:
The sequence is not empty.

The sequence contains tokens of one type, either pp_token::TOK_STRING_LIT or pp_token::TOK_WSTRING_LIT. The location for the new token is taken from the first entry.

Parameters:
ptt The type of the new token.
Returns:
String or wide string literal token.

Definition at line 201 of file string_joiner.cc.

References lestes::lang::cplus::lex::token_value::create(), lestes::lang::cplus::lex::pp_token::create(), lassert, and sequence.

Referenced by read().

00202 {
00203         sequence_type::iterator it = sequence->begin();
00204         sequence_type::iterator end = sequence->end();
00205 
00206         // process first entry
00207         lassert(it != end);
00208         ptr<pp_token> t = *it;
00209         
00210         ptr<source_location> loc = t->location_get();
00211         ucn_string str(t->value_get()->content_get());
00212         ++it;
00213         
00214         // process second and following entries
00215         for ( ;it != end; ++it) {
00216                 t = *it;
00217                 str += t->value_get()->content_get();
00218         }
00219 
00220         return pp_token::create(loc,ptt,token_value::create(str));
00221 }

string_joiner& lestes::lang::cplus::lex::string_joiner::operator= ( const string_joiner &   )  [private]

Hides assignment operator.

Member Data Documentation

state_type lestes::lang::cplus::lex::string_joiner::state [private]

Current state of the processor.

Definition at line 84 of file string_joiner.hh.

Referenced by read().

srp<pp_token> lestes::lang::cplus::lex::string_joiner::saved [private]

Buffered token.

Definition at line 86 of file string_joiner.hh.

Referenced by gc_mark(), and read().

srp<sequence_type> lestes::lang::cplus::lex::string_joiner::sequence [private]

Current sequence of adjacent literals.

Definition at line 88 of file string_joiner.hh.

Referenced by gc_mark(), join_sequence(), and read().

The documentation for this class was generated from the following files:
Generated on Mon Feb 12 18:24:19 2007 for lestes by doxygen 1.5.1-20070107