lestes::lang::cplus::lex::stringifier Class Reference

Token stringifier. More...

#include <stringifier.hh>

Inheritance diagram for lestes::lang::cplus::lex::stringifier:

List of all members.

Public Member Functions
ptr< pp_token >	process (const ptr< token_input > &input)
	Stringifies tokens.
Static Public Member Functions
static ptr< stringifier >	instance (void)
	Returns the only instance.
Protected Member Functions
	stringifier (void)
	Creates the object.
Private Member Functions
ucn_string	escape_spelling (const ptr< pp_token > &tok)
	Adds guard characters to disable escape sequences.
	stringifier (const stringifier &)
	Hides copy constructor.
stringifier &	operator= (const stringifier &)
	Hides assignment operator.
Static Private Attributes
static ptr< stringifier >	singleton
	The only instance of the class.

---

Detailed Description

Token stringifier.

Performs token stringification.

Definition at line 53 of file stringifier.hh.

---

Constructor & Destructor Documentation

lestes::lang::cplus::lex::stringifier::stringifier

(

void

)

[protected]

Creates the object.

Creates the only object of the class.

Definition at line 50 of file stringifier.cc.

Referenced by instance().

{
}

lestes::lang::cplus::lex::stringifier::stringifier

(

const stringifier &

)

[private]

Hides copy constructor.

---

Member Function Documentation

ptr< pp_token > lestes::lang::cplus::lex::stringifier::process

(

const ptr< token_input > &

input

)

Stringifies tokens.

Attempts to stringify tokens. Creates new token of type pp_token::TOK_STRING_LIT, with location taken from the first nonblank input token and value representing joined stringification of all input tokens, with character and string literals escaped. Runs of blank tokens inside the sequence are represented by single space, surrounding blanks are discarded.

Precondition:

input != NULL

Todo:

pt necessary? pre input does not contain pp_token::TOK_LINE_END tokens.

Parameters:

input

The tokens to stringify.

Returns:

New token containing the stringification, no correctness checks are done.

Definition at line 65 of file stringifier.cc.

References lestes::lang::cplus::lex::token_value::create(), lestes::lang::cplus::lex::pp_token::create(), escape_spelling(), lestes::lang::cplus::lex::pp_token::TOK_BLANK, lestes::lang::cplus::lex::pp_token::TOK_STRING_LIT, and lestes::lang::cplus::lex::pp_token::TOK_TERMINATOR.

{
        ucn_string str;
        ptr<pp_token> tok = input->read_front();
        // skip the leading blank
        if (tok->type_get() == pp_token::TOK_BLANK) tok = input->read_front();
        ptr<source_location> loc = tok->location_get();
        
        if (tok->type_get() != pp_token::TOK_TERMINATOR) {
                ptr<pp_token> last = tok;
                tok = input->read_front();
                
                while (tok->type_get() != pp_token::TOK_TERMINATOR) {
                        str += escape_spelling(last);
                        last = tok;
                        tok = input->read_front();
                }

                // add the last token iff nonblank
                if (last->type_get() != pp_token::TOK_BLANK) str += escape_spelling(last);
        }
        
        // TODO ??? set error flag, because the literal can be broken
        return pp_token::create(loc,pp_token::TOK_STRING_LIT,token_value::create(str));
}

ptr< stringifier > lestes::lang::cplus::lex::stringifier::instance

(

void

)

[static]

Returns the only instance.

Returns the only instance. Lazy initialized.

Returns:

The singleton.

Definition at line 158 of file stringifier.cc.

References singleton, and stringifier().

Referenced by lestes::lang::cplus::lex::macro_argument::stringified_get().

{
        if (!singleton) {
                singleton = new stringifier();
        }
        return singleton;
}

ucn_string lestes::lang::cplus::lex::stringifier::escape_spelling

(

const ptr< pp_token > &

tok

)

[private]

Adds guard characters to disable escape sequences.

Adds guard backslash characters before double quotes and backslash characters into the spelling of string, character and other literals to avoid interpreting them as escape sequences. Spelling of tokens with other types are returned intact.

Precondition:

tok != NULL

Parameters:

tok

The token to process.

Returns:

The spelling string with special characters escaped.

Definition at line 99 of file stringifier.cc.

References lassert, lestes::lang::cplus::lex::pp_token::TOK_CHAR_LIT, lestes::lang::cplus::lex::pp_token::TOK_IDENT, lestes::lang::cplus::lex::pp_token::TOK_OTHER, lestes::lang::cplus::lex::pp_token::TOK_STRING_LIT, lestes::lang::cplus::lex::pp_token::TOK_WCHAR_LIT, lestes::lang::cplus::lex::pp_token::TOK_WSTRING_LIT, and u.

Referenced by process().

{
        lassert(tok);

        switch (tok->type_get()) {
                case pp_token::TOK_STRING_LIT:
                case pp_token::TOK_WSTRING_LIT:
                case pp_token::TOK_CHAR_LIT:
                case pp_token::TOK_WCHAR_LIT:
                case pp_token::TOK_OTHER:
                case pp_token::TOK_IDENT:
                        break;
                default:
                        return tok->spelling_get();
        }

        ucn_string str(tok->spelling_get());
        ucn_string::size_type len = str.length();
        ucn u;
        ucn_string work;

        for (ucn_string::size_type i = 0; i < len; i++) {
                u = str[i];
                
                if (character::is_translated(u)) {
                        ulint x = character::extract_value(u);
                        work += character::ascii_backslash;
                        work += character::ascii_backslash;
                        if (x <= 0xffff) {
                                work += character::ascii_lower_u;
                                work += character::create_xdigit((x >> 12) & 0xf);
                                work += character::create_xdigit((x >> 4) & 0xf);
                                work += character::create_xdigit((x >> 8) & 0xf);
                                work += character::create_xdigit(x & 0xf);
                        } else {
                                work += character::ascii_upper_u;
                                work += character::create_xdigit((x >> 28) & 0xf);
                                work += character::create_xdigit((x >> 24) & 0xf);
                                work += character::create_xdigit((x >> 20) & 0xf);
                                work += character::create_xdigit((x >> 16) & 0xf);
                                work += character::create_xdigit((x >> 12) & 0xf);
                                work += character::create_xdigit((x >> 8) & 0xf);
                                work += character::create_xdigit((x >> 4) & 0xf);
                                work += character::create_xdigit(x & 0xf);
                        }
                } else {
                        if (u == character::ascii_dquote || u == character::ascii_backslash)
                                work += character::ascii_backslash;
                        work += u;
                }
                
        }
        return work;
}

stringifier& lestes::lang::cplus::lex::stringifier::operator=

(

const stringifier &

)

[private]

Hides assignment operator.

---

Member Data Documentation

ptr< stringifier > lestes::lang::cplus::lex::stringifier::singleton [static, private]

The only instance of the class.

The only instance of the class. Lazy initialized in the instance() method.

Definition at line 70 of file stringifier.hh.

Referenced by instance().

---

The documentation for this class was generated from the following files:

• stringifier.hh
• stringifier.cc

---