c++ - How to combine skipping and non-skipping (lexeme) rules?

Question

my parser is nearly working :) (still amazed by Spirit feature set (and compiletimes) and the very welcoming community here on stack overflow)

small sample for online try: http://coliru.stacked-crooked.com/a/1c1bf88909dce7e3

so i've learned to use more lexeme-rules and try to prevent no_skip - my rules are smaller and better to read as a result but now i stuck with combining lexeme-rules and skipping-rules what seems to be not possible (compiletime error with warning about not castable to Skipper)

my problem is the comma seperated list in subscriptions which does not skip spaces around expressions

parses:

"a.b[a,b]"

fails:

"a.b[ a , b ]"

these are my rules:

qi::rule<std::string::const_iterator, std::string()> identifier_chain;

qi::rule<std::string::const_iterator, std::string()>
    expression_list = identifier_chain >> *(qi::char_(',') >> identifier_chain);

qi::rule < std::string::const_iterator, std::string() >
    subscription = qi::char_('[') >> expression_list >> qi::char_(']');

qi::rule<std::string::const_iterator, std::string()>
    identifier = qi::ascii::alpha >> *(qi::ascii::alnum | '_');

identifier_chain = identifier >> *(('.' >> identifier) | subscription);

as you can see all rules are "lexeme" and i think the subscription rule should be a ascii::space_type skipper but that does not compile

should i add space eaters in the front and back of identifier_chains in the expression_list?

feels like writing an regex :(

expression_list = *qi::blank >> identifier_chain >> *(*qi::blank >> qi::char_(',') >> *qi::blank >> identifier_chain >> *qi::blank);

it works but i've read that this will get me to an much bigger parser in the end (handling all the space skipping by myself)

thx for any advice

btw: any idea why i can't compile if surrounding the '.' in the indentifier_chain with qi::char_('.')

identifier_chain = identifier >> *(('.' >> identifier) | subscription);

UPDATE:

i've updated my expression list as suggested by sehe

qi::rule<std::string::const_iterator, spirit::ascii::blank_type, std::string()>
expression_list = identifier_chain >> *(qi::char_(',') >> identifier_chain);

qi::rule < std::string::const_iterator, std::string() >
subscription = qi::char_('[') >> qi::skip(qi::blank)[expression_list] >> qi::char_(']');

but still get compile error due to non castable Skipper: http://coliru.stacked-crooked.com/a/adcf665742b055dd

i also tried changed the identifer_chain to

identifier_chain = identifier >> *(('.' >> identifier) | qi::skip(qi::blank)[subscription]);

but i still can't compile the example

Answer 1

The answer I linked to earlier describes all the combinations (if I remember correctly): Boost spirit skipper issues

In short:

• any rule that declares a skipper (so rule<It, Skipper[, Attr()]> or rule<It, Attr(), Skipper>) MUST be invoked with a compatible skipper (an expression that can be assigned to the type of Skipper).

• any rule that does NOT declare a skipper (so of the form rule<It[, Attr()]>) will implicitly behave like a lexeme, meaning no input characters are skipped.

That's it. The slightly subtler ramifications are that given two rules:

rule<It, blank_type> a;
rule<It> b; // b is implicitly lexeme

You can invoke b from a:

a = "test" >> b;

But when you wish to invoke a from b you will find that you have to provide the skipper:

b = "oops" >> a; // DOES NOT COMPILE
b = "okay" >> qi::skip(qi::blank) [ a ];

That's almost all there is to it. There are a few more directives around skippers and lexemes in Qi, see again the answer linked above.

Side Question:

should i add space eaters in the front and back of identifier_chains in the expression_list?

If you look closely at the answer example here Parse a '.' chained identifier list, with qi::lexeme and prevent space skipping, you can see that it already does pre- and post skipping correctly, because I used phrase_parse:

" a.b " OK: ( "a" "b" ) 
----
"a . b" Failed
Remaining unparsed: "a . b"
----

You COULD also wrap the whole thing in an "outer" rule:

rule<std::string::const_iterator> main_rule = 
     qi::skip(qi::blank) [ identifier_chain ];

That's just the same but allows users to call parse without specifying the skipper.