Custom Generator

A PHP parser is not the only thing a compiled grammar is good for. An editor plugin wants the token list; a syntax highlighter wants the pattern; a client in another language wants the rules. All of that is in CompilerResult, and the interface for turning it into a file has one method:

1public function generate(
2    CompilerResult $result,
3    OutputContext $context = new OutputContext(),
4): string;

Return a string, and everything else - withNamespaceName(), save(), creating directories - keeps working as before.

Here is a generator that writes the token list down as JSON, for a highlighter that needs to know the words of a language but not its grammar:

 1use Phplrt\Compiler\CompilerResult;
 2use Phplrt\Compiler\Generator\OutputContext;
 3use Phplrt\Compiler\Generator\OutputGeneratorInterface;
 4
 5final readonly class TokenListGenerator implements OutputGeneratorInterface
 6{
 7    public function generate(
 8        CompilerResult $result,
 9        OutputContext $context = new OutputContext(),
10    ): string {
11        $tokens = [];
12
13        foreach ($result->lexer->tokens as $id => $definition) {
14            $tokens[] = [
15                'id' => $id,
16                'name' => $result->lexer->names[$id] ?? null,
17                'channel' => $result->lexer->channels[$id] ?? 'Default',
18            ];
19        }
20
21        return \json_encode([
22            'name' => $context->class ?? 'grammar',
23            'pattern' => $result->lexer->pattern,
24            'tokens' => $tokens,
25        ], \JSON_PRETTY_PRINT | \JSON_UNESCAPED_SLASHES) . "\n";
26    }
27}

Pass it to generate() in place of the default:

1new Compiler()
2    ->load(FileSource::createFromPathname(__DIR__ . '/grammar.pp3'))
3    ->generate(new TokenListGenerator())
4        ->withClassName('Calculator')
5        ->save(__DIR__ . '/calculator.json');

Given this grammar:

1%skip  T_WHITESPACE  \s++
2%skip  T_COMMENT     //[^\n]*+
3
4%token T_NUMBER      \d++
5%token T_PLUS        \+
6
7Sum -> { return \array_sum($children); }
8  : <T_NUMBER> (::T_PLUS:: <T_NUMBER>)*
9  ;

you get calculator.json:

 1{
 2    "name": "Calculator",
 3    "pattern": "/\\G(?|(?:(?:\\s++)(*MARK:0))|(?:(?:\\/\\/[^\\n]*+)(*MARK:1))|(?:(?:\\d++)(*MARK:2))|(?:(?:\\+)(*MARK:3))|(?:(?:[^\\s]++)(*MARK:4)))/Ssum",
 4    "tokens": [
 5        {
 6            "id": 0,
 7            "name": "T_WHITESPACE",
 8            "channel": "Hidden"
 9        },
10        {
11            "id": 1,
12            "name": "T_COMMENT",
13            "channel": "Hidden"
14        },
15        {
16            "id": 2,
17            "name": "T_NUMBER",
18            "channel": "Default"
19        },
20        {
21            "id": 3,
22            "name": "T_PLUS",
23            "channel": "Default"
24        },
25        {
26            "id": 4,
27            "name": null,
28            "channel": "Unknown"
29        }
30    ]
31}

Two things the output shows that the grammar does not say out loud: the %skip tokens are ordinary tokens on the Hidden channel, and token #4 is the catch-all the compiler appends so that unrecognized input becomes an Unknown token instead of an error.

What You Get To Work With

 1$result->lexer->tokens;      // array<int, TokenDefinition>
 2$result->lexer->names;       // array<int, non-empty-string>
 3$result->lexer->channels;    // array<int, non-empty-string>
 4$result->lexer->pattern;     // the compiled regex
 5$result->lexer->transitions; // which tokens enter a nested lexer
 6$result->lexer->lexers;      // the nested lexers themselves
 7
 8$result->parser->grammar;    // list<RuleInterface> - the rules, by id
 9$result->parser->initial;    // where parsing starts
10$result->parser->reducers;   // array<int, ReducerInterface>
11$result->parser->constants;  // ['Sum' => 0, ...] - named rules
12$result->parser->lookahead;
13$result->parser->kept;

OutputContext carries what the caller asked for - $context->namespace, $context->class, $context->imports. Use what makes sense for your format and ignore the rest.

Reporting Problems

Not every grammar can be written into every format. Throw a GeneratorException when yours cannot express something, so the failure arrives as a compiler error rather than as broken output:

1use Phplrt\Compiler\Exception\GeneratorException;
2
3if ($result->lexer->lexers !== []) {
4    throw new class ('Nested lexers cannot be written as JSON')
5        extends GeneratorException {};
6}

Adjusting The PHP Output Instead

If you only want to change how the PHP looks, you do not need a generator of your own. PhpOutputGenerator renders Twig templates from the compiler's resources/php/ directory, and they are split per fragment - the file layout, the parser body, the grammar table, the reducer methods, the lexer and its states - so a change is usually confined to one small template.