phplrt 4.0

Compiling a Grammar

This package is for development only: composer require phplrt/compiler --dev

The compiler reads a grammar file - the tokens, the rules, the reducers - and gives you back a working parser. It is the friendly front end to everything the lexer builder and parser builder can do.

Reading A Grammar

1use Phplrt\Compiler\Compiler;
2use Phplrt\Source\FileSource;
3use Phplrt\Source\StringSource;
4
5$parser = new Compiler()
6    ->load(FileSource::createFromPathname(__DIR__ . '/grammar.pp3'))
7    ->getParser();
8
9echo $parser->parse(StringSource::createFromString('2 + 2'));

load() reads the grammar (and everything it %includes), and getParser() compiles it. That is the whole thing for a script or a prototype.

You can load several grammars into one compiler - they all end up in the same lexer and parser:

1$compiler = new Compiler();
2$compiler->load(FileSource::createFromPathname(__DIR__ . '/lexemes.pp3'));
3$compiler->load(FileSource::createFromPathname(__DIR__ . '/expressions.pp3'));
4
5$parser = $compiler->getParser();

Generating Code

Reading a grammar takes real time, and the grammar does not change between requests. So: do it once, write the result to a file, and commit the file.

1new Compiler()
2    ->load(FileSource::createFromPathname(__DIR__ . '/grammar.pp3'))
3    ->generate()
4        ->withNamespaceName('App\Language')
5        ->withClassName('LanguageParser')
6        ->save(__DIR__ . '/Parser.php');

Now production never sees the compiler at all:

$parser = new App\Language\LanguageParser();

Code Generation covers this in full, and the same thing without a script of your own is one command:

php vendor/bin/phplrt compile resources/grammar.pp3 \ 
    src/Parser.php \ 
    --class Parser

Command Line covers the binary.

Grammar Formats

The format is decided by the file extension:

Extension Format
.pp The legacy Hoa format - no longer supported
.pp2 The older format, described in PP2 Grammar Syntax
.pp3 The current format, described in PP3 Grammar Syntax

Write .pp3 for anything new. A .pp2 file keeps being read the way it always was, so an existing grammar needs no attention.

A grammar that did not come from a file (a StringSource, a string) is read as the newest format, since there is no extension to go by:

$compiler->load(StringSource::createFromString('%token T_DIGIT \d++  Num : <T_DIGIT> ;'));

Reading a .pp file gives you a clear error rather than a confusing one:

1error[UnsupportedFormatException]: Grammar files written in the "pp" format
2are not supported
3 --> /app/grammar.pp:1:1

Splitting A Grammar Up

Real grammars get long. %include pulls in another file, and the declarations land exactly where the %include is written:

1%include grammar/lexemes
2%include grammar/literals
3%include grammar/expressions
4
5%pragma root Expression

A few useful details:

  • the path is relative to the file the %include is written in;
  • the extension may be omitted - every known format is tried in turn;
  • a grammar reached from several places is read once, so a shared lexemes.pp3 can be included by every file that needs it.

If the file is missing, the error names both the include and the file that wanted it:

1error[GrammarNotFoundException]: grammar/missing: failed to open stream:
2No such file or directory
3 --> /app/grammar.pp3:1:1
41 | %include grammar/missing
5  | ^^^^^^^^^^^^^^^^^^^^^^^^

Errors inside an included grammar are reported the same way, with the chain of includes that led there.

Getting At The Pieces

The compiler is a thin layer over the two builders, and both are public:

 1$compiler = new Compiler();
 2$compiler->load(FileSource::createFromPathname(__DIR__ . '/grammar.pp3'));
 3
 4// Add a token the grammar file does not mention
 5$compiler->lexer->addPattern('#[^\n]*+')
 6    ->hide();
 7
 8// Add a compiler pass of your own
 9$compiler->parser->addCompilerPass(new MyValidationPass());
10
11$parser = $compiler->getParser();

build() gives you the compiled description instead of a ready parser - which is what the generator works from:

1$result = $compiler->build();
2
3$result->lexer;  // LexerBuilderResult
4$result->parser; // ParserBuilderResult

Errors

Everything that can go wrong points at the exact spot in the grammar:

1error[UnsupportedPragmaException]: Unrecognized pragma "unknown"
2 --> /app/grammar.pp3:2:1
31 | %token T_A a
42 | %pragma unknown value
5  | ^^^^^^^^^^^^^^^^^^^^^
63 | A : <T_A> ;

The message above is what printing the exception gives you - see Errors for catching and rendering them.

What's Next?