Upgrade Guide

Upgrading To 4.0 From 3.x

Version 4.0 is a rewrite, so this is a porting guide rather than a list of renames. The grammar files mostly survive, and that is where the bulk of the work usually lives.

Getters Became Properties

Likelihood Of Impact: High

Everything that was getX() is now a property. This is the single most common change you will hit.

 1// 3.x
 2$token->getName();
 3$token->getOffset();
 4$token->getValue();
 5$token->getBytes();
 6
 7$source->getContents();
 8$source->getPathname();
 9
10// 4.x
11$token->name;
12$token->offset;
13$token->value;
14$token->size;
15
16$source->content;
17$source->pathname;

Tokens Are Identified By Number

Likelihood Of Impact: High

In 3.x a token was addressed by its name. In 4.x it has an int $id, and the name is optional metadata for error messages.

1// 3.x
2if ($token->getName() === 'T_DIGIT') { /* ... */ }
3
4// 4.x
5if ($token->id === MyParser::T_DIGIT) { /* ... */ }

Generated parsers expose the ids as class constants, so you do not have to track the numbers yourself.

The Lexer Is Built, Not Configured

Likelihood Of Impact: High

Phplrt\Lexer\Lexer no longer takes a map of names and patterns. It takes a single compiled regular expression and a set of tables - which you get from LexerBuilder.

 1// 3.x
 2$lexer = new Lexer([
 3    'T_DIGIT' => '\d+',
 4    'T_PLUS'  => '\+',
 5]);
 6
 7// 4.x
 8use Phplrt\Lexer\Builder\LexerBuilder;
 9
10$builder = new LexerBuilder();
11$builder->addPattern('\d++', 'T_DIGIT');
12$builder->addValue('+', 'T_PLUS');
13
14$lexer = $builder->build()
15    ->toLexer();

append(), prepend(), prependMany() and skip() are gone; addPattern(), addValue() and hide() cover the same ground.

Channels Replaced "Skipped Tokens"

Likelihood Of Impact: Medium

The lexer no longer takes a list of names to skip. Every token carries a channel, and the lexer reports every channel except the ones it is told to leave out - Hidden alone, unless you say otherwise.

1// 3.x
2$lexer = new Lexer($tokens, ['T_WHITESPACE']);
3
4// 4.x
5$builder->addPattern('\s++', 'T_WHITESPACE')
6    ->hide();

The Parser Takes Explicit Arguments

Likelihood Of Impact: High

The Parser::CONFIG_* options are gone, replaced by constructor parameters:

 1// 3.x
 2$parser = new Parser($lexer, $grammar, [
 3    Parser::CONFIG_INITIAL_RULE => 'expression',
 4    Parser::CONFIG_AST_BUILDER  => new MyBuilder(),
 5]);
 6
 7// 4.x
 8$parser = new Parser(
 9    lexer: $lexer,
10    grammar: $grammar,
11    initial: 0,
12    reducers: [0 => $callback],
13);

Rules are keyed by integers rather than by name, and they refer to each other by index. In practice you do not write this array by hand - see the parser builder.

BuilderInterface Became Per-Rule Reducers

Likelihood Of Impact: High

The single AST builder with a switch over rule names is gone. Each rule now carries its own reducer.

 1// 3.x
 2class MyBuilder implements BuilderInterface
 3{
 4    public function build(Context $ctx, $children)
 5    {
 6        switch ($ctx->getState()) {
 7            case 'Number': return new NumberNode($children);
 8            case 'Sum':    return new SumNode($children);
 9        }
10
11        return null;
12    }
13}
14
15// 4.x - in the grammar file
16Number -> { return new \NumberNode($offset, $children->value); } 
17    : <T_DIGIT> ;
18Sum    -> { return new \SumNode($children); } 
19    : Number() ::T_PLUS:: Number() ;

The signature is still callable(Context $ctx, mixed $children): mixed, but $ctx->getState() is now $ctx->rule and holds an integer. Returning null still means "leave the children alone".

check() And Trailing Tokens Became analyze()

Likelihood Of Impact: Medium

3.x answered "is this valid?" with check(), and read a source the grammar does not describe in full through a setting, picking up where it stopped from the parser afterwards. Both are analyze() now:

 1// 3.x
 2$parser->check($source); // true or false
 3$context = $parser->getLastExecutionContext();
 4
 5// 4.x
 6use Phplrt\Parser\Analysis\Mode;
 7
 8$result = $parser->analyze($source, Mode::SyntaxCheck);
 9
10$result->value; // what has been read reduced to
11$result->token; // where the parser stopped
12$result->error; // and the exception it would be rejected with

How far the grammar got is the class of the result: SuccessfulResult, PartialResult or FailureResult. Nothing is kept on the parser between calls, so getLastExecutionContext() has no replacement.

See Analysing A Source.

A Source Is Read By Offset

Likelihood Of Impact: Medium

3.x gave you the source in one piece and nothing smaller. 4.x keeps the whole in $content and adds read(), which names the fragment it wants:

1// 3.x
2$whole = $source->getContents();
3$part = \fread($source->getStream(), 4);
4
5// 4.x
6$whole = $source->content;
7$part = $source->read(0, 4);

read() takes an absolute offset, and reading a source leaves it as it is. getStream() has no replacement - build a ResourceSource over a resource of your own where you need one.

Sources are also constructed directly now. The static factory methods on File still work, but are deprecated:

1// 3.x
2File::fromPathname('/app/x.txt');
3File::fromSources('2 + 2');
4
5// 4.x
6FileSource::createFromPathname('/app/x.txt');
7StringSource::createFromString('2 + 2');
8VirtualSource::createFromString('x.txt', '2 + 2'); // a string with a name

SourceFactory::createDefault() is there if you need the "figure out what this is" behaviour, now behind a single create() method.

See Source.

Positions Are Calculated By A Factory

Likelihood Of Impact: Medium

phplrt/position is still there, with a smaller API. A Position is the line and the column alone - the offset it was built from is not a part of it, and createFromOffset() already knows it.

createAtStarting() is spelled new Position(), createAtEnding() is an offset past the end of the source, and Interval, IntervalFactoryTrait and PositionFactoryTrait are gone.

See Position.

The .pp Format Is No Longer Read

Likelihood Of Impact: Medium

Grammars written in the original Hoa-style .pp format are not supported. A .pp file is still recognized by its extension, so you get a clear error rather than a confusing parse failure. Rewrite the grammar in one of the formats that are read - see PP3 Grammar Syntax.

Grammar Files: What To Check

Likelihood Of Impact: Medium

A grammar file keeps its extension and keeps being read the way it was, so most of them compile unchanged. What no longer works:

The old pragmas. Unification and the error levels are gone; the corresponding behaviour is either the default now or is configured in PHP. Which settings a grammar may carry is listed under Settings.

$file and $state in a reducer. Use $source and $rule, which is an int. See Results and Reducers.

Left recursion, which is now rejected at build time. It never worked at runtime either, but 3.x would let you compile it. Rewrite as a repetition:

1// ✘ rejected
2Expression : Expression() ::T_PLUS:: Number() ;
3
4// ✔
5Expression : Number() (::T_PLUS:: Number())* ;

Reducers returning null mean "no result" and pass the children through. If a rule of yours legitimately produces null, wrap it.

Reducers returning arrays are flattened into the rule above. If you relied on nesting, return an object instead. See Results and Reducers.

Everything Else That Is Gone

Likelihood Of Impact: Low

Code generation, dropped in 3.0 in favour of a config array, is back and now emits the lexer, the token constants and the reducers as a real class - see Compiling a Grammar.