Automation and CI

Rebuilding the parser after a grammar change is the kind of step that works right up until the day somebody forgets. Two composer scripts and a CI job turn it from a habit into something the build enforces.

Fitting It Into A Project

The usual layout:

1resources/grammar/
2    grammar.pp3
3    lexemes.pp3
4src/Parser/
5    LanguageParser.php   <- generated, committed

The command is short enough to type and long enough to get wrong, so it usually ends up in composer.json:

1{
2    "scripts": {
3        "grammar:check": "phplrt check resources/grammar.pp3",
4        "grammar:build": "phplrt compile resources/grammar.pp3 src/Parser/LanguageParser.php --class LanguageParser --namespace \"App\\\\Parser\""
5    }
6}

composer grammar:build

Composer puts vendor/bin on the path for its own scripts, so phplrt needs no prefix there.

Run it whenever the grammar changes, and commit the result. Two reasons to commit rather than generate on deploy: the file is what static analysis and your IDE actually see, and a broken grammar fails in a pull request instead of in production.

Checking It In CI

Two things are worth automating, and both are exit codes:

# 1. does the grammar still compile?
php vendor/bin/phplrt check resources/grammar.pp3

# 2. does the committed parser still match the grammar?
php vendor/bin/phplrt compile resources/grammar.pp3 src/Parser/LanguageParser.php \
    --class LanguageParser --namespace "App\Parser"
git diff --exit-code -- src/Parser/LanguageParser.php

The first fails on a grammar that no longer compiles. The second regenerates the parser and fails if the result differs from what is in the repository - the day someone edits the grammar and forgets to rebuild.

Run check with -v: the numbers land in the build log, so a reviewer can see the rule and token counts move in a pull request instead of guessing what a grammar change did. Command Line explains what they mean.

GitHub Actions

 1name: Grammar
 2
 3on:
 4  push:
 5    paths: [ 'resources/**', 'src/Parser/**' ]
 6  pull_request:
 7    paths: [ 'resources/**', 'src/Parser/**' ]
 8
 9jobs:
10  grammar:
11    name: Grammar
12    runs-on: ubuntu-latest
13    steps:
14      - name: Checkout
15        uses: actions/checkout@v7
16      - name: Setup PHP
17        uses: shivammathur/setup-php@v2
18        with:
19          php-version: '8.4'
20      - name: Install Dependencies
21        run: composer install --prefer-dist --no-interaction --no-progress
22      - name: Check The Grammar
23        run: php vendor/bin/phplrt check resources/grammar.pp3 -v
24      - name: Check The Parser Is Not Stale
25        run: |
26          php vendor/bin/phplrt compile resources/grammar.pp3 src/Parser/LanguageParser.php \
27              --class LanguageParser --namespace "App\Parser"
28          git diff --exit-code -- src/Parser/LanguageParser.php

The paths filter keeps the job off every unrelated commit. Drop the last step if the parser is built on deploy rather than committed.

GitLab CI

 1grammar:
 2  image: php:8.4-cli
 3  rules:
 4    - changes:
 5        - 'resources/**/*'
 6        - 'src/Parser/**/*'
 7  before_script:
 8    - apt-get update -yqq && apt-get install -yqq git unzip
 9    - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
10    - composer install --prefer-dist --no-interaction --no-progress
11  script:
12    - php vendor/bin/phplrt check resources/grammar.pp3 -v
13    - >
14      php vendor/bin/phplrt compile resources/grammar.pp3 src/Parser/LanguageParser.php
15      --class LanguageParser --namespace "App\Parser"
16    - git diff --exit-code -- src/Parser/LanguageParser.php

git and unzip are what composer needs and the bare php image does not have. git diff needs the checkout to be a repository, which the default GIT_STRATEGY gives you - a job running with GIT_STRATEGY: none can still run check, but not the staleness step.

More Than One Grammar

The obvious loop is a trap:

# WRONG - find exits 0 even when a grammar fails to compile
find resources -name '*.pp3' -exec php vendor/bin/phplrt check {} \;

xargs reports the failure instead, which is what CI reads:

find resources -name '*.pp3' -print0 \
    | xargs -0 -n1 php vendor/bin/phplrt check

Doing It From PHP

A project that generates through the API rather than the binary - a generator of its own, several grammars in one parser - runs a script instead:

1bin/
2    build-parser.php     <- runs the generator

1{
2    "scripts": {
3        "grammar:build": "php bin/build-parser.php"
4    }
5}

The staleness check has a PHP form too, since the output is Stringable:

1$expected = (string) $output;
2$actual = \file_get_contents(__DIR__ . '/LanguageParser.php');
3
4if ($expected !== $actual) {
5    throw new \RuntimeException('Parser is out of date, run "composer grammar:build"');
6}

Compiling a Grammar covers the API this leans on.