Package org.antlr.v4.test.runtime.java.api.perf.jmh

Class ComparativeParseHarness

java.lang.Object

org.antlr.v4.test.runtime.java.api.perf.jmh.ComparativeParseHarness

public final class ComparativeParseHarness
extends Object

Deterministic, machine-readable end-to-end harness for A/B performance verification across commits. Complements JMH with:

• serial vs parallel multi-file parse (Groovy-style front ends)
• warm shared DFA vs cold (clear-per-file) ATN hot paths
• SLL, LL, two-stage, and ProfilingATNSimulator strategies
• lexer reset/setInputStream reuse
• trimmed-mean timing with explicit warmup passes

Run:

   mvn -pl perf-testsuite -am package -DskipTests
   java -Xms2g -Xmx2g -cp perf-testsuite/target/classes:...(deps) \
     org.antlr.v4.test.runtime.java.api.perf.jmh.ComparativeParseHarness \
     --label 4.13.2.12 --files 80 --warmup 5 --iters 15 --threads 0
 

Output is TSV (one result row per scenario) plus a human summary, so results from two commits can be joined on the scenario column.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static final class	ComparativeParseHarness.ScenarioResult

Field Summary

Fields

Modifier and Type	Field	Description
private static final Locale	LOCALE

Constructor Summary

Constructors

Constructor	Description
ComparativeParseHarness()

Method Summary

Modifier and Type	Method	Description
static void	main(String[] args)
private static String	opt(Map<String,String> m, String key, String dflt)
private static Map<String,String>	parseArgs(String[] args)
private static double	percentile(double[] values, double p)
private static void	printHeader(PrintStream out, String label, int files, int warmup, int iters, int threads)
private static void	printSummary(PrintStream err, String label, List<ComparativeParseHarness.ScenarioResult> results)
private static void	printTsv(PrintStream out, String label, List<ComparativeParseHarness.ScenarioResult> results)
private static ComparativeParseHarness.ScenarioResult	runBatch(String name, List<CorpusLoader.SourceFile> files, ParseWorkload.PredictionStrategy strategy, boolean clearDfaPerFile, boolean clearDfaAtStart, int threads, int warmup, int iters)
private static ComparativeParseHarness.ScenarioResult	runLex(String name, CorpusLoader.SourceFile file, boolean clearDfa, int warmup, int iters)
private static ComparativeParseHarness.ScenarioResult	runLexReuse(String name, CorpusLoader.SourceFile file, int warmup, int iters)	Same source rebound onto one lexer via Lexer.setInputStream(org.antlr.v4.runtime.CharStream) (calls Lexer.reset()).
private static ParseWorkload.Stats	runOneBatch(List<CorpusLoader.SourceFile> files, ParseWorkload.PredictionStrategy strategy, boolean clearDfaPerFile, boolean clearDfaAtStart, int threads)
private static ComparativeParseHarness.ScenarioResult	runParse(String name, CorpusLoader.SourceFile file, ParseWorkload.PredictionStrategy strategy, boolean clearDfa, int warmup, int iters)
private static ComparativeParseHarness.ScenarioResult	toResult(String name, int threads, boolean clearDfa, String strategy, int files, long chars, int iters, long[] ns, long tokens, int errors)
private static double	trimmedMean(double[] values, double trimFraction)	Drop top/bottom trimFraction then average (robust to GC hiccups).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOCALE

private static final Locale LOCALE

Constructor Details

ComparativeParseHarness

public ComparativeParseHarness()

Method Details

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception

runLex

private static ComparativeParseHarness.ScenarioResult runLex(String name,
 CorpusLoader.SourceFile file,
 boolean clearDfa,
 int warmup,
 int iters)

runLexReuse

private static ComparativeParseHarness.ScenarioResult runLexReuse(String name,
 CorpusLoader.SourceFile file,
 int warmup,
 int iters)

Same source rebound onto one lexer via Lexer.setInputStream(org.antlr.v4.runtime.CharStream) (calls Lexer.reset()). Isolates the reset / interp-reset path from lexer construction.

runParse

private static ComparativeParseHarness.ScenarioResult runParse(String name,
 CorpusLoader.SourceFile file,
 ParseWorkload.PredictionStrategy strategy,
 boolean clearDfa,
 int warmup,
 int iters)

runBatch

private static ComparativeParseHarness.ScenarioResult runBatch(String name,
 List<CorpusLoader.SourceFile> files,
 ParseWorkload.PredictionStrategy strategy,
 boolean clearDfaPerFile,
 boolean clearDfaAtStart,
 int threads,
 int warmup,
 int iters)
                                                        throws Exception

Parameters:

clearDfaPerFile - clear before every file (serial ATN-path stress)

clearDfaAtStart - clear once before each measured batch (DFA rebuild)

Throws:

Exception

runOneBatch

private static ParseWorkload.Stats runOneBatch(List<CorpusLoader.SourceFile> files,
 ParseWorkload.PredictionStrategy strategy,
 boolean clearDfaPerFile,
 boolean clearDfaAtStart,
 int threads)
                                        throws Exception

Throws:

Exception

toResult

private static ComparativeParseHarness.ScenarioResult toResult(String name,
 int threads,
 boolean clearDfa,
 String strategy,
 int files,
 long chars,
 int iters,
 long[] ns,
 long tokens,
 int errors)

trimmedMean

private static double trimmedMean(double[] values,
 double trimFraction)

Drop top/bottom trimFraction then average (robust to GC hiccups).

percentile

private static double percentile(double[] values,
 double p)

printHeader

private static void printHeader(PrintStream out,
 String label,
 int files,
 int warmup,
 int iters,
 int threads)

printTsv

private static void printTsv(PrintStream out,
 String label,
 List<ComparativeParseHarness.ScenarioResult> results)

printSummary

private static void printSummary(PrintStream err,
 String label,
 List<ComparativeParseHarness.ScenarioResult> results)

parseArgs

private static Map<String,String> parseArgs(String[] args)

opt

private static String opt(Map<String,String> m,
 String key,
 String dflt)