4.9.1 bug fix release

Issues fixed

• Invalid C# code being generated (code-gen)
• netstandard 2.0 build (comp:build, target:csharp)

Pull requests grouped by target

csharp target

• Fix typo (target:csharp)
• ensure 1 statement per line to help debug (target:csharp)
• C# runtime upgrade (comp:build, target:csharp)

cpp target

• travis: upgrade c++ build to focal & llvm 10 (comp:build, target:cpp)

go target

java target

javascript target

• upgrade ini (target:javascript)
• Potential fix to #2980 (target:javascript)
• rollback partially incorrect fix for #2902 (target:javascript)
• Fixed exception clauses in JS template. (target:javascript)
• 🚑 fix bad import (target:javascript)

python2 target

• Fix incorrect python token string templates (target:python2, target:python3)
• add optional encoding for string input in python 2 (target:python2)

python3 target

• Improve python3 performance by adding slots (target:python3)
• Fix incorrect python token string templates (target:python2, target:python3)
• Py3 Runtime: distribute the tests (target:python3)

Contributors

• 8 items: ericvergnaud
• 2 items: Ivan Kochurkin (KvanTTT)
• 1 items: Alex Mykyta (amykyta3)
• 1 items: Ozzah
• 1 items: Tabari Alexander (tehbone)
• 1 items: David Phillips (electrum)
• 1 items: maxence-lefebvre
• 1 items: Jim Baltikauskas (JBaltika)
• 1 items: Felix N (felixn)
• 1 items: Michael R. Crusoe (mr-c)

4.9 New dart target, bug fixes, small enhancements

New DART target

Thanks to @lingyv-li, we have a DART target!

Issues fixed

• [Python3] Large import time cost paid for inline unittests ()
• Dart tests fail (target:dart)
• C++20 can not construct a std::string with a UTF-8 encoded string literal ()
• Warning in generated javascript code when using node 14 ()
• Python3 parser generation incorrect by ANTLR 4.7.1 and problem in Parser.py (target:python3)
• PHP target generates invalid output when $ is used as part of the literal in lexer rule (target:php)
• [C++][CMake] find_package() fails when the C++ runtime is make-install'ed ()
• [Go] Generated lexer and parser file names are always in lower case with underscore ()
• Syntax error in antlr4/runtime/Swift/boot.py (target:swift)

Improvements, features

• Avoid stepping through ANTLR generated code when debugging (C#) (target:csharp, type:improvement)
• replace gramamr with grammar (type:improvement)
• Dart Target (target:dart, type:improvement)

Pull requests grouped by target

csharp target

• Made C# Trace Listener usable for most cases (target:csharp)
• Avoid stepping through ANTLR generated code when debugging (C#) (target:csharp, type:improvement)
• refactored AntlrInputStream.cs by removing a redundand if-statement (target:csharp)
• Clarify and expand documentation for runtime ParseTreeWalker (target:csharp)
• NullReferenceException in CSharp target #2304 (target:csharp)

cpp target

• cpp-target.md link broken fix (comp:doc, target:cpp)
• Fix linker issue when compiling Cpp runtime with CMake and MSVC on Windows (comp:build, target:cpp)
• Fix regressions in travis Cpp tests (target:cpp)
• Fix clang compile error with --stdlib=libc++ (target:cpp)
• [C++ runtime] Remove codecvt header (deprecated in C++17) in cpp runtime. (target:cpp)
• Fix C++ target README example (comp:doc, target:cpp)
• [C++ codegen] Add default cases to AltBlock and OptionalBlock (target:cpp)
• [C++ codegen] Remove use of the u8"" string literal prefix (target:cpp)
• [C++ codegen] Add explicit keyword to constructor (target:cpp)
• [C++ runtime] Fix static const ODR violations (target:cpp)
• [C++ runtime] Add AntlrInputStream(std::string_view) constructor (target:cpp)
• [C++ runtime] Improvements of Vocabulary::fromTokenNames (target:cpp)
• [C++ runtime]: Improve coding style of auto usages (target:cpp)
• C++ target: don't generate extra/duplicate semicolons (target:cpp)
• C++ target: escape ? character to prevent accidental trigraphs (target:cpp)
• Fixed link to CMake example in cpp-target.md (comp:doc, target:cpp)
• Update cpp-target.md doc (tree::ParseTreeWalker::DEFAULT is now a reference) (actions, target:cpp)
• Fix CMake packaging (target:cpp)
• Fix #2727: improve setTokenFactory in Cpp target (target:cpp)

go target

• [Go] Thread-safe ANTLR codegen (target:go)
• [Golang] enable syntax highlighting (comp:doc, target:go)

java target

javascript target

• Corrected Webpack configuration in JavaScript documentation (comp:doc, target:javascript)
• Javascript es6 migration (target:javascript)
• Fix template issue (target:javascript)
• Fix #2902 (target:javascript)
• Fix a NPE (target:javascript, type:bug)
• fix for issues in #1759 (target:javascript)
• Javascript: migrate prototypical root modules to ES6 (target:javascript)
• Javascript: fix some cyclic dependencies (target:javascript)
• Javascript: migrate prototypical ATN objects to ES6 (target:javascript)
• Javascript: browser support increased with babel transformation (target:javascript)
• Drop browser tests (target:javascript)
• Javascript: migrate prototypical Tree objects to es6 classes (target:javascript)
• Javascript: migrate prototypical DFA objects to es6 classes (target:javascript)

python2 target

• Fix code generation for catch statements in python (target:python2, target:python3)

python3 target

• Changed Python runtime Lexer.py, Parser.py to support version >= 3.5 (target:python3)
• Fix undefined value (target:python3)
• changed python3 runtime Lexer.py, Parser.py to support version > 5. (target:python3)
• separate py3 runtime tests from src tree + actually run said tests in CI (target:python3)
• Fix code generation for catch statements in python (target:python2, target:python3)

swift target

• Added missing swift version of class ParseTreeProperty (target:swift)
• [Swift] Update codegen template (target:swift)
• added throws declaration missing from generated swift code (target:swift)

php target

• [PHP] Escape dollar sign in target string literal (target:php)
• fix composer command in documentation (comp:doc, target:php)

dart target

• Fix travis ci for Dart pin version (target:dart)
• Pin dart version to 2.8.4 to get unit tests passing (comp:build, target:dart)
• Dart Target (target:dart, type:improvement)

Contributors

• 7 items: Camilo Roca (carocad)
• 6 items: ericvergnaud
• 5 items: Nathan Burles (nburles)
• 3 items: cliid (cliid)
• 3 items: Felix N (felixn)
• 3 items: Larry Li (lingyv-li)
• 3 items: sullis
• 2 items: Adam Wójs (adamwojs)
• 2 items: Rob McGregor (rmcgregor1990)
• 1 items: Michel Hartmann (MichelHartmann)
• 1 items: Adarsh Bhat (adarshbhat)
• 1 items: Martin Mirchev (Marti2203)
• 1 items: Rex Jaeschke (RexJaeschke)
• 1 items: Muhammed Zakir (MuhammedZakir)
• 1 items: trenki2
• 1 items: Arunav Sanyal (Khalian)
• 1 items: Tian Siyuan (tiansiyuan)
• 1 items: Arthur Sonzogni (ArthurSonzogni)
• 1 items: demons (Clcanny)
• 1 items: ericliu (eric-buaa-cn)
• 1 items: Steven Johnstone (stevenjohnstone)
• 1 items: Terence Parr (parrt)
• 1 items: Shan M (sha-N)
• 1 items: Akos Kiss (akosthekiss)
• 1 items: Gareth Andrew Lloyd (Ignition)
• 1 items: Crasse maximum (IohannRabeson)
• 1 items: Pablo Blanco Celdrán (sigmasoldi3r)
• 1 items: Kilian Kilger (cohomology)
• 1 items: Franco Victorio (fvictorio)
• 1 items: David MARKEY (d-markey)
• 1 items: [[OX...

4.8 new PHP target, bug fix release

Thanks to @marcospassos, ANTLR 4.8 introduces a PHP target! See PHP PRs

Issues fixed

• C# Antlr4.Runtime.Standard v4.7.2.0 assembly signature problem (comp:tool, status:duplicate, target:csharp)

Improvements, features

cpp target

• C++ VS project updates for next ANTLR4 release (comp:build, comp:doc, target:cpp)
• don't unnecessary fetch all tokens when grabbing an interval of text … (target:cpp)
• fix cpp cmake for emscripten (target:cpp)

go target

• Remove unnecessary Travis files. (target:go)

javascript target

• Fix placement of @members in Javascript target lexers (target:javascript)
• Fix main entry point for JavaScript runtime npm package (target:javascript)

swift target

• [Swift] Migrate Swift runtime to Swift 5. (target:swift)
• [Swift] Check process exit status during tests. (target:swift)

Contributors

• 5 items: Ewan Mellor (ewanmellor)
• 1 items: Mike Lischke (mike-lischke)
• 1 items: Marcos Passos (marcospassos)
• 1 items: jimmy-the-eek
• 1 items: Tabari Alexander (tehbone)
• 1 items: Zhiqiang Lu (zqlu)
• 1 items: WalterCouto
• 1 items: Markus Zancolo (kuegi)

ANTLR 4.7.2

Issues fixed

• Antlrv4 failed installation (actions)
• [cpp] Setting contextSuperClass='antlr4::RuleContextWithAltNum' doesn't work (target:cpp)
• Please publish JDK 9 Automatic-Module-Name entries (comp:build)

Improvements, features

• Assign token names to string literals in parse rule contexts (code-gen, type:improvement)
• File path handling fixes (comp:tool, type:improvement)
• Fixed warning from MingGW and error from Clang Windows (target:cpp, type:improvement)
• Fix null pointers + remove unused lines (type:improvement)

Pull requests grouped by target

csharp target

• fix NPE (comp:build, target:csharp)
• Fix #1955 python (target:csharp, target:python2, target:python3)
• Updated csharp-target doc (comp:doc, target:csharp)
• Make C# NuGet package name more visible in runtime docs (comp:doc, target:csharp)

cpp target

• virtual dtor for ParseTreeProperty (target:cpp)
• Do not throw std::bad_cast from antlrcpp::Any::is (target:cpp)
• Add const accessors to antlrcpp::Any C++ class (target:cpp)
• Fix for antlr#2164. (target:cpp)
• Applied C++ changes for PR #1955. (target:cpp)
• Fix #2350 (target:cpp)
• Add support c++ target on android (comp:build, target:cpp)
• Changes to cpp runtime to make antlr work with Chromium build (target:cpp)
• Fixed warning from MingGW and error from Clang Windows (target:cpp, type:improvement)
• Fix #1855, #2211 (comp:runtime, target:cpp)
• Support for CMake Packages in the Cpp Runtime (target:cpp)
• Force usage of 'previousContext'. (target:cpp)
• Don't create exception objects if you don't throw them. (target:cpp)
• Patch for multi-threaded parsing with the C++ runtime (target:cpp)
• Add missing -std=c++11 flag when using non-appleclang compiler on macosx (comp:build, target:cpp)
• Add Visual Studio 2017 project to Cpp runtime. Also change define for static library setting to be ANTLR4CPP_STATIC, in all projects. (comp:build, target:cpp)
• C++ runtime: Make Parser::exitRule() non-virtual (comp:performance, target:cpp)

go target

java target

• Avoid StringIndexOutOfBoundException (target:java, type:bug, unicode)
• Java.stg: use static functions instead of inline arrays (code-gen, target:java)

javascript target

• Fix #1955 javascript (target:javascript)
• fix indentation in javascript target examples (comp:doc, target:javascript)
• JavaScript parse tree visitor: check if ctx.children is non-null in visitChildren (target:javascript, type:bug)
• Remove unused _loadString parameter (target:javascript)

python2 target

• Fix #1955 python (target:csharp, target:python2, target:python3)
• fixes #2188 (target:python2, target:python3)

python3 target

• Fix #1955 python (target:csharp, target:python2, target:python3)
• Fix install dependency for 'typing' library in python3.4 (target:python3)
• Update to python 3.6 (comp:build, target:python3)
• Allow importing StdinStream from antlr4 package (target:python3)
• Add StdninStream for python3 runtime. Closes: #2271 (target:python3)
• Fixes #2301 (target:python3)
• Enable ParserATNSimulator debug mode in Python3 target (target:python3, type:bug)
• fixes #2188 (target:python2, target:python3)
• [Python3] return result instead of None in visitor (target:python3)

swift target

• [Swift-runtime] Make sure mutexes are being freed accordingly (target:swift)
• [Swift] Syntax error in antlr4/runtime/Swift/boot.py (target:swift)
• [Swift] Make ParserATNSimulator.precedenceTransition return a nullable value. (target:swift)
• Move the Swift runtime to Swift 4.2 (target:swift)
• Testability and build improvements (comp:build, comp:testing, target:swift)
• Enable some performance tests on Swift. (comp:testing, target:swift)
• [Swift] Report InputMismatchException with original context information (target:swift)
• [Swift] Minor tidyups in the stream code. (target:swift)
• [Swift] Make ATNDeserializationOptions a simple struct. (target:swift)
• [Swift] Remove HashMap and fix Parser caches. (target:swift)

Contributors

• 11 items: Ewan Mellor (ewanmellor)
• 7 items: ericvergnaud
• 4 items: Mike Lischke (mike-lischke)
• 2 items: Prateek Gulati (prateekgulati)
• 2 items: Adriaan Duz (nxtstep)
• 2 items: Steve Vinoski (vinoski)
• 2 items: Vladimir Goncharov (AmatanHead)
• 2 items: Maxim Novikov (m-novikov)
• 2 items: Mark Raynsford (io7m)
• 1 items: hannemann-tamas
• 1 items: Prashast Srivastava (prashast)
• 1 items: Alex Mykyta (amykyta3)
• 1 items: Sergey Chupov (scadgek)
• 1 items: Terence Parr (parrt)
• 1 items: Ivan Kochurkin (KvanTTT)
• 1 items: Lucas Henrique (LucasHMS)
• 1 items: 吴晟 Wu Sheng (wu-sheng)
• 1 items: Ale Contenti (alecont)
• 1 items: Dan Halperin (dhalperi)
• 1 items: Stephen Vance (srvance)
• 1 items: sinopsysHK
• 1 items: johnvanderholt
• 1 items: Joe R. Smith (solussd)
• 1 items: uvguy
• 1 items: Daniel Clifford (dannoc)
• 1 items: Chase (ChaseOxide)
• 1 items: Jędrzej Biedrzycki (TAndronicus)
• 1 items: Raz Friman (razfriman)
• 1 items: Zongyuan Zuo (EternalPhane)
• 1 items: Sönke Schau (xgcssch)
• 1 items: Kaspar Emanuel (kasbah)
• 1 items: Robert (robertbrignull)
• 1 items: Corey Kosak (kosak)
• 1 items: jkmar
• 1 items: Kyle Sletten (kbsletten)
• 1 items: Ajay Panyala (ajaypanyala)
• 1 items: Nicolas Croad (niccroad)
• 1 items: janlinde
• 1 items: Enes SOYLU (enessoylu)

4.7.1

ANTLR version 4.7.1 is a minor release but with lots of little improvements and bug fixes. You can find the pull requests grouped by target language below. Also, please find below the contributor list (auto-generated from the issues and pull request).

Summary of new features and improvements

• You can now import lexers with modes into other lexer grammars [...]
• The -o and -lib commandline options didn't always do the obvious thing and in fact presented some problems. With clean that up but requires a new -Xexact-output-dir command line option to enable it to avoid breaking tools built on previous versions of ANTLR [...]
• For languages that support case insensitive keywords such as SQL, ANTLR does not have a built-in mechanism. Programmers either specify case insensitive lexers rules directly or they can build a special input stream that presents characters as uppercase to the lexer. I have provided documentation that describes both approaches including sample streams in various targets. [...]
• The Swift runtime was migrated to Swift 4. [...]
• You can generate SVG now from parse trees in Java target. [...]
• The NuGet package now includes the .NET Core DLL and supports both .NET 3.5 and up, as well as .NET Core. [...]

Issues fixed

• Python3 runtime says 4.6 not 4.7 (status:invalid, target:python3)
• Can not generate java files from Rust grammar file (actions, code-gen, type:bug)
• Unexpected code generation location behavior (cross-platform-issue, type:bug)
• Compiling seperate lexer and parser in subdirectory fails (comp:build, status:duplicate, type:bug)

Improvements, features

• Changed rotation to support 32 and 64-bit architectures as noted in issue #2060. (target:go, type:improvement)
• Case insensitivity doc (comp:doc, comp:runtime, lexers, type:improvement)
• Import lexers with modes into other lexer grammars (comp:tool, lexers, type:improvement)
• Add an accessLevel parser option. (actions, type:improvement)
• Support building C++ Runtime with Intel compilers (target:cpp, type:improvement)
• Improve the "gnu" message format (comp:tool, error-handling, type:improvement)
• Security related issue with System.getenv (comp:runtime, type:improvement)
• New unreachable token value warning (comp:tool, error-handling, type:improvement)
• Simplify output path generation; new -Xexact-output-dir option (actions, comp:build, comp:tool, type:improvement)
• Remove ANTLRError.nullPointer from the Swift runtime. (type:improvement)
• Migrate the Swift runtime to Swift 4. (target:swift, type:improvement)
• Report InputMismatchException with original context information (error-handling, target:java, type:improvement)
• Many fixes to antlr4-maven-plugin dependency analysis (comp:build, type:improvement)
• Implemented TokenStreamRewriter (comp:runtime, target:go, type:improvement)
• added SVG export of parse tree (target:java, trees-contexts, type:improvement)
• NuGet packaging for .NET Core (comp:build, target:csharp, type:improvement)
• CPP target runtime: fixed a memleak (target:cpp, type:improvement)
• Support importing multi-mode lexer grammars (comp:runtime, lexers, type:improvement)

Pull requests grouped by target

csharp target

• Add extra diagnostics and a retry to the C# runtime tests and upgrade the macOS runtime (comp:build, comp:testing, target:csharp)
• #2021 fixes nuget packaging options to avoid missing dll exceptions (comp:build, target:csharp)
• fix to the ATN deserialiser issue on big endian architecture (target:csharp)
• for linux target, bring up to jdk8 since jdk7 no longer available for Trusty; for linux and osx targets, bring up dotnet from 1.0.3 to 1.0.4; Trusty image now has mvn 3.3.9 builtin, removed install code (comp:build, target:csharp)
• Set version to 4.7.1 because 4.7.0 already exists on Nuget (target:csharp)
• NuGet packaging for .NET Core (comp:build, target:csharp, type:improvement)
• speed up tests; rename runtime dll ready for nuget (target:csharp)
• update to .netcore v1.0.1; add osx build (comp:build, target:csharp)

cpp target

• The correction for cmake depends on the demo (comp:build, target:cpp)
• Support building C++ Runtime with Intel compilers (target:cpp, type:improvement)
• CMake build fixes for Visual Studio (comp:build, target:cpp)
• CMake build fixes for Visual Studio (comp:build, target:cpp)
• Add MSVC support to c++ cmake build (comp:build, target:cpp)
• Remove the useless visibility attribute from this "enum class" declaration (target:cpp)
• C++ runtime: Fix two issues found by coverity (target:cpp)
• Build fixes after previous C++ patch (comp:build, target:cpp)
• Rework C++ Runtime Interval, IntervalSet, ATN and ATNState (atn-analysis, target:cpp)
• Build fix/correction for UTF32 conversion. (target:cpp)
• Small optimizations (target:cpp)
• C++ runtime changes for high warning levels (target:cpp)
• Fix listener example in C++ documentation. (comp:doc, target:cpp)
• Make ParseTreeWalker::DEFAULT provide an IterativeParseTreeWalker (target:cpp, trees-contexts)
• Fix the utf8 convertor for VS2017 (target:cpp, unicode)
• Update C++ target doc for latest API changes - fixes #1824. (comp:doc, target:cpp)
• Channel parameter in c-tor is now a size_t - fixes #1827. (target:cpp)
• Small fix in cmake file. Fixes #1809. (comp:build, target:cpp)
• CPP target runtime: fixed a memleak (target:cpp, type:improvement)

go target

• Export GetHiddenTokens routines (target:go)
• Fix issue with tree finding when the underlying slice needs to be resized. (target:go)
• Changed rotation to support 32 and 64-bit architectures as noted in issue #2060. (target:go, type:improvement)
• Add RuleContext interface methods to reserved words (target:go)
• Revert "[Go] Interval" (target:go)
• Placed the CommonSetStuff code inside a scope (code-gen, target:go)
• Export Go lexer methods (target:go)
• Allowing Go runtime compilation to succeed under ARM (target:go)
• Add CommonTokenStream test. (target:go)
• Interval (target:go)
• Implemented TokenStreamRewriter (comp:runtime, target:go, type:improvement)
• Add getter method to Interval. (target:go)
• Go file header complies with standardised Go 'generated code' header (code-gen, target:go)
• Fix #1925 (target:go, target:javascript, target:python2, target:python3, unicode)
• Add proper error handling semantics to FileStream. (target:go)

java target

• Fixes #2069 to catch env var security exception (target:java, type:bug)
• Fix codegen error (Java) when a lexer has only fragment tokens (code-gen, target:java)
• [Fixed empty CodePointCharStream throwing ex...

4.7

ANTLR version 4.7 is a major release with many improvements and bug fixes.

Summary of new features, improvements, fixes

• The primary improvement is that ANTLR and all code generation targets can now handle full 21-bit unicode thanks to the superhuman effort of Ben Hamilton, @bhamiltoncx. After much thought concerning the "create stream" interface, I have decided on the following end result: C++, Python, Go, and Swift APIs didn't need any changes to support full Unicode code points, so I decided to leave those alone. Java, C#, and JavaScript runtimes required changes. Rather than gutting and changing the interface of the previous ANTLRFileStream etc..., I have deprecated those and introduced a CharStreams.fromXXX factory style interface for creating streams. For example, CharStreams.fromFileName("foo.txt")). See the new unicode documentation and a complete list of all pull requests related to Unicode.

• UnbufferedCharStream for Java and C# targets was upgraded to use UTF-8 rather than the locale default encoding; they further support U+10FFFF Unicode code points. C++ is the only other target with the unbuffered stream and it worked as-is.

  • Change UnbufferedCharStream to use code points
  • Use UTF-8 by default in UnbufferedCharStream and allow specifying charset
  • C#: Change UnbufferedCharStream to use 32-bit Unicode code points and 32-bit buffer

• In addition to creating CharStreams capable of supporting 21-bit Unicode, we added notation for Unicode code points beyond 16 bits (4 hexadecimal digits). The usual notation \uABCD still works for the basic code points but you can now use \u{1FFFF}..\u{10FFFF} to access the full range:

  • New extended Unicode escape \u{10ABCD} to support Unicode literals > U+FFFF
  • Support UTF-32 escape sequences

• You can also include all characters matching Unicode properties such as [\p{Emoji}]:

  • Add new \p{Extended_Pictographic} and related Unicode property escapes
  • Also support Unicode enumerated properties via \p{Foo=Bar} syntax
  • Update docs for new Unicode literal and property escapes
  • New class EscapeSequenceParsing with \p{UnicodeProperty} support
  • New \p{Letter} Unicode property escape
  • A few last escapes: \p{EmojiPresentation=EmojiDefault} and \p{EmojiPresentation=TextDefault}
  • UnicodeData: Also support Unicode blocks

• The C# target now supports .NET Core Support, thanks to David Neumann @lecode-official and Dong Xie @xied75!

• We did quite a bit of cleaning up in the lexer escape char and char set areas (big thanks to Ivan Kochurkin @KvanTTT):

  • Escapes sequence recognition failure in character sets
  • Throw "invalid escape sequence" for double quote in string literals: '\"'
  • Fixed incorrect "used multiple times" warning.
  • error-> warnings. Fixes #1537
  • Reverted unterminated ranges: [+-], [-+], [-].
  • More accurate error messages
  • Added invalid charset error (for ranges without start or end) (comp:tool, error-handling)

• The Go runtime was significantly speeded up.

  • Go Runtime: Murmur Hash and various optimizations
  • Initialize decisionToDFA in init in Go.stg

• The XPath tree matching library used an ANTLR grammar itself, which caused a cyclic dependency in the build whereby ANTLR version v was required to build version v. I implemented a handbuilt lexer to get rid of this dependency on a grammar: XPathLexer not updated in release process, Fixes #1620. Make handbuilt lexer to avoid cyclic dependence of tool and plugin.

• The Java, Swift, and C++ targets have added a hook to the construction of parse tree leaf nodes: factor out the creation of error nodes and terminal nodes in the parser. Code new TerminalNodeImpl(..) and new ErrorNodeImpl(...) instead now calls a few factory methods in the Parser that you can override in your application.

Issues fixed

C++ target:

• SIGABRT when using TokenStreamRewrite::Delete method
• Cpp runtime: compile needed header
• compilation warning
• Segmentation fault on TokenStreamRewriter destructor call
• Runtime build failure on gcc 6.3.1
• Fixed regression introduced by wrong optimization. Fixes #1708. (target:cpp)
• Fixed crash with multi threaded parsers warming up at the same time. (target:cpp)
• Improve error message in exceptions (target:cpp)
• Using new Unicode syntax for C++ demo project. (target:cpp, unicode)
• Fix syntaxError prototype issue (target:cpp)
• Fixed a number of data type + signedness issues in the C++ target (target:cpp)
• Some compilers need the functional include. (target:cpp)
• Fix wrong include path of antlr4cpp runtime in cmake/ExternalAntlr4Cpp.cmake (target:cpp)
• Fix parse tree property (target:cpp)
• No static libs anymore for Windows C++ runtime. (comp:build, target:cpp)
• Lr loop fix (target:cpp, type:improvement)
• adapt code to compile with msys2 mingw compiler (comp:build, target:cpp, type:improvement)
• Fix for VS 2013 runtime builds (C++) (comp:doc, target:cpp)
• Implemented enhanced CommonToken::toString method
• Using new Unicode syntax for C++ demo project. (target:cpp, unicode)
• fix an issue where loading antlr from an IE web worker would fail (target:javascript)
• Issue 1483 (target:cpp, type:improvement)

JavaScript target:

• JavaScript target: super class of TraceListener is not correctly set
• BailErrorStrategy in JavaScript
• JavaScript visitors not visiting children
• Javascript Target documentation contains no reference to existing NPM package?
• Doc antlr4 / doc / javascript-target.md outdated, states there is no NPM package for antlr4 runtime
• tweak wildcard
• Fixed null pointer exception for JS target
• Fix missing variable declarations
• fix an issue where loading antlr from an IE web worker would fail
• fix typo in javascript visitor
• Update npm related docs
• missing js export

Python2/3:

• python2: Bug in IntervalSet.py:removeOne causes exception when taking complement
• Python 2 - missing IllegalStateException import in PredictionContext.py
• fix 'CommonToken' object has no attribute 'stopIndex' in Python{2|3}
• Fix a Python 2 typo
• Python 2 - missing - ErrorNode and TerminalNode - imports in Parser.py
• Python 2 - missing IllegalStateException import in PredictionContext.py

C#:

• Fixed portability problems in C# target. Also cleaned up some XML do…
• Fix C# Pair.cs, Right arrow escaping in XML Comments
• Fix #1298 for CSharp
• Remove unused C# runtime method Utils.ToCharArray
• Look in /usr/local/bin before /usr/bin for mono

Go:
...

4.6

ANTLR version 4.6 is a major release with many features and bug fixes.

Summary of new features, improvements, fixes

• NEW TARGETS: C++, Swift, Go:
  • Peter Boyer (Go target)
  • Mike Lischke (C++ completed target)
  • Dan McLaughlin (C++ initial target)
  • David Sisson (C++ initial target and test)
  • Janyou (Swift target)
  • Ewan Mellor, Hanzhou Shi (Swift target merging)
• Major performance improvement discovered by @sharwell. We were seeing a number of reports of nonterminating parses for very large expressions with grammars that used left recursive rule specifications (like e : e '*' e | INT ;). Sam noticed that the parsing engine detected an ambiguity between two choices and resolved it properly but that both would lead to a valid parse. It turns out that the first choice is what led to really slow parsing but the second choice is much faster and still yields a valid parse. Most targets have added that improvement. Here is the discussion of the issue and solution and the primary patch for java.
• Error handling improved in cases related to LL(1) decisions for optional and closure subrules. Matching sets of tokens were also sometimes considered resynchronization (error) tokens. Single token deletions that occurred at the start of expressions were sometimes not included in the parse tree as error nodes.
  • Add sync() calls back in for LL(1) decisions. (code-gen, error-handling)
  • Matched tokens being marked as error (comp:runtime, error-handling, trees-contexts, type:bug)
  • Single-token deletion error node not in parse tree (error-handling, trees-contexts, type:bug)
• The tool itself generates better error messages:
  • Crash with NullPointerException in SymbolChecks.checkForModeConflicts with empty lexer (comp:tool, grammars, type:bug)
  • Have ANTLR4 prevent conflict with user rule names by behind-the-scenes renaming of its own variables (status:cannot-reproduce)
  • Add compile-time errors/warnings for use of incompatible lexer commands (comp:tool, error-handling)
  • character set issues (comp:tool, error-handling, type:improvement)
  • Duplicated and incompatible commands (comp:tool, error-handling, type:improvement)
• We fixed a number of issues with the build system
  • Tidy poms (comp:build, type:improvement)
  • divide and conquer Travis CI (comp:build, type:improvement)
  • Version information omitted from MANIFEST.MF in antlr-4.5.1-complete.jar (comp:build, type:bug)
  • antrl4--sources.jar does not contain sources for runtime (comp:build, type:bug)
  • OSGi module name does not reflect Maven module name (comp:build, type:improvement)
  • Running a limited subset of runtime-testsuite (comp:testing, type:improvement)
• Improvements to the ANTLR4 mvn plug-in:
  • Grammar dependency management for Maven plugin (comp:build, comp:maven-plugin, type:improvement)
  • antlr4-maven-plugin: code generated in wrong encoding (comp:maven-plugin, type:improvement)
• The ANTLR unit test mechanism was completely rewritten but it should not affect anyone other than the development team
  • Refactor runtime test generation mechanism (comp:testing, type:improvement)
• We assume java 1.7 not 1.6 now.
  • Drop support for 1.6, move pom.xml etc. to JDK 1.7 (cross-platform-issue, type:improvement)
• Miscellaneous
  • make sure BSD 3-clause license prefix is consistent and on all appropriate files (type:improvement)
  • Allow paths as tokenVocab option (comp:build, comp:tool, type:improvement)
  • add TokenStreamRewriter to Python2 runtime (target:python2, type:improvement)
  • Add an iterative version of the ParseTreeWalker (comp:runtime, target:java, trees-contexts, type:improvement)
  • indicate token types as well as numbers in -tokens dump (target:java, type:improvement)
  • Binary parsing (comp:doc, type:improvement, type:question)

Issues fixed

• Python 2 Target's ErrorStrategy Creates Problematic Tokens (target:python2, type:bug)
• Prefix rules that also recurse are not detected (grammars, type:bug)
• mvn compile failing. Could not resolve dependencies (comp:build, comp:doc, type:bug)
• Repeated epsilon transitions in ATN from left-recursive grammar (atn-analysis, type:bug)
• Add sync() calls back in for LL(1) decisions. (code-gen, error-handling)
• Array2DHashSet.clear does not reset threshold (target:java, type:bug)
• ANTLR 4.5.3 & java.lang.StackOverflowError (comp:tool, type:bug)
• Matched tokens being marked as error (comp:runtime, error-handling, trees-contexts, type:bug)
• Single-token deletion error node not in parse tree (error-handling, trees-contexts, type:bug)
• $e.v yields incorrect value 0 in "e returns [int v] : '1' {$v = 1;} | '(' e ')' {$v = $e.v;} ;" (actions, grammars, type:bug)
• Strange code in JavaScript - PredictionContext.js line 191 (target:javascript, type:bug)
• A bug of the antlr4 python3 runtime 4.5.3 (target:python3, type:bug)
• Empty alternative in rule causes syntax error in *Parser.py for Python2/3 targets (target:python2, target:python3, type:bug)
• getText method with arguments not always works correctly (target:java, type:bug)
• maven plugin dependency tracking ignores imported files (comp:build, comp:maven-plugin, comp:tool)
• NullPointerException on invalid attribute reference (comp:tool, type:bug)
• getLookaheadParseTrees() should reset or create new error handler each alt but doesn't (comp:runtime, trees-contexts, type:bug)
• getAllPossibleParseTrees() doesn't work when ambig input reaches eof. (comp:runtime, trees-contexts, type:bug)
• Changing '+' to '*' crashes antlr4 (atn-analysis, parsers, type:bug)
• Crash with NullPointerException in SymbolChecks.checkForModeConflicts with empty lexer (comp:tool, grammars, type:bug)
• LB() can try to scan to i < 0 (comp:runtime, type:bug)
• Incorrect childs generation with array of element label (grammars, status:invalid)
• TokenStreamRewriter for Python (target:python2)
• getTokenTypeMap should test all token type values (comp:runtime, target:java, type:bug)
• Version information omitted from MANIFEST.MF in antlr-4.5.1-complete.jar (comp:build, type:bug)
• antrl4--sources.jar does not contain sources for runtime (comp:build, type:bug)
• template error: attribute parser isn't defined (target:javascript, type:bug)
• Add compile-time errors/warnings for use of incompatible lexer commands (comp:tool, error-handling)

4.5.3

Features / improvements

• New grammar option: contextSuperClass. All parse tree internal nodes will derive from this. Default is ParserRuleContext. Should derive from ultimately RuleContext at minimum.
  Java target can use contextSuperClass=org.antlr.v4.runtime.RuleContextWithAltNum for convenience. It adds a backing field for altNumber, the alt matched for the associated rule node.
• Added getMaxTokenType() to Vocabulary interface

Complete list of pull requests for 4.5.3 but most of those are fixing bugs.

Issues

Complete list of issues closed/solved for 4.5.2.

4.5.2

Features / improvements

• removed final (hopefully!) dependency on Java AWT lib in runtime.
• moved doc, faq from wiki to the repo.
• added more tool error checks.
• major Python speed improvement with better hash function (minutes to seconds)

Complete list of pull requests for 4.5.2 but most of those are fixing bugs.

Issues

Summary: some clean up in JavaScript and Python targets. Minor issues in Java/jar.

Complete list of issues closed/solved for 4.5.2.

4.5.1

Bug fix and repo reorganization release

We fixed number of important bugs but also combined the various target repositories, such as antlr/antlr4-python2, into the main antlr/antlr4 repository.

For the Java target only, there is also a new feature: a parser interpreter that tracks which alternative or label was match for a particular parse tree node, which is often useful during debugging. It is used in the 1.7 release of the ANTLR Intellij Plugin.

Summary of Major Changes

• Maven build was added back in, thanks to new contributor Jason van Zyl; you can just open the topmost pom.xml in your favorite IDE
• We completely overhauled runtime test generation
• We merged all existing target repositories such as antlr4-python2 into the main antlr4 repo so that everything is now included in a single spot.
• We removed runtime jar dependency on Swing. Class org.antlr.v4.runtime.misc.TestRig has moved to org.antlr.v4.gui.TestRig but we left a proxy in so that org.antlr.v4.runtime.misc.TestRig still works. The org.antlr.v4.runtime.tree.gui package moved to org.antlr.v4.gui in the tool area from the runtime. A few classes from org.antlr.v4.runtime.misc had to move. Convenience methods for saving/viewing parse trees were moved from RuleContext (parse tree) and org.antlr.v4.runtime.tree.Trees to org.antlr.v4.gui.Trees.

You can view all Issues fixed in 4.5.1, all pull requests merged and all commits for this release.

Features

• Add goto to reserved words for java
• Create functionality to get all possible parse trees for ambiguous input.
• Makes TestRig's dialog to save and restore its state
• Parser.setProfile() should keep settings like prediction mode
• Make Grammar-object-aware ParserInterpreter (This interpreter will have parse tree nodes that know which alternative or label was matched for a particular parse tree subtree root.)
• Check for nonconforming left-recursive rules.

Bug Fixes

• Null pointer in named alternative with attributes
• NullPointerException in bad recursion
• Parser.setProfile() should keep settings like prediction mode
• Allow parsing decision override for (decision, input index) in parsers
• Random ArrayIndexOutOfBounds exceptions coming from ParserATNSimulator
• Empty rule matched first gives invalid source interval
• EOF leaf not included in parent tree source interval
• Rule parameter in left recursive rules disallowed
• wrong location on import error message reference to undefined rule
• NotNull import is not deleted in Java.stg

Download Binaries

Download the ANTLR tool and all target runtimes at the antlr.org site.

The Java jars are OSGi compatible so you should be able to use them within Eclipse.

Runtime Libraries for Language Targets

• Java is embedded with the tool, but is also available from maven-central
• C# (zipped .dll) is a download from antlr.org. Might be in NuGet too in the future.
• JavaScript is a download from antlr.org. Might also be registered as a npm package for node.js
• Python2 and Python 3 are in PyPi
• Sam Harwell's alternative C# target is available through NuGet. Note that this alternative target is distributed with a standalone copy of the ANTLR Tool, and only works with code generated by that standalone copy.

Documentation

• Java runtime API (4.5.1)
• Java Tool API (4.5.1)
• ANTLR 4 Maven plugin (4.5.1)

Language Targets Info

As of 4.5, the standard distribution of ANTLR can generate code in the following languages:

• Java
• Python2
• Python3
• JavaScript
• C#

In addition, the following languages are supported by standalone release(s) of the tool.

• C# (Sam Harwell's)