CL Language Module
This is a parser for IBM Control Language (CL), with support for ILE CL and OPM CL.
It is intended to be used mainly as a JVM library. It can also be used as a command line tool.
The parser works in three different stages:
- First a list of tokens are created from some input stream (typically a File)
- Then a parse-tree is obtained by grouping the tokens and recognizing structures
- Finally an Abstract Syntax Tree (AST) is derived from the parse-tree. While the AST
resembles closely the parse-tree it contains re-arrangements and simplifications that makes
easier working with it. Control-flow constructs such as
IF/DO/SELECT/PGM/SUBRare nested in a second pass after the flat command stream is mapped.
The AST is built using the Kolasu library. This open-source library has been built by Strumenta. This tutorial on Kolasu explains how the library can be used: Building advanced parsers using Kolasu.
The AST could be used to perform analysis on the code or transformations of the code.
Typical CL source extensions recognized by the language module are .cl, .clp, and .clle.
Contents of the package
- Full JAR file with all dependencies (distribution shadow JAR)
- Windows and Linux scripts to execute the JAR
- Documentation in HTML format
How to use the parser
You can use the parser as a library (i.e., from code) or as a standalone program, from the command line.
How to use from the command line
This document describes how to use the CL parser from the command line. The distribution CLI is implemented in
com.strumenta.cl.distribution.cli.DistributionCLI and is packaged as a standalone fat JAR (Gradle task
:distribution:shadowJar, classifier all). It includes the same commands as the original AST-module CLI
(com.strumenta.cl.cli.CLKolasuParserCLI) plus codebase snapshot export.
Prerequisites
- Java Runtime Environment (JRE) on the classpath. The parser has been tested with JVM 11.
- A valid Strumenta license file for the
parseandexportsnapshotcommands. Those commands will not run unless a license is registered with--license.
Invoking the CLI
Run the JAR with java -jar:
java -jar clparser-distribution-<version>-all.jar [GLOBAL_OPTIONS] COMMAND [COMMAND_OPTIONS] [ARGUMENTS...]
Running the JAR without a subcommand prints the top-level help.
Command-name casing
Subcommand names are case-sensitive. Use the exact spelling below:
| Correct name | Common mistake |
|---|---|
downloadlicense | downloadLicense, download-license |
parse | — |
metamodel | — |
exportsnapshot | exportSnapshot, export-snapshot |
Global options
| Option | Description |
|---|---|
-h, --help | Show top-level help. |
License
Most parsing commands require a commercial license.
Commands that require --license
| Command | Option | Requirement |
|---|---|---|
parse | --license PATH | Required. Path to an existing, readable license file. |
exportsnapshot | --license PATH | Required. Path to an existing, readable license file. |
The license path can be passed as --license path/to/file.license or --license=path/to/file.license.
The license file must exist and be readable. Symlinks are accepted.
Commands that do not require a license
| Command | Purpose |
|---|---|
downloadlicense | Download a new license from the Strumenta license service. |
metamodel | Generate the EMF metamodel for the CL AST (no parsing of user CL files). |
Downloading a license
Use downloadlicense to obtain or refresh a license. Provide your credentials and the destination path:
java -jar clparser-distribution-<version>-all.jar downloadlicense USERNAME PASSWORD path/to/license.file
On success, the CLI prints confirmation and the license validity period:
License successfully downloaded at path/to/license.file
StartDate: Mon Nov 04 01:00:00 CET 2024
EndDate: Sat Dec 07 01:00:00 CET 2024
Downloaded licenses expire depending on your subscription or contract. Anytime you download a license, it will be downloaded with the updated expiration date.
Commands
parse — parse CL files to JSON or XML
Parses one or more CL files or directories and writes one AST file per input file.
java -jar clparser-distribution-<version>-all.jar parse --license=my.license [OPTIONS] INPUT...
| Option | Description |
|---|---|
--license PATH | Required. License file. |
-c, --charset TEXT | Charset used to read input files. Default: UTF-8. |
-f, --format [json|xml] | Output format. Default: json. |
-o, --output PATH | Output directory. Default: current directory (.). |
--no-output | Parse without writing AST files. |
-v, --verbose | Print additional progress messages. |
-h, --help | Show command help. |
Arguments
| Argument | Description |
|---|---|
INPUT | Required. One or more files or directories to parse. Directories are walked recursively. |
Output layout
- For each input file, writes
<output-dir>/<filename>.jsonor<output-dir>/<filename>.xml. - For directory inputs, the output mirrors the input directory structure under
--output.
Example
java -jar clparser-distribution-<version>-all.jar parse --license=strumenta.license -o ./out examples/basic.cl
The serialized result contains the AST (root) and any parsing issues.
exportsnapshot — export a LionWeb snapshot
Parses a CL codebase, runs semantic enrichment, and writes a StarLasu-style LionWeb snapshot: a ZIP whose entries are {partitionId}.binpb ProtoBuf chunks.
The snapshot includes traditional partitions:
- languages (AST + CL)
- codebase metadata
- codebase files with source
code,tokens, and (semantics-enriched) ASTs
java -jar clparser-distribution-<version>-all.jar exportsnapshot --license=my.license [OPTIONS] INPUT_DIR
| Option | Description |
|---|---|
--license PATH | Required. License file. |
-o, --output PATH | Destination file for the snapshot (typically .zip or .sps). Default: cl-snapshot.pb.zip. Parent directories are created if needed. |
--codebase-name TEXT | Logical name assigned to sources in the codebase. Default: UnnamedCodebase. |
-v, --verbose | Print additional progress messages (codebase path and file count). |
-h, --help | Show command help. |
Arguments
| Argument | Description |
|---|---|
INPUT_DIR | Required. Directory containing the CL codebase. The directory must exist and be readable. Files are discovered using the usual CL extensions (.cl, .clp, .clle). |
Examples
Export a snapshot of a codebase to the default output file:
java -jar clparser-distribution-<version>-all.jar exportsnapshot --license=strumenta.license ./cl-sources
Export a snapshot to a chosen .sps file, with a codebase name and verbose logging:
java -jar clparser-distribution-<version>-all.jar exportsnapshot --license=strumenta.license --codebase-name=MyApp -o ./out/myapp.pb.sps -v ./cl-sources
On success, the CLI prints the absolute path of the written snapshot file.
metamodel — generate the AST metamodel
Generates the EMF metamodel for the CL language. This command does not parse user CL code and does not require a license.
java -jar clparser-distribution-<version>-all.jar metamodel [OPTIONS]
| Option | Description |
|---|---|
-e, --export PATH | Directory where the metamodel is stored. Default: export. |
--mmxmi | Save the metamodel as metamodel.xmi instead of metamodel.json. |
-v, --verbose | Print additional progress messages. |
-h, --help | Show command help. |
Example
java -jar clparser-distribution-<version>-all.jar metamodel -e build/metamodel -v
downloadlicense — download a license
Described in Downloading a license above.
Getting help
Help is available at every level:
# Top-level help and command list
java -jar clparser-distribution-<version>-all.jar --help
# Help for a specific command
java -jar clparser-distribution-<version>-all.jar parse --help
java -jar clparser-distribution-<version>-all.jar exportsnapshot --help
java -jar clparser-distribution-<version>-all.jar metamodel --help
java -jar clparser-distribution-<version>-all.jar downloadlicense --help
Quick reference
| Goal | Command |
|---|---|
| Write AST JSON/XML files | parse --license=… INPUT… |
| Export LionWeb snapshot | exportsnapshot --license=… INPUT_DIR |
| Generate the language metamodel | metamodel -e PATH |
| Obtain a license | downloadlicense USER PASS PATH |
How to use as a library
You can use this parser as any other library, directly from your code.
Download a license
You can also download an updated license using the method download license in the LicenseValidation class.
LicenseValidation lv = new LicenseValidation();
lv.downloadLicense("user", "pass", "location.license");
Using a license
This parser requires a license to be used.
You can use the static method registerLicense to register the license by providing the license file as argument.
You need to register the license once in the program.
import com.strumenta.cl.parser.CLKolasuParser;
import com.strumenta.kolasu.commercial.LicenseManager;
[..]
LicenseManager.INSTANCE.registerLicense(new File("data/example.license"));
You can then instantiate the parser to parse your CL code (either as a string or as a file).
CLKolasuParser parser = new CLKolasuParser();
ParsingResult<CLCompilationUnit> result = parser.parse(input);
Symbol resolution (semantics module)
The optional semantics module (com.strumenta.langmodules.kolasu-cl.semantics) resolves
CLReferenceExpr names against declarations in scope: DCL variables (DeclareVariableCommand),
subroutine names (Subroutine / CALLSUBR), and labels (GOTO targets).
CLCompilationUnit cu = parser.parse(input).getRoot();
CLSemantics semantics = new CLSemantics();
semantics.resolve(cu);
Consult the cookbook for more examples of uses of the parser.