A Cobol Parser

By Bernard Pinon - August 2002. This document describes an implementation of a COBOL 85 parser for JavaCC.

• Introduction
  • What for ?
  • What's new
• Reference
  • Source documentation
  • Specificities
• License and how to contribute
  • Status
• Download
• Background
• About the author

Introduction division.

The venerable COBOL programming language is probably one of the most difficult language for which to write a compiler. Having a strict LL(1) grammar was certainly out of the mind of those who created it, it is even likely that few of them had a clear idea bout the interest of implementing a LL(1) grammar at that time. The language is extremely verbose. There are several input formats, and many variants. Lexical parsing is context dependant. In other terms, it is both a challenge and a nightmare.

What for section.

A COBOL parser can be used as a start point if you intend to write a COBOL compiler (have fun!) but can also be used to write analysis tools, source formatters, converters between various COBOL dialects, automatic documentation, translation to other languages like C or Java.

What's new section.

• 12-aug-2002 :
  • Modified the grammar so it works better with the tree builder, and fixed some little problems. Now the pretty printer actually prints pretty well, with syntax coloring (using HTML tags).
  • Added a page on how to build your abstract syntax tree yourself in five easy steps.
  • Started some reflection on grammar "factorization", for instance treating the "AT END" clause as a separate production instead of doing it in the statement production itself.
• 9-aug-2002 :
  • Investigated on Java Tree Builder, an abstract syntax tree (AST) generator for JavaCC. Interesting result : I was able to generate the AST, then write a quick and dirty pretty-print program.
• 8-aug-2002 :
  • Added a "brute force" pre-processor.
  • The parser has successfully passed the first test of the NIST test suite. This test suite consist of an enormous chunk of COBOL source files (25,4Mo) used to test COBOL compilers. After several bug fixes - many typing mistakes, wrong order of production rules, etc (see source for details), the first program of the suite (EXEC85) was parsed correctly. Now I consider that the grammar is no longer in the experimental state, but rather in the promising state ;-)

Reference division.

Source Documentation section.

The documentation I started from was the Tandem (oops: Compaq (oops: HP today ) ) Non-Stop Cobol 85, the COBOL variant I am much used to. In order to have a comparison - Tandem Cobol is very specific, as it includes extensions to support fault-tolerance and system messaging - I installed the excellent - and free - Fujitsu COBOL that comes with a very good set of documentation, and started to write a limited grammar and test it against simple COBOL example.

Then a miracle occurred : I came across a full COBOL grammar developed by Ralf Lämmel & Chris Verhoef : VS COBOL II grammar Version 1.0.3
which is available at http://www.cwi.nl/ and http://www.wins.uva.nl/. This is a very good grammar, and many lines of my parser are simple copy-and-paste of their work, but was automatically generated, and thus had many imperfections - and one lovely typing mistake : USE WITH DEGUGGING MODE. My grammar is about 80% similar to their, but it has been largely modified mainly in the areas of lexical analysis, and also to remove many ambiguities.

Specificities section.

The current implementation violates the strict COBOL orthodoxy : 

• It's a free format COBOL. Forget about areas, columns, etc. Text can be written from column 1. You can even write the entire program on a single line.
• Comments starts with *> and ends at the end of the current line. You can put comments anywhere. I also plan to introduce Java-like comments, just for aesthetic reasons.
• There is no need for continuation lines. But unfortunately debugging lines are ignored.
• Preprocessing instructions COPY and REPLACE are not implemented and will not be.
• Same remark for compiler options.
• The parser does not check for clause unicity in a statement. For instance you could write "PERFORM UNTIL condition UNTIL other-condition" which is illegal. If you want to create a compiler from this parser, you will have to check for this, or allow it assuming that only the last clause of its kind will have an effect.
• Some Tandem (oops...) extensions and IBM (nobody has bought them recently ?) extensions are present. And I would love to add the excellent Fujitsu extensions (screen section, methods, etc...)

In other word, if you want to parse a real-life COBOL program, you will need to write a preprocessor that will for each input line :

1. remove the "sequence number area" of the COBOL "cards" (columns 1 to 6)
2. process the flags in column 7 :
   1. if it is a comment ("*" or "/" ), insert "*>" before the line
   2. if it is a debugging line, well do what you please, include it or not
   3. ignore the other flags or treat them as comments
3. for non comment lines, scan the rest of the line with a simple parsing - e.g. looking for spaces, commas, etc. You can also use a simplified version of this parser to do so.
   1. if the first word of the line is "COPY", process the COPY clause
   2. if the first word is "REPLACE", add the substitutions to a hash table
   3. otherwise, scan the entire line to search for substitutions.
4. write the processed line to the output file.

(NEW) For the lazy ones, you can use my brute force preprocessor.

License & how to contribute division.

License section.

This program is distributed under the Free Software Foundation LGPL license, which means that you can do almost whatever you want with it, including selling it or including this work in a closed-source, proprietary software, with fair restrictions. For instance, you are requested to publish all changes and fixes made within (not around) this program. You must also give your customer a way to access freely this source code. You cannot distribute it under another license without my consent.

How to contribute section.

Contributions are welcome, click on my name at the top of this page to send me an email with your contribution. If this project becomes successful, I will maybe install it in newsforge with the as-usual mailing list, bug track, and so.

Status section.

Version 0.0 was purely experimental - It parsed only my programs. 

Version 0.1 parses the first program of the test suite, but still do not rely on it for real life apps.

Download division.

• Download cobol.jj

Note that the program has no main function. You will have to compile it using JavaCC. Then use a regular Java compiler. You will have to write your mainline, then call the CompilationUnit() method to start parsing a file. See also the abstract syntax tree page which gives a mainline example. You can download also the preprocessor source file.

Background division.

A bit of background about COBOL. The aim of COBOL was to make the source code look like a text written in English. COBOL is the ideal language for those allergic to anything that looks like a mathematical formula.

A COBOL program is divided in divisions, sections, paragraphs, and phrases (with a final dot). Depending on the division, phrases will be either composed of description entries and clauses, or statements and clauses. 

There are four divisions : 

• identification :  the program name and some comments
• environment : everything that the program needs to know about the rest of the world : what library it uses, which files, devices, etc
• data : fields and parameters
• procedure (imperative statements). 

In the first three divisions, the sections and the corresponding entries are predefined. In the procedure division, the author can create sections, paragraphs, etc.. at will. A section in the procedure division can be seen as a sub-program without parameters, while a paragraph can be seen as a labeled block of statements.

Each entry, statement or clause have its own proper syntax, and sometime its own lexical rules.

Here is an example of a very simple COBOL program for those who never saw one. The programs you will come across in the real world are likely to be much more cryptic than this one... Note than the colors and fonts decoration have been manually added.

About the author division.

I am a free-lance Tandem (oops: Compaq Non-Stop (oops: HP-Compaq Non-Stop) ) system programmer since the B40 release of this absolutely fabulous OS - the best I have seen so far, and I have seen a lot. I have also made developments on HP-UX (in C, a good system), Solaris (in Java, no comment), various flagrances of Linux (in C and PHP, a very good system, just not home-trained yet), and I have also a very bad experience of development under Windows NT4 with ASP/VB/C++ (we throw the code away after three months of struggle and switched to Linux). You can contact me by clicking on my name on the top of this page.

COBOL is not dead, it just smells funny