Flex and Bison are used together to create parsers, usually for programming-related tasks like parsing source files or SQL statements. While not as easy on the programmer as newer libraries like Spirit OR ANTLR, they are more efficient since they generate raw C code that can be compiled into your application. They are also capable of outputting C++ code, but there is a lack of clear documentation / examples available demonstrating this.
What I present here are my efforts over the last couple of days towards creating a program that uses the two together in C++ mode. The result is trivial – numbers are recognized and divided by five – since the focus of this program is to demonstrate using Flex and Bison. If you do not know Flex and Bison, I recommend Lex & Yacc by Tom Niemann (Flex is a clone of Lex, Bison is a clone of Yacc).
Download the example source code here.
This is an example, not a tutorial – I’m not going to take you through the code line by line. I do have some notes for you, though:
- The require line tells Bison that this is meant for Bison 2.4.1. While it might work on other versions of Bison, the C++ code generated is ‘experimental’ and subject to change between versions.
- The skeleton line specifies an alternate template to use – lalr1.cc is the C++ version.
- The parse-param options tell Bison that we want the class to have an additional member variable – the scanner. Since Bison calls the scanner to get a token, the Bison class needs to have a reference to the scanner. The lex-param option tells Bison that when it calls yylex, it should pass the scanner as an additional argument. Our implementation of yylex invokes the passed scanner and returns the result.
- There are two sections of code: the first is ‘%code requires’. The code inside of this block is put in the generated header file as well as the c file. The other %code block contains code that only goes in the c file, and since we don’t want anything else calling the yylex global function we make it static to the file.
- This is a convenience class that bundles the parser and the scanner together; it is good Object-Oriented design to do this, and is recommended in the Bison documentation.
- FlexLexer.h is provided by Flex, and defines the base class that generated Flex scanner classes inherit from. The preprocessor directive surrounding it is necessary because FlexLexer.h is a mess (it says so right in the Flex generated code on line 16).
- The yylex function has to be overloaded because Bison passes a pointer to the yylval variable. It would be nice if we could just specify this in YY_DECL, but the base Flex class has yylval with no arguments defined as pure virtual, forcing us to implement it anyway; I just went ahead and used it.
I used the following resources to construct this example:
Please post any suggestions or questions – I am no expert on either Flex or Bison and am always interested in improving my code!
Update: After you have gleaned all you can from this example, check out this followup about creating an INI file parser using Flex and Bison in C++.