Lexical analyzer code in java

Lexical Analyzer Assignment

Implement a lexical analyzer for the assignment statement language called ada21s.

LexicalAnalyzer is the main class of the lexical analyzer. You must declare private, all the data members of the LexicalAnalyzer class. You must declare private, all the methods in the LexicalAnalyzer class, except for the constructor and the method getToken. The constructor has one parameter, a string, which contains the filename of the ada21s program source code file the lexical analyzer processes. The getToken method returns the next token in the ada21s program source code file. You can create any additional classes for your lexical analyzer, as needed.

2. Download the files Symbol.java and Token.java provided below The file Symbol.java contains the enumerated type which includes the token symbols for all lexemes in ada21s. The file Token.java contains the Token class used by the getToken method of the LexicalAnalyzer class to return a token which consists of three values: the token symbol, a string if the token is an identifier, and an integer if the token is a numerical literal. You cannot modify any of the code in these two files. (token.java and symbol.java are below

3. Page 3 lists the table of the lexemes for ada21s. All lexemes (except for identifier, numerical literal, not a lexeme, and end of input) are individual lexemes consisting of the character string listed in the Lexeme column and their corresponding token symbol listed in the Token Symbol column. For example, the lexeme :=corresponds to the token symbol BECOMES. A description of the other four lexemes are given below.

4. An identifier lexeme starts with a letter and the remaining characters are either letters or digits. The length of an identifier is capped at 20 characters and the lexical analyzer ignores any additional letters and digits. Identifier lexemes are case sensitive; which means, for example, Count and count are two different identifier names and therefore, two different identifiers.

5. A numerical literal lexeme consists only of digits. The length of a numerical literal is capped at 10 digits and the lexical analyzer ignores any additional digits.

6. The not a lexeme lexeme is a single character lexeme for any character not found in any of the other lexemes of the ada21s language as well as for characters which are not the first character in any of the other lexemes in the ada21s language. The lexical analyzer returns the NAL token symbol for the not a lexeme lexeme. For example, the lexical analyzer returns the NAL token symbol for the ~ character. Another example where the lexical analyzer returns the NAL token symbol is the = character. It is found in the := lexeme but is not the first character in any of the other lexemes in the ada21s language.

7. The lexical analyzer returns the end of input lexeme when it reaches the end of the ada21s program source code file.

8. Create a driver class and make the name of the driver class Assignment1 containing only one method:

public static void main(String args[]). The main method receives, via the command line arguments, the name of the ada21s program source code file that your lexical analyzer processes. The main method itself is fairly short containing a loop to call the lexical analyzer’s getToken method to get the next token and print it out to a text file called output1.txt with each token printed on a separate line. The command to launch your program, assuming example1.ada21s contains an ada21s program, is: java Assignment1 example1.ada21s

9. You must declare public each class you create which means you define each class in its own file.

10. You must declare private all the data members in every class you create.

11. You cannot use extends in this assignment to extend any class.

12. Tip: Make your program as modular as possible, not placing all your code in one .java file. You can create as many classes as you need in addition to the classes described above. Methods being reasonably small follow the guidance that "A function does one thing and does it well." You will lose a lot of points for code readability if you don’t make your program as modular as possible. But, do not go overboard on creating classes and methods. Your common sense guides your creation of classes and methods.

13. Do NOT use your own packages in your program. If you see the keyword package on the top line of any of your .java files then you created a package. Create every .java file in the src folder of your Eclipse project, if you’re using Eclipse.

14. Do NOT use any graphical user interface code in your program!

Pls add some comment so I can understand it

Token.java

public class Token

{

private Symbol sym;

private String id;

private long num;

public Token(Symbol s, String i, long n)

{

sym = s;

id = i;

num = n;

}

public Symbol getSym()

{

return sym;

}

public String getId()

{

return id;

}

public long getNum()

{

return num;

}

public void setSym(Symbol s)

{

sym = s;

}

public void setId(String i)

{

id = i;

}

public void setNum(long n)

{

num = n;

}

public String toString()

{

return ("Symbol: " + sym + ", id: " + id + ", num: " + num);

}

}

Symbol.java

public enum Symbol
{
IDENT, NUMLIT, LPAREN, RPAREN, POSTINC, POSTDEC,
MULT, DIV, MOD, PLUS, MINUS, SLL, SRA, SRL,
BITAND, BITXOR, BITOR, BECOMES, SEMICOLON,
NAL, EOI
}