chapter10lexicalanalyzer(lex)(已修改)
2024-11-09 18:01 本頁面
【正文】 Chapter 10 lexical analyzer (lex) Speaker: LungSheng Chien Reference book: John R. Levine, lex amp。 yacc 中譯本 , 林偉豪譯 Reference ppt: Lecture 2: Lexical Analysis, CS 440/540, Gee Mason university Reference URL: Online manual: OutLine ? What is lex ? Regular expression ? Finite state machine ? Content of flex ? Application Recall Exercise 7 in the midterm Question: can we write more pact code to obtain integers? for each line in a file if line contains “//” not in a string, then remove remaining characters after “//”. if line contains “/*” not in a string, then find conjugate pair “*/” and remove all characters in between endfor Exercise 7: remove ments in a file in Clanguage, ment is delimited by a pair of /* and */ whereas in C++, ment starts from //. write a program to remove all ments of a given file. You can show result in screen or to another file. Pseudocode Question: can we have other tool to identify Cment ? What is lex ? Lex is a program generator designed for lexical (語彙的 ) processing of character input streams. It accepts a highlevel, problem oriented specification for character string matching, and produces a program in a general purpose language which recognizes regular expressions (正規(guī)表示法 ). ? The regular expressions are specified by the user in the source specifications given to Lex. ? Lex generates a deterministic finite automaton (DFA, 有限自動(dòng)機(jī) ) from the regular expressions in the source. ? The Lex written code recognizes these expressions in an input stream and partitions the input stream into strings matching the expressions. From definition Token Pattern Lexeme(詞彙 ) integer (09)+ 234 identifier [azAZ]?[azAZ09]* x1 string Characters between “ “ “hello world” ? Token: set of strings defining an atomic element with a defined meaning ? Pattern: a rule describing a set of string ? Lexeme: a sequence of characters that match some pattern Lexical analyzer Syntax analyzer (文法分析 ) Semantic analyzer (語意分析 ) Intermediate code generator Code optimizer Code generator Source code machine code Phases of a Compiler Lex is a crucial tool to extract token token Input file Scanner yylex() parser yyparse() symbol table ask next character character ask next token token Input file Scanner yylex() ask next character character File processor of Linear programming ask next token token Role of scanner: find token flex Lex specification + source file gcc c g++ input token flex : lexical analyzer generator ? Ccode is kernel to extract token, one just need to call function yylex(). To use in different platforms, we need to solve several technical problems don?t use library don?t include specific header file mix C with C++ code flex in RedHat 9 Link with library Library Generate source Ccode 按 Ctrl+D T h i s i s a b o o k \n b y e b y e \n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Example in the manual of Flex Count number of lines and number of characters 按 enter definition section %% rule section %% user code grammar of input file User code Lex copy data enclosed by %{ and %} into C source file pattern action \n { ++num_lines 。 ++ num_chars 。 } pattern action
教學(xué)課件相關(guān)推薦
鄂ICP備17016276號(hào)-1