Scanner unit contains functions that processes source files and compiler directives and extracts tokens – basic elements of the language.
The NextNonCommentCharacter function reads and processes characters from the source line buffer. It returns pointer to the next non-comment character. The null (#0) character means end of line. Turbo Pascal tries to read the next source line. If this fails it tries to return to the previous source file (if $I include file was loaded). …
This procedure processes compiler directives and skips comment. Here also conditional compilation directives (with support for nested directives) are processed.
Turbo Pascal contains few procedures and functions that create and check conditional defines. The first function creates initial conditional defines if certain conditions are met: MSDOS, VER70, CPU86 and CPU87.
Parser operates only with tokens – basic elements of the language. Tokens like identifiers and constants have also additional data structures where actual data is stored.
These two functions check and process decimal and hexadecimal integer constants. The value is stored in the IntegerNumericConstant variable.
This function checks and processes string constants. It processes characters enclosed with single quotes, control characters and characters defined with the ASCII code (#code).
Turbo Pascal uses few special words as directives for special functions: ABSOLUTE, ASSEMBLER, EXTERNAL, FAR, FORWARD, INTERRUPT, NEAR, PRIVATE, PUBLIC, VIRTUAL and System as unit name.
This function case insensitive compares two identifiers and returns false if they are not equal. Otherwise returns true and Id2 points to byte after its end. This function must be as fast as possible! It is called whenever Turbo Pascal needs to compare two identifiers.