diff options
| author | Carl Worth <[email protected]> | 2014-07-01 16:31:07 -0700 |
|---|---|---|
| committer | Ian Romanick <[email protected]> | 2014-08-07 16:08:29 -0700 |
| commit | 04e40fd337a244ee77ef9553985e9398ff0344af (patch) | |
| tree | b2600fffceb1f27dc3a5ab55fbe7f1b019cb11b7 | |
| parent | f4ddd026c6f155a1050b142f2e88225305ecdd90 (diff) | |
glsl/glcpp: Treat CR+LF pair as a single newline
The GLSL specification says that either carriage-return, line-feed, or both
together can be used to terminate lines. Further, it says that when used
together, the pair of terminators shall be interpreted as a single line.
This final requirement has not been respected by glcpp up until now, (it has
been emitting two newlines for every CR+LF pair).
Here, we fix the lexer by using a regular expression for NEWLINE that eats
up both "\r\n" (or even "\n\r") if possible before also considering a single
'\n' or a single '\r' as a line terminator.
Before this commit, the test results are as follows:
\r: 135/143 tests pass
\r\n: 4/143 tests pass
\n\r: 4/143 tests pass
After this commit, the test results are as follows:
\r: 135/143 tests pass
\r\n: 140/143 tests pass
\n\r: 139/143 tests pass
So, obviously, a dramatic improvement.
Reviewed-by: Ian Romanick <[email protected]>
| -rw-r--r-- | src/glsl/glcpp/glcpp-lex.l | 11 | ||||
| -rw-r--r-- | src/glsl/glcpp/tests/.gitignore | 3 |
2 files changed, 9 insertions, 5 deletions
diff --git a/src/glsl/glcpp/glcpp-lex.l b/src/glsl/glcpp/glcpp-lex.l index aeaf8abdf8a..48cc62c1109 100644 --- a/src/glsl/glcpp/glcpp-lex.l +++ b/src/glsl/glcpp/glcpp-lex.l @@ -172,6 +172,7 @@ SPACE [[:space:]] NONSPACE [^[:space:]] HSPACE [ \t] HASH # +NEWLINE (\r\n|\n\r|\r|\n) IDENTIFIER [_a-zA-Z][_a-zA-Z0-9]* PP_NUMBER [.]?[0-9]([._a-zA-Z0-9]|[eEpP][-+])* PUNCTUATION [][(){}.&*~!/%<>^|;,=+-] @@ -256,9 +257,9 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]? /* Multi-line comments */ <INITIAL,DEFINE,HASH>"/*" { yy_push_state(COMMENT, yyscanner); } <COMMENT>[^*\r\n]* -<COMMENT>[^*\r\n]*[\r\n] { yylineno++; yycolumn = 0; parser->commented_newlines++; } +<COMMENT>[^*\r\n]*{NEWLINE} { yylineno++; yycolumn = 0; parser->commented_newlines++; } <COMMENT>"*"+[^*/\r\n]* -<COMMENT>"*"+[^*/\r\n]*[\r\n] { yylineno++; yycolumn = 0; parser->commented_newlines++; } +<COMMENT>"*"+[^*/\r\n]*{NEWLINE} { yylineno++; yycolumn = 0; parser->commented_newlines++; } <COMMENT>"*"+"/" { yy_pop_state(yyscanner); /* In the <HASH> start condition, we don't want any SPACE token. */ @@ -289,7 +290,7 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]? /* Swallow empty #pragma directives, (to avoid confusing the * downstream compiler). */ -<HASH>pragma{HSPACE}*/[\r\n] { +<HASH>pragma{HSPACE}*/{NEWLINE} { BEGIN INITIAL; } @@ -305,7 +306,7 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]? RETURN_TOKEN (LINE); } -<HASH>\n { +<HASH>{NEWLINE} { BEGIN INITIAL; RETURN_TOKEN_NEVER_SKIP (NEWLINE); } @@ -521,7 +522,7 @@ HEXADECIMAL_INTEGER 0[xX][0-9a-fA-F]+[uU]? /* We preserve all newlines, even between #if 0..#endif, so no skipping.. */ -<*>[\r\n] { +<*>{NEWLINE} { if (parser->commented_newlines) { BEGIN NEWLINE_CATCHUP; } else { diff --git a/src/glsl/glcpp/tests/.gitignore b/src/glsl/glcpp/tests/.gitignore new file mode 100644 index 00000000000..f18a9bb4b3a --- /dev/null +++ b/src/glsl/glcpp/tests/.gitignore @@ -0,0 +1,3 @@ +subtest-cr/ +subtest-cr-lf/ +subtest-lf-cr/ |