Notice that a similar problem still exists for which we
need to fix the lexer when there's a C style commend which
ends with a backslash-newline.
Task-number: QTCREATORBUG-7713
Change-Id: I0f6d561703984f917fa5ed29de020ad0bdc5aaf0
Reviewed-by: David Schulz <david.schulz@nokia.com>
Reviewed-by: hjk <qthjk@ovi.com>
Summary of most relevant items:
- Preprocessor output format change. No more gen true/false. Instead
a more intuitive and natural expansion (like from a real compiler) is
performed directly corresponding to the macro invocation. Notice that
information about the generated tokens is not lost, because it's now
embedded in the expansion section header (in terms of lines and columns
as explained in the code). In addition the location on where the macro
expansion happens is also documented for future use.
- Fix line control directives and associated token line numbers.
This was not detected in tests cases because some of them were
actually wrong: Within expansions the line information was being
considered as originally computed in the macro definition, while
the desired and expected for Creator's reporting mechanism (just
like regular compilers) is the line from the expanded version
of the tokens.
- Do not allow for eager expansion. This was previously being done
inside define directives. However, it's not allowed and might
lead to incorrect results, since the argument substitution should
only happen upon the macro invocation (and following nested ones).
At least GCC and clang are consistent with that. See test case
tst_Preprocessor:dont_eagerly_expand for a detailed explanation.
- Revive the 'expanded' token flag. This is used to mark every token
that originates from a macro expansion. Notice, however, that
expanded tokens are not necessarily generated tokens (although
every generated token is a expanded token). Expanded tokens that
are not generated are those which are still considered by our
code model features, since they are visible on the editor. The
translation unit is smart enough to calculate line/column position
for such tokens based on the information from the expansion section
header.
- How expansions are tracked has also changed. Now, we simply add
two surrounding marker tokens to each "top-level" expansion
sequence. There is an enumeration that control expansion states.
Also, no "previous" token is kept around.
- Preprocessor client methods suffered a change in signature so
they now receive the line number of the action in question as
a paramater. Previously such line could be retrieved by the client
implementation by accessing the environment line. However, this
is not reliable because we try to avoid synchronization of the
output/environment lines in order to avoid unnecessary output,
while expanding macros or handling preprocessor directives.
- Although macros are not expanded during define directives (as
mentioned above) the preprocessor client is now "notified"
when it sees a macro. This is to allow usage tracking.
- Other small stuff.
This is all in one patch because the fixes are a consequence
of the change in preprocessing control.
Change-Id: I8f4c6e6366f37756ec65d0a93b79f72a3ac4ed50
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
This reverts commit 44d8e7bef7.
The original test case was actually right. I guess I was mislead
by the recently added preprocessor tests, which are the ones
that should be corrected (together with the preprocessor, which
seems to have issues with # lineno generation).
Conflicts:
tests/auto/cplusplus/preprocessor/tst_preprocessor.cpp
Change-Id: I47f82ed23a37086d0d81c4b3ea2cac48fb753451
Reviewed-by: hjk <qthjk@ovi.com>
Reviewed-by: Francois Ferrand <thetypz@gmail.com>
Do not expand function-like macros at all when there's a mismatch
in the parameter/argument count.
The report below raises the issue but its expected result is not
correct. This would be the more appropriate fix.
Task-number: QTCREATORBUG-7225
Change-Id: Ide8580faa7b724d3e8b396ec1f899cc5ca7f9e7e
Reviewed-by: hjk <qthjk@ovi.com>
Make sure the environment line is consistent during preprocessor
directives and identifier handling so clients can rely on consistent
information. Particularly important for macro usages.
New tests also added.
Change-Id: I962a39a86cd17b8d945d2959c2c95e2d258ea3e6
Reviewed-by: hjk <qthjk@ovi.com>
I believe those were actually incorrect, since the # mark
from the generated tokens `int f` should be relative to <stdin>
on the line they are defined, which is 1.
Change-Id: I663ef49ad75eb8bb0a4a4b18d4899a952011536c
Reviewed-by: Francois Ferrand <thetypz@gmail.com>
Reviewed-by: hjk <qthjk@ovi.com>
Generated tokens do not have a position in any source file, so not try
to indent them. Previously, the 'source' used was the scratch buffer,
which would not contain newlines, so the indent depth would be the
length of the scratch buffer at that point.
Task-number: QTCREATORBUG-7262
Change-Id: If94213d6dffd13dd2b47c7038ec2398ad925d904
Reviewed-by: Yuchen Deng <loaden@gmail.com>
Reviewed-by: Thomas Hartmann <Thomas.Hartmann@nokia.com>
The line corresponding to the macro definition is simply the token's
line. This caused errors in the macro definitions, since there were
multiple macros defined on the same line of the same file.
Task-number: QTCREATORBUG-7217
Change-Id: I56d17eeba677ac51f9eed283c0e964019bdfe4b7
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
- This enables some tests which had been unexpectedly disabled in
earlier test.
- Tests to show issue QTCREATORBUG-7225.
- Tests which validates somehow behavior of preprocessor for
QTCREATORBUG-7217: the test passes, so the problem is likely not in the
startExpandingMacro() notification.
Change-Id: Ib7c1433e4dcdd8c1d19fa117371f94c1645ec3fb
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
- Notify end of macro if function expansion is skipped.
- Do not notify client of generated macros expansion.
Change-Id: Ic027fc13ee391425a5ebadc8e84b9305912dbcf0
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
This rewrite fixes a couple of issues with the pre-processor. It now
supports:
- macros in macro bodies
- stringification of parameters [cpp.stringize]
- the concatenation operator [cpp.concat]
- #include MACRO_HERE
- defined() inside macro bodies used in pp-conditions.
Change-Id: Ifdb78041fb6afadf44f939a4bd66ce2832b8601f
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
The macro used to be at the same line as the first token after the
closing parenthesis.
Change-Id: I4c2d7317632baf612900e84572017aeb247d0585
Reviewed-by: Erik Verbruggen <erik.verbruggen@nokia.com>
Expanding macro arguments can add or remove argument for a nested macro,
so the actual parameters list was not correct.
Also, remove unused arguments reference list and reserve space for the
expected number of arguments.
Change-Id: I4cf369cbb3909927c6bf65750bd715fa6f070fad
Reviewed-by: Erik Verbruggen <erik.verbruggen@nokia.com>
First empty argument used to be dropped: e.g. MACRO(,test) would be
expanded with one parameter only, with value 'test'.
Change-Id: I693fbb7faf1360f62266fa04c4b39c2de0d159a7
Reviewed-by: Erik Verbruggen <erik.verbruggen@nokia.com>
When a PP line ended with a \ (join line) and the next line was empty
(ending th joining), the following token would be incorrectly marked as
joined.
Done-with: Roberto Raggi
