feedc0de/qt-creator
1
0
Fork 0
forked from qt-creator/qt-creator
Code Pull Requests Activity
Files
0fc9e7c87735d5cc4ecadbe6d4656f620e2259cd
qt-creator/src/libs/cplusplus/PPToken.cpp

34 lines
600 B
C++
Raw Normal View History

[C++] Rewrite of the preprocessor. This rewrite fixes a couple of issues with the pre-processor. It now supports: - macros in macro bodies - stringification of parameters [cpp.stringize] - the concatenation operator [cpp.concat] - #include MACRO_HERE - defined() inside macro bodies used in pp-conditions. Change-Id: Ifdb78041fb6afadf44f939a4bd66ce2832b8601f Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
2012-03-26 15:18:01 +02:00
#include "PPToken.h"
#include <cstring>
preprocessor: make CPlusPlus::Internal::ByteArrayRef public Change-Id: Ief6ea71f750b6ba19b650050bae376f4ad82d9d2 Reviewed-by: hjk <qthjk@ovi.com>
2012-04-18 17:35:28 +02:00
using namespace CPlusPlus;
[C++] Rewrite of the preprocessor. This rewrite fixes a couple of issues with the pre-processor. It now supports: - macros in macro bodies - stringification of parameters [cpp.stringize] - the concatenation operator [cpp.concat] - #include MACRO_HERE - defined() inside macro bodies used in pp-conditions. Change-Id: Ifdb78041fb6afadf44f939a4bd66ce2832b8601f Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
2012-03-26 15:18:01 +02:00
bool ByteArrayRef::startsWith(const char *s) const
{
int l = std::strlen(s);
if (l > m_length)
return false;
return !qstrncmp(start(), s, l);
}
int ByteArrayRef::count(char ch) const
{
int num = 0;
const char *b = start();
const char *i = b + m_length;
while (i != b)
if (*--i == ch)
++num;
return num;
}
C++: Core changes in preprocessing Summary of most relevant items: - Preprocessor output format change. No more gen true/false. Instead a more intuitive and natural expansion (like from a real compiler) is performed directly corresponding to the macro invocation. Notice that information about the generated tokens is not lost, because it's now embedded in the expansion section header (in terms of lines and columns as explained in the code). In addition the location on where the macro expansion happens is also documented for future use. - Fix line control directives and associated token line numbers. This was not detected in tests cases because some of them were actually wrong: Within expansions the line information was being considered as originally computed in the macro definition, while the desired and expected for Creator's reporting mechanism (just like regular compilers) is the line from the expanded version of the tokens. - Do not allow for eager expansion. This was previously being done inside define directives. However, it's not allowed and might lead to incorrect results, since the argument substitution should only happen upon the macro invocation (and following nested ones). At least GCC and clang are consistent with that. See test case tst_Preprocessor:dont_eagerly_expand for a detailed explanation. - Revive the 'expanded' token flag. This is used to mark every token that originates from a macro expansion. Notice, however, that expanded tokens are not necessarily generated tokens (although every generated token is a expanded token). Expanded tokens that are not generated are those which are still considered by our code model features, since they are visible on the editor. The translation unit is smart enough to calculate line/column position for such tokens based on the information from the expansion section header. - How expansions are tracked has also changed. Now, we simply add two surrounding marker tokens to each "top-level" expansion sequence. There is an enumeration that control expansion states. Also, no "previous" token is kept around. - Preprocessor client methods suffered a change in signature so they now receive the line number of the action in question as a paramater. Previously such line could be retrieved by the client implementation by accessing the environment line. However, this is not reliable because we try to avoid synchronization of the output/environment lines in order to avoid unnecessary output, while expanding macros or handling preprocessor directives. - Although macros are not expanded during define directives (as mentioned above) the preprocessor client is now "notified" when it sees a macro. This is to allow usage tracking. - Other small stuff. This is all in one patch because the fixes are a consequence of the change in preprocessing control. Change-Id: I8f4c6e6366f37756ec65d0a93b79f72a3ac4ed50 Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
2012-06-20 15:22:02 +02:00
void Internal::PPToken::squeezeSource()
[C++] Rewrite of the preprocessor. This rewrite fixes a couple of issues with the pre-processor. It now supports: - macros in macro bodies - stringification of parameters [cpp.stringize] - the concatenation operator [cpp.concat] - #include MACRO_HERE - defined() inside macro bodies used in pp-conditions. Change-Id: Ifdb78041fb6afadf44f939a4bd66ce2832b8601f Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
2012-03-26 15:18:01 +02:00
{
C++: Core changes in preprocessing Summary of most relevant items: - Preprocessor output format change. No more gen true/false. Instead a more intuitive and natural expansion (like from a real compiler) is performed directly corresponding to the macro invocation. Notice that information about the generated tokens is not lost, because it's now embedded in the expansion section header (in terms of lines and columns as explained in the code). In addition the location on where the macro expansion happens is also documented for future use. - Fix line control directives and associated token line numbers. This was not detected in tests cases because some of them were actually wrong: Within expansions the line information was being considered as originally computed in the macro definition, while the desired and expected for Creator's reporting mechanism (just like regular compilers) is the line from the expanded version of the tokens. - Do not allow for eager expansion. This was previously being done inside define directives. However, it's not allowed and might lead to incorrect results, since the argument substitution should only happen upon the macro invocation (and following nested ones). At least GCC and clang are consistent with that. See test case tst_Preprocessor:dont_eagerly_expand for a detailed explanation. - Revive the 'expanded' token flag. This is used to mark every token that originates from a macro expansion. Notice, however, that expanded tokens are not necessarily generated tokens (although every generated token is a expanded token). Expanded tokens that are not generated are those which are still considered by our code model features, since they are visible on the editor. The translation unit is smart enough to calculate line/column position for such tokens based on the information from the expansion section header. - How expansions are tracked has also changed. Now, we simply add two surrounding marker tokens to each "top-level" expansion sequence. There is an enumeration that control expansion states. Also, no "previous" token is kept around. - Preprocessor client methods suffered a change in signature so they now receive the line number of the action in question as a paramater. Previously such line could be retrieved by the client implementation by accessing the environment line. However, this is not reliable because we try to avoid synchronization of the output/environment lines in order to avoid unnecessary output, while expanding macros or handling preprocessor directives. - Although macros are not expanded during define directives (as mentioned above) the preprocessor client is now "notified" when it sees a macro. This is to allow usage tracking. - Other small stuff. This is all in one patch because the fixes are a consequence of the change in preprocessing control. Change-Id: I8f4c6e6366f37756ec65d0a93b79f72a3ac4ed50 Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
2012-06-20 15:22:02 +02:00
if (hasSource()) {
m_src = m_src.mid(offset, f.length);
[C++] Rewrite of the preprocessor. This rewrite fixes a couple of issues with the pre-processor. It now supports: - macros in macro bodies - stringification of parameters [cpp.stringize] - the concatenation operator [cpp.concat] - #include MACRO_HERE - defined() inside macro bodies used in pp-conditions. Change-Id: Ifdb78041fb6afadf44f939a4bd66ce2832b8601f Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
2012-03-26 15:18:01 +02:00
m_src.squeeze();
offset = 0;
}
}
Copy Permalink