This prevents a whole lot of re-allocations when the output byte array
needs to grow. It also prevents some heap fragmentation for big files.
Because the preprocessed output is short lived (it will be parsed
immediately after, and then discarded), it is not squeezed to the
minimal size. This would result in another allocation.
Change-Id: I4974be5144f88cdfc4ddc9d8330200725aa90803
Reviewed-by: hjk <hjk121@nokiamail.com>
Reviewed-by: Eike Ziller <eike.ziller@digia.com>
... and adjust INCLUDEPATH accordingly.
while i'm at messing with include statements, also re-order the include
blocks according to policy and sort them within bigger blocks.
Change-Id: I7762abfd7c4ecf59432b99db2f424e4fa25733a5
Reviewed-by: Tobias Hunger <tobias.hunger@digia.com>
Reviewed-by: Eike Ziller <eike.ziller@digia.com>
Track the typical #ifndef/#define/#endif usage in header files to see if
the macro is an include guard. If so, store it in the Document. No
behavioural change, just recording the name.
This can be used in the future to track if a file needs to be re-parsed
when a macro changes: if it was used in the file, and not defined in it
nor being the include-guard, a file should be re-preprocessed and
re-parsed.
It can also be used to check if two files have the same include guard.
Change-Id: I2715f529997a7b24a11bdbc6150652e2669f1a46
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>
When in 'keep comments' mode, the preprocessor does not properly handle macro calls with
comments between the macro name and the opening parenthesis: "FOO /*something to say*/
(45)".
Change-Id: I6fe733242e4d2ccff2985d17399d0a084917415a
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
- Fixed blocking macro check
- Enabled buffer "compression": when tokens are generated and no new
macro is being blocked, then prepend the tokens to the previous buffer.
This happens a lot when undo-ing look-ahead.
- Added documentation
Change-Id: I6fa816d94ce4696e473bdbc4f3bf477d77e4dd51
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>
This does not yet resolve the file using the proper mechanism.
Change-Id: I04913e8b01ae0c3411961f0c1cffe07202f06a0a
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
#!/usr/bin/env ruby
Dir.glob('**/*.cpp') { |file|
# skip ast (excluding paste, astpath, and canv'ast'imer)
next if file =~ /ast[^eip]|keywords\.|qualifiers|preprocessor|names.cpp/i
s = File.read(file)
next if s.include?('qlalr')
orig = s.dup
s.gsub!(/\n *if [^\n]*{\n[^\n]*\n\s+}(\s+else if [^\n]* {\n[^\n]*\n\s+})*(\s+else {\n[^\n]*\n\s+})?\n/m) { |m|
res = $&
if res =~ /^\s*(\/\/|[A-Z_]{3,})/ # C++ comment or macro (Q_UNUSED, SDEBUG), do not touch braces
res
else
res.gsub!('} else', 'else')
res.gsub!(/\n +} *\n/m, "\n")
res.gsub(/ *{$/, '')
end
}
s.gsub!(/ *$/, '')
File.open(file, 'wb').write(s) if s != orig
}
Change-Id: I3b30ee60df0986f66c02132c65fc38a3fbb6bbdc
Reviewed-by: hjk <qthjk@ovi.com>
Turns out QByteArray::setNum() is unnecessarily slow (will be fixed
independently), but even then, we can be faster.
Change-Id: I663bd2b8cc844bbe800879bccfa57999d020ba3b
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
Record revisions of documents in macro definitions and usages. Then,
when searching for usages, check the revision of the documents against
the revision of the macros. If they are out-of-sync, repreprocess the
documents to get up-to-date info.
Task-number: QTCREATORBUG-7872
Change-Id: I846bb52ec660024728ab117a9fb7e43382a50e63
Reviewed-by: Christian Stenger <christian.stenger@digia.com>
This addresses the main memory leak revealed in QTCREATORBUG-7645.
The other leaks seem to have their origin in Qt.
Task-Number: QTCREATORBUG-7645.
Change-Id: I77f45449416c143b222ed5f5c905cba9674f95bb
Reviewed-by: Christian Kamm <kamm@incasoftware.de>
Even if "expand funcion-like macros" is unset we still
perform the expansion in the case it's already doing
so - when it originally started from an object-like macro.
Task-number: QTCREATORBUG-7712
Change-Id: Ie2a24de227f757d195146477d48246472082d28a
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
Reviewed-by: hjk <qthjk@ovi.com>
Notice that a similar problem still exists for which we
need to fix the lexer when there's a C style commend which
ends with a backslash-newline.
Task-number: QTCREATORBUG-7713
Change-Id: I0f6d561703984f917fa5ed29de020ad0bdc5aaf0
Reviewed-by: David Schulz <david.schulz@nokia.com>
Reviewed-by: hjk <qthjk@ovi.com>
The parser now understands emit/Q_EMIT as an expression statement.
Also, the recent fixes in the preprocessor introduced a side-effect
in the hanlding of code such as: emit signal(); Member signal started
being treated as a local use (parsed as a declaration) and possibily
being highlighted as unused variable.
Previously that worked by accident since there was an inconsistency
in the preprocessor on which only object-like macros were being
expanded even when the "no expand" flag was set. Then, the code
mentioned above was being parsed as an expression, what kind of worked.
Change-Id: I47a68ed4c1c1702872620b8ed7c7264fb0997034
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
Summary of most relevant items:
- Preprocessor output format change. No more gen true/false. Instead
a more intuitive and natural expansion (like from a real compiler) is
performed directly corresponding to the macro invocation. Notice that
information about the generated tokens is not lost, because it's now
embedded in the expansion section header (in terms of lines and columns
as explained in the code). In addition the location on where the macro
expansion happens is also documented for future use.
- Fix line control directives and associated token line numbers.
This was not detected in tests cases because some of them were
actually wrong: Within expansions the line information was being
considered as originally computed in the macro definition, while
the desired and expected for Creator's reporting mechanism (just
like regular compilers) is the line from the expanded version
of the tokens.
- Do not allow for eager expansion. This was previously being done
inside define directives. However, it's not allowed and might
lead to incorrect results, since the argument substitution should
only happen upon the macro invocation (and following nested ones).
At least GCC and clang are consistent with that. See test case
tst_Preprocessor:dont_eagerly_expand for a detailed explanation.
- Revive the 'expanded' token flag. This is used to mark every token
that originates from a macro expansion. Notice, however, that
expanded tokens are not necessarily generated tokens (although
every generated token is a expanded token). Expanded tokens that
are not generated are those which are still considered by our
code model features, since they are visible on the editor. The
translation unit is smart enough to calculate line/column position
for such tokens based on the information from the expansion section
header.
- How expansions are tracked has also changed. Now, we simply add
two surrounding marker tokens to each "top-level" expansion
sequence. There is an enumeration that control expansion states.
Also, no "previous" token is kept around.
- Preprocessor client methods suffered a change in signature so
they now receive the line number of the action in question as
a paramater. Previously such line could be retrieved by the client
implementation by accessing the environment line. However, this
is not reliable because we try to avoid synchronization of the
output/environment lines in order to avoid unnecessary output,
while expanding macros or handling preprocessor directives.
- Although macros are not expanded during define directives (as
mentioned above) the preprocessor client is now "notified"
when it sees a macro. This is to allow usage tracking.
- Other small stuff.
This is all in one patch because the fixes are a consequence
of the change in preprocessing control.
Change-Id: I8f4c6e6366f37756ec65d0a93b79f72a3ac4ed50
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
Do not expand function-like macros at all when there's a mismatch
in the parameter/argument count.
The report below raises the issue but its expected result is not
correct. This would be the more appropriate fix.
Task-number: QTCREATORBUG-7225
Change-Id: Ide8580faa7b724d3e8b396ec1f899cc5ca7f9e7e
Reviewed-by: hjk <qthjk@ovi.com>
Make sure the environment line is consistent during preprocessor
directives and identifier handling so clients can rely on consistent
information. Particularly important for macro usages.
New tests also added.
Change-Id: I962a39a86cd17b8d945d2959c2c95e2d258ea3e6
Reviewed-by: hjk <qthjk@ovi.com>
Due to latest changes the macro arguments were no longer being tracked.
Then they were no available in the document's macro uses. The patch also
makes sure that the preprocessor condition to be expanded is spelled
exactly as in the source code (this guarantees that offsets will be
properly calculated).
Change-Id: I8aff0c3aca0c528ef2c4bcfa56ff1c3da2961060
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
Generated tokens do not have a position in any source file, so not try
to indent them. Previously, the 'source' used was the scratch buffer,
which would not contain newlines, so the indent depth would be the
length of the scratch buffer at that point.
Task-number: QTCREATORBUG-7262
Change-Id: If94213d6dffd13dd2b47c7038ec2398ad925d904
Reviewed-by: Yuchen Deng <loaden@gmail.com>
Reviewed-by: Thomas Hartmann <Thomas.Hartmann@nokia.com>
