Inserting to a std::vector (possibly) invalidates all iterators and
references for it.
In this piece of code tok is a reference to a member in _tokens and it
should not be used after an insertion is made to _tokens. Switched the
order of two code lines to prevent this.
Change-Id: Ia9d1ce66fc67406f56b426e7431402c6fe68a38f
Reviewed-by: Eike Ziller <eike.ziller@theqtcompany.com>
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@theqtcompany.com>
For the semantic info document we do not expand function like macros and
because of that certain macro invocations lead to invalid code that we
need to handle, e.g.:
Q_GLOBAL_STATIC(CppTools::SymbolFinder, symbolFinder)
class Foo {};
This change makes parsing Foo in the semantic info document successfully
again, which affects highlighting of that class.
Change-Id: I389265ac64d3f0b8b8f406d38fa58d78820b14ba
Reviewed-by: Erik Verbruggen <erik.verbruggen@theqtcompany.com>
See [global.names] (17.6.4.3.2 in the C++11 spec.)
Change-Id: I8434496dbe392b52d339d5f17cfaeee8dbd88995
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>
In TranslationUnit, the "normal" lines are based on utf16char offsets,
but the preprocessor lines were based on byte/latin1 offsets.
The preprocessor lines are now based on utf16char offsets, too.
Task-number: QTCREATORBUG-7356
Change-Id: I3c41d1dcee8e9e487210f36da806b0229d3f4cd0
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
In most cases we need to work with the utf16 indices. Only in
cppfindreferences the byte interface is still needed since there we read
in files and work on a QByteArray to save memory.
Change-Id: I6ef6a93fc1875a8c9a305c075d51a9ca034c41bb
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
...and not byte offsets anymore. This is necessary in order to calculate
the line and column numbers correctly with respect to unicode code
points.
Change-Id: I5d79857b3eaefeb8d563b4f1e3938a64debc5e08
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
This will avoid confusion when later more length and indices methods are
added.
In Token:
length() --> bytes()
begin() --> bytesBegin()
end() --> bytesEnd()
Change-Id: I244c69b022e239ee762b4114559e707f93ff344f
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
They are already inlined. Now it's easier to find read-only accesses.
Change-Id: I9aaeca3bc5860e3a20a536a2484925e4334c005f
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
Until now std::vector::at() was used to access the elements. This is
handy for debugging since an exception is thrown for invalid indices,
but it does not stop Qt Creator from crashing because exceptions are not
caught.
This is especially a problem for the parser, which has to look ahead via
LA(n), which accesses TranslationUnit::_tokens.
With this patch, explicit bounds checking is done before accessing the
elements and thus calls to
std::vector::at() // bounds checking, throwing out_of_range
were replaced by calls to
std::vector::operator[]() // no bounds checking, not throwing out_of_range
Measuring the parse time for the Qt Creator project shows that there is
no slowdown. In both cases, with and without the patch, about 15s are
needed on the authors machine.
Task-number: QTCREATORBUG-10453
Change-Id: I32b12a526ff7199bcadfc21a3deb5354063a3e3b
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
Be more environment friendly by using less trees.
Change-Id: I54d3b23d697a3b72a6a803688a8da0eddaad3e17
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>
Summary of most relevant items:
- Preprocessor output format change. No more gen true/false. Instead
a more intuitive and natural expansion (like from a real compiler) is
performed directly corresponding to the macro invocation. Notice that
information about the generated tokens is not lost, because it's now
embedded in the expansion section header (in terms of lines and columns
as explained in the code). In addition the location on where the macro
expansion happens is also documented for future use.
- Fix line control directives and associated token line numbers.
This was not detected in tests cases because some of them were
actually wrong: Within expansions the line information was being
considered as originally computed in the macro definition, while
the desired and expected for Creator's reporting mechanism (just
like regular compilers) is the line from the expanded version
of the tokens.
- Do not allow for eager expansion. This was previously being done
inside define directives. However, it's not allowed and might
lead to incorrect results, since the argument substitution should
only happen upon the macro invocation (and following nested ones).
At least GCC and clang are consistent with that. See test case
tst_Preprocessor:dont_eagerly_expand for a detailed explanation.
- Revive the 'expanded' token flag. This is used to mark every token
that originates from a macro expansion. Notice, however, that
expanded tokens are not necessarily generated tokens (although
every generated token is a expanded token). Expanded tokens that
are not generated are those which are still considered by our
code model features, since they are visible on the editor. The
translation unit is smart enough to calculate line/column position
for such tokens based on the information from the expansion section
header.
- How expansions are tracked has also changed. Now, we simply add
two surrounding marker tokens to each "top-level" expansion
sequence. There is an enumeration that control expansion states.
Also, no "previous" token is kept around.
- Preprocessor client methods suffered a change in signature so
they now receive the line number of the action in question as
a paramater. Previously such line could be retrieved by the client
implementation by accessing the environment line. However, this
is not reliable because we try to avoid synchronization of the
output/environment lines in order to avoid unnecessary output,
while expanding macros or handling preprocessor directives.
- Although macros are not expanded during define directives (as
mentioned above) the preprocessor client is now "notified"
when it sees a macro. This is to allow usage tracking.
- Other small stuff.
This is all in one patch because the fixes are a consequence
of the change in preprocessing control.
Change-Id: I8f4c6e6366f37756ec65d0a93b79f72a3ac4ed50
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
Those are the types char16_t and char32_t along with the new
char/string literals u'', U'', u"", u8"", and U"".
This is particularly important for the use of QStringLiteral
since in some platforms it relies on expansion such as above.
Note: The string literals quickfixes still need some tunning.
Task-number: QTCREATORBUG-7449
Change-Id: Iebcfea15677dc8e0ebb6143def89a5477e1be7d4
Reviewed-by: hjk <qthjk@ovi.com>
they are lying. nokia has no copyright on this code. and the double
license in a single file looks weird. that's why we moved it to
3rdparty/, so it is clear it is not nokia's.
Approved-by: legal
