In most cases we need to work with the utf16 indices. Only in
cppfindreferences the byte interface is still needed since there we read
in files and work on a QByteArray to save memory.
Change-Id: I6ef6a93fc1875a8c9a305c075d51a9ca034c41bb
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
...and not byte offsets anymore. This is necessary in order to calculate
the line and column numbers correctly with respect to unicode code
points.
Change-Id: I5d79857b3eaefeb8d563b4f1e3938a64debc5e08
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
This will save us toLatin1() conversations in CppTools (which already
holds UTF-8 encoded QByteArrays) and thus loss of information (see
QTCREATORBUG-7356). It also gives us support for non-latin1 identifiers.
API-wise the following functions are added to Token. In follow-up
patches these will become handy in combination with QStrings.
utf16chars() - aequivalent of bytes()
utf16charsBegin() - aequivalent of bytesBegin()
utf16charsEnd() - aequivalent of bytesEnd()
Next steps:
* Adapt functions from TranslationUnit. They should work with utf16
chars in order to calculate lines and columns correctly also for
UTF-8 multi-byte code points.
* Adapt the higher level clients:
* Cpp{Tools,Editor} should expect UTF-8 encoded Literals.
* Cpp{Tools,Editor}: When dealing with identifiers on the
QString/QTextDocument layer, code points
represendet by two QChars need to be respected, too.
* Ensure Macro::offsets() and Document::MacroUse::{begin,end}() report
offsets usable in CppEditor/CppTools.
Addresses QTCREATORBUG-7356.
Change-Id: I0791b5236be8215d24fb8e38a1f7cb0d279454c0
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
This will avoid confusion when later more length and indices methods are
added.
In Token:
length() --> bytes()
begin() --> bytesBegin()
end() --> bytesEnd()
Change-Id: I244c69b022e239ee762b4114559e707f93ff344f
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
The necessary data can be retrieved by the resulting Token.
Change-Id: I79afb23183c156240c690beff30bb11dfe943e61
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
Also fix false positive line continuation on blank line
e.g.
"foo \
bar"
Change-Id: Ic6d345a4b578c955411d119b8438c8dc5065c072
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
Almost most useful feature ever.
Task-number: QTCREATORBUG-2474
Change-Id: If1ad661fab58ffb4a0b9ddb8ba771f2fde3b54ec
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>
See 2.5/2.12. Most useful feature ever.
Change-Id: I3f92256733e15078df56205dba9746ffff8ed145
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>
Some qmake versions ignore "DEFINES += NDEBUG" on Windows.
Change-Id: Ibdf3b1036c2331dcef61dcd278463c02754044d2
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
Now everything following '///' or '//!' is a doxygen comment.
This simplification fixes also the bug ('\n' was considered as part of
the doxygen comment and led to strange highlighting).
Task-number: QTCREATORBUG-8921
Change-Id: I6dae3b80ec11400f2f623897257782a80860a7f3
Reviewed-by: Eike Ziller <eike.ziller@digia.com>
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
Although they are now supported by the lexer
and parser, it is worth to remind that we still
need to address an issue concerning the highlight
of multiline literals (which with the advent of
the new raw strings will become more common).
Task-number: QTCREATORBUG-6722
Change-Id: I137337a9ac0152a1f8b9faded0b960c6fe3dd38a
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
This makes things slightly more efficient. But it will be more
significant when we introduce R"rawliterals" since we would avoid
an even further lookahead for cases like u8R"string".
Change-Id: Id4bad8b917752d23daf2f4989330434979cf602f
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
Reviewed-by: hjk <qthjk@ovi.com>
Previously it was considering invalid things `such as U8"literal".
Change-Id: Icf4d051a26617ac2c6cb35d5a98f8af0ed801556
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
Reviewed-by: hjk <qthjk@ovi.com>
Actually this behavior was already in the Lexer for regular string
literals (although erroneously not for wide ones) and was lost
in the recent commit 23c637c4f6.
Change-Id: I36609038ce22dc6389e9da5b078f7bf7f6c031be
Reviewed-by: hjk <qthjk@ovi.com>
Those are the types char16_t and char32_t along with the new
char/string literals u'', U'', u"", u8"", and U"".
This is particularly important for the use of QStringLiteral
since in some platforms it relies on expansion such as above.
Note: The string literals quickfixes still need some tunning.
Task-number: QTCREATORBUG-7449
Change-Id: Iebcfea15677dc8e0ebb6143def89a5477e1be7d4
Reviewed-by: hjk <qthjk@ovi.com>
The change brokes the Lexer.
This reverts commit e46a5579d3.
Change-Id: I3363c6eff74b53a7f2d9f417941cde07aaa92619
Reviewed-by: Leandro Melo <leandro.melo@nokia.com>
The whitespaces are now highlighted with the same backgroundcolor as the characters.
Also added a test for the Lexer.
Task-number: QTCREATORBUG-5802
Change-Id: Ic1bcd8cfe30d6b8a0281b7963eaab310f972b9d2
Reviewed-by: Roberto Raggi <roberto.raggi@nokia.com>
they are lying. nokia has no copyright on this code. and the double
license in a single file looks weird. that's why we moved it to
3rdparty/, so it is clear it is not nokia's.
Approved-by: legal
