feedc0de/qt-creator
1
0
Fork 0
forked from qt-creator/qt-creator
Code Pull Requests Activity

C++: fix multi-byte character handling in input.

Browse Source 
Temporary fix: if a single byte is found with the highest bit set, then
convert from utf8 to latin1. This can be removed when the lexer can
handle multi-byte characters.

Task-number: QTCREATORBUG-10141
Change-Id: I36a17aa18bd1b2378f12d0cecf4fd4957b38d8f2
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
This commit is contained in:
Erik Verbruggen
2013-09-20 12:41:25 +02:00
parent 4eb633fd77
commit 8350c4bdc2
2 changed files with 28 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
src/libs/cplusplus/FastPreprocessor.cpp
View File

@@ -41,6 +41,18 @@ FastPreprocessor::FastPreprocessor(const Snapshot &snapshot)
, _preproc(this, &_env)
{ }
// This is a temporary fix to handle non-ascii characters. This can be removed when the lexer can
// handle multi-byte characters.
static QByteArray convertToLatin1(const QByteArray &contents)
{
const char *p = contents.constData();
while (char ch = *p++)
if (ch & 0x80)
return QString::fromUtf8(contents).toLatin1();
return contents;
}
QByteArray FastPreprocessor::run(Document::Ptr newDoc, const QByteArray &source)
{
std::swap(newDoc, _currentDoc);
@@ -56,7 +68,9 @@ QByteArray FastPreprocessor::run(Document::Ptr newDoc, const QByteArray &source)
mergeEnvironment(i.resolvedFileName());
}
const QByteArray preprocessed = _preproc.run(fileName, source);
QByteArray src = convertToLatin1(source);
const QByteArray preprocessed = _preproc.run(fileName, src);
// qDebug("FastPreprocessor::run for %s produced [[%s]]", fileName.toUtf8().constData(), preprocessed.constData());
std::swap(newDoc, _currentDoc);
return preprocessed;