forked from qt-creator/qt-creator
70122b3061
This will save us toLatin1() conversations in CppTools (which already
holds UTF-8 encoded QByteArrays) and thus loss of information (see
QTCREATORBUG-7356). It also gives us support for non-latin1 identifiers.
API-wise the following functions are added to Token. In follow-up
patches these will become handy in combination with QStrings.
utf16chars() - aequivalent of bytes()
utf16charsBegin() - aequivalent of bytesBegin()
utf16charsEnd() - aequivalent of bytesEnd()
Next steps:
* Adapt functions from TranslationUnit. They should work with utf16
chars in order to calculate lines and columns correctly also for
UTF-8 multi-byte code points.
* Adapt the higher level clients:
* Cpp{Tools,Editor} should expect UTF-8 encoded Literals.
* Cpp{Tools,Editor}: When dealing with identifiers on the
QString/QTextDocument layer, code points
represendet by two QChars need to be respected, too.
* Ensure Macro::offsets() and Document::MacroUse::{begin,end}() report
offsets usable in CppEditor/CppTools.
Addresses QTCREATORBUG-7356.
Change-Id: I0791b5236be8215d24fb8e38a1f7cb0d279454c0
Reviewed-by: Erik Verbruggen <erik.verbruggen@digia.com>
80 lines
2.6 KiB
C++
80 lines
2.6 KiB
C++
/****************************************************************************
|
|
**
|
|
** Copyright (C) 2014 Digia Plc and/or its subsidiary(-ies).
|
|
** Contact: http://www.qt-project.org/legal
|
|
**
|
|
** This file is part of Qt Creator.
|
|
**
|
|
** Commercial License Usage
|
|
** Licensees holding valid commercial Qt licenses may use this file in
|
|
** accordance with the commercial license agreement provided with the
|
|
** Software or, alternatively, in accordance with the terms contained in
|
|
** a written agreement between you and Digia. For licensing terms and
|
|
** conditions see http://qt.digia.com/licensing. For further information
|
|
** use the contact form at http://qt.digia.com/contact-us.
|
|
**
|
|
** GNU Lesser General Public License Usage
|
|
** Alternatively, this file may be used under the terms of the GNU Lesser
|
|
** General Public License version 2.1 as published by the Free Software
|
|
** Foundation and appearing in the file LICENSE.LGPL included in the
|
|
** packaging of this file. Please review the following information to
|
|
** ensure the GNU Lesser General Public License version 2.1 requirements
|
|
** will be met: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html.
|
|
**
|
|
** In addition, as a special exception, Digia gives you certain additional
|
|
** rights. These rights are described in the Digia Qt LGPL Exception
|
|
** version 1.1, included in the file LGPL_EXCEPTION.txt in this package.
|
|
**
|
|
****************************************************************************/
|
|
#ifndef CPLUSPLUS_SIMPLELEXER_H
|
|
#define CPLUSPLUS_SIMPLELEXER_H
|
|
|
|
#include <cplusplus/CPlusPlusForwardDeclarations.h>
|
|
#include <cplusplus/Token.h>
|
|
|
|
#include <QString>
|
|
#include <QList>
|
|
|
|
namespace CPlusPlus {
|
|
|
|
class SimpleLexer;
|
|
class Token;
|
|
|
|
class CPLUSPLUS_EXPORT SimpleLexer
|
|
{
|
|
public:
|
|
SimpleLexer();
|
|
~SimpleLexer();
|
|
|
|
bool skipComments() const;
|
|
void setSkipComments(bool skipComments);
|
|
|
|
LanguageFeatures languageFeatures() const { return _languageFeatures; }
|
|
void setLanguageFeatures(LanguageFeatures features) { _languageFeatures = features; }
|
|
|
|
bool endedJoined() const;
|
|
|
|
QList<Token> operator()(const QString &text, int state = 0, bool convertToUtf8 = false);
|
|
|
|
int state() const
|
|
{ return _lastState; }
|
|
|
|
static int tokenAt(const QList<Token> &tokens, unsigned offset);
|
|
static Token tokenAt(const QString &text,
|
|
unsigned offset,
|
|
int state,
|
|
bool qtMocRunEnabled = false);
|
|
|
|
static int tokenBefore(const QList<Token> &tokens, unsigned offset);
|
|
|
|
private:
|
|
int _lastState;
|
|
LanguageFeatures _languageFeatures;
|
|
bool _skipComments: 1;
|
|
bool _endedJoined: 1;
|
|
};
|
|
|
|
} // namespace CPlusPlus
|
|
|
|
#endif // CPLUSPLUS_SIMPLELEXER_H
|