feedc0de/qt-creator
1
0
Fork 0
forked from qt-creator/qt-creator
Code Pull Requests Activity

qmljs: handle string templates in scanner

Browse Source 
* add (multiline) template string support
  templates can be nested, which means that the scanner/lexer cannot
  be a simple state machine anymore, but should have a stack to store
  the state (the number of open braces in the current template
  string).
  The lexer stare is currently stored in an int, so we abuse that and
  store a the number of open braces (maximum 0x7f = 127) for at most 5
  nested templates in the int after the flags for the multiline
  comments and strings.
* improve representation of delimiters (==, <=, ||,... not split in
separate delimiters)
* (QmlDom backport)

Change-Id: I2b4d23b65febedef29a748f4c5f377fde27bd7fd
Fixes: QTCREATORBUG-22766
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
This commit is contained in:
Fawzi Mohamed
2021-02-19 16:30:21 +01:00
parent 28d6be9867
commit bd05384765
3 changed files with 170 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
src/libs/qmljs/qmljsscanner.h
View File

@@ -62,23 +62,60 @@ public:
inline bool is(int k) const { return k == kind; }
inline bool isNot(int k) const { return k != kind; }
static int compare(const Token &t1, const Token &t2) {
if (int c = t1.offset - t2.offset)
return c;
if (int c = t1.length - t2.length)
return c;
return int(t1.kind) - int(t2.kind);
}
public:
int offset;
int length;
Kind kind;
int offset = 0;
int length = 0;
Kind kind = EndOfFile;
};
inline int operator == (const Token &t1, const Token &t2) {
return Token::compare(t1, t2) == 0;
}
inline int operator != (const Token &t1, const Token &t2) {
return Token::compare(t1, t2) != 0;
}
class QMLJS_EXPORT Scanner
{
public:
enum {
FlagsBits = 4,
BraceCounterBits = 7
};
enum {
Normal = 0,
MultiLineComment = 1,
MultiLineStringDQuote = 2,
MultiLineStringSQuote = 3,
MultiLineMask = 3,
MultiLineStringBQuote = 4,
MultiLineMask = 7,
RegexpMayFollow = 4 // flag that may be combined with the above
RegexpMayFollow = 8, // flag that may be combined with the above
// templates can be nested, which means that the scanner/lexer cannot
// be a simple state machine anymore, but should have a stack to store
// the state (the number of open braces in the current template
// string).
// The lexer stare is currently stored in an int, so we abuse that and
// store a the number of open braces (maximum 0x7f = 127) for at most 5
// nested templates in the int after the flags for the multiline
// comments and strings.
TemplateExpression = 0x1 << 4,
TemplateExpressionOpenBracesMask0 = 0x7F,
TemplateExpressionOpenBracesMask1 = 0x7F << 4,
TemplateExpressionOpenBracesMask2 = 0x7F << 11,
TemplateExpressionOpenBracesMask3 = 0x7F << 18,
TemplateExpressionOpenBracesMask4 = 0x7F << 25,
TemplateExpressionOpenBracesMask = TemplateExpressionOpenBracesMask1 | TemplateExpressionOpenBracesMask2
| TemplateExpressionOpenBracesMask3 | TemplateExpressionOpenBracesMask4
};
Scanner();