Commit Graph

163 Commits

Author SHA1 Message Date
Marco Bubke
a86867eb8a Clang: Introduce parallel indexing
Change-Id: I522cb18e6d24b7dbed5d5dfa3a732e5b3b5113bb
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-09-03 12:13:10 +00:00
Marco Bubke
2f41886c4b ClangRefactoring: Process tasks after a task has been finished
It could be that processTasks is executed before the future is finished
but in that case there are other tasks which will be called later.

Change-Id: I9b1bfb6fdd642f23842b9c70d60d5b1552193b99
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-09-03 09:40:03 +00:00
Marco Bubke
b2c3e683cf ClangRefactoring: Add SymbolIndexerTaskScheduler
The scheduler is managing the asynchronous tasks by using the symbols
collector manager. Every symbols collector can be used by only one thread,
so the we have to pass the symbols collector around by the future interface
to make the available again after a task is finished.

Change-Id: Ic2eeaa986c2d93978d043216c46e8cb38cea769f
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-27 13:46:34 +00:00
Marco Bubke
5bd7af7a90 ClangRefactoring: Add SymbolIndexerTaskQueue
A first step for concurrent index task.

Change-Id: I9a0dba9f4a67ee605281516785697045b34e2694
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-27 09:17:39 +00:00
Marco Bubke
6fba0be280 ClangRefactoring: Improve symbol parsing
Declarations are only indexed if their file has been changed and references
has to be indexed if the file or any included file has been changed.

Change-Id: I07c6de1379bce2462c1e0fad34d4378a3da4397b
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-27 08:51:29 +00:00
Marco Bubke
d02d9b0f43 ClangRefactoring: Remove FileCache from SymbolStorage
It is not used.

Change-Id: Iaff5a34dfe613f0a627e0a3a727af1130ad2b522
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-23 08:11:29 +00:00
Marco Bubke
39e78bd3f6 Clang: Don't index already indexed symbols
We don't index system headers any more and introduce a first step to
decrease double indexing. For that we introduces the SourcesManager which
so far tells you only if a file was already indexed for a certain time
stamp.

Change-Id: Icde54465693ca84a622764c595635cac365c0111
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-20 13:16:52 +00:00
Marco Bubke
cc0db43c34 Clang: Extent sources manager with dependency
For many index cases such a function call it is needed to check if
any of the included files are changed because the function which is called
could be changed too.

Change-Id: Ibe0f43426c735d39072f284cad075dd4dc6f99c0
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-20 13:16:19 +00:00
Marco Bubke
6a541fd692 Clang: Add sources manager
Indexing source files multiple times is unneeded overhead in many cases.
The source manager is a support class to handle that cases.

Change-Id: Id737eaa9a691c54563279562833493a221eb3431
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-20 08:57:28 +00:00
Marco Bubke
b592339b4d Clang: Don't parse system headers
Change-Id: I6474fbe4f43daaac930ad6ba49fd9cb3145a3bbd
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-14 15:03:41 +00:00
Marco Bubke
4195fce68f ClangRefactoring: Integrate generated files
This is an intermediate step to handle the indexing of the project parts
completely. The generated files are now independently handled from the
project parts. We still not handle the case the a file is indexed but the
generated file is not provided. This will be done in a different patch.
All provided data is now sorted too to improve merging.

Change-Id: I09712b99217a881ec0a233d09aea8659fb787324
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-14 12:47:03 +00:00
Marco Bubke
0bef7610a1 ClangRefactoring: Move generated files to own messages
There must be a management about it but this will be added in a different
change set.

Change-Id: I47b6ce7f671f1c8ae07083d6d99c534069e3cc1a
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-08 13:13:35 +00:00
Marco Bubke
613db15cea ClangRefactoring: Add project part queue
Change-Id: Id137233a042181553ff7bb766c9dafa6eb9cf046
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-08-08 13:07:26 +00:00
Nikolai Kosjar
29110b68f7 Clang: Fix qbs build for libtooling components
Change-Id: I320e23ffd8a244a7582b9d4115f6c8616ba44022
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
2018-04-17 14:25:00 +00:00
Ivan Donchevskii
851726aad3 ClangRefactoring: Do not save macros with invalid fileId
Change-Id: I2933fc3695ebf03b3e4f05a0435ca4d629877d2c
Reviewed-by: Marco Bubke <marco.bubke@qt.io>
2018-04-12 13:14:38 +00:00
Marco Bubke
4b0bcbdcb6 Clang: Locator filter for the symbol database
There are no symbol queries for the locator filters. The signature
generation is still not implemented but for simple cases it should work.

Change-Id: Ic6b04fbe1e7e057892f194ac139615c47d6ec33f
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-04-12 13:12:24 +00:00
Marco Bubke
ee85cf4518 Clang: Store the symbol kind in the database
It is cleaning up some other stuff too.

Change-Id: I75274356fd35f2ee8c84aedf8839c67506ab2355
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-04-12 13:12:08 +00:00
Marco Bubke
b0fd6c30ce Clang: Distinguish between Enum and Records
Creator is distinguishing enumerations and records, so we should do too.

Change-Id: I114cfd207464abd9afd96c26c7504cf8a3a1cb8c
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-04-12 13:11:51 +00:00
Marco Bubke
1b9469aa85 Clang: Use DeclarationName for indexing
Change-Id: Iefb2cb8a21c0f76d935bca56a4e9e02d6626e936
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-04-09 13:09:39 +00:00
Marco Bubke
d4ef90e632 Clang: Filter duplicate locations
An unique index is prevent double locations in the database.

Change-Id: I167cafe1a707dd8a8a9754b8d69790a8382f4eea
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-04-09 13:09:28 +00:00
Orgad Shaneh
5e483d04bc Clang: Fix build
This amends commit cf4dbb4bb6.

Change-Id: Ib74fba38348a02f6bb9d254b7a5e9013a2013942
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2018-04-09 05:25:40 +00:00
hjk
cf4dbb4bb6 ClangSupport: Use simpler structures in some cases
The patch is mostly mechanical, but contains also a few spurious changes
from values references for some local variables, foreach -> ranged for
etc that I coulnd't resist.

Change-Id: I58f0bd972546895eb318607cbfbd7ac35caf3f23
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-04-06 12:55:23 +00:00
Nikolai Kosjar
39306d55eb Clang: Fix build with QTC_NO_CLANG_LIBTOOLING
Change-Id: I8b3f4db10c43fc1620b765933915e3c4fd0a594a
Reviewed-by: Marco Bubke <marco.bubke@qt.io>
2018-03-28 11:47:52 +00:00
Marco Bubke
de4f6301e6 Clang: Process SymbolKind and SymbolTags with the indexer
We add the infrastructure to compute the SymbolKind and SymbolTags in the
indexer. Later we have to add more for templates, virtual functions etc..

Change-Id: I9203c5cfbfffed3065337292010de5fce5736453
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-03-27 14:45:58 +00:00
Marco Bubke
a8b11cb2c5 Clang: Improve access to caching structure
The clang file id is counted from zero, so it better to use a vector
instead of a hash with all the overhead.

Change-Id: Iaf201898e9e16005d196b5b49065f15f9d3d2dfa
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-03-27 14:45:41 +00:00
Marco Bubke
576eb3370c Clang: Use the clang indexer for indexing
It is not removing so much code but will makes somethings easier in the
future. We added the SymbolType::Definition too.

Change-Id: I4e106b8518e6bfed0c6a4aa6be61c4a5fe5f8bef
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-03-27 14:45:35 +00:00
Marco Bubke
d6df537ae2 Clang: Remove old CPP code
It was there to suppress warnings but this is now done with compiler
arguments.

Change-Id: I62edae9c115c09aa07601547159c04e038943a9b
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-03-26 13:39:58 +00:00
Marco Bubke
9c4bfbe20a Clang: Improve interfaces
The interfaces should never used to handle ownership. So it is now using
protected destructors. Copy operations are forbidden too.

https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#c35-a-base-class-destructor-should-be-either-public-and-virtual-or-protected-and-nonvirtual

Change-Id: Ib0b60a73a7ec130973b5cb0095cc5b2f10fa0758
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-03-26 13:39:36 +00:00
Marco Bubke
53454b0f79 Clang: Use PCHs for indexing
As generating the AST is quite expensive it would be very useful to cache
the not changed include. So we generate PCHs for include outside of a
project part. With this change this PCHs are used by the indexer.

For that they are save to the symbol database by the PCH manager and when
fetched by the symbol indexer.

Change-Id: I7a5b07cfb32d72d50dc52d2b108cd41727a7bfc7
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-03-22 13:26:24 +00:00
Christian Stenger
092d5068bd ClangRefactoring: Fix compile for Qt < 5.8
Change-Id: I61a848c9a7bad42697dd6de4f5b002cc655381c6
Reviewed-by: Marco Bubke <marco.bubke@qt.io>
2018-02-23 06:24:17 +00:00
Marco Bubke
73963cc9c7 Clang: Add time stamp based filtering for project parts
The source ids are now filtered by the modified time. If the modified time
in the database is older than the modified time of the file it will be
parsed. If it is not newer it will be not parsed.

Change-Id: I4ade3443dd66573ac88053a2cafa600e54cfe973
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-22 10:24:59 +00:00
Marco Bubke
a6f47e872f Clang: Add isInPrecompiledHeader to the file status table
It will useful to find out if we have to wait for an updated precompiled
header.

Change-Id: I6a314f278485965571cc6e46982bbd6f5523c581
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-22 10:24:31 +00:00
Marco Bubke
49bdde2ab0 Clang: Fix formatting
Change-Id: I77fb233f101f7046d5cfab71933c413ae15835d4
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-22 10:24:23 +00:00
Marco Bubke
157c74e1f1 Clang: Use C++ condition instead of preprocessor
It is now possible to refactor the code. This pattern is favorable to the
preprocessor.

Change-Id: Ic85db43e23e16412d938a47892f1812b5883d949
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-22 10:24:16 +00:00
Marco Bubke
6922f549d5 Clang: Check if the include search path has changed
If the include search paths and the compiler macros have not changed it is
save to assume that we don't need to update the symbol database. This saves
us from executing a very expensive task. Later we have to test the
modification time of the files too.

Change-Id: I6b958075024a811c2abd3d7918263fd74bba090b
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-22 10:23:39 +00:00
Marco Bubke
0172631b48 Clang: Don't reparse if the macros haven't changed
We have to extend that to include paths too, which will be happen in a
follow up patch.

Change-Id: I7f8ac663ae8588e647fc6a6b5d689a629a28ef65
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-21 14:17:19 +00:00
Marco Bubke
b5c3d5a40d Clang: Improve ProjectPartArtefact
Empty strings were only handled by accident and wrongly formatted ones
were never handled. Now we do nothing for empty strings and throw an
exception from wrongly formatted strings. The exceptions are very helpful
in the test code because the show errors the the testing data.

Change-Id: I87d1678eda7502fdc3f74f51fad491803d28582b
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-21 14:17:09 +00:00
Marco Bubke
56f79feebf Clang: Introduce CompilerMacro
We want not only the name but the value of the macro too. So we can
compare if anything has changed.

Change-Id: Ie59caf8cbf54d108f9e15299d25306a406b5c40d
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-15 09:58:53 +00:00
Marco Bubke
29a4419c07 Clang: Update symbol database if file has changed
The code in the symbol indexer is quite similar, but still different
enough to prevent easy reuse of the function.

Change-Id: I47907d90066da922eafe8ff3cce124ea47ea4a0a
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-14 15:06:58 +00:00
Marco Bubke
dfb9e4355f Sqlite: Transaction catch exception in destructor
Fixes coverity warning. We want to throw an exception in the destructor
in this case, which is save because it is a RAII pattern and holds
only trivial values. Actually this is the only way to communicate that
rollback was failing. It is quite rare that it will be happen without an
exception is already thrown.

There is a non throwing variant too if you want to use it as a class
member.

Change-Id: Ie71ead19678619465c6ad9334d36ee666225dce7
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-14 15:03:22 +00:00
Marco Bubke
c4220041a6 Clang: Rename FileInformation in FileStatus
It's more near to the Unix convention and it has a plural too.

Change-Id: I53f85911d8fcbaadba9947c70e9a375dc6995ed5
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-08 16:49:43 +00:00
Marco Bubke
98774d7b92 Clang: Remove export filters
Lets try to find a better way to do it later.

Change-Id: I960abf65c910f639cc860b8e1291b0c235b7db07
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-08 16:49:27 +00:00
Nikita Baryshnikov
cbcbaaa2f3 ClangRefactoringBackend fix reorder warning
Change-Id: I7cd957c2c172021768c98b7fa87c5a53fa6da54e
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-03 06:56:59 +00:00
Marco Bubke
497900eb3f Clang: Handle source dependencies in symbol indexer
Get the source dependencies from the collector and save them in the
symbol storage.

Change-Id: Ibc141970a100625398c1526f5bc644cc14d25c0c
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-01 15:06:20 +00:00
Marco Bubke
048224bef1 Clang: Collect source dependencies
It is quite easy because we track the include directives in the
preprocessor callbacks.

Change-Id: I2d7bd67b31f50c0d8d4a46c57e83dffa0c558dc7
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-01 15:06:09 +00:00
Marco Bubke
b2d3951bde Clang: Clear used macros and file information for symbol collection
We have to clear them too otherwise we will collect them which can lead
to wrong data in the symbol database.

Change-Id: Iad7b87344caec0f27a5f8f24c214573a274db911
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-01 15:06:00 +00:00
Marco Bubke
89b9dfed84 Clang: Add source dependency saving to the symbol storage
The source dependencies are simply a table which connects the include file
with the included file.

Change-Id: I5454e81a2b5b98f05c7ff3f6740a6d45e01772c3
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-01 15:05:52 +00:00
Marco Bubke
5dd24110f5 Clang: SymbolIndexer is saving the file information to the symbol storage
Change-Id: Id34ce9901c1a938e7ea14da444a9ef1b052286e5
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-01 15:05:10 +00:00
Marco Bubke
95ce58cf32 Clang: Add insert file information to symbol storage
We use INSERT OR REPLACE because it is simpler than the insert and update
and we provide the primary key by our self.

Change-Id: I868e040918cc6c2ed183cecc746c89bec136f974
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-01 15:04:51 +00:00
Marco Bubke
70ab43ca62 Clang: Get file informations from symbols collector
We need the same information like file size and the last modification time
to check if a file has changed. This information will be saved to the
database.

Change-Id: Icf8eb9b6553945a563887e11aff9539c9b300b66
Reviewed-by: Ivan Donchevskii <ivan.donchevskii@qt.io>
2018-02-01 15:03:56 +00:00