Commit Graph

40 Commits

Author SHA1 Message Date
Christian Stenger
239a59c596 AutoTest: Introduce ITestConfiguration
Preparation for an additional simpler test configuration
which needs to have the same common base.

Change-Id: I1800ed5d7301f1aea99eba6ef588a204697bd569
Reviewed-by: David Schulz <david.schulz@qt.io>
2020-11-16 08:10:04 +00:00
Christian Stenger
540ea616ec AutoTest: Fix handling of GTEST_SKIP
Fixes: QTCREATORBUG-23736
Change-Id: I3d52f3caf6b28e07d7d8d072fb33c782d33575c6
Reviewed-by: David Schulz <david.schulz@qt.io>
2020-03-25 13:38:43 +00:00
Christian Stenger
a57bfda5d8 AutoTest: Handle gtest internal logging
One may use gtest internal logging inside the tests.
Handle its output correctly if needed.

Fixes: QTCREATORBUG-23354
Change-Id: If7b56b7eddf266600dcd815a145cf3b29f8581b3
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-12-16 06:27:57 +00:00
Christian Stenger
333b8f9812 AutoTest: Allow colored commandline output
Some test frameworks allow to print their output colorful
to further indicate meanings of messages or test results.
Provide a highlighter for the textual output of the results
and enable this functionality for GTest and Boost UTF.
Keep at least a small backdoor for overwriting this by
the user.

Fixes: QTCREATORBUG-22297
Change-Id: Iddd2b734416de807635d90c6519553081f7372f2
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-11-13 13:09:00 +00:00
Christian Stenger
2c7e769e31 AutoTest: Clear up and generalize
Instead of transforming forth and back the output
try to handle the output once correctly and pass it
line-wise around.
This also ensures that we always get a single line
when appending the output which will be necessary
later on.

Change-Id: I3e9c6db5f81172997dfe566eee9a86bfe2f17a1f
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-11-13 13:07:25 +00:00
Christian Stenger
a2fa8ce2e8 AutoTest: Replace QRegExp by QRegularExpression
Change-Id: I0e7de5482786105d21765fdf7c1180c16414bc01
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-05-27 13:10:22 +00:00
Christian Stenger
5c6eb0a2ba AutoTest: Further adaption following official terms
GoogleTest started using the term Test Suite for grouping
related tests and Test Case for general tests.
This patch adapts the visual strings as well as some
internal API which used the same terms.
This completes 6189745a3b.

Change-Id: I0307c10b8b5f8574f31a68ee4320732f3e829532
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-05-24 06:24:44 +00:00
Christian Stenger
cbab457cf4 AutoTest: Refactor TestResult
* use enum class
* remove superfluous enum value
* remove superfluous subclass
* remove superfluous constructors
* simplify handling

Change-Id: Iecf11e62eb842c9d455d9238939244496f6b66f4
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-04-26 12:19:05 +00:00
Orgad Shaneh
39ba01da71 Merge remote-tracking branch 'origin/4.9'
Change-Id: I7d1912cd5c4d824fd40d3454c5f1bb796f2c21d8
2019-04-07 23:13:17 +03:00
Christian Stenger
efbe62b389 AutoTest: Fix handling of internal messages
When order results by applications is enabled some internal
messages might end up in a wrong (or at least useless) position
inside the results pane.
For non-pure-global results we need to pass the id / application
to avoid this.

Change-Id: I4221326f9729547a1ee49eeb0ee4f82807444ae7
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-04-04 12:52:22 +00:00
Christian Stenger
19f2da8048 AutoTest: Redo reporting of disabled tests
Provide a proper way of reporting disabled tests instead
of the hacky one.

Change-Id: Id3a775d57ac20c9db8ad0f504cb708021117aa2f
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-04-03 08:11:57 +00:00
hjk
6f37348b4c Replace static_casts by QOverload where possible
Mainly to get rid of the QProcess::finished deprecation warning.

Also adjust coding style in the surrounding connects when needed.

Change-Id: I12f9b248c7974b892c4a069356e578e80f8c59e9
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2019-02-26 15:18:14 +00:00
Christian Kandeler
c6a6c12f05 Canonicalize some includes
Our canonical style is
    #include <utils/fileutils.h>
rather than
    #include "utils/fileutils.h"
Which makes sense, as such headers will never be found in the local
directory.

Change-Id: I4ca46e90d6c4d19d4b1f235f4c79caad864ef222
Reviewed-by: hjk <hjk@qt.io>
2019-01-10 09:04:46 +00:00
Christian Stenger
1cfaa828a8 AutoTest: Fix outputparser
This patch amends 3056105c66. This patch fixes
handling of output while debugging and appending
to the "real" output if the user interacts with
the output while the test is running.

Change-Id: I1db54382f1df3e2b9a5a860002aac8fb208ee5b9
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-11-08 06:33:09 +00:00
Christian Stenger
3056105c66 AutoTest: Fix displaying XML content
CDATA sections that have line breaks at their line ends
within a multiline CDATA section should get handled
that these (inner) line breaks do not get lost.
This patch fixes displaying failed Qt tests when using
the XML output.

Change-Id: I3aa7100836613372ac5b2b409c5cfacfb34209ba
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-11-06 09:40:18 +00:00
Sergey Morozov
db85693690 AutoTest: Create tree item for each failure
Introduce special location item for this purpose.

Task-number: QTCREATORBUG-20967
Change-Id: Icb6c6ec8ff9d2e4e8e3d13834427bae50d74bd2a
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2018-08-29 16:57:05 +00:00
Christian Stenger
1d894c0f7a AutoTest: Code cosmetics
* use nullptr
* override
* remove old style casts
* fix most of the code model warnings

Change-Id: Ia08e846f9326ae28ca7e7d66748e25a8b817b9b1
Reviewed-by: Xing Xiong
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-07-20 08:34:48 +00:00
Christian Stenger
78bd3f9ec3 AutoTest: Rename results executable
Rename member to better reflect its meaning. It is used
as an identifier and might be something else than an
executable.

Change-Id: I082e3caaa3d94fcd7ab26ebc43bb0eb6da136d57
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-05-15 05:28:45 +00:00
Christian Stenger
3b7d550a7c AutoTest: Do not connect on nullptr object
The application can be a nullptr. If so, do not try to
establish a connection.

Change-Id: Icdcd04a016105f163691144277f44c65f96d3847
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-05-08 05:40:59 +00:00
Christian Stenger
8c8313094f AutoTest: Make message translatable
Change-Id: I987fc5da3155040a4580546ac2eca1257d229d5b
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-04-23 06:34:59 +00:00
Eike Ziller
185fe0c705 Merge remote-tracking branch 'origin/4.6'
Conflicts:
	src/plugins/android/androidrunnable.h
	src/plugins/android/androidrunner.cpp
	src/plugins/qmakeandroidsupport/qmakeandroidrunconfiguration.cpp
	src/plugins/qmakeprojectmanager/qmakeproject.cpp
	src/plugins/qmlprofiler/qmlprofilerruncontrol.cpp

Change-Id: I68093d44cfd672347eab82459ff70c21a32297ce
2018-04-17 10:09:35 +02:00
Christian Stenger
54f024dba4 AutoTest: Fix handling exceptions while environment setup
If setting up the environment for gtest fails due to an exception
we might end up having no reportable result but we get a return
code of 1 for the test application.
Inform the results pane instead of just ignoring the return code
and explicitly send a fatal result for this.

Task-number: QTCREATORBUG-20280
Change-Id: I05e522764d6302c5b0760c4bc10e01a2248a4494
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-04-16 08:39:06 +00:00
Claus Steuer
27190cd50a AutoTest: Set gtest-result line number even if file can't be resolved
Task-number: QTCREATORBUG-18725
Change-Id: Id9c4749feafcd3dc3c54fcb25589f45e9cebe283
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2018-04-09 05:45:53 +00:00
Christian Stenger
f60641f3d3 AutoTest: Fix wrong handling of gtest output
Task-number: QTCREATORBUG-19912
Change-Id: I23b47fb6ac2e54f0b9980da8c68c41d3ce7d9511
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-02-23 13:45:55 +00:00
Christian Stenger
c57b06b9e6 AutoTest: Fix noticing crashes on Windows for gtest
gtest based tests do not end the QProcess with CrashExit
when abort() is triggered but it results in a special exit
code.

Task-number: QTCREATORBUG-19565
Change-Id: I4ffc67cb43b05406ab79a7b3061e75c88b87ba98
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-01-16 11:47:30 +00:00
David Schulz
d11a724047 AutoTest: generalize create default result
Add a pure virtual function to TestOutputReader returning
a pointer to a default constructed result.

Change-Id: Icd8090346f2c5115a7c548da7ba69ea54fe39fc4
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2018-01-16 09:11:23 +00:00
Christian Stenger
3eb8c88974 AutoTest: Improve process error handling
If the test executable cannot be started or it seems
not to be the correct one or wrong arguments have been
passed to it the user now gets a respective message
inside the results pane.

Task-number: QTCREATORBUG-18955
Change-Id: Ica68cdbb9e401c8d48a9ce8b23b65d5410d2075a
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
2017-10-04 12:29:01 +00:00
Claus Steuer
4c04cff7c1 AutoTest: Test execution from result output pane
Enable test execution via context menu in the result output pane.
At the moment only gtest is supported.

Task-number: QTCREATORBUG-16695
Change-Id: Ib39164c3cb44d249647b11e25dc51c9ac5db89c5
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2017-09-27 09:47:28 +00:00
Christian Stenger
cd51ff7c0f Utils: Fix typo in function name
Change-Id: Ibd973f68d291731dab06ef1b7eee902be19601a5
Reviewed-by: hjk <hjk@qt.io>
2017-09-05 13:49:23 +00:00
Claus Steuer
4e4118dd26 AutoTest: Add location info to all GTest results
Currently location information are only set for failed gtest results.
One can't jump to the location of a successful test by doing a double
click on the corresponding entry in the test result pane.

Use the TestTreeModel to obtain the location information for successful
tests in the GTestOutputReader.

Change-Id: I3fad2d0540edb653ff186011cb92db220748b4f5
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2017-09-01 05:02:38 +00:00
Christian Stenger
625129d29c AutoTest: Fix appending test results with the same name
If two test cases had the same name but came from a different
executable the results pane still might have shown any later one
(badly) cascaded inside the first one. Avoid this by providing
an additional unique information (the respective executable for
the test case)

Task-number: QTCREATORBUG-18502
Change-Id: Ib071e389758b6269a9a90cc4c4afbcf86ca583ac
Reviewed-by: David Schulz <david.schulz@qt.io>
2017-07-24 11:05:08 +00:00
Christian Stenger
c1555b1d7e AutoTest: Provide way to access the original test output
When running tests the original output is processed and not presented
to the user at all. For crashing tests this could mean that output
was not able to get processed completely (e.g. when having XML as
output and relying on well-formed code)
Unhandled output could also lead to incorrect results.
This patch adds another view to the results pane which contains
the complete output of the last test run.

Change-Id: I923496e9c440de4ea68bee55415777ea5c2379c2
Reviewed-by: David Schulz <david.schulz@qt.io>
2017-06-13 12:50:22 +00:00
Orgad Shaneh
da7cb91446 Merge remote-tracking branch 'origin/4.2' into 4.3
Change-Id: I4931dcc81be872d7712e67123e94d15ee696459f
2017-03-14 16:20:14 +02:00
Samuel Gaist
4033471aa9 QRegExp include cleanup
This patch adds the missing include statements for QRegExp.

Change-Id: Ibb03b929940adb84ae190b5090cb6b88653cc14c
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
2017-03-13 22:05:30 +00:00
Christian Stenger
23398229e9 AutoTest: Modernize code
* remove unnecessary QLatin1String where possible
* foreach
* nullptr
* bad naming of static members

Change-Id: If0f7c6839be4d2efbfef6f7f0775d2034775bd90
Reviewed-by: David Schulz <david.schulz@qt.io>
2017-02-17 08:01:36 +00:00
Christian Stenger
b9fc628bb9 AutoTest: Fix result tree for google test
Change-Id: I1f4a361a3c4ceb77c221d6549e42a1b7803ef305
Reviewed-by: David Schulz <david.schulz@qt.io>
2017-01-03 07:01:37 +00:00
Christian Stenger
b0bf07c64a AutoTest: Modernize code a bit
Change-Id: Ie56f6e5cb8a4f1962bd8df5e57214911ae609921
Reviewed-by: David Schulz <david.schulz@qt.io>
2016-09-29 12:27:14 +00:00
Christian Stenger
bb643a3cdc AutoTest: Refactor output handling
Handle getting output from application's process inside base class
and just process output inside the sub classes.
Additionally this is a preparation for being able to process output
for debugging tests as well.

Change-Id: I8a2289dc7faab25afe08530b5021a0318f3ba6a6
Reviewed-by: David Schulz <david.schulz@qt.io>
2016-08-15 10:26:40 +00:00
Christian Stenger
e10189efb3 AutoTest: Fix extraction of failure location
It seems the failure location output is slightly different when
using an MSVC compiler.

Change-Id: I8f07debac505a0429f4ea1644d076ffcc3304f25
Reviewed-by: David Schulz <david.schulz@theqtcompany.com>
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2016-06-17 04:59:28 +00:00
Christian Stenger
2f8b4b3fbf AutoTest: Move test framework related code...
...into files and respective sub folders.

Change-Id: Ic80d693bd73993a6e74b6d422349e47276e8cb6e
Reviewed-by: David Schulz <david.schulz@theqtcompany.com>
2016-06-01 11:37:32 +00:00