Commit Graph

32 Commits

Author SHA1 Message Date
Christian Stenger
ac14a1e038 AutoTest: Clean up header mess
Better decoupling and reduced binary size.

Change-Id: I4f6239979d9d7dae4ad92f19ec8420be38372c07
Reviewed-by: David Schulz <david.schulz@qt.io>
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2020-11-09 08:24:01 +00:00
Christian Stenger
8806ae3bdf AutoTest: Fix output handling
Connect to the correct signal to explicitly react on
output coming on stdout.
Necessary as we are changing the read channel while
processing output.

Fixes: QTCREATORBUG-23939
Change-Id: Ibf62de62fb64bd9c2395f93643e39d11d5b4e0d5
Reviewed-by: David Schulz <david.schulz@qt.io>
2020-04-29 10:30:52 +00:00
Christian Stenger
333b8f9812 AutoTest: Allow colored commandline output
Some test frameworks allow to print their output colorful
to further indicate meanings of messages or test results.
Provide a highlighter for the textual output of the results
and enable this functionality for GTest and Boost UTF.
Keep at least a small backdoor for overwriting this by
the user.

Fixes: QTCREATORBUG-22297
Change-Id: Iddd2b734416de807635d90c6519553081f7372f2
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-11-13 13:09:00 +00:00
Christian Stenger
e58f376068 AutoTest: Tweak output handling
..to be able to distinguish between the output
channels. Some test frameworks use the stderr
stream for printing relevant output.
This is a preparation for coloring the output
correctly later on.

Change-Id: I3bfea9e552bde3621df99611a124b4f2d3b7d1da
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-11-13 13:07:35 +00:00
Christian Stenger
2c7e769e31 AutoTest: Clear up and generalize
Instead of transforming forth and back the output
try to handle the output once correctly and pass it
line-wise around.
This also ensures that we always get a single line
when appending the output which will be necessary
later on.

Change-Id: I3e9c6db5f81172997dfe566eee9a86bfe2f17a1f
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-11-13 13:07:25 +00:00
Christian Stenger
03b80025a9 AutoTest: Shuffle namespaces for clearer separation
Change-Id: I6ac440c6d72c3d0dbcd75907f8a6ae756f3cf5ee
Reviewed-by: hjk <hjk@qt.io>
2019-09-02 09:04:16 +00:00
Christian Stenger
cbab457cf4 AutoTest: Refactor TestResult
* use enum class
* remove superfluous enum value
* remove superfluous subclass
* remove superfluous constructors
* simplify handling

Change-Id: Iecf11e62eb842c9d455d9238939244496f6b66f4
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-04-26 12:19:05 +00:00
Christian Stenger
d846722264 AutoTest: Move signal emission
... to location where the output gets processed.

Change-Id: I293f1dbd67e4c8969f60305768e1ce5bc8cba063
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-11-08 07:40:12 +00:00
Christian Stenger
1cfaa828a8 AutoTest: Fix outputparser
This patch amends 3056105c66. This patch fixes
handling of output while debugging and appending
to the "real" output if the user interacts with
the output while the test is running.

Change-Id: I1db54382f1df3e2b9a5a860002aac8fb208ee5b9
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-11-08 06:33:09 +00:00
Christian Stenger
3056105c66 AutoTest: Fix displaying XML content
CDATA sections that have line breaks at their line ends
within a multiline CDATA section should get handled
that these (inner) line breaks do not get lost.
This patch fixes displaying failed Qt tests when using
the XML output.

Change-Id: I3aa7100836613372ac5b2b409c5cfacfb34209ba
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-11-06 09:40:18 +00:00
Christian Stenger
78bd3f9ec3 AutoTest: Rename results executable
Rename member to better reflect its meaning. It is used
as an identifier and might be something else than an
executable.

Change-Id: I082e3caaa3d94fcd7ab26ebc43bb0eb6da136d57
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-05-15 05:28:45 +00:00
Christian Stenger
54f024dba4 AutoTest: Fix handling exceptions while environment setup
If setting up the environment for gtest fails due to an exception
we might end up having no reportable result but we get a return
code of 1 for the test application.
Inform the results pane instead of just ignoring the return code
and explicitly send a fatal result for this.

Task-number: QTCREATORBUG-20280
Change-Id: I05e522764d6302c5b0760c4bc10e01a2248a4494
Reviewed-by: David Schulz <david.schulz@qt.io>
2018-04-16 08:39:06 +00:00
David Schulz
8accfddb17 AutoTest: Delegate reporting crashes to output reader
Let the output reader report crashes of the test
instead of just reporting crashes globally.
This allows to reflect the crash also inside summary items
to avoid having tests displayed as passed if no fail happened
until the crash.

Task-number: QTCREATORBUG-19565
Change-Id: Idd9fe28f7486cbb31b1aa6556530718a2278722f
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2018-01-16 11:47:06 +00:00
Christian Stenger
3eb8c88974 AutoTest: Improve process error handling
If the test executable cannot be started or it seems
not to be the correct one or wrong arguments have been
passed to it the user now gets a respective message
inside the results pane.

Task-number: QTCREATORBUG-18955
Change-Id: Ica68cdbb9e401c8d48a9ce8b23b65d5410d2075a
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
2017-10-04 12:29:01 +00:00
Christian Stenger
c1555b1d7e AutoTest: Provide way to access the original test output
When running tests the original output is processed and not presented
to the user at all. For crashing tests this could mean that output
was not able to get processed completely (e.g. when having XML as
output and relying on well-formed code)
Unhandled output could also lead to incorrect results.
This patch adds another view to the results pane which contains
the complete output of the last test run.

Change-Id: I923496e9c440de4ea68bee55415777ea5c2379c2
Reviewed-by: David Schulz <david.schulz@qt.io>
2017-06-13 12:50:22 +00:00
Christian Stenger
053f5bf3d3 AutoTest: Show results for debugging tests as well
At least display results for using gdb or lldb and inform
user that we are not able to do so when using cdb.

Task-number: QTCREATORBUG-16693
Change-Id: I1a98a79aaf76de47b5dcb58d4dca9247f1642d36
Reviewed-by: David Schulz <david.schulz@qt.io>
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2016-09-05 12:40:59 +00:00
Christian Stenger
bb643a3cdc AutoTest: Refactor output handling
Handle getting output from application's process inside base class
and just process output inside the sub classes.
Additionally this is a preparation for being able to process output
for debugging tests as well.

Change-Id: I8a2289dc7faab25afe08530b5021a0318f3ba6a6
Reviewed-by: David Schulz <david.schulz@qt.io>
2016-08-15 10:26:40 +00:00
Christian Stenger
b4c263c974 AutoTest: Give some hint about the warning's origin
Change-Id: I4e26eea953d3e4e840678bd45eb2e4961e270eb2
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@qt.io>
2016-07-11 13:03:58 +00:00
Christian Stenger
2f8b4b3fbf AutoTest: Move test framework related code...
...into files and respective sub folders.

Change-Id: Ic80d693bd73993a6e74b6d422349e47276e8cb6e
Reviewed-by: David Schulz <david.schulz@theqtcompany.com>
2016-06-01 11:37:32 +00:00
Christian Stenger
87a6f003e9 AutoTest: Split off TestResult...
...to avoid dragging around members that are not used in sub-classes.

Change-Id: I3e30b403491b1ae9219a0b461c3034cb854701a6
Reviewed-by: David Schulz <david.schulz@theqtcompany.com>
2016-05-12 13:49:00 +00:00
Takumi ASAKI
8d3e0a1017 AutoTest: tr()-Fixes
Do not use QObject::tr().

Change-Id: If92f36166f29defd3501410efeed8e6f68dc896d
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2016-05-03 13:28:26 +00:00
Christian Stenger
104b434c8d AutoTest: Fix processing of output
In case a (Qt) test process produces output that does not belong
to the standard test output we might ended without processable
output at all.
Avoid this by just handling the output that is given on stdout
for now.

Change-Id: If5ee70140b95cb61422e58a7610b7fe2d09915c2
Reviewed-by: David Schulz <david.schulz@theqtcompany.com>
2016-04-18 11:01:50 +00:00
Christian Stenger
19c8c9bc4b AutoTest: Avoid passing around pointer without owner...
...when using signals and slots between threads.

Change-Id: Ib1ce607e225cb8d9973393c5bdaeebaa6e30bb41
Reviewed-by: Eike Ziller <eike.ziller@theqtcompany.com>
2016-02-26 09:55:48 +00:00
Christian Stenger
ba61f2946d AutoTest: Add some gtest related settings
This patch enables execution of disabled tests, repetition of
test runs and shuffling of test sets.

Change-Id: I3668afde5766d18acdf1afb7edd913655b13a76c
Reviewed-by: David Schulz <david.schulz@theqtcompany.com>
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@theqtcompany.com>
2016-02-26 09:40:46 +00:00
Friedemann Kleint
97af238656 Preferably use QStringRef::toInt().
Avoid allocations for converting to int.

Change-Id: Id8c79334f4809ec075ffe7e6b7635be4873eafd8
Reviewed-by: Eike Ziller <eike.ziller@theqtcompany.com>
2016-02-04 07:56:58 +00:00
Christian Stenger
b2c6ea47b1 AutoTest: Display output of passes on test results pane...
...if possible. If passing tests had output beside the result
it had been omitted so far.

Change-Id: I37fd14714d00fe6a28f079f3fe2ca6e82606fa31
Reviewed-by: Niels Weber <niels.weber@theqtcompany.com>
2016-01-29 08:34:41 +00:00
Christian Stenger
09bc4baf3c AutoTest: Fix source file construction on Windows
Task-number: QTCREATORBUG-15667
Change-Id: I4d0e81a39a5031403d77fb3e2179ed17451346b4
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
2016-01-28 07:03:53 +00:00
Christian Stenger
9ff5786d53 AutoTest: Fix parsing of file/line information for gtest...
...this was wrong in cases where we have additional output before
this information.

Change-Id: I6f55c48815217da98078d40e5a0f1f4ce4ad8a77
Reviewed-by: Niels Weber <niels.weber@theqtcompany.com>
2016-01-26 16:27:43 +00:00
Orgad Shaneh
621c834023 AutoTest: Clear description on test start
Task-number: QTCREATORBUG-15668
Change-Id: I031941938f59e59e0e05d3284b12ef09993f6541
Reviewed-by: Christian Stenger <christian.stenger@theqtcompany.com>
2016-01-26 11:42:29 +00:00
Orgad Shaneh
6062d3953d AutoTest: Fix pass/fail parsing on Windows
Chop \r

Change-Id: Ibd1ce9025a681aefcdce0c01a842903fda8a8e30
Reviewed-by: Christian Stenger <christian.stenger@theqtcompany.com>
2016-01-25 06:00:10 +00:00
Orgad Shaneh
e3d4e1c825 AutoTest: Fix invalid escape sequence
Change-Id: I9cd09327a561eb8691e358d77a455714838397bb
Reviewed-by: Christian Stenger <christian.stenger@theqtcompany.com>
2016-01-25 05:58:48 +00:00
Christian Stenger
15b4e6fc91 Delete unneeded and prepare directory structure...
...to get plugin's repository more easily integrated
into QC's repository.

Change-Id: I10243bb4283623ae951e799feacaa15915678e04
Reviewed-by: Eike Ziller <eike.ziller@theqtcompany.com>
2016-01-22 14:07:19 +00:00