Commit Graph

18 Commits

Author SHA1 Message Date
Christian Stenger
b28f907114 AutoTest: Use QtcProcess for test runner
Change-Id: I29b15a08a260aaf8d2c4071d5ea71d49902d8ef6
Reviewed-by: David Schulz <david.schulz@qt.io>
2022-06-14 13:33:20 +00:00
Christian Stenger
5525f2b112 AutoTest: Use Utils::FilePath for files and directories
Still some missing bits as some QString members had different meanings
depending on their context.

Change-Id: Ib48eab54498974a26bbd5123cbffeefee5f7e79c
Reviewed-by: hjk <hjk@qt.io>
2021-05-27 12:24:05 +00:00
Robert Löhning
cd0a140bde Improve translatable strings
Change-Id: I0104209bb0f865a48f9ca2524f62457f7524bba9
Reviewed-by: André Hartmann <aha_1980@gmx.de>
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
2021-03-25 16:53:40 +00:00
Christian Stenger
ca769055e0 AutoTest: Handle sanitizer output
This is an attempt to handle output coming from
sanitizers appropriate.

Task-number: QTCREATORBUG-24328
Change-Id: Ifb22d465c9f99d767cc7702b9c459076ab39d567
Reviewed-by: Robert Loehning <robert.loehning@qt.io>
2020-12-11 07:20:08 +00:00
Christian Stenger
239a59c596 AutoTest: Introduce ITestConfiguration
Preparation for an additional simpler test configuration
which needs to have the same common base.

Change-Id: I1800ed5d7301f1aea99eba6ef588a204697bd569
Reviewed-by: David Schulz <david.schulz@qt.io>
2020-11-16 08:10:04 +00:00
Eike Ziller
8b031c8c05 Merge remote-tracking branch 'origin/4.13' into master
Change-Id: I3f2a6c553819e992da7e9f538dc44b95b482359e
2020-10-02 10:47:07 +02:00
Christian Stenger
d10d25b3f3 AutoTest: Improve Boost test results handling
This improves constructing the results tree and additionally fixes
matching free functions to their respective test tree items.

Change-Id: I79490507ba7a1934a7be010a00cb341374bf93ad
Reviewed-by: David Schulz <david.schulz@qt.io>
2020-10-01 08:08:06 +00:00
Christian Stenger
dba75f3d2c AutoTest: Improve matching test results to tree item
...when using Boost UTF. We need to take parameterized and
templated tests into account differently and we may have
results that do not provide file information at all, so try
to find a matching tree item from what we have got.

Change-Id: Ia0b1894d5dd729d39e5724d9fdeadd574a9cfde5
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Reviewed-by: David Schulz <david.schulz@qt.io>
2020-09-25 12:35:27 +00:00
Christian Stenger
479f6c8129 AutoTest: Fix handling of summary
Avoid overwriting already gathered results and handle
special cases like parameterized boost tests correctly.
Beside this take into account that fatals on the boost
side are treated as failures when printing the summary.
Amends 6ab7013579.

Change-Id: I3815f79cbb4d16fd7b3d286617d79b30e94ccdd8
Reviewed-by: David Schulz <david.schulz@qt.io>
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
2020-09-24 11:39:17 +00:00
Christian Stenger
6ab7013579 AutoTest: Fix summary for boost
Do not take skipped tests into account when computing
passes out of the available information.
These are handled separately and may lead to a negative
passes amount.

Change-Id: I47bb1f054468dc5374d0d4e7fdeb3ad87efd0dff
Reviewed-by: David Schulz <david.schulz@qt.io>
2020-09-14 08:56:10 +00:00
Christian Stenger
333b8f9812 AutoTest: Allow colored commandline output
Some test frameworks allow to print their output colorful
to further indicate meanings of messages or test results.
Provide a highlighter for the textual output of the results
and enable this functionality for GTest and Boost UTF.
Keep at least a small backdoor for overwriting this by
the user.

Fixes: QTCREATORBUG-22297
Change-Id: Iddd2b734416de807635d90c6519553081f7372f2
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-11-13 13:09:00 +00:00
Christian Stenger
e58f376068 AutoTest: Tweak output handling
..to be able to distinguish between the output
channels. Some test frameworks use the stderr
stream for printing relevant output.
This is a preparation for coloring the output
correctly later on.

Change-Id: I3bfea9e552bde3621df99611a124b4f2d3b7d1da
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-11-13 13:07:35 +00:00
Christian Stenger
2c7e769e31 AutoTest: Clear up and generalize
Instead of transforming forth and back the output
try to handle the output once correctly and pass it
line-wise around.
This also ensures that we always get a single line
when appending the output which will be necessary
later on.

Change-Id: I3e9c6db5f81172997dfe566eee9a86bfe2f17a1f
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-11-13 13:07:25 +00:00
Christian Stenger
c23fe4f403 AutoTest: Remove wrong todos
These todos have been resolved already or turned out
to be wrong / not necessary.

Change-Id: Iefa46bd8144834a60fbb9d523ac73d3372d4a6b6
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-06-07 06:40:52 +00:00
Christian Stenger
77902649a8 AutoTest: Use reporting summary from Boost UTF
Handle all combinations of LogFormat and ReportFormat
correctly and be able to summarize the test result
correctly by reporting the correct number of Passes,
Fails and Skips.

Change-Id: I47e6fefa2bb2cbb7a25f2ed474a3bb2da1e1e769
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-06-07 06:40:11 +00:00
Christian Stenger
c911be190f AutoTest: Fix handling of test output messages
Send all results after they have been created instead of
relying on having some further output that might send an
old result that had been created but not sent yet.

Change-Id: I0c64b702712509264ee5e86a6af15d6411839f43
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-06-05 13:25:22 +00:00
Christian Stenger
07b1d2aa56 AutoTest: Fix regular expression
All test cases are surrounded by the quotes and the
quotes themselves do not belong to the test case name.

Change-Id: Ie38b0bbaed915ea38f8bd11174510c5096de3de8
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-05-27 13:10:04 +00:00
Christian Stenger
339f26aec1 AutoTest: Add basic boost test support
Provide experimental support for Boost UTF.
This patch adds the basic implementation for
 * parsing the code for Boost tests
 * executing the found tests
 * displaying respective results

This is just a basic and limited support which
needs to be enhanced and improved later on.

Task-number: QTCREATORBUG-21169
Change-Id: Ie0da5f51f90fb1fa7217eac461ebfc5214395ef6
Reviewed-by: David Schulz <david.schulz@qt.io>
2019-05-20 13:09:08 +00:00