...especially for the case where the test output is suppressed.
Change-Id: I7294af9e941d800b8aae65f7c2c6f898f2ac4e46
Reviewed-by: Jarek Kobus <jaroslaw.kobus@qt.io>
Get rid of QFutureInterface argument from
ITestConfiguration::createOutputReader() and from
TestOutputReader c'tor.
The fine-grained progress reporting was broken anyway:
1. The assumption was that testCaseCount was meant to be
the total number of test functions executed. It didn't
include the initTestCase() and cleanupTestCase(),
while those were reported on runtime apparently
(and exceeding the max progress by 2).
2. In case of tst_qtcprocess, when the whole test was run,
the testCaseCount reported 41, while the real
number of functions was 26 (+2 = 28 for init/cleanup).
3. While the max progress was set to testCaseCount initially,
the corresponding FutureProgress rendered the progress
always in 0-100 range, what didn't match the reality.
Instead, rely on TaskTree progress, which resolution
is per test as a whole. So, when executing a series
of tests this should scale fine. In addition, the
progress advances fluently according to the expected
run time - with 10 seconds hardcoded.
The original code locations, where progress was bumped,
are left with a TODO comment for any possible future tweaks.
Like in case of result reporting, fine-grained progress
reporting may be implemented by providing additional signal,
so there is no need for QFutureInterface inside
TestOutputReader.
Change-Id: Idc11d55e3a49dac8d1788948b9a82f68199203c6
Reviewed-by: <github-actions-qt-creator@cristianadam.eu>
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
The goal is to make the TestResult a value type without a need
to create these objects on heap.
Introduce a ResultHooks class that will replace all 5 virtual
methods.
Step 1 - implement outputStringHook.
Reorder constructor arguments of TestResult's subclasses so that
the first 2 args are always: id and name.
Change-Id: Iae93e5a348328414f057d1471afad8020b3c1171
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Since we also license under GPL-3.0 WITH Qt-GPL-exception-1.0,
this applies only to a hypothetical newer version of GPL, that doesn't
exist yet. If such a version emerges, we can still decide to relicense...
While at it, replace (deprecated) GPL-3.0 with more explicit GPL-3.0-only
Change was done by running
find . -type f -exec perl -pi -e "s/LicenseRef-Qt-Commercial OR GPL-3.0\+ OR GPL-3.0 WITH Qt-GPL-exception-1.0/LicenseRef-Qt-Commercial OR GPL-3.0-only WITH Qt-GPL-exception-1.0/g" {} \;
Change-Id: I5097e6ce8d10233993ee30d7e25120e2659eb10b
Reviewed-by: Eike Ziller <eike.ziller@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Task-number: QTBUG-67283
Change-Id: I708fd1f9f2b73d60f57cc3568646929117825813
Reviewed-by: Eike Ziller <eike.ziller@qt.io>
QtcProcess::done() is also emitted when process failed to start.
Change-Id: I08f6f104014d1c90c0f761de352dada620207d86
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Still some missing bits as some QString members had different meanings
depending on their context.
Change-Id: Ib48eab54498974a26bbd5123cbffeefee5f7e79c
Reviewed-by: hjk <hjk@qt.io>
This is an attempt to handle output coming from
sanitizers appropriate.
Task-number: QTCREATORBUG-24328
Change-Id: Ifb22d465c9f99d767cc7702b9c459076ab39d567
Reviewed-by: Robert Loehning <robert.loehning@qt.io>
Preparation for an additional simpler test configuration
which needs to have the same common base.
Change-Id: I1800ed5d7301f1aea99eba6ef588a204697bd569
Reviewed-by: David Schulz <david.schulz@qt.io>
This improves constructing the results tree and additionally fixes
matching free functions to their respective test tree items.
Change-Id: I79490507ba7a1934a7be010a00cb341374bf93ad
Reviewed-by: David Schulz <david.schulz@qt.io>
...when using Boost UTF. We need to take parameterized and
templated tests into account differently and we may have
results that do not provide file information at all, so try
to find a matching tree item from what we have got.
Change-Id: Ia0b1894d5dd729d39e5724d9fdeadd574a9cfde5
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Reviewed-by: David Schulz <david.schulz@qt.io>
Avoid overwriting already gathered results and handle
special cases like parameterized boost tests correctly.
Beside this take into account that fatals on the boost
side are treated as failures when printing the summary.
Amends 6ab7013579.
Change-Id: I3815f79cbb4d16fd7b3d286617d79b30e94ccdd8
Reviewed-by: David Schulz <david.schulz@qt.io>
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Do not take skipped tests into account when computing
passes out of the available information.
These are handled separately and may lead to a negative
passes amount.
Change-Id: I47bb1f054468dc5374d0d4e7fdeb3ad87efd0dff
Reviewed-by: David Schulz <david.schulz@qt.io>
Some test frameworks allow to print their output colorful
to further indicate meanings of messages or test results.
Provide a highlighter for the textual output of the results
and enable this functionality for GTest and Boost UTF.
Keep at least a small backdoor for overwriting this by
the user.
Fixes: QTCREATORBUG-22297
Change-Id: Iddd2b734416de807635d90c6519553081f7372f2
Reviewed-by: David Schulz <david.schulz@qt.io>
..to be able to distinguish between the output
channels. Some test frameworks use the stderr
stream for printing relevant output.
This is a preparation for coloring the output
correctly later on.
Change-Id: I3bfea9e552bde3621df99611a124b4f2d3b7d1da
Reviewed-by: David Schulz <david.schulz@qt.io>
Instead of transforming forth and back the output
try to handle the output once correctly and pass it
line-wise around.
This also ensures that we always get a single line
when appending the output which will be necessary
later on.
Change-Id: I3e9c6db5f81172997dfe566eee9a86bfe2f17a1f
Reviewed-by: David Schulz <david.schulz@qt.io>
These todos have been resolved already or turned out
to be wrong / not necessary.
Change-Id: Iefa46bd8144834a60fbb9d523ac73d3372d4a6b6
Reviewed-by: David Schulz <david.schulz@qt.io>
Handle all combinations of LogFormat and ReportFormat
correctly and be able to summarize the test result
correctly by reporting the correct number of Passes,
Fails and Skips.
Change-Id: I47e6fefa2bb2cbb7a25f2ed474a3bb2da1e1e769
Reviewed-by: David Schulz <david.schulz@qt.io>
Send all results after they have been created instead of
relying on having some further output that might send an
old result that had been created but not sent yet.
Change-Id: I0c64b702712509264ee5e86a6af15d6411839f43
Reviewed-by: David Schulz <david.schulz@qt.io>
All test cases are surrounded by the quotes and the
quotes themselves do not belong to the test case name.
Change-Id: Ie38b0bbaed915ea38f8bd11174510c5096de3de8
Reviewed-by: David Schulz <david.schulz@qt.io>
Provide experimental support for Boost UTF.
This patch adds the basic implementation for
* parsing the code for Boost tests
* executing the found tests
* displaying respective results
This is just a basic and limited support which
needs to be enhanced and improved later on.
Task-number: QTCREATORBUG-21169
Change-Id: Ie0da5f51f90fb1fa7217eac461ebfc5214395ef6
Reviewed-by: David Schulz <david.schulz@qt.io>
