AutoTest: Restructure settings documentation

This patch restructures the settings part of the
documentation to have all related information is
at one location instead of scattered across the
whole page.
Additionally added some information for so far
undocumented features and fix URLs to referenced
googletest web pages.

Change-Id: Ic897237ea67e9cc11f7d0b55059c47108a6e0f3a
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
This commit is contained in:
Christian Stenger
2021-01-20 15:02:06 +01:00
parent 30c6f44031
commit ba667a342b
6 changed files with 68 additions and 53 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.5 KiB

After

Width:  |  Height:  |  Size: 4.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.1 KiB

After

Width:  |  Height:  |  Size: 4.7 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.3 KiB

After

Width:  |  Height:  |  Size: 4.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

View File

@@ -1,6 +1,6 @@
/****************************************************************************
**
** Copyright (C) 2019 The Qt Company Ltd.
** Copyright (C) 2021 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the Qt Creator documentation.
@@ -30,13 +30,13 @@
\title Running Autotests
\QC supports special handling for a couple of testing frameworks which
is strongly tied to the underlying code models or specialized parsers
(\e {code based tests}).
Beside this there is also support for build system based testing which
is independent from any testing frameworks and retrieves its information
directly from the underlying build system and uses these or even the
build system as such to execute the respective tests.
\QC supports both \e {code based tests} and \e {build system based tests}.
Code based testing provides special handling for particular testing
frameworks that is strongly tied to the underlying code models or
specialized parsers. Build system based testing is independent from any
testing frameworks. It retrieves information directly from the underlying
build system and uses it or even the build system as such to execute the
respective tests.
\QC integrates the \l{Qt Test} framework,
\l{https://github.com/google/googletest}{Google C++ Testing Framework},
@@ -52,10 +52,10 @@
\section1 Build System Based Tests
The handling of build system based tests has to get enabled explicitly
as these are off by default to avoid interference with code based parsers.
To enable the tests, select the respective test tool in
\uicontrol Options > \uicontrol Testing > \uicontrol General.
The handling of build system based tests is disabled by default to avoid
interference with code based parsers. To enable build system based tests,
select the respective test tool in \uicontrol Options > \uicontrol Testing
> \uicontrol General.
The detection of tests is usually much faster for build system based
tests as this normally does not involve scanning or parsing.
@@ -158,7 +158,7 @@
\QC creates the test in the specified project directory.
For more information about creating Google tests, see the
\l{https://github.com/google/googletest/blob/master/googletest/docs/primer.md}
\l{https://github.com/google/googletest/blob/master/docs/primer.md}
{Google Test Primer}.
\section2 Creating Boost Tests
@@ -361,29 +361,6 @@
test list for the currently active test frameworks when you edit tests.
To refresh the view, select \uicontrol {Rescan Tests} in the context menu.
To group related test cases for an active test framework, select
\uicontrol Tools > \uicontrol Options > \uicontrol {Testing} >
\uicontrol General, and then select the \uicontrol Group check box next to
the framework name in the \uicontrol {Active Test Frameworks} list.
By default, tests are grouped based on the directory where they are located.
To group Google tests by using a GTest filter, select
\uicontrol {Google Test}. In the \uicontrol {Group mode} field,
select \uicontrol {GTest Filter} and specify the filter to use in the
\uicontrol {Active filter} field. For more information about GTest
filters, see
\l{https://github.com/google/googletest/blob/master/googletest/docs/advanced.md#running-a-subset-of-the-tests}
{Running a Subset of the Tests}.
\image qtcreator-autotests-options.png
In some special setups, \QC cannot deduce which executable or run
configuration it should use. If \QC repeatedly asks you to select the
tests to run when trying to execute tests, you can enable it to cache
your choices and use them were appropriate. The cached information is
cleared when you switch to another project, close the current one, or
select \uicontrol {Reset Cached Choices}.
To show or hide init and cleanup or data functions in the \uicontrol Tests
view, select \inlineimage filtericon.png
(\uicontrol {Filter Test Tree}), and then select \uicontrol {Show Init and
@@ -410,6 +387,53 @@
\note Availability for build system based tests depends on the build system
tool's abilities.
\section1 Specifying Test Settings
To customize the handling of tests, test frameworks, and test tools, select
\uicontrol Tools > \uicontrol Options > \uicontrol {Testing} >
\uicontrol General.
\image qtcreator-autotests-options.png
You can customize some settings at project level. To change settings
for the current project instead of globally, select \uicontrol Projects >
\uicontrol {Project Settings} > \uicontrol {Testing}.
In the \uicontrol {Active Test Frameworks} list you can select which tests
\QC will handle. To improve the performance of full scans for tests, disable
test frameworks you are not using.
To group related test cases for an active test framework, select the
\uicontrol Group check box next to the framework name in the
\uicontrol {Active Test Frameworks} list.
By default, tests are grouped based on the directory where they are located.
Internal messages and run configuration warnings for deduced configurations
are omitted by default. To view them, deselect the \uicontrol {Omit internal
messages} and \uicontrol {Omit run configuration warnings} check boxes.
By default, test result output is limited to 100,000 characters. The output
pane is automatically scrolled down when new results are added. To display
full results, deselect the \uicontrol {Limit result output} check box.
To disable automatic scrolling, deselect the
\uicontrol {Automatically scroll results} check box.
Test results can be grouped by the executable path that was used to run the
tests. This is useful if you have multiple test executables and run them all
at once. To enable this functionality you need to select the
\uicontrol {Group results by application} check box.
It is possible to automatically run the currently available tests after
successfully building the current project. In \uicontrol {Automatically run},
select which tests should be run after a successful build.
In some special setups, \QC cannot deduce which executable or run
configuration it should use. If \QC repeatedly asks you to select the
tests to run when trying to execute tests, you can enable it to cache
your choices and use them were appropriate. The cached information is
cleared when you switch to another project, close the current one, or
select \uicontrol {Reset Cached Choices}.
\section2 Specifying Settings for Running Qt Tests
The code inside a benchmark test is measured, and possibly also repeated
@@ -451,6 +475,13 @@
failures into C++ exceptions, select the \uicontrol {Throw on failure} check
box.
To group Google tests by using a GTest filter, select
\uicontrol {GTest Filter} in the \uicontrol {Group mode} field,
and specify the filter to use in the \uicontrol {Active filter}
field. For more information about GTest filters, see
\l{https://github.com/google/googletest/blob/master/docs/advanced.md#running-a-subset-of-the-tests}
{Running a Subset of the Tests}.
\section2 Specifying Settings for Running Boost Tests
\list 1
@@ -480,7 +511,7 @@
\li To specify settings for running Catch2 tests, select
\uicontrol Tools > \uicontrol Options > \uicontrol {Testing} >
\uicontrol {Catch Test}.
//! insert image here
\image qtcreator-autotests-options-catch2.png
\li Select the \uicontrol {Show success} check box to show succeeding
expressions as well. By default Catch2 will print only fails.
\li Select the \uicontrol {Break on failure while debugging} check box
@@ -598,20 +629,4 @@
(\uicontrol {Filter Test Results}), and then select the types of messages to
show. To show all messages, select \uicontrol {Check All Filters}. To
deselect all message types, select \uicontrol {Uncheck All Filters}.
By default, test result output is limited to 100,000 characters. The output
pane is automatically scrolled down when new results are added. To display
full results, select \uicontrol Tools > \uicontrol Options >
\uicontrol {Testing} > \uicontrol General, and then deselect the
\uicontrol {Limit result output} check box. To disable automatic scrolling,
deselect the \uicontrol {Automatically scroll results} check box.
Test results can be grouped by the executable path that was used to run the
tests. This is useful if you have multiple test executables and run them all
at once. To enable this functionality you need to select the
\uicontrol {Group results by application} check box.
Internal messages and run configuration warnings for deduced configurations
are omitted by default. To view them, deselect the \uicontrol {Omit internal
messages} and \uicontrol {Omit run configuration warnings} check boxes.
*/