Doc: Describe Squish plugin
Fixes: QTCREATORBUG-27890 Change-Id: Icc203a96f08c785f016130480c45c988d26ac371 Reviewed-by: <github-actions-qt-creator@cristianadam.eu> Reviewed-by: Christian Stenger <christian.stenger@qt.io>
BIN
doc/qtcreator/images/icons/jumpto.png
Normal file
|
After Width: | Height: | Size: 399 B |
BIN
doc/qtcreator/images/icons/objectsmap.png
Normal file
|
After Width: | Height: | Size: 140 B |
|
After Width: | Height: | Size: 1.6 KiB |
|
After Width: | Height: | Size: 1.3 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-create-test-suite-aut.png
Normal file
|
After Width: | Height: | Size: 8.5 KiB |
|
After Width: | Height: | Size: 10 KiB |
|
After Width: | Height: | Size: 9.2 KiB |
|
After Width: | Height: | Size: 12 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-create-test-suite.png
Normal file
|
After Width: | Height: | Size: 11 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-debugging-view.png
Normal file
|
After Width: | Height: | Size: 1.4 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-locals.png
Normal file
|
After Width: | Height: | Size: 1.6 KiB |
|
After Width: | Height: | Size: 6.3 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-output-test-results.png
Normal file
|
After Width: | Height: | Size: 24 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-preferences.png
Normal file
|
After Width: | Height: | Size: 7.2 KiB |
|
After Width: | Height: | Size: 3.9 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-server-settings.png
Normal file
|
After Width: | Height: | Size: 13 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-symbolic-names.png
Normal file
|
After Width: | Height: | Size: 13 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-test-script-edit.png
Normal file
|
After Width: | Height: | Size: 6.4 KiB |
|
After Width: | Height: | Size: 6.9 KiB |
BIN
doc/qtcreator/images/qtcreator-squish-view.png
Normal file
|
After Width: | Height: | Size: 3.3 KiB |
@@ -4,7 +4,7 @@
|
||||
/*!
|
||||
\previouspage creator-ctf-visualizer.html
|
||||
\page creator-autotest.html
|
||||
\nextpage creator-advanced.html
|
||||
\nextpage creator-squish.html
|
||||
|
||||
\title Running Autotests
|
||||
|
||||
|
||||
330
doc/qtcreator/src/howto/creator-only/creator-squish.qdoc
Normal file
@@ -0,0 +1,330 @@
|
||||
// Copyright (C) 2022 The Qt Company Ltd.
|
||||
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only
|
||||
|
||||
/*!
|
||||
\previouspage creator-autotest.html
|
||||
\page creator-squish.html
|
||||
\nextpage creator-advanced.html
|
||||
|
||||
\title Using Squish
|
||||
|
||||
\l{https://doc.froglogic.com/squish/latest/}{Squish} is an automated GUI
|
||||
testing framework for testing Android, iOS, Java, Qt, Tk, Windows, and
|
||||
XView applications, as well as HTML-based web applications running in
|
||||
browsers, such as Apple Safari, Mozilla Firefox, Google Chrome, and
|
||||
Microsoft Internet Explorer and Edge.
|
||||
|
||||
The experimental Squish plugin integrates Squish into \QC. You can:
|
||||
|
||||
\list
|
||||
\li Open existing Squish test suites.
|
||||
\li Create new test suites and test cases.
|
||||
\li Record test cases (in a very limited way compared to what you can do
|
||||
inside the Squish IDE).
|
||||
\li Use Squish Runner and Server to run test suites or cases and view
|
||||
the results in the \uicontrol Squish \l{Viewing Output}{output}.
|
||||
\li Set breakpoints before running tests to stop at certain locations and
|
||||
inspect the local variables, similarly to when debugging a test.
|
||||
\endlist
|
||||
|
||||
When running test suites or cases, the Squish Runner instructs the Squish
|
||||
Server to start the application under test (AUT). The server can be running
|
||||
on multiple machines, and the AUT can be located on a different path on each
|
||||
of them. Therefore, you must either map AUTs to their corresponding paths or
|
||||
specify AUT paths to search from in the server settings.
|
||||
|
||||
In addition, you can test an already running application by attaching to it.
|
||||
This enables you to test your application using a Squish Server running on
|
||||
another machine. However, you can have only one server attached to your
|
||||
application at a time. Also, the attached application is not closed when the
|
||||
test case is completed.
|
||||
|
||||
To use the plugin, you must download and install Squish, create a connection
|
||||
to the Squish Server, and specify AUTs to run.
|
||||
|
||||
\section1 Enabling the Squish Plugin
|
||||
|
||||
To enable the Squish plugin:
|
||||
|
||||
\list 1
|
||||
\li Select \uicontrol Help > \uicontrol {About Plugins} >
|
||||
\uicontrol Utilities > \uicontrol Squish to enable the plugin.
|
||||
\li Select \uicontrol {Restart Now} to restart \QC and load the plugin.
|
||||
\endlist
|
||||
|
||||
\section1 Specifying a Squish Server
|
||||
|
||||
To specify a Squish Server to run:
|
||||
|
||||
\list 1
|
||||
\li Select \uicontrol Edit > \uicontrol Preferences > \uicontrol Squish.
|
||||
\image qtcreator-squish-preferences.png "Squish general preferences"
|
||||
\li In the \uicontrol {Squish path} field, specify the path to the Squish
|
||||
installation directory.
|
||||
\li In the \uicontrol {License path} field, specify the path to your
|
||||
Squish license file if it is not located in your home folder. For
|
||||
example, if you have a global installation with several users, where
|
||||
the license key is installed in the global folder.
|
||||
\li Select the \uicontrol {Local server} check box to use a locally
|
||||
installed \c {squishserver.exe}. To use a server running in another
|
||||
machine, deselect the check box and specify the server address in the
|
||||
\uicontrol {Server host} field and the port number in the
|
||||
\uicontrol {Server port} field. If no port is specified, \QC starts
|
||||
\c squishserver in a way that enables it to automatically select an
|
||||
open port.
|
||||
\li Select the \uicontrol {Verbose log} check box to include additional
|
||||
logging levels in the log output.
|
||||
\li Select the \uicontrol {Minimize IDE} check box to automatically
|
||||
minimize \QC when running or recording test cases.
|
||||
\endlist
|
||||
|
||||
\section1 Specifying AUTs
|
||||
|
||||
To specify applications to test using Squish, select \uicontrol {Tools} >
|
||||
\uicontrol {Squish} > \uicontrol {Server Settings}.
|
||||
|
||||
\image qtcreator-squish-server-settings.png "Squish Server Settings"
|
||||
|
||||
\section2 Mapping AUTs
|
||||
|
||||
To specify the path to an AUT to run, select \uicontrol {Mapped AUTs} >
|
||||
\uicontrol Add and locate the AUT.
|
||||
|
||||
The Squish server checks whether the name of the AUT to run is mapped to a
|
||||
path and starts the AUT using the mapped path. This way, it does not need
|
||||
to search from all the specified AUT paths.
|
||||
|
||||
Mapping AUTs prevents the server from accidentally executing the wrong AUT
|
||||
if two different executables have the same name, as the server executes the
|
||||
first matching AUT it finds in the \uicontrol {AUT Paths} list.
|
||||
|
||||
To change the path to the selected AUT, select \uicontrol {Edit}.
|
||||
|
||||
To remove the mapping to the selected AUT, select \uicontrol {Remove}.
|
||||
|
||||
\section2 Specifying AUT Paths
|
||||
|
||||
To specify a path to search AUTs from, select \uicontrol {AUT Paths} >
|
||||
\uicontrol Add.
|
||||
|
||||
The Squish Server searches for the executable to run from the specified
|
||||
AUT paths and runs the first one with the specified name that it finds in
|
||||
any path.
|
||||
|
||||
To change the selected AUT path, select \uicontrol {Edit}.
|
||||
|
||||
To remove the selected AUT path, select \uicontrol {Remove}.
|
||||
|
||||
\section2 Adding Attachable AUTs
|
||||
|
||||
To specify the path to a running AUT, select \uicontrol {Attachable AUTs} >
|
||||
\uicontrol Add. In the \uicontrol {Add Attachable AUT} dialog, specify a
|
||||
connection to a running application to register an attachable AUT.
|
||||
|
||||
\image qtcreator-squish-server-settings-add-attachable-aut.png "Add Attachable AUT dialog"
|
||||
|
||||
To change the connection to the selected AUT, select \uicontrol {Edit}.
|
||||
|
||||
To remove the connection to the selected AUT, select \uicontrol {Remove}.
|
||||
|
||||
\section1 Managing Test Suites and Cases
|
||||
|
||||
You can manage Squish test suites and cases in the \uicontrol Squish
|
||||
\l {Working with Sidebars}{view}.
|
||||
|
||||
\image qtcreator-squish-view.png "Squish sidebar view"
|
||||
|
||||
To show existing test suites in \uicontrol {Test Suites}, select
|
||||
\uicontrol {Open Squish Suites} in the context menu.
|
||||
|
||||
\image qtcreator-squish-view-open-squish-suites.png "Open Squish Test Suites dialog"
|
||||
|
||||
You can open the \uicontrol {Squish Test Suite} wizard for creating a new
|
||||
test suite by selecting \uicontrol {Create New Test Suite} in the context
|
||||
menu.
|
||||
|
||||
To add a test case to a test suite, select it and then select
|
||||
\uicontrol {Add New Test Case} in the context menu.
|
||||
|
||||
To close all test suites, select \uicontrol {Close All Test Suites} in the
|
||||
context menu.
|
||||
|
||||
To add a shared folder to \uicontrol {Shared Folders}, select
|
||||
\uicontrol {Add Shared Folder} in the context menu. To remove all
|
||||
shared folders, select \uicontrol {Remove All Shared Folders}.
|
||||
|
||||
Double-click a test suite in \uicontrol {Test Suites} to open the
|
||||
\c {suite.conf} configuration file for editing.
|
||||
|
||||
\section2 Creating Test Suites
|
||||
|
||||
To create a new test suite:
|
||||
|
||||
\list 1
|
||||
\li Select \uicontrol File > \uicontrol {New Project}
|
||||
> \uicontrol {Squish} > \uicontrol {Squish Test Suite} >
|
||||
\uicontrol Choose.
|
||||
\image qtcreator-squish-create-test-suite.png "Create Squish Test Suite wizard"
|
||||
\li On the \uicontrol {Location} page, in \uicontrol {Test Suite Name},
|
||||
enter a name for the test suite.
|
||||
\image qtcreator-squish-create-test-suite-location.png "Location page"
|
||||
\li In \uicontrol {Test Suite folder's parent folder}, enter the path to
|
||||
the folder to create the test suite folder, and then select
|
||||
\uicontrol Next.
|
||||
\li On the \uicontrol Setup page, select the GUI toolkit used by the AUT,
|
||||
and then select \uicontrol Next.
|
||||
\image qtcreator-squish-create-test-suite-setup.png "Setup page"
|
||||
Currently, only desktop GUI toolkits are supported.
|
||||
\li On the \uicontrol {Script Language} page, select the scripting
|
||||
language to use for the test suite's test script, and then select
|
||||
\uicontrol Next.
|
||||
\image qtcreator-squish-create-test-suite-language.png "Languages page"
|
||||
\li On the \uicontrol {AUT} page, select the AUT to test, and then select
|
||||
\uicontrol Next.
|
||||
\image qtcreator-squish-create-test-suite-aut.png "AUT page"
|
||||
\li On the \uicontrol {Summary} page review the test suite settings, and
|
||||
then select \uicontrol Finish to create the test suite.
|
||||
\endlist
|
||||
|
||||
The test suite is listed in \uicontrol {Test Suites} in the \uicontrol Squish
|
||||
sidebar view.
|
||||
|
||||
\section2 Recording Test Cases
|
||||
|
||||
Squish records tests using the scripting language that you specified for the
|
||||
test suite. Recordings are made into existing test cases.
|
||||
|
||||
In \uicontrol {Test Suites}, select the \inlineimage icons/recordfill.png
|
||||
(\uicontrol {Record Test Case}) button next to the test case name. The AUT
|
||||
that you selected for the test suite is displayed and you can start recording
|
||||
the test case.
|
||||
|
||||
\image qtcreator-squish-control-bar-record-test-case.png "Squish control bar for recording test cases"
|
||||
|
||||
When you are done, select the \inlineimage icons/stop_small.png
|
||||
(\uicontrol {Stop}) button in the \uicontrol {Control Bar}.
|
||||
|
||||
You can edit recorded test scripts or copy parts of them into manually
|
||||
created test scripts.
|
||||
|
||||
\image qtcreator-squish-test-script-edit.png "A test script open in the editor"
|
||||
|
||||
\section2 Running Test Suites
|
||||
|
||||
You can run a recorded test case to have Squish repeat all the actions that
|
||||
you applied when recording the test, but without the pauses that humans are
|
||||
prone to but which computers don't need.
|
||||
To run a test case, select the \inlineimage icons/run_small.png
|
||||
(\uicontrol {Run}) button next to the test case in \uicontrol {Test Suites}.
|
||||
|
||||
\image qtcreator-squish-control-bar-run-test-case.png "Squish control bar for running test cases"
|
||||
|
||||
While the test is running, you can view test results as well as interrupt and
|
||||
stop tests in the \uicontrol {Control Bar}.
|
||||
|
||||
\section2 Mapping Symbolic Names
|
||||
|
||||
When Squish records a test, it uses \e {symbolic names} to identify the UI
|
||||
elements. Symbolic names are stored in an object map that can be either
|
||||
text-based or script-based. Text-based symbolic names are plain strings
|
||||
starting with a colon (:), whereas script-based symbolic names are script
|
||||
variables.
|
||||
|
||||
Squish generates symbolic names programmatically, but you can use them in
|
||||
hand-written code, or when you edit test cases or use extracts from recorded
|
||||
test cases.
|
||||
|
||||
Symbolic names have one major advantage over real names: if a property
|
||||
that a real name depends on changes in the AUT, the real name becomes
|
||||
invalid, and you must update all occurrences of it in test scripts.
|
||||
When using symbolic names, you only need to update the real name in the
|
||||
object map. You do not need to make any changes to the tests.
|
||||
|
||||
To edit the object map of a test suite, select the
|
||||
\inlineimage icons/objectsmap.png
|
||||
button next to the test suite in \uicontrol {Test Suites}.
|
||||
|
||||
\image qtcreator-squish-symbolic-names.png "Symbolic Names view"
|
||||
|
||||
You can filter the symbolic names in the \uicontrol {Symbolic Names} view.
|
||||
To edit a symbolic name or the names or values of its properties,
|
||||
double-click the name or value in the view and enter a new one.
|
||||
|
||||
To add a new symbolic name, select \uicontrol New. Double-click the
|
||||
placeholder for the name and enter a new name. Then select \uicontrol New
|
||||
next to \uicontrol Properties to enter properties for the symbolic name.
|
||||
|
||||
To remove the selected symbolic name or property, select \uicontrol Remove.
|
||||
|
||||
To jump to the symbolic name associated to the selected
|
||||
property, select \inlineimage icons/jumpto.png
|
||||
.
|
||||
|
||||
\section2 Inspecting Local Variables
|
||||
|
||||
If you set breakpoints in the test code before running the test, the test
|
||||
execution is automatically interrupted when a breakpoint is hit. You can
|
||||
inspect the contents of local variables in the \uicontrol {Squish Locals}
|
||||
view.
|
||||
|
||||
\image qtcreator-squish-locals.png "Squish Locals view"
|
||||
|
||||
Use the \uicontrol {Step Into}, \uicontrol {Step Over}, and
|
||||
\uicontrol {Step Out} buttons in the \uicontrol Squish debugging
|
||||
view to step through the code.
|
||||
|
||||
\image qtcreator-squish-debugging-view.png "Squish debugging view"
|
||||
|
||||
\section2 Specifying Settings for Running Tests
|
||||
|
||||
To specify settings for running tests, select \uicontrol Tools >
|
||||
\uicontrol Squish > \uicontrol {Server Settings}:
|
||||
|
||||
\list
|
||||
\li In the \uicontrol {Maximum startup time} field, set the maximum time
|
||||
to wait for the AUT to start before throwing an error.
|
||||
\li In the \uicontrol {Maximum response time} field, set the maximum time
|
||||
to wait for the AUT to respond before throwing an error.
|
||||
\li In the \uicontrol {Maximum post-mortem wait time} field, set the
|
||||
maximum time to wait after the main AUT has exited. This is useful
|
||||
for AUTs invoked through launcher applications, such as shell scripts
|
||||
or batch files.
|
||||
\li Select the \uicontrol {Animate mouse cursor} check box to to animate
|
||||
the mouse cursor when playing back a test.
|
||||
\endlist
|
||||
|
||||
\section1 Viewing Test Results
|
||||
|
||||
Squish uses compare, verify, and exception functions to record the results of
|
||||
tests applied to a running AUT in the test log as \e {passes} or \e {fails}.
|
||||
In addition, any kind of test results can be recorded in the test log.
|
||||
|
||||
You can view the test log in the \uicontrol Squish output,
|
||||
\uicontrol {Test Results} tab.
|
||||
|
||||
\image qtcreator-squish-output-test-results.png "Test Results output"
|
||||
|
||||
The \uicontrol Result column displays the time when each test run started
|
||||
and finished, log information and warnings, and test result status:
|
||||
|
||||
\list
|
||||
\li \uicontrol Pass - The test passed.
|
||||
\li \uicontrol Fail - The test failed.
|
||||
\li \uicontrol ExpectedFail - The test failed, as expected. For example,
|
||||
a known bug in the AUT caused a comparison or verification to fail.
|
||||
\li \uicontrol UnexpectedPass - The test passed unexpectedly. For
|
||||
example, a comparison or verification succeeded even though it was
|
||||
expected to fail.
|
||||
\endlist
|
||||
|
||||
The \uicontrol Message column displays log messages and information about
|
||||
the type of the operation that was performed: comparison, verification, or
|
||||
exception.
|
||||
|
||||
\section1 Viewing Squish Runner and Server Logs
|
||||
|
||||
You can view the Squish Runner and Server logs in the \uicontrol Squish
|
||||
output, \uicontrol {Runner/Server Log} tab.
|
||||
|
||||
\image qtcreator-squish-output-runner-server-log.png "Runner and Server Log output"
|
||||
*/
|
||||
@@ -8,7 +8,7 @@
|
||||
// **********************************************************************
|
||||
|
||||
/*!
|
||||
\previouspage creator-autotest.html
|
||||
\previouspage creator-squish.html
|
||||
\page creator-advanced.html
|
||||
\nextpage creator-os-supported-platforms.html
|
||||
|
||||
|
||||
@@ -37,9 +37,13 @@
|
||||
|
||||
\li \l{Running Autotests}
|
||||
|
||||
You can create, build and run Qt tests, Qt Quick tests, Google
|
||||
Create, build and run Qt tests, Qt Quick tests, Google
|
||||
tests, and Boost tests using \QC.
|
||||
|
||||
\li \l{Using Squish}
|
||||
|
||||
Map AUTs to \QC and run Squish test suites and cases from it.
|
||||
|
||||
\endlist
|
||||
|
||||
*/
|
||||
|
||||
@@ -208,6 +208,11 @@
|
||||
\li Imports an existing project that does not use any of the
|
||||
supported build systems: qmake, Qbs, CMake, or Autotools.
|
||||
This enables you to use \QC as a code editor.
|
||||
\row
|
||||
\li Squish
|
||||
\li Squish Test Suite
|
||||
\li Creates a new \l {Using Squish}{Squish test suite}.
|
||||
|
||||
\endtable
|
||||
|
||||
To create a new project, select \uicontrol File > \uicontrol{New Project} and
|
||||
|
||||
@@ -187,6 +187,7 @@
|
||||
\li \l{Visualizing Chrome Trace Events}
|
||||
\endlist
|
||||
\li \l{Running Autotests}
|
||||
\li \l{Using Squish}
|
||||
\endlist
|
||||
\li \l{Advanced Use}
|
||||
\list
|
||||
|
||||
@@ -87,6 +87,7 @@
|
||||
\li \l{Debugging}
|
||||
\li \l{Analyzing Code}
|
||||
\li \l{Running Autotests}
|
||||
\li \l{Using Squish}
|
||||
\endlist
|
||||
\li \b {\l{Advanced Use}}
|
||||
\list
|
||||
|
||||
@@ -379,9 +379,13 @@
|
||||
\li \uicontrol{Version Control}
|
||||
|
||||
\if defined(qtcreator)
|
||||
\li \uicontrol{Test Results}
|
||||
\li \l{Running Autotests}{Test Results}
|
||||
|
||||
\li \l{Using Squish}{Squish} test results and Squish Server and
|
||||
Runner logs
|
||||
|
||||
\li \uicontrol {To-Do Entries}
|
||||
|
||||
\endif
|
||||
|
||||
\endlist
|
||||
|
||||