Compare commits

...
18 Commits
Author SHA1 Message Date
Martin Hořeňovský 317ac1ed4c v3.16.0 2026-08-25 09:32:22 +02:00
Martin Hořeňovský fc5bc1fa8b Improve String matcher handling of case sensitivity
The old implementation of String matchers worked by making a, potentially
lower-cased, copy of both the constructor arg, and the arg to `match` calls.

The new implementation does not make copies, and instead does per-character
lowering if needed. This is a win in **most** cases, with two exceptions:

1) Case-insensitive `Equals` between long strings. The old implementation
   is easy to autovectorize by the compiler, which wins over the extra copy
   with sufficiently long inputs.
2) Case-insensitive `ContainsSubstring` with inputs that trigger the worst
   case complexity of O(m*n) of the naive implementation.

The latter is fixable with some elbow grease by implementing a proper
string search algorithm, but the former is a cost we have to live with.
2026-08-24 15:11:48 +02:00
Martin Hořeňovský 499442d7ef Further reduce stamped out types for each templated test case
This provides further speed-ups for TUs with many test cases templated
over the same set of types. For especially test-heavy (but still
realistic) TUs, the speed up is up to 20%.
2026-08-23 20:46:39 +02:00
Martin Hořeňovský 9a2b2ee8d6 Avoid stamping out common type helpers in each templated test cases
This provides small (but measurable) improvements in compilation
times when a TU has multiple test cases templated over the same types.
2026-08-23 11:31:50 +02:00
Martin Hořeňovský ef3a728265 Don't store extra copy of all TestCaseInfo* in TestRegistry
We used to store a full vector of "views" of the registered test infos,
so that it can be retrieved externally. However, the only user of that
interface left was `Session`, specifically `Session::applyFilenameAsTags`.

By moving the adding of the filename tags into the `ITestCaseRegistry`
interface, we avoid storing the vector and make the initial test registration
faster.

The new code is ugly and shows a design/layering issue, but no worse than
the previous approach did.
2026-08-22 23:28:11 +02:00
Martin Hořeňovský 69da66ee73 Don't instantiate the default implementation of benchmarks in every TU
The underlying benchmarking machinery is templated, so that it can use
different clock implementations (either for testing, or to support user
with platform-specific clocks). However, 99.99% of all users interact
with it using the `BENCHMARK` macro, which means that they use it with
`std::chrono::steady_clock`.

By adding outlined implementation of benchmarking helpers specialized
onto `std::chrono::steady_clock`, we save some amount of time per every
TU that uses benchmarks.
2026-08-22 23:01:51 +02:00
Martin Hořeňovský 45f9ea1e79 Add optimizer barrier to benchmark calls of void-returning functions
The goal is to do the equivalent of clobbering memory, which forces
the compiler to keep side effects that are external to the benchmarked
function, while allowing it to optimize inside the benchmarked function.

E.g. this cannot be optimized away:
```cpp
BENCHMARK("foo") {
    global_count += 1;
};
```

while this can:
```cpp
BENCHMARK("bar") {
    size_t local_count = 0;
    for (size_t i = 0; i < 100; ++i) {
        local_count += 1;
    }
};
```
2026-08-22 20:43:13 +02:00
Martin Hořeňovský 9915f7250d Preallocate test case vectors in TestRegistry
Preallocating to some reasonable and small number of tests avoids
much of the geometric-reallocation threashing at low sizes, without
taking up too much memory for tiny test binaries.
2026-08-22 20:42:23 +02:00
Martin Hořeňovský f9dfb10315 Tiny formatting fix in catch_template_test_registry.hpp 2026-08-22 20:41:59 +02:00
Matt Van HornandMatt Van Horn fdfc07e571 fix: work around clang 20/21 + libc++ compile failure in TEMPLATE_PRODUCT_TEST_CASE with differing arities (#3173)
* fix: work around clang 20/21 + libc++ compile failure in TEMPLATE_PRODUCT_TEST_CASE with differing arities

Fixes #3115

* docs: reference llvm/llvm-project#130778 in the clang 20/21 workaround comment

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-08-20 17:39:01 +02:00
Martin Hořeňovský 8582805f54 Support TEST_{PREFIX,SUFFIX} with leading/trailing whitespace
Closes #2149
2026-08-20 16:24:18 +02:00
Matt Rasa 0aeb818520 Emit warning when using --shard-count (> 1) with --order rand 2026-08-11 00:08:23 +02:00
Martin Hořeňovský 97ec4e8e2e catch_discover_tests: Escape test-invariant parts of CTest script only once
Previously, the `catch_discover_tests` would prepare the entire CTest
command (e.g. `add_test(...)` or `set_tests_properties(...)`) first, and
then escape it when finished. However, this caused lot of the command args
to be escaped over and over again (e.g. executable name or Catch2's reporter
args), for no reason, as they were always the same, and thus their escaping
was always the same.

Until recently, the performance overhead didn't matter as there were many
spots which had quadratic runtime in number of tests. However, the recent
refactorings fixed these, and this commit now improves the throughput by
10-20%.

Also extended the benchmarked COUNTS in `benchmark_discovery.py`,
because the performance is now good enough that it is reasonable
to benchmark 16k tests.
2026-08-10 11:53:22 +02:00
Martin Hořeňovský 0136276e15 Avoid n**2 runtime when creating list of tests in catch_discover_tests
This was another place where the script triggered the quadratic runtime
from calling `string(APPEND` (or `list(APPEND`) repeatedly. As in
60c8b87, we avoid this by flushing the test list into a file every
50kB of text input.

As the string with the test list never grew quite as much as the
string that contains all the test definitions, this only provides
significant savings for high number of tests. It starts being
properly measurable around 4k tests at ~100ms, but grows to ~4s at
32k tests.
2026-08-10 10:36:04 +02:00
Martin Hořeňovský 630840c500 Fix <target>_TESTS variable generated by catch_discover_tests
There were 2 separate bugs, one recently added and one ancient
(since roughly the first version of the registration script).

1) The recent refactoring of how the JSON output from Catch2 is
   parsed caused shadowing between the variable containing the
   per-test JSON fragments and the accumulator of test names for
   <target>_TESTS variable. This led to the variable containing
   both the names and the JSON fragments, and thus being completely
   wrong.

2) The test name accumulation has never accounted for characters
   that need escaping to be present in a CMake list. This means
   that e.g. test names with semicolon in them would end up with
   two elements in the test list.

Both of these are now fixed, at the cost of extra complexity and
my sanity as I had to learn more about CMake escaping rules.

(CMake escaping rules are dumb)
2026-08-10 00:17:26 +02:00
Martin Hořeňovský 4cde128517 JSONReporter considers verbosity when listing tests
* Quiet verbosity provides just the test names
* Normal verbosity adds tags
* High verbosity add the source location of the tests

Listing tests with the quiet verbosity results in about 1/4 of the
previous output, which leads to measurably faster execution of
`catch_discover_tests` when it does not need tags for labels.
(If it does need tags, the output is about 1/2 of previous).
2026-08-08 00:05:18 +02:00
Martin Hořeňovský 0b4d7a5a16 JSONReporter's listTests only lists class-name if it is non-empty 2026-08-07 23:07:46 +02:00
Martin Hořeňovský a0eba50e9d Make verbosity per-reporter 2026-08-07 20:57:42 +02:00
57 changed files with 1536 additions and 665 deletions
+1 -1
View File
@@ -35,7 +35,7 @@ if(CMAKE_BINARY_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
endif()
project(Catch2
VERSION 3.15.3 # CML version placeholder, don't delete
VERSION 3.16.0 # CML version placeholder, don't delete
LANGUAGES CXX
HOMEPAGE_URL "https://github.com/catchorg/Catch2"
DESCRIPTION "A modern, C++-native, unit test framework."
@@ -8,6 +8,7 @@
# SPDX-License-Identifier: BSL-1.0
import argparse
import copy
import json
import os
import re
@@ -20,7 +21,7 @@ import time
HERE = os.path.dirname(os.path.abspath(__file__))
TEMPLATE = os.path.join(HERE, "listing_template.json")
SHIM = os.path.join(HERE, "copy_shim.cmake")
COUNTS = [1, 10, 500, 1000, 2000, 4000, 8000]
COUNTS = [1, 10, 500, 1000, 2000, 4000, 8000, 16000]
#COUNTS = [1, 10, 100]
REPEATS = 5
@@ -33,14 +34,19 @@ def load_template(path: str) -> dict:
return catch_out
def synthesize_listing(template_doc: dict, count: int, file) -> str:
def synthesize_listing(template_doc: dict, count: int, file, with_tags: bool) -> str:
"""Writes JSON listing with exactly `count` test cases into `file`.
The template entries are cycled through, and each name is made unique by
appending an index, so that every registered CTest test has a distinct name.
"""
doc_copy = template_doc.copy()
template_tests = template_doc['listings']['tests']
# To avoid messing up the template tests by deletion, we have
# to take a deepcopy before the changes.
doc_copy = copy.deepcopy(template_doc)
template_tests = doc_copy['listings']['tests']
if not with_tags:
for test in template_tests:
del test['tags']
num_original = len(template_tests)
out_tests = []
@@ -176,10 +182,10 @@ def main():
os.makedirs(workdir, exist_ok=True)
for count in COUNTS:
listing_path = os.path.join(workdir, f"listing-{count}.json")
with open(listing_path, "w", encoding="utf-8") as f:
synthesize_listing(template_doc, count, f)
for add_tags in tag_modes:
with open(listing_path, "w", encoding="utf-8") as f:
synthesize_listing(template_doc, count, f, add_tags)
mode = "on" if add_tags else "off"
ctest_file = os.path.join(workdir, f"ctest-{count}-{mode}.cmake")
cmd = build_command(args, listing_path, ctest_file, add_tags)
+11 -60
View File
@@ -1,5 +1,5 @@
{
"version": 1,
"version": 2,
"metadata": {
"name": "benchmark-template",
"catch2-version": "3.15.2"
@@ -8,66 +8,41 @@
"tests": [
{
"name": "Comparing function pointers",
"class-name": "",
"tags": [
"function pointer",
"Tricky"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/UsageTests/Tricky.tests.cpp",
"line": 266
}
]
},
{
"name": "Testing checked-if 4",
"class-name": "",
"tags": [
"!shouldfail",
"checked-if"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/UsageTests/Misc.tests.cpp",
"line": 216
}
]
},
{
"name": "count_equidistant_floats - double",
"class-name": "",
"tags": [
"approvals",
"distance",
"floating-point"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/IntrospectiveTests/FloatingPoint.tests.cpp",
"line": 103
}
]
},
{
"name": "Usage of AllTrue range matcher",
"class-name": "",
"tags": [
"matchers",
"quantifiers",
"templated"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/UsageTests/MatchersRanges.tests.cpp",
"line": 372
}
]
},
{
"name": "Exception matchers that succeed",
"class-name": "",
"tags": [
"!throws",
"exceptions",
"matchers"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/UsageTests/Matchers.tests.cpp",
"line": 428
}
]
},
{
"name": "A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float",
@@ -75,59 +50,35 @@
"tags": [
"class",
"template"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/UsageTests/Class.tests.cpp",
"line": 82
}
]
},
{
"name": "Approximate PI",
"class-name": "",
"tags": [
"Approx",
"PI"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/UsageTests/Approx.tests.cpp",
"line": 134
}
]
},
{
"name": "TextFlow::Column respects width setting",
"class-name": "",
"tags": [
"approvals",
"column",
"TextFlow"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/IntrospectiveTests/TextFlow.tests.cpp",
"line": 38
}
]
},
{
"name": "Generators internals",
"class-name": "",
"tags": [
"generators",
"internals"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/IntrospectiveTests/GeneratorsImpl.tests.cpp",
"line": 24
}
]
},
{
"name": "Mayfail test case with nested sections",
"class-name": "",
"tags": [
"!mayfail"
],
"source-location": {
"filename": "/mnt/c/ubuntu/Catch2/tests/SelfTest/UsageTests/Condition.tests.cpp",
"line": 83
}
]
}
]
}
+7 -2
View File
@@ -202,8 +202,13 @@ as many times as you want, e.g. `--reporter xml::out=someFile.xml` or
The keys must either be prefixed by "X", in which case they are not parsed
by Catch2 and are only passed down to the reporter, or one of options
hardcoded into Catch2. Currently there are only 2,
["out"](#sending-output-to-a-file), and ["colour-mode"](#colour-mode).
hardcoded into Catch2. Currently there are 3 supported options:
* ["out"](#sending-output-to-a-file)
* ["colour-mode"](#colour-mode)
* ["verbosity"](#output-verbosity)
> Support for per-reporter verbosity option was added in Catch2 3.16.0
_Note that the reporter might still check the X-prefixed options for
validity, and throw an error if they are wrong._
+30
View File
@@ -2,6 +2,7 @@
# Release notes
**Contents**<br>
[3.16.0](#3160)<br>
[3.15.3](#3153)<br>
[3.15.2](#3152)<br>
[3.15.1](#3151)<br>
@@ -79,6 +80,35 @@
## 3.16.0
### Fixes
* Multiple fixes in `catch_discover_tests`:
* Fixed `<target>_TESTS` variable accumulating JSON fragments alongside test names.
* This was introduced during the refactoring in last release.
* Fixed `<target>_TESTS` variable from `catch_discover_tests` not escaping test names to be properly parsed by CMake.
* This means that e.g. test names with semicolons will not be split into multiple partial test names.
* This bug has existed since the first version of the script.
* Fixed `TEST_PREFIX`/`TEST_SUFFIX` args having leading/trailing whitespace stripped.
* Added workaround for Clang 20-21 compile error with `TEMPLATE_PRODUCT_TEST_CASE` (#3115, #3173)
### Improvements
* Verbosity option is now handled per reporter.
* The standalone `--verbosity` flag is propagated to all reporters as default, just like `--colour-mode`.
* The JSON reporter considers verbosity when listing tests.
* Another set of performance improvements for `catch_discover_tests` performance
* The newest version can register about 3k tests in 1 second, up from 1k previously.
* The initial `TEST_CASE` registration is slightly faster.
* Reduced overhead from first instantiating `BENCHMARK` machinery in a TU.
* Improved compilation speed when multiple templated test case macros use the same types.
* Rewrote implementation of the string matchers
* Case-sensitive matching (the default) is significantly faster.
* Case-insensitive matching is faster in most cases.
* Added optimizer barrier to calls into benchmarks without return values
* This limits the optimizations compiler can perform **between calls** into the benchmarked function, improving the accuracy.
* Added warning that checks for using sharding without deterministic test order (#3186)
## 3.15.3
### Fixes
+4 -4
View File
@@ -43,13 +43,13 @@ them write into different destinations. The two main uses of this are
Specifying multiple reporter looks like this:
```
--reporter JUnit::out=result-junit.xml --reporter console::out=-::colour-mode=ansi
--reporter JUnit::out=result-junit.xml --reporter console::out=-::colour-mode=ansi::verbosity=quiet
```
This tells Catch2 to use two reporters, `JUnit` reporter that writes
its machine-readable XML output to file `result-junit.xml`, and the
`console` reporter that writes its user-friendly output to stdout and
uses ANSI colour codes for colouring the output.
`console` reporter that writes its user-friendly output to stdout, uses
ANSI colour codes for colouring the output and is set to "quiet" verbosity.
Using multiple reporters (or one reporter and one-or-more [event
listeners](event-listeners.md#top)) can have surprisingly complex semantics
@@ -110,7 +110,7 @@ passing and failing assertions.
_Generally we recommend that if you override a member function from either
of the bases, you call into the base's implementation first. This is not
necessarily in all cases, but it is safer and easier._
necessary in all cases, but it is safer and easier._
Writing your own reporter then looks like this:
+9 -4
View File
@@ -58,6 +58,11 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
directory property. The set of discovered tests is made accessible to such a
script via the ``<target>_TESTS`` variable.
Note that ``<target>_TESTS`` variable contains test names with brackets
("[", "]") escaped into ASCII char 2, 3 respectively, to work around CMake's
list parsing rules. You have to unescape them back for each element to get
the original names.
The options are:
``target``
@@ -226,8 +231,8 @@ function(catch_discover_tests TARGET)
-D "TEST_SPEC=${_TEST_SPEC}"
-D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
-D "TEST_PROPERTIES=${_PROPERTIES}"
-D "TEST_PREFIX=${_TEST_PREFIX}"
-D "TEST_SUFFIX=${_TEST_SUFFIX}"
-D "TEST_PREFIX='${_TEST_PREFIX}'"
-D "TEST_SUFFIX='${_TEST_SUFFIX}'"
-D "TEST_LIST=${_TEST_LIST}"
-D "TEST_REPORTER=${_REPORTER}"
-D "TEST_OUTPUT_DIR=${_OUTPUT_DIR}"
@@ -272,8 +277,8 @@ function(catch_discover_tests TARGET)
" TEST_SPEC" " [==[" "${_TEST_SPEC}" "]==]" "\n"
" TEST_EXTRA_ARGS" " [==[" "${_EXTRA_ARGS}" "]==]" "\n"
" TEST_PROPERTIES" " [==[" "${_PROPERTIES}" "]==]" "\n"
" TEST_PREFIX" " [==[" "${_TEST_PREFIX}" "]==]" "\n"
" TEST_SUFFIX" " [==[" "${_TEST_SUFFIX}" "]==]" "\n"
" TEST_PREFIX" " [==['" "${_TEST_PREFIX}" "']==]" "\n"
" TEST_SUFFIX" " [==['" "${_TEST_SUFFIX}" "']==]" "\n"
" TEST_LIST" " [==[" "${_TEST_LIST}" "]==]" "\n"
" TEST_REPORTER" " [==[" "${_REPORTER}" "]==]" "\n"
" TEST_OUTPUT_DIR" " [==[" "${_OUTPUT_DIR}" "]==]" "\n"
+154 -68
View File
@@ -12,14 +12,28 @@
# exist in JSON unescaped.
#
# 0x01 <=> ';' (CMake list separator)
# 0x02 == element boundary marker used while splitting the tests array
# 0x03 <-> '[' (opens a CMake bracket-argument context)
# 0x04 <-> ']' (closes a CMake bracket-argument context)
# 0x02 <-> '[' (opens a CMake bracket-argument context)
# 0x03 <-> ']' (closes a CMake bracket-argument context)
# 0x04 == element boundary marker used while splitting the tests array
#
string(ASCII 1 _SemicolonEscape)
string(ASCII 2 _BoundaryEscape)
string(ASCII 3 _OpenBracketEscape)
string(ASCII 4 _CloseBracketEscape)
string(ASCII 2 _OpenBracketEscape)
string(ASCII 3 _CloseBracketEscape)
string(ASCII 4 _BoundaryEscape)
# As far as I can tell, the only way of having unclosed '[' or ']' in the
# list of test names we define for CTest, is to escape them into completely
# different characters. Otherwise, they will break parsing of unrelated
# semicolons (inserted by CMake after parsing the list from the user-facing
# space separated format). This means that the consumers of the list have
# to unescape them back, but that's CMake :v
#
# 0x02 <-> '['
# 0x03 <-> ']'
#
string(ASCII 2 _OpenBracketListingEscape)
string(ASCII 3 _CloseBracketListingEscape)
# Placeholder bytes in the listing would break our parsing hack, so
@@ -126,35 +140,28 @@ function(split_json_array json_array_var out_var)
set(${out_var} "${array_elements}" PARENT_SCOPE)
endfunction()
# TBD: Further possible optimization is that most arguments for per-test
# `prepare_command` call are constant across one invocation of
# `catch_discover_tests`, and thus need checking and escaping only
# once, instead of for each test.
# This would provide nice speed-up of the actual command preparation,
# but it will make the script much harder to read, and it is utterly
# dwarfed by the quadratic scaling of parsing JSON arrays in CMake.
# Prepare command with escaped (bracketed) arguments and return it via `_Command` out variable.
# Prepare (a part of) command with bracketed arguments and return it via `out_var`.
#
# To avoid quadratic performance when concatenating all commands together,
# the actual concatenation must be done by the caller, by appending it
# into a string of all other commands.
function(prepare_command NAME)
# This allows the registration script to escape parts of the test script
# only once, instead of escaping the unchanged arguments over and over again.
function(prepare_command_fragment out_var)
set(_args "")
# use ARGV* instead of ARGN, because ARGN splits arrays into multiple arguments
math(EXPR _last_arg ${ARGC}-1)
foreach(_n RANGE 1 ${_last_arg})
set(_arg "${ARGV${_n}}")
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]") # form a bracket_argument
else()
set(_args "${_args} ${_arg}")
endif()
endforeach()
set(_Command "${NAME}(${_args})\n" PARENT_SCOPE)
if(_last_arg GREATER_EQUAL 1)
foreach(_n RANGE 1 ${_last_arg})
set(_arg "${ARGV${_n}}")
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]") # form a bracket_argument
else()
set(_args "${_args} ${_arg}")
endif()
endforeach()
endif()
set(${out_var} "${_args}" PARENT_SCOPE)
endfunction()
# Generates random filename in the temp folder.
# Temp folder is retrieved by checking env vars from various platforms.
function(make_temp_file_path OUT_VARIABLE FALLBACK_PATH)
@@ -190,6 +197,23 @@ function(make_temp_file_path OUT_VARIABLE FALLBACK_PATH)
set(${OUT_VARIABLE} "${FINAL_TEMP_PATH}" PARENT_SCOPE)
endfunction()
# Computes the path of the test-list file based on the path for the main
# CTest script file (passed in `CTEST_FILE`).
#
# It works by replacing the `_tests` part of the original name with
# `_test-list`, or failing that, it appends `-list.cmake` instead.
#
# <base>_tests.cmake -> <base>_test-list.cmake
# <base>_tests-Debug.cmake -> <base>_test-list-Debug.cmake
function(make_test_list_file_path CTEST_FILE OUT_VARIABLE)
if("${CTEST_FILE}" MATCHES "_tests(.*)\\.cmake$")
string(REGEX REPLACE "_tests(.*)\\.cmake$" "_test-list\\1.cmake" list_file "${CTEST_FILE}")
else()
set(list_file "${CTEST_FILE}-list.cmake")
endif()
set(${OUT_VARIABLE} "${list_file}" PARENT_SCOPE)
endfunction()
function(catch_discover_tests_impl)
cmake_parse_arguments(
""
@@ -199,15 +223,22 @@ function(catch_discover_tests_impl)
${ARGN}
)
# We periodically append to the output file below, so we have to ensure
# We write the list of all discovered tests into a separate file
make_test_list_file_path("${_CTEST_FILE}" _CTEST_LIST_FILE)
# We periodically append to the output files below, so we have to ensure
# that it is empty at the start, or we get duplicated test scripts.
file(REMOVE "${_CTEST_FILE}")
file(REMOVE "${_CTEST_LIST_FILE}")
# Size (in Bytes) at which the intermediate `script` var is dumped to file.
set(_WriteToFileThreshold 50000)
set(add_tags "${_ADD_TAGS_AS_LABELS}")
set(prefix "${_TEST_PREFIX}")
set(suffix "${_TEST_SUFFIX}")
# TEST_{PREFIX,SUFFIX} is enclosed in single quotes to keep ensure
# leading/trailing whitespace isn't trimmed by CMake's argument passing.
string(REGEX REPLACE "^'(.*)'$" "\\1" prefix "${_TEST_PREFIX}")
string(REGEX REPLACE "^'(.*)'$" "\\1" suffix "${_TEST_SUFFIX}")
set(spec ${_TEST_SPEC})
set(extra_args ${_TEST_EXTRA_ARGS})
set(properties ${_TEST_PROPERTIES})
@@ -221,6 +252,10 @@ function(catch_discover_tests_impl)
set(script)
set(suite)
set(tests)
# Holds the list of all registered test names **as a string**, not list.
# This avoids issue of CMake removing semicolon escapes even inside
# bracket-quoted strings.
set(_test_names)
if(WIN32)
set(dl_paths_variable_name PATH)
@@ -253,10 +288,15 @@ function(catch_discover_tests_impl)
make_temp_file_path(listing_output_path "${_TEST_WORKING_DIR}")
set(_Verbosity "quiet")
if (add_tags)
set(_Verbosity "normal")
endif()
execute_process(
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" ${spec}
--list-tests
--reporter json
--reporter "json::verbosity=${_Verbosity}"
--out "${listing_output_path}"
--order lex # Make sure the output order, and thus test registration order, is consistent across runs.
OUTPUT_VARIABLE listing_output
@@ -326,7 +366,7 @@ function(catch_discover_tests_impl)
# Parse JSON output for list of tests/class names/tags
string(JSON version GET "${listing_output}" "version")
if(NOT version STREQUAL "1")
if(NOT version STREQUAL "2")
message(FATAL_ERROR "Unsupported catch output version: '${version}'")
endif()
@@ -337,10 +377,54 @@ function(catch_discover_tests_impl)
# Exit early if no tests are detected
if(NOT tests)
file(WRITE "${_CTEST_FILE}" "")
# Still emit an (empty) test list file and have the main script include
# it, so that consumers relying on the `${_TEST_LIST}` variable and on the
# include structure get consistent behavior regardless of test count.
file(WRITE "${_CTEST_LIST_FILE}" "set(${_TEST_LIST})\n")
file(WRITE "${_CTEST_FILE}" "include(\"${_CTEST_LIST_FILE}\")\n")
return()
endif()
# The 'set(VAR` header for the test list has to be written separately,
# so that each test name can be appended file without further processing.
set(test_names "set(${_TEST_LIST}")
# Most of the commands/arguments in the CTest script are identical
# for every test registered with one `catch_discover_tests` call.
# To avoid repeating the work in escaping them, we escape them before
# the per-test loop and reuse the escaped fragments.
#
# `add_test` calls are
# add_test(<name><exec><exe><escaped_name><extra_args><reporter><out_dir>)
# Of these, <exec>,<exe>,<extra_args>, and <reporter> are the same between
# all tests.
prepare_command_fragment(_exec_exe_fragment
${_TEST_EXECUTOR}
"${_TEST_EXECUTABLE}"
)
prepare_command_fragment(_args_reporter_fragment
${extra_args}
"${reporter_arg}"
)
# `set_tests_properties` calls are
# set_tests_properties(<name> PROPERTIES WORKING_DIRECTORY <dir> <properties>)
# set_tests_properties(<name> PROPERTIES ENVIRONMENT_MODIFICATION <env_mod>)
# Of these, only the <name> changes between tests.
prepare_command_fragment(_properties_fragment
PROPERTIES
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
${properties}
)
# Env modification is optional, so we prepare it in a separate command
if(environment_modifications)
prepare_command_fragment(_env_modification_fragment
PROPERTIES
ENVIRONMENT_MODIFICATION "${environment_modifications}"
)
endif()
# Each element in the tests is JSON-string representing one test object.
# We have to parse it and then turn it into CTest script commands.
foreach(single_test IN LISTS tests)
@@ -351,6 +435,11 @@ function(catch_discover_tests_impl)
file(APPEND "${_CTEST_FILE}" "${script}")
set(script "")
endif()
string(LENGTH "${test_names}" names_len)
if (names_len GREATER _WriteToFileThreshold)
file(APPEND "${_CTEST_LIST_FILE}" "${test_names}")
set(test_names "")
endif()
# The elements are still escaped and contain JSON-invalid characters,
# they have to be unescaped before parsing them as JSON.
@@ -372,24 +461,15 @@ function(catch_discover_tests_impl)
set(output_dir_arg "--out ${output_dir}/${output_prefix}${escaped_name_clean}${output_suffix}")
endif()
# ...and add to script
prepare_command(add_test
"${prefix}${plain_name}${suffix}"
${_TEST_EXECUTOR}
"${_TEST_EXECUTABLE}"
"${escaped_name}"
${extra_args}
"${reporter_arg}"
"${output_dir_arg}"
)
string(APPEND script "${_Command}")
prepare_command(set_tests_properties
"${prefix}${plain_name}${suffix}"
PROPERTIES
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
${properties}
)
string(APPEND script "${_Command}")
set(full_name "${prefix}${plain_name}${suffix}")
prepare_command_fragment(_full_name_fragment "${full_name}")
prepare_command_fragment(_escaped_name_fragment "${escaped_name}")
prepare_command_fragment(_outdir_fragment "${output_dir_arg}")
string(APPEND script
"add_test(${_full_name_fragment}${_exec_exe_fragment}${_escaped_name_fragment}${_args_reporter_fragment}${_outdir_fragment})\n")
string(APPEND script
"set_tests_properties(${_full_name_fragment}${_properties_fragment})\n")
if(add_tags)
string(JSON num_tags LENGTH "${test_tags}")
@@ -409,33 +489,39 @@ function(catch_discover_tests_impl)
list(APPEND tag_list "${a_tag}")
endforeach()
prepare_command(set_tests_properties
"${prefix}${plain_name}${suffix}"
prepare_command_fragment(_labels_fragment
PROPERTIES
LABELS "${tag_list}"
)
string(APPEND script "${_Command}")
string(APPEND script "set_tests_properties(${_full_name_fragment}${_labels_fragment})\n")
endif()
endif(add_tags)
if(environment_modifications)
prepare_command(set_tests_properties
"${prefix}${plain_name}${suffix}"
PROPERTIES
ENVIRONMENT_MODIFICATION "${environment_modifications}")
string(APPEND script "${_Command}")
string(APPEND script "set_tests_properties(${_full_name_fragment}${_env_modification_fragment})\n")
endif()
list(APPEND tests "${prefix}${plain_name}${suffix}")
# The test name has to be escaped using the same rules as prepare_command
# uses for the arguments, so that it keeps being single element in list
# even with weird characters and semicolons.
set(full_list_name "${prefix}${plain_name}${suffix}")
string(REPLACE ";" "\\;" full_list_name "${full_list_name}")
string(REPLACE "[" "${_OpenBracketListingEscape}" full_list_name "${full_list_name}")
string(REPLACE "]" "${_CloseBracketListingEscape}" full_list_name "${full_list_name}")
if(full_list_name MATCHES "[^-./:a-zA-Z0-9_]")
# The space before the start of quote is important, so that we get
# space-separated list in the final file.
string(APPEND test_names " [==[${full_list_name}]==]")
else()
string(APPEND test_names " ${full_list_name}")
endif()
endforeach()
# Create a list of all discovered tests, which users may use to e.g. set
# properties on the tests
prepare_command(set ${_TEST_LIST} ${tests})
string(APPEND script "${_Command}")
# Write any script leftovers we have
# Write any test names leftovers we have
file(APPEND "${_CTEST_LIST_FILE}" "${test_names})\n")
# Write any main script leftovers we have, and append the include of test names
file(APPEND "${_CTEST_FILE}" "${script}")
file(APPEND "${_CTEST_FILE}" "include(\"${_CTEST_LIST_FILE}\")\n")
endfunction()
# To enable `include`ing this file in the unit test scripts, we only run
+212 -60
View File
@@ -6,8 +6,8 @@
// SPDX-License-Identifier: BSL-1.0
// Catch v3.15.3
// Generated: 2026-07-26 22:17:52.418168
// Catch v3.16.0
// Generated: 2026-08-25 09:29:23.172704
// ----------------------------------------------------------
// This file is an amalgamation of multiple different files.
// You probably shouldn't edit it directly.
@@ -40,6 +40,54 @@
#include <algorithm>
#include <chrono>
#include <cmath>
namespace Catch {
namespace Benchmark {
namespace Detail {
Environment measure_environment_default() {
return Detail::measure_environment<default_clock>();
}
ExecutionPlan prepare_default( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun ) {
// This mirrors Benchmark::prepare<default_clock>(), but with the
// clock fixed so it is instantiated once here in the library.
auto min_time = env.clock_resolution.mean * Detail::minimum_ticks;
auto run_time = std::max(
min_time,
std::chrono::duration_cast<decltype( min_time )>(
cfg.benchmarkWarmupTime() ) );
auto&& test = Detail::run_for_at_least<default_clock>(
std::chrono::duration_cast<IDuration>( run_time ), 1, fun );
int new_iters = static_cast<int>(
std::ceil( min_time * test.iterations / test.elapsed ) );
return { new_iters,
test.elapsed / test.iterations * new_iters *
cfg.benchmarkSamples(),
CATCH_MOVE( fun ),
std::chrono::duration_cast<FDuration>(
cfg.benchmarkWarmupTime() ),
Detail::warmup_iterations };
}
std::vector<FDuration> run_plan_default( ExecutionPlan const& plan,
const IConfig& cfg,
Environment env ) {
return plan.run<default_clock>( cfg, env );
}
} // namespace Detail
} // namespace Benchmark
} // namespace Catch
namespace Catch {
namespace Benchmark {
namespace Detail {
@@ -795,6 +843,7 @@ namespace Catch {
return lhs.name == rhs.name &&
lhs.outputFilename == rhs.outputFilename &&
lhs.colourMode == rhs.colourMode &&
lhs.verbosity == rhs.verbosity &&
lhs.customOptions == rhs.customOptions;
}
@@ -863,6 +912,7 @@ namespace Catch {
reporterSpec.outputFile() ? *reporterSpec.outputFile()
: data.defaultOutputFilename,
reporterSpec.colourMode().valueOr( data.defaultColourMode ),
reporterSpec.verbosity().valueOr( data.verbosity ),
reporterSpec.customOptions() } );
}
}
@@ -913,6 +963,7 @@ namespace Catch {
double Config::minDuration() const { return m_data.minDuration; }
TestRunOrder Config::runOrder() const { return m_data.runOrder; }
uint32_t Config::rngSeed() const { return m_data.rngSeed; }
bool Config::rngSeedWasFixed() const { return m_data.rngSeedWasFixed; }
unsigned int Config::shardCount() const { return m_data.shardCount; }
unsigned int Config::shardIndex() const { return m_data.shardIndex; }
ColourMode Config::defaultColourMode() const { return m_data.defaultColourMode; }
@@ -938,7 +989,7 @@ namespace Catch {
if ( bazelOutputFile ) {
m_data.reporterSpecifications.push_back(
{ "junit", std::string( bazelOutputFile ), {}, {} } );
{ "junit", std::string( bazelOutputFile ), {}, {}, {} } );
}
const auto bazelTestSpec = Detail::getEnv( "TESTBRIDGE_TEST_ONLY" );
@@ -977,6 +1028,7 @@ namespace Catch {
<< bazelRandomSeed << "') as proper seed.\n";
} else {
m_data.rngSeed = *parsedSeed;
m_data.rngSeedWasFixed = true;
}
}
}
@@ -1162,6 +1214,10 @@ namespace Catch {
#endif
}
ITestCaseRegistry& getMutableTestCaseRegistry() override {
return m_testCaseRegistry;
}
private:
TestRegistry m_testCaseRegistry;
ReporterRegistry m_reporterRegistry;
@@ -1217,6 +1273,7 @@ namespace Catch {
ReporterConfig( config,
makeStream( spec.outputFilename ),
spec.colourMode,
spec.verbosity,
spec.customOptions ) );
}
@@ -1233,6 +1290,7 @@ namespace Catch {
ReporterConfig( config,
makeStream( reporterSpec.outputFilename ),
reporterSpec.colourMode,
reporterSpec.verbosity,
reporterSpec.customOptions ) ) );
}
@@ -1303,9 +1361,7 @@ namespace Catch {
};
void applyFilenamesAsTags() {
for (auto const& testInfo : getRegistryHub().getTestCaseRegistry().getAllInfos()) {
testInfo->addFilenameTag();
}
getMutableRegistryHub().getMutableTestCaseRegistry().enableFilenameTags();
}
// Creates empty file at path. The path must be writable, we do not
@@ -1511,6 +1567,19 @@ namespace Catch {
CATCH_TRY {
config(); // Force config to be constructed
if ( m_config->shardCount() > 1 &&
m_config->runOrder() == TestRunOrder::Randomized &&
!m_config->rngSeedWasFixed() ) {
Catch::cerr()
<< "Warning: using sharding (--shard-count) with random "
"order (--order rand, the default) and without a fixed "
"numeric --rng-seed does not guarantee disjoint coverage "
"between shard invocations. Pass the same numeric "
"--rng-seed to every shard, or use --order decl or "
"--order lex instead.\n"
<< std::flush;
}
// We need to retrieve potential Bazel config with the full Config
// constructor, so we have to create the guard file after it is created.
setUpGuardFile( m_config->getExitGuardFilePath() );
@@ -2394,7 +2463,7 @@ namespace Catch {
}
Version const& libraryVersion() {
static Version version( 3, 15, 3, "", 0 );
static Version version( 3, 16, 0, "", 0 );
return version;
}
@@ -2591,10 +2660,12 @@ namespace Catch {
IConfig const* _fullConfig,
Detail::unique_ptr<IStream> _stream,
ColourMode colourMode,
Verbosity verbosity,
std::map<std::string, std::string> customOptions ):
m_stream( CATCH_MOVE(_stream) ),
m_fullConfig( _fullConfig ),
m_colourMode( colourMode ),
m_verbosity( verbosity ),
m_customOptions( CATCH_MOVE( customOptions ) ) {}
Detail::unique_ptr<IStream> ReporterConfig::takeStream() && {
@@ -2603,6 +2674,7 @@ namespace Catch {
}
IConfig const * ReporterConfig::fullConfig() const { return m_fullConfig; }
ColourMode ReporterConfig::colourMode() const { return m_colourMode; }
Verbosity ReporterConfig::verbosity() const { return m_verbosity; }
std::map<std::string, std::string> const&
ReporterConfig::customOptions() const {
@@ -3297,9 +3369,11 @@ namespace Catch {
auto const setRngSeed = [&]( std::string const& seed ) {
if( seed == "time" ) {
config.rngSeed = generateRandomSeed(GenerateFrom::Time);
config.rngSeedWasFixed = false;
return ParserResult::ok(ParseResultType::Matched);
} else if (seed == "random-device") {
config.rngSeed = generateRandomSeed(GenerateFrom::RandomDevice);
config.rngSeedWasFixed = false;
return ParserResult::ok(ParseResultType::Matched);
}
@@ -3310,6 +3384,7 @@ namespace Catch {
return ParserResult::runtimeError( "Could not parse '" + seed + "' as seed" );
}
config.rngSeed = *parsedSeed;
config.rngSeedWasFixed = true;
return ParserResult::ok( ParseResultType::Matched );
};
auto const setDefaultColourMode = [&]( std::string const& colourMode ) {
@@ -5753,6 +5828,18 @@ namespace Catch {
return {};
}
}
Optional<Verbosity> stringToVerbosity( StringRef verbosity ) {
if (verbosity == "quiet") { return Verbosity::Quiet;
} else if ( verbosity == "normal" ) {
return Verbosity::Normal;
} else if ( verbosity == "high" ) {
return Verbosity::High;
} else {
return {};
}
}
} // namespace Detail
@@ -5760,6 +5847,7 @@ namespace Catch {
return lhs.m_name == rhs.m_name &&
lhs.m_outputFileName == rhs.m_outputFileName &&
lhs.m_colourMode == rhs.m_colourMode &&
lhs.m_verbosity == rhs.m_verbosity &&
lhs.m_customOptions == rhs.m_customOptions;
}
@@ -5771,6 +5859,7 @@ namespace Catch {
std::map<std::string, std::string> kvPairs;
Optional<std::string> outputFileName;
Optional<ColourMode> colourMode;
Optional<Verbosity> verbosity;
// First part is always reporter name, so we skip it
for ( size_t i = 1; i < parts.size(); ++i ) {
@@ -5808,6 +5897,12 @@ namespace Catch {
if ( !colourMode ) {
return {};
}
} else if ( key == "verbosity" ) {
// Duplicated key
if ( verbosity ) { return {}; }
verbosity = Detail::stringToVerbosity( value );
// Parsing failed
if ( !verbosity ) { return {}; }
} else {
// Unrecognized option
return {};
@@ -5817,6 +5912,7 @@ namespace Catch {
return ReporterSpec{ CATCH_MOVE( parts[0] ),
CATCH_MOVE( outputFileName ),
CATCH_MOVE( colourMode ),
CATCH_MOVE( verbosity),
CATCH_MOVE( kvPairs ) };
}
@@ -5824,10 +5920,12 @@ ReporterSpec::ReporterSpec(
std::string name,
Optional<std::string> outputFileName,
Optional<ColourMode> colourMode,
Optional<Verbosity> verbosity,
std::map<std::string, std::string> customOptions ):
m_name( CATCH_MOVE( name ) ),
m_outputFileName( CATCH_MOVE( outputFileName ) ),
m_colourMode( CATCH_MOVE( colourMode ) ),
m_verbosity( CATCH_MOVE( verbosity ) ),
m_customOptions( CATCH_MOVE( customOptions ) ) {}
} // namespace Catch
@@ -7234,6 +7332,9 @@ namespace Catch {
namespace Catch {
namespace {
// Picked small-ish number at random
static size_t kInitialTestCount = 120;
static void enforceNoDuplicateTestCases(
std::vector<TestCaseHandle> const& tests ) {
auto testInfoCmp = []( TestCaseInfo const* lhs,
@@ -7335,17 +7436,26 @@ namespace Catch {
return getRegistryHub().getTestCaseRegistry().getAllTestsSorted( config );
}
TestRegistry::TestRegistry() {
// We pre-reserve some reasonable number of tests to avoid the
// initial geometric growth churning during test registration.
m_handles.reserve( kInitialTestCount );
m_test_infos.reserve( kInitialTestCount );
m_invokers.reserve( kInitialTestCount );
}
TestRegistry::~TestRegistry() = default;
void TestRegistry::registerTest(Detail::unique_ptr<TestCaseInfo> testInfo, Detail::unique_ptr<ITestInvoker> testInvoker) {
m_handles.emplace_back(testInfo.get(), testInvoker.get());
m_viewed_test_infos.push_back(testInfo.get());
m_owned_test_infos.push_back(CATCH_MOVE(testInfo));
m_test_infos.push_back(CATCH_MOVE(testInfo));
m_invokers.push_back(CATCH_MOVE(testInvoker));
}
std::vector<TestCaseInfo*> const& TestRegistry::getAllInfos() const {
return m_viewed_test_infos;
void TestRegistry::enableFilenameTags() {
for (auto& info : m_test_infos) {
info->addFilenameTag();
}
}
std::vector<TestCaseHandle> const& TestRegistry::getAllTests() const {
@@ -9035,66 +9145,101 @@ namespace Catch {
#include <regex>
namespace Catch {
namespace {
constexpr StringRef caseSensitivitySuffix( CaseSensitive caseSensitivity ) {
return caseSensitivity == CaseSensitive::Yes
? StringRef{}
: " (case insensitive)"_sr;
}
} // namespace
namespace Matchers {
CasedString::CasedString( std::string const& str, CaseSensitive caseSensitivity )
: m_caseSensitivity( caseSensitivity ),
m_str( adjustString( str ) )
{}
std::string CasedString::adjustString( std::string const& str ) const {
return m_caseSensitivity == CaseSensitive::No
? toLower( str )
: str;
}
StringRef CasedString::caseSensitivitySuffix() const {
return m_caseSensitivity == CaseSensitive::Yes
? StringRef()
: " (case insensitive)"_sr;
}
StringMatcherBase::StringMatcherBase( std::string target,
StringRef operation,
CaseSensitive caseSensitivity ):
m_target( CATCH_MOVE( target ) ),
m_operation( operation ),
m_caseSensitivity( caseSensitivity ) {}
StringMatcherBase::StringMatcherBase( StringRef operation, CasedString const& comparator )
: m_comparator( comparator ),
m_operation( operation ) {
}
std::string StringMatcherBase::describe() const {
std::string description;
description.reserve(5 + m_operation.size() + m_comparator.m_str.size() +
m_comparator.caseSensitivitySuffix().size());
description.reserve(5 + m_operation.size() + m_target.size() +
caseSensitivitySuffix(m_caseSensitivity).size());
description += m_operation;
description += ": \"";
description += m_comparator.m_str;
description += m_target;
description += '"';
description += m_comparator.caseSensitivitySuffix();
description += caseSensitivitySuffix(m_caseSensitivity);
return description;
}
StringEqualsMatcher::StringEqualsMatcher( CasedString const& comparator ) : StringMatcherBase( "equals"_sr, comparator ) {}
StringEqualsMatcher::StringEqualsMatcher( std::string comparator, CaseSensitive caseSensitivity ):
StringMatcherBase( CATCH_MOVE( comparator ), "equals"_sr, caseSensitivity ) {}
bool StringEqualsMatcher::match( std::string const& source ) const {
return m_comparator.adjustString( source ) == m_comparator.m_str;
if (m_caseSensitivity == CaseSensitive::Yes) {
return m_target == source;
}
if (m_target.size() != source.size()) { return false; }
Catch::Detail::CaseInsensitiveEqualTo eq;
return eq( m_target, source );
}
StringContainsMatcher::StringContainsMatcher( CasedString const& comparator ) : StringMatcherBase( "contains"_sr, comparator ) {}
StringContainsMatcher::StringContainsMatcher(
std::string comparator, CaseSensitive caseSensitivity ):
StringMatcherBase( CATCH_MOVE( comparator ), "contains"_sr, caseSensitivity ) {}
bool StringContainsMatcher::match( std::string const& source ) const {
return contains( m_comparator.adjustString( source ), m_comparator.m_str );
if ( m_caseSensitivity == CaseSensitive::Yes ) {
return contains( source, m_target );
}
if ( source.size() < m_target.size() ) { return false; }
StringRef as_ref( source );
// The worst case of this is O(m*n), which is terrible, BUT:
// * The average case is much better, the worst case only happens rarely
// * We can implement BMH/other better searchers later if it matters
Catch::Detail::CaseInsensitiveEqualTo eq;
for (size_t i = 0; i < source.size(); ++i) {
const auto substr = as_ref.substr( i, m_target.size() );
bool found = eq( substr, m_target );
if ( found ) { return true; }
}
return false;
}
StartsWithMatcher::StartsWithMatcher( CasedString const& comparator ) : StringMatcherBase( "starts with"_sr, comparator ) {}
StartsWithMatcher::StartsWithMatcher( std::string comparator,
CaseSensitive caseSensitivity ):
StringMatcherBase( CATCH_MOVE( comparator ), "starts with"_sr, caseSensitivity ) {}
bool StartsWithMatcher::match( std::string const& source ) const {
return startsWith( m_comparator.adjustString( source ), m_comparator.m_str );
if ( m_caseSensitivity == CaseSensitive::Yes ) {
return startsWith( source, m_target );
}
if (source.size() < m_target.size()) { return false; }
Catch::Detail::CaseInsensitiveEqualTo eq;
return eq(
StringRef( source ).substr( 0, m_target.size() ), m_target );
}
EndsWithMatcher::EndsWithMatcher( CasedString const& comparator ) : StringMatcherBase( "ends with"_sr, comparator ) {}
EndsWithMatcher::EndsWithMatcher( std::string comparator,
CaseSensitive caseSensitivity ):
StringMatcherBase( CATCH_MOVE( comparator ), "ends with"_sr, caseSensitivity ) {}
bool EndsWithMatcher::match( std::string const& source ) const {
return endsWith( m_comparator.adjustString( source ), m_comparator.m_str );
if ( m_caseSensitivity == CaseSensitive::Yes ) {
return endsWith( source, m_target );
}
if ( source.size() < m_target.size() ) { return false; }
Catch::Detail::CaseInsensitiveEqualTo eq;
const size_t start_point = source.size() - m_target.size();
return eq( StringRef( source ).substr( start_point, m_target.size() ), m_target );
}
@@ -9115,21 +9260,21 @@ namespace Matchers {
}
StringEqualsMatcher Equals( std::string const& str, CaseSensitive caseSensitivity ) {
return StringEqualsMatcher( CasedString( str, caseSensitivity) );
StringEqualsMatcher Equals( std::string str, CaseSensitive caseSensitivity ) {
return StringEqualsMatcher( CATCH_MOVE( str ), caseSensitivity );
}
StringContainsMatcher ContainsSubstring( std::string const& str, CaseSensitive caseSensitivity ) {
return StringContainsMatcher( CasedString( str, caseSensitivity) );
StringContainsMatcher ContainsSubstring( std::string str, CaseSensitive caseSensitivity ) {
return StringContainsMatcher( CATCH_MOVE( str ), caseSensitivity );
}
EndsWithMatcher EndsWith( std::string const& str, CaseSensitive caseSensitivity ) {
return EndsWithMatcher( CasedString( str, caseSensitivity) );
EndsWithMatcher EndsWith( std::string str, CaseSensitive caseSensitivity ) {
return EndsWithMatcher( CATCH_MOVE( str ), caseSensitivity );
}
StartsWithMatcher StartsWith( std::string const& str, CaseSensitive caseSensitivity ) {
return StartsWithMatcher( CasedString( str, caseSensitivity) );
StartsWithMatcher StartsWith( std::string str, CaseSensitive caseSensitivity ) {
return StartsWithMatcher( CATCH_MOVE( str ), caseSensitivity );
}
RegexMatcher Matches(std::string const& regex, CaseSensitive caseSensitivity) {
return RegexMatcher(regex, caseSensitivity);
RegexMatcher Matches(std::string regex, CaseSensitive caseSensitivity) {
return RegexMatcher( CATCH_MOVE( regex ), caseSensitivity );
}
} // namespace Matchers
@@ -9231,6 +9376,7 @@ namespace Catch {
m_wrapped_stream( CATCH_MOVE(config).takeStream() ),
m_stream( m_wrapped_stream->stream() ),
m_colour( makeColourImpl( config.colourMode(), m_wrapped_stream.get() ) ),
m_verbosity( config.verbosity() ),
m_customOptions( config.customOptions() )
{}
@@ -9238,12 +9384,12 @@ namespace Catch {
void ReporterBase::listReporters(
std::vector<ReporterDescription> const& descriptions ) {
defaultListReporters( m_stream, descriptions, m_config->verbosity() );
defaultListReporters( m_stream, descriptions, m_verbosity );
}
void ReporterBase::listListeners(
std::vector<ListenerDescription> const& descriptions ) {
defaultListListeners( m_stream, descriptions, m_config->verbosity() );
defaultListListeners( m_stream, descriptions, m_verbosity );
}
void ReporterBase::listTests(std::vector<TestCaseHandle> const& tests) {
@@ -9251,11 +9397,11 @@ namespace Catch {
m_colour.get(),
tests,
m_config->hasTestFilters(),
m_config->verbosity());
m_verbosity);
}
void ReporterBase::listTags(std::vector<TagInfo> const& tags) {
defaultListTags( m_stream, tags, m_config->hasTestFilters(), m_config->verbosity() );
defaultListTags( m_stream, tags, m_config->hasTestFilters(), m_verbosity );
}
} // namespace Catch
@@ -10705,6 +10851,8 @@ namespace Catch {
namespace Catch {
namespace {
static size_t kJsonOutputVersion = 2;
void writeSourceInfo( JsonObjectWriter& writer,
SourceLineInfo const& sourceInfo ) {
auto source_location_writer =
@@ -10747,7 +10895,7 @@ namespace Catch {
m_writers.emplace( Writer::Object );
auto& writer = m_objectWriters.top();
writer.write( "version"_sr ).write( 1 );
writer.write( "version"_sr ).write( kJsonOutputVersion );
{
auto metadata_writer = writer.write( "metadata"_sr ).writeObject();
@@ -11033,14 +11181,18 @@ namespace Catch {
auto const& info = test.getTestCaseInfo();
desc_writer.write( "name"_sr ).write( info.name );
desc_writer.write( "class-name"_sr ).write( info.className );
{
if (!info.className.empty()) {
desc_writer.write( "class-name"_sr ).write( info.className );
}
if ( m_verbosity >= Verbosity::Normal ) {
auto tag_writer = desc_writer.write( "tags"_sr ).writeArray();
for ( auto const& tag : info.tags ) {
tag_writer.write( tag.original );
}
}
writeSourceInfo( desc_writer, info.lineInfo );
if ( m_verbosity >= Verbosity::High) {
writeSourceInfo( desc_writer, info.lineInfo );
}
}
}
void JsonReporter::listTags( std::vector<TagInfo> const& tags ) {
+223 -86
View File
@@ -6,8 +6,8 @@
// SPDX-License-Identifier: BSL-1.0
// Catch v3.15.3
// Generated: 2026-07-26 22:17:52.004020
// Catch v3.16.0
// Generated: 2026-08-25 09:29:22.652020
// ----------------------------------------------------------
// This file is an amalgamation of multiple different files.
// You probably shouldn't edit it directly.
@@ -1317,6 +1317,8 @@ namespace Catch {
virtual void registerTranslator( Detail::unique_ptr<IExceptionTranslator>&& translator ) = 0;
virtual void registerTagAlias( std::string const& alias, std::string const& tag, SourceLineInfo const& lineInfo ) = 0;
virtual void registerStartupException() noexcept = 0;
virtual ITestCaseRegistry& getMutableTestCaseRegistry() = 0;
};
IRegistryHub const& getRegistryHub();
@@ -1506,7 +1508,7 @@ namespace Catch {
} // namespace Detail
#elif defined(_MSC_VER) || defined(__IAR_SYSTEMS_ICC__)
#if defined(_MSVC_VER)
#if defined(_MSC_VER)
#pragma optimize("", off)
#elif defined(__IAR_SYSTEMS_ICC__)
// For IAR the pragma only affects the following function
@@ -1543,6 +1545,13 @@ namespace Catch {
template <typename Fn, typename... Args>
inline auto invoke_deoptimized(Fn&& fn, Args&&... args) -> std::enable_if_t<std::is_same<void, decltype(fn(args...))>::value> {
CATCH_FORWARD((fn)) (CATCH_FORWARD(args)...);
// In the non-void case, we pass the result through `deoptimize_value`
// to force the compiler to keep it. We have no return value here,
// but add an optimizer barrier (ideally a memory clobber) to force
// the _side effects_ of the loop be visible (e.g. writes to globals).
// Note that writes to benchmark-locals can be optimized away, as
// we would expect in normal code.
Detail::optimizer_barrier();
}
} // namespace Benchmark
} // namespace Catch
@@ -2159,9 +2168,94 @@ namespace Catch {
#include <exception>
#include <string>
#include <cmath>
#include <type_traits>
namespace Catch {
namespace Benchmark {
namespace Detail {
template <typename Clock>
ExecutionPlan prepare( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun ) {
auto min_time =
env.clock_resolution.mean * Detail::minimum_ticks;
auto run_time =
std::max( min_time,
std::chrono::duration_cast<decltype( min_time )>(
cfg.benchmarkWarmupTime() ) );
auto&& test = Detail::run_for_at_least<Clock>(
std::chrono::duration_cast<IDuration>( run_time ), 1, fun );
int new_iters = static_cast<int>(
std::ceil( min_time * test.iterations / test.elapsed ) );
return { new_iters,
test.elapsed / test.iterations * new_iters *
cfg.benchmarkSamples(),
CATCH_MOVE( fun ),
std::chrono::duration_cast<FDuration>(
cfg.benchmarkWarmupTime() ),
Detail::warmup_iterations };
}
// These are wrappers for their respective function templated
// over `default_clock`. This allows outlining the usual use
// of the template into single TU and save on compilation costs.
Environment measure_environment_default();
ExecutionPlan prepare_default( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun );
std::vector<FDuration> run_plan_default( ExecutionPlan const& plan,
const IConfig& cfg,
Environment env );
template <typename Clock>
std::enable_if_t<std::is_same<Clock, default_clock>::value,
Environment>
measureEnvironmentDispatch() {
return measure_environment_default();
}
template <typename Clock>
std::enable_if_t<!std::is_same<Clock, default_clock>::value,
Environment>
measureEnvironmentDispatch() {
return measure_environment<Clock>();
}
template <typename Clock>
std::enable_if_t<std::is_same<Clock, default_clock>::value,
std::vector<FDuration>>
runPlanDispatch( ExecutionPlan const& plan,
const IConfig& cfg,
Environment env ) {
return run_plan_default( plan, cfg, env );
}
template <typename Clock>
std::enable_if_t<!std::is_same<Clock, default_clock>::value,
std::vector<FDuration>>
runPlanDispatch( ExecutionPlan const& plan,
const IConfig& cfg,
Environment env ) {
return plan.template run<Clock>( cfg, env );
}
template <typename Clock>
std::enable_if_t<std::is_same<Clock, default_clock>::value,
ExecutionPlan>
prepareDispatch( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun ) {
return prepare_default( cfg, env, CATCH_MOVE( fun ) );
}
template <typename Clock>
std::enable_if_t<!std::is_same<Clock, default_clock>::value,
ExecutionPlan>
prepareDispatch( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun ) {
return prepare<Clock>( cfg, env, CATCH_MOVE( fun ) );
}
} // namespace Detail
struct Benchmark {
Benchmark(std::string&& benchmarkName)
: name(CATCH_MOVE(benchmarkName)) {}
@@ -2170,27 +2264,18 @@ namespace Catch {
Benchmark(std::string&& benchmarkName , FUN &&func)
: fun(CATCH_MOVE(func)), name(CATCH_MOVE(benchmarkName)) {}
template <typename Clock>
ExecutionPlan prepare(const IConfig &cfg, Environment env) {
auto min_time = env.clock_resolution.mean * Detail::minimum_ticks;
auto run_time = std::max(min_time, std::chrono::duration_cast<decltype(min_time)>(cfg.benchmarkWarmupTime()));
auto&& test = Detail::run_for_at_least<Clock>(std::chrono::duration_cast<IDuration>(run_time), 1, fun);
int new_iters = static_cast<int>(std::ceil(min_time * test.iterations / test.elapsed));
return { new_iters, test.elapsed / test.iterations * new_iters * cfg.benchmarkSamples(), CATCH_MOVE(fun), std::chrono::duration_cast<FDuration>(cfg.benchmarkWarmupTime()), Detail::warmup_iterations };
}
template <typename Clock = default_clock>
void run() {
static_assert( Clock::is_steady,
"Benchmarking clock should be steady" );
auto const* cfg = getCurrentContext().getConfig();
auto env = Detail::measure_environment<Clock>();
auto env = Detail::measureEnvironmentDispatch<Clock>();
getResultCapture().benchmarkPreparing(name);
CATCH_TRY{
auto plan = user_code([&] {
return prepare<Clock>(*cfg, env);
return Detail::prepareDispatch<Clock>( *cfg, env, CATCH_MOVE(fun) );
});
BenchmarkInfo info {
@@ -2206,7 +2291,7 @@ namespace Catch {
getResultCapture().benchmarkStarting(info);
auto samples = user_code([&] {
return plan.template run<Clock>(*cfg, env);
return Detail::runPlanDispatch<Clock>( plan, *cfg, env );
});
auto analysis = Detail::analyse(*cfg, samples.data(), samples.data() + samples.size());
@@ -3736,6 +3821,7 @@ namespace Catch {
std::vector<std::string> splitReporterSpec( StringRef reporterSpec );
Optional<ColourMode> stringToColourMode( StringRef colourMode );
Optional<Verbosity> stringToVerbosity( StringRef verbosity );
}
/**
@@ -3750,6 +3836,7 @@ namespace Catch {
std::string m_name;
Optional<std::string> m_outputFileName;
Optional<ColourMode> m_colourMode;
Optional<Verbosity> m_verbosity;
std::map<std::string, std::string> m_customOptions;
friend bool operator==( ReporterSpec const& lhs,
@@ -3764,6 +3851,7 @@ namespace Catch {
std::string name,
Optional<std::string> outputFileName,
Optional<ColourMode> colourMode,
Optional<Verbosity> verbosity,
std::map<std::string, std::string> customOptions );
std::string const& name() const { return m_name; }
@@ -3774,13 +3862,15 @@ namespace Catch {
Optional<ColourMode> const& colourMode() const { return m_colourMode; }
Optional<Verbosity> const& verbosity() const { return m_verbosity; }
std::map<std::string, std::string> const& customOptions() const {
return m_customOptions;
}
};
/**
* Parses provided reporter spec string into
* Parses provided reporter spec string into actual `ReporterSpec`
*
* Returns empty optional on errors, e.g.
* * field that is not first and not a key+value pair
@@ -3813,6 +3903,7 @@ namespace Catch {
std::string name;
std::string outputFilename;
ColourMode colourMode;
Verbosity verbosity;
std::map<std::string, std::string> customOptions;
friend bool operator==( ProcessedReporterSpec const& lhs,
ProcessedReporterSpec const& rhs );
@@ -3840,6 +3931,7 @@ namespace Catch {
int abortAfter = -1;
uint32_t rngSeed = generateRandomSeed(GenerateFrom::Default);
bool rngSeedWasFixed = false;
unsigned int shardCount = 1;
unsigned int shardIndex = 0;
@@ -3911,6 +4003,7 @@ namespace Catch {
double minDuration() const override;
TestRunOrder runOrder() const override;
uint32_t rngSeed() const override;
bool rngSeedWasFixed() const;
unsigned int shardCount() const override;
unsigned int shardIndex() const override;
ColourMode defaultColourMode() const override;
@@ -6624,6 +6717,51 @@ namespace Catch {
struct priority_tag : priority_tag<N - 1> {};
template <>
struct priority_tag<0> {};
// This is a bunch of helpers for the templated test case handling.
// They should live elsewhere in the long run, but as an in-between
// step we toss them all here.
template <typename...> struct TypeList {};
template <typename... Ts>
constexpr auto get_wrapper( priority_tag<1> ) noexcept -> TypeList<Ts...> { return {}; }
template <template <typename...> class...> struct TemplateTypeList {};
// Clang 20 and 21 cannot handle an explicitly specified all-pack
// template-template parameter here ("conflicting deduction" regression,
// llvm/llvm-project#130778; fixed for Clang 22).
// Remove get_template_wrapper once Clang 21 is no longer supported.
template <template <typename...> class C, template <typename...> class... Cs>
constexpr auto get_template_wrapper( priority_tag<1> ) noexcept -> TemplateTypeList<C, Cs...> { return {}; }
template <typename...>
struct append;
template <typename T>
struct append<T> { using type = T; };
template <template <typename...> class L1, typename... E1, template <typename...> class L2, typename... E2, typename... Rest>
struct append<L1<E1...>, L2<E2...>, Rest...> { using type = typename append<L1<E1..., E2...>, Rest...>::type; };
template <template <typename...> class L1, typename... E1, typename... Rest>
struct append<L1<E1...>, TypeList<mpl_::na>, Rest...> { using type = L1<E1...>; };
template <template <typename...> class, typename>
struct convert;
template <template <typename...> class Final, template <typename...> class List, typename... Ts>
struct convert<Final, List<Ts...>> { using type = typename append<Final<>, TypeList<Ts>...>::type; };
// These are helpers for the PRODUCT templated test cases.
// Note that the _SIG macros (for NTTPs) also use specializations
// of these, but they have to use their own instances due to needing
// per-sig specializations and we have to keep these in their own
// unnamed namespace.
template <typename...>
struct rewrap;
template <template <typename...> class Container, template <typename...> class List, typename... elems>
struct rewrap<TemplateTypeList<Container>, List<elems...>> { using type = TypeList<Container<elems...>>; };
template <template <typename...> class Container, template <typename...> class List, class... Elems, typename... Elements>
struct rewrap<TemplateTypeList<Container>, List<Elems...>, Elements...> { using type = typename append<TypeList<Container<Elems...>>, typename rewrap<TemplateTypeList<Container>, Elements...>::type>::type; };
template <template <typename...> class, typename...>
struct create;
template <template <typename...> class Final, template <typename...> class... Containers, typename... Types>
struct create<Final, TemplateTypeList<Containers...>, TypeList<Types...>> { using type = typename append<Final<>, typename rewrap<TemplateTypeList<Containers>, Types...>::type...>::type; };
}
}
@@ -6710,46 +6848,42 @@ namespace Catch {
#define INTERNAL_CATCH_VA_NARGS_IMPL(_0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, N, ...) N
#define INTERNAL_CATCH_TYPE_GEN\
template<typename...> struct TypeList {};\
template<typename... Ts>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<1>) noexcept -> TypeList<Ts...> { return {}; }\
template<template<typename...> class...> struct TemplateTypeList{};\
template<template<typename...> class...Cs>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<1>) noexcept -> TemplateTypeList<Cs...> { return {}; }\
template<typename...>\
struct append;\
template<typename...>\
struct rewrap;\
template<template<typename...> class, typename...>\
struct create;\
template<template<typename...> class, typename>\
struct convert;\
\
template<typename T> \
struct append<T> { using type = T; };\
template< template<typename...> class L1, typename...E1, template<typename...> class L2, typename...E2, typename...Rest>\
struct append<L1<E1...>, L2<E2...>, Rest...> { using type = typename append<L1<E1...,E2...>, Rest...>::type; };\
template< template<typename...> class L1, typename...E1, typename...Rest>\
struct append<L1<E1...>, TypeList<mpl_::na>, Rest...> { using type = L1<E1...>; };\
#define INTERNAL_CATCH_TYPE_GEN \
/* We moved these into a central location and no longer create them
in each templated test's unnamed namespace, but we pull them in
with using to avoid qualifying all the references. */ \
using Catch::Detail::TypeList; \
using Catch::Detail::get_wrapper; \
using Catch::Detail::TemplateTypeList; \
using Catch::Detail::get_template_wrapper; \
using Catch::Detail::append; \
using Catch::Detail::convert;
// This stamps out the per test case specializations of wrapper handlers
// for _SIG (NTTP) macros inside their own namespace, so they can add
// their required specializations
#define INTERNAL_CATCH_NTTP_1( signature, ... ) \
template<INTERNAL_CATCH_REMOVE_PARENS(signature)> struct Nttp{};\
template<INTERNAL_CATCH_REMOVE_PARENS(signature)>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<0>) noexcept -> Nttp<__VA_ARGS__> { return {}; } \
template<template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class...> struct NttpTemplateTypeList{};\
template<template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class C, template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class...Cs>\
constexpr auto get_template_wrapper(Catch::Detail::priority_tag<0>) noexcept -> NttpTemplateTypeList<C, Cs...> { return {}; }
// This stamps out the per test case specializations of type-product
// machinery for the NTTP product macros.
#define INTERNAL_CATCH_NTTP_REWRAP_1( signature, ... ) \
template<typename...> \
struct rewrap; \
template<template<typename...> class, typename...> \
struct create; \
\
template< template<typename...> class Container, template<typename...> class List, typename...elems>\
struct rewrap<TemplateTypeList<Container>, List<elems...>> { using type = TypeList<Container<elems...>>; };\
template< template<typename...> class Container, template<typename...> class List, class...Elems, typename...Elements>\
struct rewrap<TemplateTypeList<Container>, List<Elems...>, Elements...> { using type = typename append<TypeList<Container<Elems...>>, typename rewrap<TemplateTypeList<Container>, Elements...>::type>::type; };\
\
template<template <typename...> class Final, template< typename...> class...Containers, typename...Types>\
struct create<Final, TemplateTypeList<Containers...>, TypeList<Types...>> { using type = typename append<Final<>, typename rewrap<TemplateTypeList<Containers>, Types...>::type...>::type; };\
template<template <typename...> class Final, template <typename...> class List, typename...Ts>\
struct convert<Final, List<Ts...>> { using type = typename append<Final<>,TypeList<Ts>...>::type; };
#define INTERNAL_CATCH_NTTP_1(signature, ...)\
template<INTERNAL_CATCH_REMOVE_PARENS(signature)> struct Nttp{};\
template<INTERNAL_CATCH_REMOVE_PARENS(signature)>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<0>) noexcept -> Nttp<__VA_ARGS__> { return {}; } \
template<template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class...> struct NttpTemplateTypeList{};\
template<template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class...Cs>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<0>) noexcept -> NttpTemplateTypeList<Cs...> { return {}; } \
\
template< template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class Container, template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class List, INTERNAL_CATCH_REMOVE_PARENS(signature)>\
struct rewrap<NttpTemplateTypeList<Container>, List<__VA_ARGS__>> { using type = TypeList<Container<__VA_ARGS__>>; };\
@@ -6826,7 +6960,9 @@ namespace Catch {
#ifndef CATCH_CONFIG_TRADITIONAL_MSVC_PREPROCESSOR
#define INTERNAL_CATCH_NTTP_0
#define INTERNAL_CATCH_NTTP_0_REWRAP using Catch::Detail::rewrap; using Catch::Detail::create;
#define INTERNAL_CATCH_NTTP_GEN(...) INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_1( __VA_ARGS__),INTERNAL_CATCH_NTTP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_0)
#define INTERNAL_CATCH_NTTP_REWRAP_GEN(...) INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__),INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_0_REWRAP)
#define INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD(TestName, ...) INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD1, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD0)(TestName, __VA_ARGS__)
#define INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD(TestName, ClassName, ...) INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD1, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD0)(TestName, ClassName, __VA_ARGS__)
#define INTERNAL_CATCH_NTTP_REG_METHOD_GEN(TestName, ...) INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD0, INTERNAL_CATCH_NTTP_REGISTER_METHOD0)(TestName, __VA_ARGS__)
@@ -6836,7 +6972,9 @@ namespace Catch {
#define INTERNAL_CATCH_REMOVE_PARENS_GEN(...) INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_REMOVE_PARENS_11_ARG,INTERNAL_CATCH_REMOVE_PARENS_10_ARG,INTERNAL_CATCH_REMOVE_PARENS_9_ARG,INTERNAL_CATCH_REMOVE_PARENS_8_ARG,INTERNAL_CATCH_REMOVE_PARENS_7_ARG,INTERNAL_CATCH_REMOVE_PARENS_6_ARG,INTERNAL_CATCH_REMOVE_PARENS_5_ARG,INTERNAL_CATCH_REMOVE_PARENS_4_ARG,INTERNAL_CATCH_REMOVE_PARENS_3_ARG,INTERNAL_CATCH_REMOVE_PARENS_2_ARG,INTERNAL_CATCH_REMOVE_PARENS_1_ARG)(__VA_ARGS__)
#else
#define INTERNAL_CATCH_NTTP_0(signature)
#define INTERNAL_CATCH_NTTP_0_REWRAP(signature) using Catch::Detail::rewrap; using Catch::Detail::create;
#define INTERNAL_CATCH_NTTP_GEN(...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1,INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_0)( __VA_ARGS__))
#define INTERNAL_CATCH_NTTP_REWRAP_GEN(...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1,INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_0_REWRAP)( __VA_ARGS__))
#define INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD(TestName, ...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD1, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD0)(TestName, __VA_ARGS__))
#define INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD(TestName, ClassName, ...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD1, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD0)(TestName, ClassName, __VA_ARGS__))
#define INTERNAL_CATCH_NTTP_REG_METHOD_GEN(TestName, ...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD0, INTERNAL_CATCH_NTTP_REGISTER_METHOD0)(TestName, __VA_ARGS__))
@@ -6961,6 +7099,7 @@ namespace Catch {
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestName) { \
INTERNAL_CATCH_TYPE_GEN \
INTERNAL_CATCH_NTTP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature)) \
INTERNAL_CATCH_NTTP_REWRAP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature)) \
template<typename... Types> \
struct TestName { \
void reg_tests() { \
@@ -6973,7 +7112,7 @@ namespace Catch {
} \
}; \
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){ \
using TestInit = typename create<TestName, decltype(get_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type; \
using TestInit = typename create<TestName, decltype(get_template_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type; \
TestInit t; \
t.reg_tests(); \
return 0; \
@@ -7057,7 +7196,7 @@ namespace Catch {
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
TestNameClass<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(__VA_ARGS__)>();\
return 0;\
}();\
}();\
}\
}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
@@ -7093,6 +7232,7 @@ namespace Catch {
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestNameClass) {\
INTERNAL_CATCH_TYPE_GEN \
INTERNAL_CATCH_NTTP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature))\
INTERNAL_CATCH_NTTP_REWRAP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature))\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
@@ -7105,7 +7245,7 @@ namespace Catch {
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename create<TestNameClass, decltype(get_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type;\
using TestInit = typename create<TestNameClass, decltype(get_template_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
@@ -7571,8 +7711,8 @@ namespace Catch {
#define CATCH_VERSION_MACROS_HPP_INCLUDED
#define CATCH_VERSION_MAJOR 3
#define CATCH_VERSION_MINOR 15
#define CATCH_VERSION_PATCH 3
#define CATCH_VERSION_MINOR 16
#define CATCH_VERSION_PATCH 0
#endif // CATCH_VERSION_MACROS_HPP_INCLUDED
@@ -9159,6 +9299,7 @@ namespace Catch {
ReporterConfig( IConfig const* _fullConfig,
Detail::unique_ptr<IStream> _stream,
ColourMode colourMode,
Verbosity verbosity,
std::map<std::string, std::string> customOptions );
ReporterConfig( ReporterConfig&& ) = default;
@@ -9168,12 +9309,14 @@ namespace Catch {
Detail::unique_ptr<IStream> takeStream() &&;
IConfig const* fullConfig() const;
ColourMode colourMode() const;
Verbosity verbosity() const;
std::map<std::string, std::string> const& customOptions() const;
private:
Detail::unique_ptr<IStream> m_stream;
IConfig const* m_fullConfig;
ColourMode m_colourMode;
Verbosity m_verbosity;
std::map<std::string, std::string> m_customOptions;
};
@@ -9426,8 +9569,7 @@ namespace Catch {
class ITestCaseRegistry {
public:
virtual ~ITestCaseRegistry(); // = default
// TODO: this exists only for adding filenames to test cases -- let's expose this in a saner way later
virtual std::vector<TestCaseInfo* > const& getAllInfos() const = 0;
virtual void enableFilenameTags() = 0;
virtual std::vector<TestCaseHandle> const& getAllTests() const = 0;
virtual std::vector<TestCaseHandle> const& getAllTestsSorted( IConfig const& config ) const = 0;
};
@@ -11290,20 +11432,20 @@ namespace Catch {
class TestRegistry final : public ITestCaseRegistry {
public:
void registerTest( Detail::unique_ptr<TestCaseInfo> testInfo, Detail::unique_ptr<ITestInvoker> testInvoker );
void enableFilenameTags() override;
std::vector<TestCaseInfo*> const& getAllInfos() const override;
std::vector<TestCaseHandle> const& getAllTests() const override;
std::vector<TestCaseHandle> const& getAllTestsSorted( IConfig const& config ) const override;
TestRegistry();
~TestRegistry() override; // = default
private:
std::vector<Detail::unique_ptr<TestCaseInfo>> m_owned_test_infos;
// Keeps a materialized vector for `getAllInfos`.
// We should get rid of that eventually (see interface note)
std::vector<TestCaseInfo*> m_viewed_test_infos;
// Owns the test infos for handles
std::vector<Detail::unique_ptr<TestCaseInfo>> m_test_infos;
// Owns the test invokers for handles
std::vector<Detail::unique_ptr<ITestInvoker>> m_invokers;
std::vector<TestCaseHandle> m_handles;
mutable TestRunOrder m_currentSortOrder = TestRunOrder::Declared;
mutable std::vector<TestCaseHandle> m_sortedFunctions;
@@ -13301,44 +13443,40 @@ namespace Catch {
namespace Catch {
namespace Matchers {
struct CasedString {
CasedString( std::string const& str, CaseSensitive caseSensitivity );
std::string adjustString( std::string const& str ) const;
StringRef caseSensitivitySuffix() const;
CaseSensitive m_caseSensitivity;
std::string m_str;
};
class StringMatcherBase : public MatcherBase<std::string> {
protected:
CasedString m_comparator;
std::string m_target;
StringRef m_operation;
CaseSensitive m_caseSensitivity;
StringMatcherBase( std::string target,
StringRef operation,
CaseSensitive caseSensitivity );
public:
StringMatcherBase( StringRef operation,
CasedString const& comparator );
std::string describe() const override;
};
class StringEqualsMatcher final : public StringMatcherBase {
public:
StringEqualsMatcher( CasedString const& comparator );
StringEqualsMatcher( std::string comparator, CaseSensitive caseSensitivity );
bool match( std::string const& source ) const override;
};
class StringContainsMatcher final : public StringMatcherBase {
public:
StringContainsMatcher( CasedString const& comparator );
StringContainsMatcher( std::string comparator,
CaseSensitive caseSensitivity );
bool match( std::string const& source ) const override;
};
class StartsWithMatcher final : public StringMatcherBase {
public:
StartsWithMatcher( CasedString const& comparator );
StartsWithMatcher( std::string comparator,
CaseSensitive caseSensitivity );
bool match( std::string const& source ) const override;
};
class EndsWithMatcher final : public StringMatcherBase {
public:
EndsWithMatcher( CasedString const& comparator );
EndsWithMatcher( std::string comparator,
CaseSensitive caseSensitivity );
bool match( std::string const& source ) const override;
};
@@ -13353,15 +13491,15 @@ namespace Matchers {
};
//! Creates matcher that accepts strings that are exactly equal to `str`
StringEqualsMatcher Equals( std::string const& str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
StringEqualsMatcher Equals( std::string str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
//! Creates matcher that accepts strings that contain `str`
StringContainsMatcher ContainsSubstring( std::string const& str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
StringContainsMatcher ContainsSubstring( std::string str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
//! Creates matcher that accepts strings that _end_ with `str`
EndsWithMatcher EndsWith( std::string const& str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
EndsWithMatcher EndsWith( std::string str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
//! Creates matcher that accepts strings that _start_ with `str`
StartsWithMatcher StartsWith( std::string const& str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
StartsWithMatcher StartsWith( std::string str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
//! Creates matcher that accepts strings matching `regex`
RegexMatcher Matches( std::string const& regex, CaseSensitive caseSensitivity = CaseSensitive::Yes );
RegexMatcher Matches( std::string regex, CaseSensitive caseSensitivity = CaseSensitive::Yes );
} // namespace Matchers
} // namespace Catch
@@ -13615,6 +13753,8 @@ namespace Catch {
std::ostream& m_stream;
//! Colour implementation this reporter was configured for
Detail::unique_ptr<ColourImpl> m_colour;
//! Verbosity configured for this reporter
Verbosity m_verbosity;
//! The custom reporter options user passed down to the reporter
std::map<std::string, std::string> m_customOptions;
@@ -14206,9 +14346,6 @@ namespace Catch {
std::stack<Writer> m_writers{};
bool m_startedListing = false;
// std::size_t m_sectionDepth = 0;
// std::size_t m_sectionStarted = 0;
};
} // namespace Catch
+1 -1
View File
@@ -8,7 +8,7 @@
project(
'catch2',
'cpp',
version: '3.15.3', # CML version placeholder, don't delete
version: '3.16.0', # CML version placeholder, don't delete
license: 'BSL-1.0',
meson_version: '>=0.54.1',
)
+1
View File
@@ -32,6 +32,7 @@ set(BENCHMARK_HEADERS
${SOURCES_DIR}/benchmark/detail/catch_timing.hpp
)
set(BENCHMARK_SOURCES
${SOURCES_DIR}/benchmark/catch_benchmark.cpp
${SOURCES_DIR}/benchmark/catch_chronometer.cpp
${SOURCES_DIR}/benchmark/detail/catch_analyse.cpp
${SOURCES_DIR}/benchmark/detail/catch_benchmark_function.cpp
+54
View File
@@ -0,0 +1,54 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
#include <catch2/benchmark/catch_benchmark.hpp>
#include <algorithm>
#include <chrono>
#include <cmath>
namespace Catch {
namespace Benchmark {
namespace Detail {
Environment measure_environment_default() {
return Detail::measure_environment<default_clock>();
}
ExecutionPlan prepare_default( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun ) {
// This mirrors Benchmark::prepare<default_clock>(), but with the
// clock fixed so it is instantiated once here in the library.
auto min_time = env.clock_resolution.mean * Detail::minimum_ticks;
auto run_time = std::max(
min_time,
std::chrono::duration_cast<decltype( min_time )>(
cfg.benchmarkWarmupTime() ) );
auto&& test = Detail::run_for_at_least<default_clock>(
std::chrono::duration_cast<IDuration>( run_time ), 1, fun );
int new_iters = static_cast<int>(
std::ceil( min_time * test.iterations / test.elapsed ) );
return { new_iters,
test.elapsed / test.iterations * new_iters *
cfg.benchmarkSamples(),
CATCH_MOVE( fun ),
std::chrono::duration_cast<FDuration>(
cfg.benchmarkWarmupTime() ),
Detail::warmup_iterations };
}
std::vector<FDuration> run_plan_default( ExecutionPlan const& plan,
const IConfig& cfg,
Environment env ) {
return plan.run<default_clock>( cfg, env );
}
} // namespace Detail
} // namespace Benchmark
} // namespace Catch
+88 -12
View File
@@ -33,9 +33,94 @@
#include <exception>
#include <string>
#include <cmath>
#include <type_traits>
namespace Catch {
namespace Benchmark {
namespace Detail {
template <typename Clock>
ExecutionPlan prepare( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun ) {
auto min_time =
env.clock_resolution.mean * Detail::minimum_ticks;
auto run_time =
std::max( min_time,
std::chrono::duration_cast<decltype( min_time )>(
cfg.benchmarkWarmupTime() ) );
auto&& test = Detail::run_for_at_least<Clock>(
std::chrono::duration_cast<IDuration>( run_time ), 1, fun );
int new_iters = static_cast<int>(
std::ceil( min_time * test.iterations / test.elapsed ) );
return { new_iters,
test.elapsed / test.iterations * new_iters *
cfg.benchmarkSamples(),
CATCH_MOVE( fun ),
std::chrono::duration_cast<FDuration>(
cfg.benchmarkWarmupTime() ),
Detail::warmup_iterations };
}
// These are wrappers for their respective function templated
// over `default_clock`. This allows outlining the usual use
// of the template into single TU and save on compilation costs.
Environment measure_environment_default();
ExecutionPlan prepare_default( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun );
std::vector<FDuration> run_plan_default( ExecutionPlan const& plan,
const IConfig& cfg,
Environment env );
template <typename Clock>
std::enable_if_t<std::is_same<Clock, default_clock>::value,
Environment>
measureEnvironmentDispatch() {
return measure_environment_default();
}
template <typename Clock>
std::enable_if_t<!std::is_same<Clock, default_clock>::value,
Environment>
measureEnvironmentDispatch() {
return measure_environment<Clock>();
}
template <typename Clock>
std::enable_if_t<std::is_same<Clock, default_clock>::value,
std::vector<FDuration>>
runPlanDispatch( ExecutionPlan const& plan,
const IConfig& cfg,
Environment env ) {
return run_plan_default( plan, cfg, env );
}
template <typename Clock>
std::enable_if_t<!std::is_same<Clock, default_clock>::value,
std::vector<FDuration>>
runPlanDispatch( ExecutionPlan const& plan,
const IConfig& cfg,
Environment env ) {
return plan.template run<Clock>( cfg, env );
}
template <typename Clock>
std::enable_if_t<std::is_same<Clock, default_clock>::value,
ExecutionPlan>
prepareDispatch( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun ) {
return prepare_default( cfg, env, CATCH_MOVE( fun ) );
}
template <typename Clock>
std::enable_if_t<!std::is_same<Clock, default_clock>::value,
ExecutionPlan>
prepareDispatch( const IConfig& cfg,
Environment env,
BenchmarkFunction&& fun ) {
return prepare<Clock>( cfg, env, CATCH_MOVE( fun ) );
}
} // namespace Detail
struct Benchmark {
Benchmark(std::string&& benchmarkName)
: name(CATCH_MOVE(benchmarkName)) {}
@@ -44,27 +129,18 @@ namespace Catch {
Benchmark(std::string&& benchmarkName , FUN &&func)
: fun(CATCH_MOVE(func)), name(CATCH_MOVE(benchmarkName)) {}
template <typename Clock>
ExecutionPlan prepare(const IConfig &cfg, Environment env) {
auto min_time = env.clock_resolution.mean * Detail::minimum_ticks;
auto run_time = std::max(min_time, std::chrono::duration_cast<decltype(min_time)>(cfg.benchmarkWarmupTime()));
auto&& test = Detail::run_for_at_least<Clock>(std::chrono::duration_cast<IDuration>(run_time), 1, fun);
int new_iters = static_cast<int>(std::ceil(min_time * test.iterations / test.elapsed));
return { new_iters, test.elapsed / test.iterations * new_iters * cfg.benchmarkSamples(), CATCH_MOVE(fun), std::chrono::duration_cast<FDuration>(cfg.benchmarkWarmupTime()), Detail::warmup_iterations };
}
template <typename Clock = default_clock>
void run() {
static_assert( Clock::is_steady,
"Benchmarking clock should be steady" );
auto const* cfg = getCurrentContext().getConfig();
auto env = Detail::measure_environment<Clock>();
auto env = Detail::measureEnvironmentDispatch<Clock>();
getResultCapture().benchmarkPreparing(name);
CATCH_TRY{
auto plan = user_code([&] {
return prepare<Clock>(*cfg, env);
return Detail::prepareDispatch<Clock>( *cfg, env, CATCH_MOVE(fun) );
});
BenchmarkInfo info {
@@ -80,7 +156,7 @@ namespace Catch {
getResultCapture().benchmarkStarting(info);
auto samples = user_code([&] {
return plan.template run<Clock>(*cfg, env);
return Detail::runPlanDispatch<Clock>( plan, *cfg, env );
});
auto analysis = Detail::analyse(*cfg, samples.data(), samples.data() + samples.size());
+8 -1
View File
@@ -34,7 +34,7 @@ namespace Catch {
} // namespace Detail
#elif defined(_MSC_VER) || defined(__IAR_SYSTEMS_ICC__)
#if defined(_MSVC_VER)
#if defined(_MSC_VER)
#pragma optimize("", off)
#elif defined(__IAR_SYSTEMS_ICC__)
// For IAR the pragma only affects the following function
@@ -71,6 +71,13 @@ namespace Catch {
template <typename Fn, typename... Args>
inline auto invoke_deoptimized(Fn&& fn, Args&&... args) -> std::enable_if_t<std::is_same<void, decltype(fn(args...))>::value> {
CATCH_FORWARD((fn)) (CATCH_FORWARD(args)...);
// In the non-void case, we pass the result through `deoptimize_value`
// to force the compiler to keep it. We have no return value here,
// but add an optimizer barrier (ideally a memory clobber) to force
// the _side effects_ of the loop be visible (e.g. writes to globals).
// Note that writes to benchmark-locals can be optimized away, as
// we would expect in normal code.
Detail::optimizer_barrier();
}
} // namespace Benchmark
} // namespace Catch
+5 -1
View File
@@ -89,6 +89,7 @@ namespace Catch {
return lhs.name == rhs.name &&
lhs.outputFilename == rhs.outputFilename &&
lhs.colourMode == rhs.colourMode &&
lhs.verbosity == rhs.verbosity &&
lhs.customOptions == rhs.customOptions;
}
@@ -157,6 +158,7 @@ namespace Catch {
reporterSpec.outputFile() ? *reporterSpec.outputFile()
: data.defaultOutputFilename,
reporterSpec.colourMode().valueOr( data.defaultColourMode ),
reporterSpec.verbosity().valueOr( data.verbosity ),
reporterSpec.customOptions() } );
}
}
@@ -207,6 +209,7 @@ namespace Catch {
double Config::minDuration() const { return m_data.minDuration; }
TestRunOrder Config::runOrder() const { return m_data.runOrder; }
uint32_t Config::rngSeed() const { return m_data.rngSeed; }
bool Config::rngSeedWasFixed() const { return m_data.rngSeedWasFixed; }
unsigned int Config::shardCount() const { return m_data.shardCount; }
unsigned int Config::shardIndex() const { return m_data.shardIndex; }
ColourMode Config::defaultColourMode() const { return m_data.defaultColourMode; }
@@ -232,7 +235,7 @@ namespace Catch {
if ( bazelOutputFile ) {
m_data.reporterSpecifications.push_back(
{ "junit", std::string( bazelOutputFile ), {}, {} } );
{ "junit", std::string( bazelOutputFile ), {}, {}, {} } );
}
const auto bazelTestSpec = Detail::getEnv( "TESTBRIDGE_TEST_ONLY" );
@@ -271,6 +274,7 @@ namespace Catch {
<< bazelRandomSeed << "') as proper seed.\n";
} else {
m_data.rngSeed = *parsedSeed;
m_data.rngSeedWasFixed = true;
}
}
}
+3
View File
@@ -35,6 +35,7 @@ namespace Catch {
std::string name;
std::string outputFilename;
ColourMode colourMode;
Verbosity verbosity;
std::map<std::string, std::string> customOptions;
friend bool operator==( ProcessedReporterSpec const& lhs,
ProcessedReporterSpec const& rhs );
@@ -62,6 +63,7 @@ namespace Catch {
int abortAfter = -1;
uint32_t rngSeed = generateRandomSeed(GenerateFrom::Default);
bool rngSeedWasFixed = false;
unsigned int shardCount = 1;
unsigned int shardIndex = 0;
@@ -133,6 +135,7 @@ namespace Catch {
double minDuration() const override;
TestRunOrder runOrder() const override;
uint32_t rngSeed() const override;
bool rngSeedWasFixed() const;
unsigned int shardCount() const override;
unsigned int shardIndex() const override;
ColourMode defaultColourMode() const override;
+4
View File
@@ -72,6 +72,10 @@ namespace Catch {
#endif
}
ITestCaseRegistry& getMutableTestCaseRegistry() override {
return m_testCaseRegistry;
}
private:
TestRegistry m_testCaseRegistry;
ReporterRegistry m_reporterRegistry;
+16 -3
View File
@@ -52,6 +52,7 @@ namespace Catch {
ReporterConfig( config,
makeStream( spec.outputFilename ),
spec.colourMode,
spec.verbosity,
spec.customOptions ) );
}
@@ -68,6 +69,7 @@ namespace Catch {
ReporterConfig( config,
makeStream( reporterSpec.outputFilename ),
reporterSpec.colourMode,
reporterSpec.verbosity,
reporterSpec.customOptions ) ) );
}
@@ -138,9 +140,7 @@ namespace Catch {
};
void applyFilenamesAsTags() {
for (auto const& testInfo : getRegistryHub().getTestCaseRegistry().getAllInfos()) {
testInfo->addFilenameTag();
}
getMutableRegistryHub().getMutableTestCaseRegistry().enableFilenameTags();
}
// Creates empty file at path. The path must be writable, we do not
@@ -346,6 +346,19 @@ namespace Catch {
CATCH_TRY {
config(); // Force config to be constructed
if ( m_config->shardCount() > 1 &&
m_config->runOrder() == TestRunOrder::Randomized &&
!m_config->rngSeedWasFixed() ) {
Catch::cerr()
<< "Warning: using sharding (--shard-count) with random "
"order (--order rand, the default) and without a fixed "
"numeric --rng-seed does not guarantee disjoint coverage "
"between shard invocations. Pass the same numeric "
"--rng-seed to every shard, or use --order decl or "
"--order lex instead.\n"
<< std::flush;
}
// We need to retrieve potential Bazel config with the full Config
// constructor, so we have to create the guard file after it is created.
setUpGuardFile( m_config->getExitGuardFilePath() );
+1 -1
View File
@@ -36,7 +36,7 @@ namespace Catch {
}
Version const& libraryVersion() {
static Version version( 3, 15, 3, "", 0 );
static Version version( 3, 16, 0, "", 0 );
return version;
}
+2 -2
View File
@@ -9,7 +9,7 @@
#define CATCH_VERSION_MACROS_HPP_INCLUDED
#define CATCH_VERSION_MAJOR 3
#define CATCH_VERSION_MINOR 15
#define CATCH_VERSION_PATCH 3
#define CATCH_VERSION_MINOR 16
#define CATCH_VERSION_PATCH 0
#endif // CATCH_VERSION_MACROS_HPP_INCLUDED
@@ -53,6 +53,8 @@ namespace Catch {
virtual void registerTranslator( Detail::unique_ptr<IExceptionTranslator>&& translator ) = 0;
virtual void registerTagAlias( std::string const& alias, std::string const& tag, SourceLineInfo const& lineInfo ) = 0;
virtual void registerStartupException() noexcept = 0;
virtual ITestCaseRegistry& getMutableTestCaseRegistry() = 0;
};
IRegistryHub const& getRegistryHub();
@@ -19,10 +19,12 @@ namespace Catch {
IConfig const* _fullConfig,
Detail::unique_ptr<IStream> _stream,
ColourMode colourMode,
Verbosity verbosity,
std::map<std::string, std::string> customOptions ):
m_stream( CATCH_MOVE(_stream) ),
m_fullConfig( _fullConfig ),
m_colourMode( colourMode ),
m_verbosity( verbosity ),
m_customOptions( CATCH_MOVE( customOptions ) ) {}
Detail::unique_ptr<IStream> ReporterConfig::takeStream() && {
@@ -31,6 +33,7 @@ namespace Catch {
}
IConfig const * ReporterConfig::fullConfig() const { return m_fullConfig; }
ColourMode ReporterConfig::colourMode() const { return m_colourMode; }
Verbosity ReporterConfig::verbosity() const { return m_verbosity; }
std::map<std::string, std::string> const&
ReporterConfig::customOptions() const {
@@ -12,6 +12,7 @@
#include <catch2/catch_test_run_info.hpp>
#include <catch2/catch_totals.hpp>
#include <catch2/catch_assertion_result.hpp>
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/internal/catch_message_info.hpp>
#include <catch2/internal/catch_stringref.hpp>
#include <catch2/internal/catch_unique_ptr.hpp>
@@ -36,6 +37,7 @@ namespace Catch {
ReporterConfig( IConfig const* _fullConfig,
Detail::unique_ptr<IStream> _stream,
ColourMode colourMode,
Verbosity verbosity,
std::map<std::string, std::string> customOptions );
ReporterConfig( ReporterConfig&& ) = default;
@@ -45,12 +47,14 @@ namespace Catch {
Detail::unique_ptr<IStream> takeStream() &&;
IConfig const* fullConfig() const;
ColourMode colourMode() const;
Verbosity verbosity() const;
std::map<std::string, std::string> const& customOptions() const;
private:
Detail::unique_ptr<IStream> m_stream;
IConfig const* m_fullConfig;
ColourMode m_colourMode;
Verbosity m_verbosity;
std::map<std::string, std::string> m_customOptions;
};
@@ -19,8 +19,7 @@ namespace Catch {
class ITestCaseRegistry {
public:
virtual ~ITestCaseRegistry(); // = default
// TODO: this exists only for adding filenames to test cases -- let's expose this in a saner way later
virtual std::vector<TestCaseInfo* > const& getAllInfos() const = 0;
virtual void enableFilenameTags() = 0;
virtual std::vector<TestCaseHandle> const& getAllTests() const = 0;
virtual std::vector<TestCaseHandle> const& getAllTestsSorted( IConfig const& config ) const = 0;
};
@@ -75,9 +75,11 @@ namespace Catch {
auto const setRngSeed = [&]( std::string const& seed ) {
if( seed == "time" ) {
config.rngSeed = generateRandomSeed(GenerateFrom::Time);
config.rngSeedWasFixed = false;
return ParserResult::ok(ParseResultType::Matched);
} else if (seed == "random-device") {
config.rngSeed = generateRandomSeed(GenerateFrom::RandomDevice);
config.rngSeedWasFixed = false;
return ParserResult::ok(ParseResultType::Matched);
}
@@ -88,6 +90,7 @@ namespace Catch {
return ParserResult::runtimeError( "Could not parse '" + seed + "' as seed" );
}
config.rngSeed = *parsedSeed;
config.rngSeedWasFixed = true;
return ParserResult::ok( ParseResultType::Matched );
};
auto const setDefaultColourMode = [&]( std::string const& colourMode ) {
+79 -33
View File
@@ -9,6 +9,7 @@
#define CATCH_PREPROCESSOR_HPP_INCLUDED
#include <catch2/internal/catch_preprocessor_remove_parens.hpp>
#include <catch2/internal/catch_meta.hpp>
#if defined(__GNUC__)
// We need to silence "empty __VA_ARGS__ warning", and using just _Pragma does not work
@@ -22,6 +23,51 @@ namespace Catch {
struct priority_tag : priority_tag<N - 1> {};
template <>
struct priority_tag<0> {};
// This is a bunch of helpers for the templated test case handling.
// They should live elsewhere in the long run, but as an in-between
// step we toss them all here.
template <typename...> struct TypeList {};
template <typename... Ts>
constexpr auto get_wrapper( priority_tag<1> ) noexcept -> TypeList<Ts...> { return {}; }
template <template <typename...> class...> struct TemplateTypeList {};
// Clang 20 and 21 cannot handle an explicitly specified all-pack
// template-template parameter here ("conflicting deduction" regression,
// llvm/llvm-project#130778; fixed for Clang 22).
// Remove get_template_wrapper once Clang 21 is no longer supported.
template <template <typename...> class C, template <typename...> class... Cs>
constexpr auto get_template_wrapper( priority_tag<1> ) noexcept -> TemplateTypeList<C, Cs...> { return {}; }
template <typename...>
struct append;
template <typename T>
struct append<T> { using type = T; };
template <template <typename...> class L1, typename... E1, template <typename...> class L2, typename... E2, typename... Rest>
struct append<L1<E1...>, L2<E2...>, Rest...> { using type = typename append<L1<E1..., E2...>, Rest...>::type; };
template <template <typename...> class L1, typename... E1, typename... Rest>
struct append<L1<E1...>, TypeList<mpl_::na>, Rest...> { using type = L1<E1...>; };
template <template <typename...> class, typename>
struct convert;
template <template <typename...> class Final, template <typename...> class List, typename... Ts>
struct convert<Final, List<Ts...>> { using type = typename append<Final<>, TypeList<Ts>...>::type; };
// These are helpers for the PRODUCT templated test cases.
// Note that the _SIG macros (for NTTPs) also use specializations
// of these, but they have to use their own instances due to needing
// per-sig specializations and we have to keep these in their own
// unnamed namespace.
template <typename...>
struct rewrap;
template <template <typename...> class Container, template <typename...> class List, typename... elems>
struct rewrap<TemplateTypeList<Container>, List<elems...>> { using type = TypeList<Container<elems...>>; };
template <template <typename...> class Container, template <typename...> class List, class... Elems, typename... Elements>
struct rewrap<TemplateTypeList<Container>, List<Elems...>, Elements...> { using type = typename append<TypeList<Container<Elems...>>, typename rewrap<TemplateTypeList<Container>, Elements...>::type>::type; };
template <template <typename...> class, typename...>
struct create;
template <template <typename...> class Final, template <typename...> class... Containers, typename... Types>
struct create<Final, TemplateTypeList<Containers...>, TypeList<Types...>> { using type = typename append<Final<>, typename rewrap<TemplateTypeList<Containers>, Types...>::type...>::type; };
}
}
@@ -108,46 +154,42 @@ namespace Catch {
#define INTERNAL_CATCH_VA_NARGS_IMPL(_0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, N, ...) N
#define INTERNAL_CATCH_TYPE_GEN\
template<typename...> struct TypeList {};\
template<typename... Ts>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<1>) noexcept -> TypeList<Ts...> { return {}; }\
template<template<typename...> class...> struct TemplateTypeList{};\
template<template<typename...> class...Cs>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<1>) noexcept -> TemplateTypeList<Cs...> { return {}; }\
template<typename...>\
struct append;\
template<typename...>\
struct rewrap;\
template<template<typename...> class, typename...>\
struct create;\
template<template<typename...> class, typename>\
struct convert;\
\
template<typename T> \
struct append<T> { using type = T; };\
template< template<typename...> class L1, typename...E1, template<typename...> class L2, typename...E2, typename...Rest>\
struct append<L1<E1...>, L2<E2...>, Rest...> { using type = typename append<L1<E1...,E2...>, Rest...>::type; };\
template< template<typename...> class L1, typename...E1, typename...Rest>\
struct append<L1<E1...>, TypeList<mpl_::na>, Rest...> { using type = L1<E1...>; };\
#define INTERNAL_CATCH_TYPE_GEN \
/* We moved these into a central location and no longer create them
in each templated test's unnamed namespace, but we pull them in
with using to avoid qualifying all the references. */ \
using Catch::Detail::TypeList; \
using Catch::Detail::get_wrapper; \
using Catch::Detail::TemplateTypeList; \
using Catch::Detail::get_template_wrapper; \
using Catch::Detail::append; \
using Catch::Detail::convert;
// This stamps out the per test case specializations of wrapper handlers
// for _SIG (NTTP) macros inside their own namespace, so they can add
// their required specializations
#define INTERNAL_CATCH_NTTP_1( signature, ... ) \
template<INTERNAL_CATCH_REMOVE_PARENS(signature)> struct Nttp{};\
template<INTERNAL_CATCH_REMOVE_PARENS(signature)>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<0>) noexcept -> Nttp<__VA_ARGS__> { return {}; } \
template<template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class...> struct NttpTemplateTypeList{};\
template<template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class C, template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class...Cs>\
constexpr auto get_template_wrapper(Catch::Detail::priority_tag<0>) noexcept -> NttpTemplateTypeList<C, Cs...> { return {}; }
// This stamps out the per test case specializations of type-product
// machinery for the NTTP product macros.
#define INTERNAL_CATCH_NTTP_REWRAP_1( signature, ... ) \
template<typename...> \
struct rewrap; \
template<template<typename...> class, typename...> \
struct create; \
\
template< template<typename...> class Container, template<typename...> class List, typename...elems>\
struct rewrap<TemplateTypeList<Container>, List<elems...>> { using type = TypeList<Container<elems...>>; };\
template< template<typename...> class Container, template<typename...> class List, class...Elems, typename...Elements>\
struct rewrap<TemplateTypeList<Container>, List<Elems...>, Elements...> { using type = typename append<TypeList<Container<Elems...>>, typename rewrap<TemplateTypeList<Container>, Elements...>::type>::type; };\
\
template<template <typename...> class Final, template< typename...> class...Containers, typename...Types>\
struct create<Final, TemplateTypeList<Containers...>, TypeList<Types...>> { using type = typename append<Final<>, typename rewrap<TemplateTypeList<Containers>, Types...>::type...>::type; };\
template<template <typename...> class Final, template <typename...> class List, typename...Ts>\
struct convert<Final, List<Ts...>> { using type = typename append<Final<>,TypeList<Ts>...>::type; };
#define INTERNAL_CATCH_NTTP_1(signature, ...)\
template<INTERNAL_CATCH_REMOVE_PARENS(signature)> struct Nttp{};\
template<INTERNAL_CATCH_REMOVE_PARENS(signature)>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<0>) noexcept -> Nttp<__VA_ARGS__> { return {}; } \
template<template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class...> struct NttpTemplateTypeList{};\
template<template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class...Cs>\
constexpr auto get_wrapper(Catch::Detail::priority_tag<0>) noexcept -> NttpTemplateTypeList<Cs...> { return {}; } \
\
template< template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class Container, template<INTERNAL_CATCH_REMOVE_PARENS(signature)> class List, INTERNAL_CATCH_REMOVE_PARENS(signature)>\
struct rewrap<NttpTemplateTypeList<Container>, List<__VA_ARGS__>> { using type = TypeList<Container<__VA_ARGS__>>; };\
@@ -224,7 +266,9 @@ namespace Catch {
#ifndef CATCH_CONFIG_TRADITIONAL_MSVC_PREPROCESSOR
#define INTERNAL_CATCH_NTTP_0
#define INTERNAL_CATCH_NTTP_0_REWRAP using Catch::Detail::rewrap; using Catch::Detail::create;
#define INTERNAL_CATCH_NTTP_GEN(...) INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_1( __VA_ARGS__),INTERNAL_CATCH_NTTP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_0)
#define INTERNAL_CATCH_NTTP_REWRAP_GEN(...) INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1(__VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__),INTERNAL_CATCH_NTTP_REWRAP_1( __VA_ARGS__), INTERNAL_CATCH_NTTP_0_REWRAP)
#define INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD(TestName, ...) INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD1, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD0)(TestName, __VA_ARGS__)
#define INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD(TestName, ClassName, ...) INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD1, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD0)(TestName, ClassName, __VA_ARGS__)
#define INTERNAL_CATCH_NTTP_REG_METHOD_GEN(TestName, ...) INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD0, INTERNAL_CATCH_NTTP_REGISTER_METHOD0)(TestName, __VA_ARGS__)
@@ -234,7 +278,9 @@ namespace Catch {
#define INTERNAL_CATCH_REMOVE_PARENS_GEN(...) INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_REMOVE_PARENS_11_ARG,INTERNAL_CATCH_REMOVE_PARENS_10_ARG,INTERNAL_CATCH_REMOVE_PARENS_9_ARG,INTERNAL_CATCH_REMOVE_PARENS_8_ARG,INTERNAL_CATCH_REMOVE_PARENS_7_ARG,INTERNAL_CATCH_REMOVE_PARENS_6_ARG,INTERNAL_CATCH_REMOVE_PARENS_5_ARG,INTERNAL_CATCH_REMOVE_PARENS_4_ARG,INTERNAL_CATCH_REMOVE_PARENS_3_ARG,INTERNAL_CATCH_REMOVE_PARENS_2_ARG,INTERNAL_CATCH_REMOVE_PARENS_1_ARG)(__VA_ARGS__)
#else
#define INTERNAL_CATCH_NTTP_0(signature)
#define INTERNAL_CATCH_NTTP_0_REWRAP(signature) using Catch::Detail::rewrap; using Catch::Detail::create;
#define INTERNAL_CATCH_NTTP_GEN(...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_1,INTERNAL_CATCH_NTTP_1, INTERNAL_CATCH_NTTP_0)( __VA_ARGS__))
#define INTERNAL_CATCH_NTTP_REWRAP_GEN(...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL(__VA_ARGS__, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_REWRAP_1,INTERNAL_CATCH_NTTP_REWRAP_1, INTERNAL_CATCH_NTTP_0_REWRAP)( __VA_ARGS__))
#define INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD(TestName, ...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD1, INTERNAL_CATCH_DEFINE_SIG_TEST_METHOD0)(TestName, __VA_ARGS__))
#define INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD(TestName, ClassName, ...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X,INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD_X, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD1, INTERNAL_CATCH_DECLARE_SIG_TEST_METHOD0)(TestName, ClassName, __VA_ARGS__))
#define INTERNAL_CATCH_NTTP_REG_METHOD_GEN(TestName, ...) INTERNAL_CATCH_EXPAND_VARGS(INTERNAL_CATCH_VA_NARGS_IMPL( "dummy", __VA_ARGS__, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD, INTERNAL_CATCH_NTTP_REGISTER_METHOD0, INTERNAL_CATCH_NTTP_REGISTER_METHOD0)(TestName, __VA_ARGS__))
@@ -93,6 +93,18 @@ namespace Catch {
return {};
}
}
Optional<Verbosity> stringToVerbosity( StringRef verbosity ) {
if (verbosity == "quiet") { return Verbosity::Quiet;
} else if ( verbosity == "normal" ) {
return Verbosity::Normal;
} else if ( verbosity == "high" ) {
return Verbosity::High;
} else {
return {};
}
}
} // namespace Detail
@@ -100,6 +112,7 @@ namespace Catch {
return lhs.m_name == rhs.m_name &&
lhs.m_outputFileName == rhs.m_outputFileName &&
lhs.m_colourMode == rhs.m_colourMode &&
lhs.m_verbosity == rhs.m_verbosity &&
lhs.m_customOptions == rhs.m_customOptions;
}
@@ -111,6 +124,7 @@ namespace Catch {
std::map<std::string, std::string> kvPairs;
Optional<std::string> outputFileName;
Optional<ColourMode> colourMode;
Optional<Verbosity> verbosity;
// First part is always reporter name, so we skip it
for ( size_t i = 1; i < parts.size(); ++i ) {
@@ -148,6 +162,12 @@ namespace Catch {
if ( !colourMode ) {
return {};
}
} else if ( key == "verbosity" ) {
// Duplicated key
if ( verbosity ) { return {}; }
verbosity = Detail::stringToVerbosity( value );
// Parsing failed
if ( !verbosity ) { return {}; }
} else {
// Unrecognized option
return {};
@@ -157,6 +177,7 @@ namespace Catch {
return ReporterSpec{ CATCH_MOVE( parts[0] ),
CATCH_MOVE( outputFileName ),
CATCH_MOVE( colourMode ),
CATCH_MOVE( verbosity),
CATCH_MOVE( kvPairs ) };
}
@@ -164,10 +185,12 @@ ReporterSpec::ReporterSpec(
std::string name,
Optional<std::string> outputFileName,
Optional<ColourMode> colourMode,
Optional<Verbosity> verbosity,
std::map<std::string, std::string> customOptions ):
m_name( CATCH_MOVE( name ) ),
m_outputFileName( CATCH_MOVE( outputFileName ) ),
m_colourMode( CATCH_MOVE( colourMode ) ),
m_verbosity( CATCH_MOVE( verbosity ) ),
m_customOptions( CATCH_MOVE( customOptions ) ) {}
} // namespace Catch
@@ -25,6 +25,7 @@ namespace Catch {
std::vector<std::string> splitReporterSpec( StringRef reporterSpec );
Optional<ColourMode> stringToColourMode( StringRef colourMode );
Optional<Verbosity> stringToVerbosity( StringRef verbosity );
}
/**
@@ -39,6 +40,7 @@ namespace Catch {
std::string m_name;
Optional<std::string> m_outputFileName;
Optional<ColourMode> m_colourMode;
Optional<Verbosity> m_verbosity;
std::map<std::string, std::string> m_customOptions;
friend bool operator==( ReporterSpec const& lhs,
@@ -53,6 +55,7 @@ namespace Catch {
std::string name,
Optional<std::string> outputFileName,
Optional<ColourMode> colourMode,
Optional<Verbosity> verbosity,
std::map<std::string, std::string> customOptions );
std::string const& name() const { return m_name; }
@@ -63,13 +66,15 @@ namespace Catch {
Optional<ColourMode> const& colourMode() const { return m_colourMode; }
Optional<Verbosity> const& verbosity() const { return m_verbosity; }
std::map<std::string, std::string> const& customOptions() const {
return m_customOptions;
}
};
/**
* Parses provided reporter spec string into
* Parses provided reporter spec string into actual `ReporterSpec`
*
* Returns empty optional on errors, e.g.
* * field that is not first and not a key+value pair
@@ -127,6 +127,7 @@
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestName) { \
INTERNAL_CATCH_TYPE_GEN \
INTERNAL_CATCH_NTTP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature)) \
INTERNAL_CATCH_NTTP_REWRAP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature)) \
template<typename... Types> \
struct TestName { \
void reg_tests() { \
@@ -139,7 +140,7 @@
} \
}; \
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){ \
using TestInit = typename create<TestName, decltype(get_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type; \
using TestInit = typename create<TestName, decltype(get_template_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type; \
TestInit t; \
t.reg_tests(); \
return 0; \
@@ -223,7 +224,7 @@
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
TestNameClass<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(__VA_ARGS__)>();\
return 0;\
}();\
}();\
}\
}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
@@ -259,6 +260,7 @@
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestNameClass) {\
INTERNAL_CATCH_TYPE_GEN \
INTERNAL_CATCH_NTTP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature))\
INTERNAL_CATCH_NTTP_REWRAP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature))\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
@@ -271,7 +273,7 @@
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename create<TestNameClass, decltype(get_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type;\
using TestInit = typename create<TestNameClass, decltype(get_template_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
@@ -22,6 +22,9 @@
namespace Catch {
namespace {
// Picked small-ish number at random
static size_t kInitialTestCount = 120;
static void enforceNoDuplicateTestCases(
std::vector<TestCaseHandle> const& tests ) {
auto testInfoCmp = []( TestCaseInfo const* lhs,
@@ -123,17 +126,26 @@ namespace Catch {
return getRegistryHub().getTestCaseRegistry().getAllTestsSorted( config );
}
TestRegistry::TestRegistry() {
// We pre-reserve some reasonable number of tests to avoid the
// initial geometric growth churning during test registration.
m_handles.reserve( kInitialTestCount );
m_test_infos.reserve( kInitialTestCount );
m_invokers.reserve( kInitialTestCount );
}
TestRegistry::~TestRegistry() = default;
void TestRegistry::registerTest(Detail::unique_ptr<TestCaseInfo> testInfo, Detail::unique_ptr<ITestInvoker> testInvoker) {
m_handles.emplace_back(testInfo.get(), testInvoker.get());
m_viewed_test_infos.push_back(testInfo.get());
m_owned_test_infos.push_back(CATCH_MOVE(testInfo));
m_test_infos.push_back(CATCH_MOVE(testInfo));
m_invokers.push_back(CATCH_MOVE(testInvoker));
}
std::vector<TestCaseInfo*> const& TestRegistry::getAllInfos() const {
return m_viewed_test_infos;
void TestRegistry::enableFilenameTags() {
for (auto& info : m_test_infos) {
info->addFilenameTag();
}
}
std::vector<TestCaseHandle> const& TestRegistry::getAllTests() const {
@@ -31,20 +31,20 @@ namespace Catch {
class TestRegistry final : public ITestCaseRegistry {
public:
void registerTest( Detail::unique_ptr<TestCaseInfo> testInfo, Detail::unique_ptr<ITestInvoker> testInvoker );
void enableFilenameTags() override;
std::vector<TestCaseInfo*> const& getAllInfos() const override;
std::vector<TestCaseHandle> const& getAllTests() const override;
std::vector<TestCaseHandle> const& getAllTestsSorted( IConfig const& config ) const override;
TestRegistry();
~TestRegistry() override; // = default
private:
std::vector<Detail::unique_ptr<TestCaseInfo>> m_owned_test_infos;
// Keeps a materialized vector for `getAllInfos`.
// We should get rid of that eventually (see interface note)
std::vector<TestCaseInfo*> m_viewed_test_infos;
// Owns the test infos for handles
std::vector<Detail::unique_ptr<TestCaseInfo>> m_test_infos;
// Owns the test invokers for handles
std::vector<Detail::unique_ptr<ITestInvoker>> m_invokers;
std::vector<TestCaseHandle> m_handles;
mutable TestRunOrder m_currentSortOrder = TestRunOrder::Declared;
mutable std::vector<TestCaseHandle> m_sortedFunctions;
+77 -41
View File
@@ -9,70 +9,106 @@
#include <catch2/internal/catch_string_manip.hpp>
#include <catch2/catch_tostring.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <catch2/internal/catch_case_insensitive_comparisons.hpp>
#include <regex>
namespace Catch {
namespace {
constexpr StringRef caseSensitivitySuffix( CaseSensitive caseSensitivity ) {
return caseSensitivity == CaseSensitive::Yes
? StringRef{}
: " (case insensitive)"_sr;
}
} // namespace
namespace Matchers {
CasedString::CasedString( std::string const& str, CaseSensitive caseSensitivity )
: m_caseSensitivity( caseSensitivity ),
m_str( adjustString( str ) )
{}
std::string CasedString::adjustString( std::string const& str ) const {
return m_caseSensitivity == CaseSensitive::No
? toLower( str )
: str;
}
StringRef CasedString::caseSensitivitySuffix() const {
return m_caseSensitivity == CaseSensitive::Yes
? StringRef()
: " (case insensitive)"_sr;
}
StringMatcherBase::StringMatcherBase( std::string target,
StringRef operation,
CaseSensitive caseSensitivity ):
m_target( CATCH_MOVE( target ) ),
m_operation( operation ),
m_caseSensitivity( caseSensitivity ) {}
StringMatcherBase::StringMatcherBase( StringRef operation, CasedString const& comparator )
: m_comparator( comparator ),
m_operation( operation ) {
}
std::string StringMatcherBase::describe() const {
std::string description;
description.reserve(5 + m_operation.size() + m_comparator.m_str.size() +
m_comparator.caseSensitivitySuffix().size());
description.reserve(5 + m_operation.size() + m_target.size() +
caseSensitivitySuffix(m_caseSensitivity).size());
description += m_operation;
description += ": \"";
description += m_comparator.m_str;
description += m_target;
description += '"';
description += m_comparator.caseSensitivitySuffix();
description += caseSensitivitySuffix(m_caseSensitivity);
return description;
}
StringEqualsMatcher::StringEqualsMatcher( CasedString const& comparator ) : StringMatcherBase( "equals"_sr, comparator ) {}
StringEqualsMatcher::StringEqualsMatcher( std::string comparator, CaseSensitive caseSensitivity ):
StringMatcherBase( CATCH_MOVE( comparator ), "equals"_sr, caseSensitivity ) {}
bool StringEqualsMatcher::match( std::string const& source ) const {
return m_comparator.adjustString( source ) == m_comparator.m_str;
if (m_caseSensitivity == CaseSensitive::Yes) {
return m_target == source;
}
if (m_target.size() != source.size()) { return false; }
Catch::Detail::CaseInsensitiveEqualTo eq;
return eq( m_target, source );
}
StringContainsMatcher::StringContainsMatcher( CasedString const& comparator ) : StringMatcherBase( "contains"_sr, comparator ) {}
StringContainsMatcher::StringContainsMatcher(
std::string comparator, CaseSensitive caseSensitivity ):
StringMatcherBase( CATCH_MOVE( comparator ), "contains"_sr, caseSensitivity ) {}
bool StringContainsMatcher::match( std::string const& source ) const {
return contains( m_comparator.adjustString( source ), m_comparator.m_str );
if ( m_caseSensitivity == CaseSensitive::Yes ) {
return contains( source, m_target );
}
if ( source.size() < m_target.size() ) { return false; }
StringRef as_ref( source );
// The worst case of this is O(m*n), which is terrible, BUT:
// * The average case is much better, the worst case only happens rarely
// * We can implement BMH/other better searchers later if it matters
Catch::Detail::CaseInsensitiveEqualTo eq;
for (size_t i = 0; i < source.size(); ++i) {
const auto substr = as_ref.substr( i, m_target.size() );
bool found = eq( substr, m_target );
if ( found ) { return true; }
}
return false;
}
StartsWithMatcher::StartsWithMatcher( CasedString const& comparator ) : StringMatcherBase( "starts with"_sr, comparator ) {}
StartsWithMatcher::StartsWithMatcher( std::string comparator,
CaseSensitive caseSensitivity ):
StringMatcherBase( CATCH_MOVE( comparator ), "starts with"_sr, caseSensitivity ) {}
bool StartsWithMatcher::match( std::string const& source ) const {
return startsWith( m_comparator.adjustString( source ), m_comparator.m_str );
if ( m_caseSensitivity == CaseSensitive::Yes ) {
return startsWith( source, m_target );
}
if (source.size() < m_target.size()) { return false; }
Catch::Detail::CaseInsensitiveEqualTo eq;
return eq(
StringRef( source ).substr( 0, m_target.size() ), m_target );
}
EndsWithMatcher::EndsWithMatcher( CasedString const& comparator ) : StringMatcherBase( "ends with"_sr, comparator ) {}
EndsWithMatcher::EndsWithMatcher( std::string comparator,
CaseSensitive caseSensitivity ):
StringMatcherBase( CATCH_MOVE( comparator ), "ends with"_sr, caseSensitivity ) {}
bool EndsWithMatcher::match( std::string const& source ) const {
return endsWith( m_comparator.adjustString( source ), m_comparator.m_str );
if ( m_caseSensitivity == CaseSensitive::Yes ) {
return endsWith( source, m_target );
}
if ( source.size() < m_target.size() ) { return false; }
Catch::Detail::CaseInsensitiveEqualTo eq;
const size_t start_point = source.size() - m_target.size();
return eq( StringRef( source ).substr( start_point, m_target.size() ), m_target );
}
@@ -93,21 +129,21 @@ namespace Matchers {
}
StringEqualsMatcher Equals( std::string const& str, CaseSensitive caseSensitivity ) {
return StringEqualsMatcher( CasedString( str, caseSensitivity) );
StringEqualsMatcher Equals( std::string str, CaseSensitive caseSensitivity ) {
return StringEqualsMatcher( CATCH_MOVE( str ), caseSensitivity );
}
StringContainsMatcher ContainsSubstring( std::string const& str, CaseSensitive caseSensitivity ) {
return StringContainsMatcher( CasedString( str, caseSensitivity) );
StringContainsMatcher ContainsSubstring( std::string str, CaseSensitive caseSensitivity ) {
return StringContainsMatcher( CATCH_MOVE( str ), caseSensitivity );
}
EndsWithMatcher EndsWith( std::string const& str, CaseSensitive caseSensitivity ) {
return EndsWithMatcher( CasedString( str, caseSensitivity) );
EndsWithMatcher EndsWith( std::string str, CaseSensitive caseSensitivity ) {
return EndsWithMatcher( CATCH_MOVE( str ), caseSensitivity );
}
StartsWithMatcher StartsWith( std::string const& str, CaseSensitive caseSensitivity ) {
return StartsWithMatcher( CasedString( str, caseSensitivity) );
StartsWithMatcher StartsWith( std::string str, CaseSensitive caseSensitivity ) {
return StartsWithMatcher( CATCH_MOVE( str ), caseSensitivity );
}
RegexMatcher Matches(std::string const& regex, CaseSensitive caseSensitivity) {
return RegexMatcher(regex, caseSensitivity);
RegexMatcher Matches(std::string regex, CaseSensitive caseSensitivity) {
return RegexMatcher( CATCH_MOVE( regex ), caseSensitivity );
}
} // namespace Matchers
+17 -21
View File
@@ -17,44 +17,40 @@
namespace Catch {
namespace Matchers {
struct CasedString {
CasedString( std::string const& str, CaseSensitive caseSensitivity );
std::string adjustString( std::string const& str ) const;
StringRef caseSensitivitySuffix() const;
CaseSensitive m_caseSensitivity;
std::string m_str;
};
class StringMatcherBase : public MatcherBase<std::string> {
protected:
CasedString m_comparator;
std::string m_target;
StringRef m_operation;
CaseSensitive m_caseSensitivity;
StringMatcherBase( std::string target,
StringRef operation,
CaseSensitive caseSensitivity );
public:
StringMatcherBase( StringRef operation,
CasedString const& comparator );
std::string describe() const override;
};
class StringEqualsMatcher final : public StringMatcherBase {
public:
StringEqualsMatcher( CasedString const& comparator );
StringEqualsMatcher( std::string comparator, CaseSensitive caseSensitivity );
bool match( std::string const& source ) const override;
};
class StringContainsMatcher final : public StringMatcherBase {
public:
StringContainsMatcher( CasedString const& comparator );
StringContainsMatcher( std::string comparator,
CaseSensitive caseSensitivity );
bool match( std::string const& source ) const override;
};
class StartsWithMatcher final : public StringMatcherBase {
public:
StartsWithMatcher( CasedString const& comparator );
StartsWithMatcher( std::string comparator,
CaseSensitive caseSensitivity );
bool match( std::string const& source ) const override;
};
class EndsWithMatcher final : public StringMatcherBase {
public:
EndsWithMatcher( CasedString const& comparator );
EndsWithMatcher( std::string comparator,
CaseSensitive caseSensitivity );
bool match( std::string const& source ) const override;
};
@@ -69,15 +65,15 @@ namespace Matchers {
};
//! Creates matcher that accepts strings that are exactly equal to `str`
StringEqualsMatcher Equals( std::string const& str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
StringEqualsMatcher Equals( std::string str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
//! Creates matcher that accepts strings that contain `str`
StringContainsMatcher ContainsSubstring( std::string const& str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
StringContainsMatcher ContainsSubstring( std::string str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
//! Creates matcher that accepts strings that _end_ with `str`
EndsWithMatcher EndsWith( std::string const& str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
EndsWithMatcher EndsWith( std::string str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
//! Creates matcher that accepts strings that _start_ with `str`
StartsWithMatcher StartsWith( std::string const& str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
StartsWithMatcher StartsWith( std::string str, CaseSensitive caseSensitivity = CaseSensitive::Yes );
//! Creates matcher that accepts strings matching `regex`
RegexMatcher Matches( std::string const& regex, CaseSensitive caseSensitivity = CaseSensitive::Yes );
RegexMatcher Matches( std::string regex, CaseSensitive caseSensitivity = CaseSensitive::Yes );
} // namespace Matchers
} // namespace Catch
+1
View File
@@ -47,6 +47,7 @@ benchmark_headers = [
]
benchmark_sources = files(
'benchmark/catch_benchmark.cpp',
'benchmark/catch_chronometer.cpp',
'benchmark/detail/catch_analyse.cpp',
'benchmark/detail/catch_benchmark_function.cpp',
@@ -19,6 +19,7 @@ namespace Catch {
m_wrapped_stream( CATCH_MOVE(config).takeStream() ),
m_stream( m_wrapped_stream->stream() ),
m_colour( makeColourImpl( config.colourMode(), m_wrapped_stream.get() ) ),
m_verbosity( config.verbosity() ),
m_customOptions( config.customOptions() )
{}
@@ -26,12 +27,12 @@ namespace Catch {
void ReporterBase::listReporters(
std::vector<ReporterDescription> const& descriptions ) {
defaultListReporters( m_stream, descriptions, m_config->verbosity() );
defaultListReporters( m_stream, descriptions, m_verbosity );
}
void ReporterBase::listListeners(
std::vector<ListenerDescription> const& descriptions ) {
defaultListListeners( m_stream, descriptions, m_config->verbosity() );
defaultListListeners( m_stream, descriptions, m_verbosity );
}
void ReporterBase::listTests(std::vector<TestCaseHandle> const& tests) {
@@ -39,11 +40,11 @@ namespace Catch {
m_colour.get(),
tests,
m_config->hasTestFilters(),
m_config->verbosity());
m_verbosity);
}
void ReporterBase::listTags(std::vector<TagInfo> const& tags) {
defaultListTags( m_stream, tags, m_config->hasTestFilters(), m_config->verbosity() );
defaultListTags( m_stream, tags, m_config->hasTestFilters(), m_verbosity );
}
} // namespace Catch
@@ -35,6 +35,8 @@ namespace Catch {
std::ostream& m_stream;
//! Colour implementation this reporter was configured for
Detail::unique_ptr<ColourImpl> m_colour;
//! Verbosity configured for this reporter
Verbosity m_verbosity;
//! The custom reporter options user passed down to the reporter
std::map<std::string, std::string> m_customOptions;
+10 -4
View File
@@ -16,6 +16,8 @@
namespace Catch {
namespace {
static size_t kJsonOutputVersion = 2;
void writeSourceInfo( JsonObjectWriter& writer,
SourceLineInfo const& sourceInfo ) {
auto source_location_writer =
@@ -58,7 +60,7 @@ namespace Catch {
m_writers.emplace( Writer::Object );
auto& writer = m_objectWriters.top();
writer.write( "version"_sr ).write( 1 );
writer.write( "version"_sr ).write( kJsonOutputVersion );
{
auto metadata_writer = writer.write( "metadata"_sr ).writeObject();
@@ -344,14 +346,18 @@ namespace Catch {
auto const& info = test.getTestCaseInfo();
desc_writer.write( "name"_sr ).write( info.name );
desc_writer.write( "class-name"_sr ).write( info.className );
{
if (!info.className.empty()) {
desc_writer.write( "class-name"_sr ).write( info.className );
}
if ( m_verbosity >= Verbosity::Normal ) {
auto tag_writer = desc_writer.write( "tags"_sr ).writeArray();
for ( auto const& tag : info.tags ) {
tag_writer.write( tag.original );
}
}
writeSourceInfo( desc_writer, info.lineInfo );
if ( m_verbosity >= Verbosity::High) {
writeSourceInfo( desc_writer, info.lineInfo );
}
}
}
void JsonReporter::listTags( std::vector<TagInfo> const& tags ) {
@@ -85,9 +85,6 @@ namespace Catch {
std::stack<Writer> m_writers{};
bool m_startedListing = false;
// std::size_t m_sectionDepth = 0;
// std::size_t m_sectionStarted = 0;
};
} // namespace Catch
+48 -15
View File
@@ -184,6 +184,13 @@ set_tests_properties(List::Tests::Output PROPERTIES
PASS_REGULAR_EXPRESSION "[0-9]+ test cases"
FAIL_REGULAR_EXPRESSION "Hidden Test"
)
add_test(NAME VerbosityIsPerReporter COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high --reporter console::verbosity=quiet)
set_tests_properties(VerbosityIsPerReporter
PROPERTIES
FAIL_REGULAR_EXPRESSION "\.cpp"
)
# This should be equivalent to the old --list-test-names-only and be usable
# with --input-file.
add_test(NAME List::Tests::Quiet COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity quiet)
@@ -193,6 +200,7 @@ set_tests_properties(List::Tests::Quiet PROPERTIES
PASS_REGULAR_EXPRESSION "\"#1905 -- test spec parser properly clears internal state between compound tests\"[\r\n]"
FAIL_REGULAR_EXPRESSION "[ \t]\"#1905 -- test spec parser properly clears internal state between compound tests\""
)
add_test(NAME List::Tests::ExitCode COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high)
add_test(NAME List::Tests::XmlOutput COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high -r xml)
set_tests_properties(List::Tests::XmlOutput PROPERTIES
@@ -200,6 +208,21 @@ set_tests_properties(List::Tests::XmlOutput PROPERTIES
FAIL_REGULAR_EXPRESSION "[0-9]+ test cases"
)
add_test(NAME List::Tests::Json::Normal COMMAND $<TARGET_FILE:SelfTest> --list-tests -r json)
set_tests_properties(List::Tests::Json::Normal PROPERTIES
PASS_REGULAR_EXPRESSION "\"tags\": \\["
FAIL_REGULAR_EXPRESSION "\"source-location\": {"
)
add_test(NAME List::Tests::Json::Quiet COMMAND $<TARGET_FILE:SelfTest> --list-tests -r json::verbosity=quiet)
set_tests_properties(List::Tests::Json::Quiet PROPERTIES
PASS_REGULAR_EXPRESSION "\"name\": \""
FAIL_REGULAR_EXPRESSION "\"tags\": \\["
)
add_test(NAME List::Tests::Json::High COMMAND $<TARGET_FILE:SelfTest> --list-tests -r json::verbosity=high)
set_tests_properties(List::Tests::Json::High PROPERTIES
PASS_REGULAR_EXPRESSION "\"source-location\": {"
)
add_test(NAME List::Tags::Output COMMAND $<TARGET_FILE:SelfTest> --list-tags)
set_tests_properties(List::Tags::Output PROPERTIES
PASS_REGULAR_EXPRESSION "[0-9]+ tags"
@@ -210,6 +233,11 @@ set_tests_properties(List::Tags::XmlOutput PROPERTIES
PASS_REGULAR_EXPRESSION "<Count>18</Count>"
FAIL_REGULAR_EXPRESSION "[0-9]+ tags"
)
add_test(NAME List::Tags::JsonOutput COMMAND $<TARGET_FILE:SelfTest> --list-tags -r json)
set_tests_properties(List::Tags::JsonOutput PROPERTIES
PASS_REGULAR_EXPRESSION "\"count\": 18"
FAIL_REGULAR_EXPRESSION "[0-9]+ tags"
)
add_test(NAME List::Reporters::Output COMMAND $<TARGET_FILE:SelfTest> --list-reporters)
set_tests_properties(List::Reporters::Output PROPERTIES PASS_REGULAR_EXPRESSION "Available reporters:")
@@ -219,6 +247,12 @@ set_tests_properties(List::Reporters::XmlOutput PROPERTIES
PASS_REGULAR_EXPRESSION "<Name>compact</Name>"
FAIL_REGULAR_EXPRESSION "Available reporters:"
)
add_test(NAME List::Reporters::JsonOutput COMMAND $<TARGET_FILE:SelfTest> --list-reporters -r json)
set_tests_properties(List::Reporters::JsonOutput PROPERTIES
PASS_REGULAR_EXPRESSION "\"name\": \"compact\""
FAIL_REGULAR_EXPRESSION "Available reporters:"
)
add_test(NAME List::Listeners::Output
COMMAND
@@ -243,6 +277,18 @@ set_tests_properties(List::Listeners::XmlOutput
PASS_REGULAR_EXPRESSION "<RegisteredListeners>"
FAIL_REGULAR_EXPRESSION "Registered listeners:"
)
add_test(NAME List::Listeners::JsonOutput
COMMAND
$<TARGET_FILE:SelfTest>
--list-listeners
--reporter json
)
set_tests_properties(List::Listeners::JsonOutput
PROPERTIES
PASS_REGULAR_EXPRESSION "\"listeners\": \\["
FAIL_REGULAR_EXPRESSION "Registered listeners:"
)
add_test(NAME NoAssertions COMMAND $<TARGET_FILE:SelfTest> -w NoAssertions "An empty test with no assertions")
set_tests_properties(NoAssertions PROPERTIES PASS_REGULAR_EXPRESSION "No assertions in test case")
@@ -440,19 +486,6 @@ set_tests_properties("Benchmarking::SkipBenchmarkMacros"
FAIL_REGULAR_EXPRESSION "benchmark name"
)
add_test(NAME "Benchmarking::FailureReporting::OptimizedOut"
COMMAND
$<TARGET_FILE:SelfTest> "Failing benchmarks" -c "empty" -r xml
# This test only makes sense with the optimizer being enabled when
# the tests are being compiled.
CONFIGURATIONS Release
)
set_tests_properties("Benchmarking::FailureReporting::OptimizedOut"
PROPERTIES
PASS_REGULAR_EXPRESSION "could not measure benchmark\, maybe it was optimized away"
FAIL_REGULAR_EXPRESSION "successes=\"1\""
)
add_test(NAME "Benchmarking::FailureReporting::ThrowingBenchmark"
COMMAND
$<TARGET_FILE:SelfTest> "Failing benchmarks" -c "throw" -r xml
@@ -626,11 +659,11 @@ if(CATCH_ENABLE_CMAKE_HELPER_TESTS)
LABELS "uses-python"
)
add_test(NAME "CMakeHelper::PrepareCommand"
add_test(NAME "CMakeHelper::PrepareCommandFragment"
COMMAND
"${CMAKE_COMMAND}"
"-DCATCH_ADD_TESTS_SCRIPT=${CATCH_DIR}/extras/CatchAddTests.cmake"
-P "${CMAKE_CURRENT_LIST_DIR}/TestScripts/DiscoverTests/TestPrepareCommand.cmake"
-P "${CMAKE_CURRENT_LIST_DIR}/TestScripts/DiscoverTests/TestPrepareCommandFragment.cmake"
)
add_test(NAME "CMakeHelper::DecomposeJsonArray"
+31
View File
@@ -21,6 +21,37 @@ set_tests_properties(TestSharding::OverlyLargeShardIndex
PASS_REGULAR_EXPRESSION "The shard count \\(5\\) must be greater than the shard index \\(5\\)"
)
set(CATCH_SHARDING_WARNING_REGEX "Warning: using sharding .* with random order")
add_test(
NAME TestSharding::WarningOnRandomOrderWithoutFixedSeed
COMMAND $<TARGET_FILE:SelfTest> --shard-index 0 --shard-count 2 --list-tests
)
set_tests_properties(TestSharding::WarningOnRandomOrderWithoutFixedSeed
PROPERTIES
PASS_REGULAR_EXPRESSION "${CATCH_SHARDING_WARNING_REGEX}"
)
add_test(
NAME TestSharding::NoWarningOnRandomOrderWithFixedSeed
COMMAND $<TARGET_FILE:SelfTest> --shard-index 0 --shard-count 2 --rng-seed 12345 --list-tests
)
set_tests_properties(TestSharding::NoWarningOnRandomOrderWithFixedSeed
PROPERTIES
FAIL_REGULAR_EXPRESSION "${CATCH_SHARDING_WARNING_REGEX}"
)
foreach(shardOrder decl lex)
add_test(
NAME TestSharding::NoWarningOn${shardOrder}OrderWithoutFixedSeed
COMMAND $<TARGET_FILE:SelfTest> --shard-index 0 --shard-count 2 --order ${shardOrder} --list-tests
)
set_tests_properties(TestSharding::NoWarningOn${shardOrder}OrderWithoutFixedSeed
PROPERTIES
FAIL_REGULAR_EXPRESSION "${CATCH_SHARDING_WARNING_REGEX}"
)
endforeach()
# The MinDuration reporting tests do not need separate compilation, but
# they have non-trivial execution time, so they are categorized as
# extra tests, so that they are run less.
@@ -631,7 +631,7 @@ Condition.tests.cpp:<line number>: passed: x == Approx( 1.3 ) for: 1.30000000000
==
Approx( 1.30000000000000004 )
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), Equals( "this string contains 'abc' as a substring" ) for: "this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring" (case insensitive)
Matchers.tests.cpp:<line number>: failed: testStringForMatching(), Equals( "this string contains 'ABC' as a substring" ) for: "this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring"
Matchers.tests.cpp:<line number>: failed: testStringForMatching(), Equals( "something else", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "something else" (case insensitive)
ToStringGeneral.tests.cpp:<line number>: passed: ::Catch::Detail::stringify(WhatException{}) == "This exception has overridden what() method" for: "This exception has overridden what() method"
@@ -654,11 +654,11 @@ Matchers.tests.cpp:<line number>: passed: throwsDerivedException(), DerivedExcep
Matchers.tests.cpp:<line number>: passed: throwsDerivedException(), DerivedException, MessageMatches( !StartsWith( "::what" ) ) for: DerivedException::what matches "not starts with: "::what""
Matchers.tests.cpp:<line number>: passed: throwsSpecialException( 2 ), SpecialException, MessageMatches( StartsWith( "Special" ) ) for: SpecialException::what matches "starts with: "Special""
Exception.tests.cpp:<line number>: passed: thisThrows(), "expected exception" for: "expected exception" equals: "expected exception"
Exception.tests.cpp:<line number>: passed: thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) for: "expected exception" equals: "expected exception" (case insensitive)
Exception.tests.cpp:<line number>: passed: thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) for: "expected exception" equals: "expecteD Exception" (case insensitive)
Exception.tests.cpp:<line number>: passed: thisThrows(), StartsWith( "expected" ) for: "expected exception" starts with: "expected"
Exception.tests.cpp:<line number>: passed: thisThrows(), EndsWith( "exception" ) for: "expected exception" ends with: "exception"
Exception.tests.cpp:<line number>: passed: thisThrows(), ContainsSubstring( "except" ) for: "expected exception" contains: "except"
Exception.tests.cpp:<line number>: passed: thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) for: "expected exception" contains: "except" (case insensitive)
Exception.tests.cpp:<line number>: passed: thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) for: "expected exception" contains: "exCept" (case insensitive)
ToString.tests.cpp:<line number>: passed: tos == tos for: { stringification failed with an exception: "Invalid" }
==
{ stringification failed with an exception: "Invalid" }
@@ -1531,9 +1531,9 @@ CmdLine.tests.cpp:<line number>: passed: config.noThrow == false for: false == f
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications.empty() for: true
CmdLine.tests.cpp:<line number>: passed: !(cfg.hasTestFilters()) for: !false
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs().size() == 1 for: 1 == 1
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs().size() == 1 for: 1 == 1
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: result for: {?}
CmdLine.tests.cpp:<line number>: passed: cfg.hasTestFilters() for: true
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("notIncluded")) == false for: false == false
@@ -1547,21 +1547,21 @@ CmdLine.tests.cpp:<line number>: passed: cfg.hasTestFilters() for: true
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("test1")) == false for: false == false
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("alwaysIncluded")) for: true
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: !result for: true
CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Unrecognized reporter") for: "Unrecognized reporter, 'unsupported'. Check available with --list-reporters" contains: "Unrecognized reporter"
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }) for: {?}
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }) for: {?}
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: !result for: true
CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Only one reporter may have unspecified output file.") for: "Only one reporter may have unspecified output file." contains: "Only one reporter may have unspecified output file."
CmdLine.tests.cpp:<line number>: passed: cli.parse({"test", "-b"}) for: {?}
@@ -1696,7 +1696,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fa
" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: console'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fakeTag"s) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1713,7 +1713,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fak
]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fake reporter"s) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1728,7 +1728,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fak
]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1738,14 +1738,9 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fa
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
@@ -1917,11 +1912,11 @@ ToStringGeneral.tests.cpp:<line number>: passed: Catch::Detail::stringify(arr) =
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "string" ) for: "this string contains 'abc' as a substring" contains: "string"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "string", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "string" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "abc" ) for: "this string contains 'abc' as a substring" contains: "abc"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "abc" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "aBC" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), StartsWith( "this" ) for: "this string contains 'abc' as a substring" starts with: "this"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" starts with: "this" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" starts with: "THIS" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), EndsWith( "substring" ) for: "this string contains 'abc' as a substring" ends with: "substring"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: " substring" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: " SuBsTrInG" (case insensitive)
String.tests.cpp:<line number>: passed: empty.empty() for: true
String.tests.cpp:<line number>: passed: empty.size() == 0 for: 0 == 0
String.tests.cpp:<line number>: passed: std::strcmp( empty.data(), "" ) == 0 for: 0 == 0
@@ -629,7 +629,7 @@ Condition.tests.cpp:<line number>: passed: x == Approx( 1.3 ) for: 1.30000000000
==
Approx( 1.30000000000000004 )
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), Equals( "this string contains 'abc' as a substring" ) for: "this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring" (case insensitive)
Matchers.tests.cpp:<line number>: failed: testStringForMatching(), Equals( "this string contains 'ABC' as a substring" ) for: "this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring"
Matchers.tests.cpp:<line number>: failed: testStringForMatching(), Equals( "something else", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "something else" (case insensitive)
ToStringGeneral.tests.cpp:<line number>: passed: ::Catch::Detail::stringify(WhatException{}) == "This exception has overridden what() method" for: "This exception has overridden what() method"
@@ -652,11 +652,11 @@ Matchers.tests.cpp:<line number>: passed: throwsDerivedException(), DerivedExcep
Matchers.tests.cpp:<line number>: passed: throwsDerivedException(), DerivedException, MessageMatches( !StartsWith( "::what" ) ) for: DerivedException::what matches "not starts with: "::what""
Matchers.tests.cpp:<line number>: passed: throwsSpecialException( 2 ), SpecialException, MessageMatches( StartsWith( "Special" ) ) for: SpecialException::what matches "starts with: "Special""
Exception.tests.cpp:<line number>: passed: thisThrows(), "expected exception" for: "expected exception" equals: "expected exception"
Exception.tests.cpp:<line number>: passed: thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) for: "expected exception" equals: "expected exception" (case insensitive)
Exception.tests.cpp:<line number>: passed: thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) for: "expected exception" equals: "expecteD Exception" (case insensitive)
Exception.tests.cpp:<line number>: passed: thisThrows(), StartsWith( "expected" ) for: "expected exception" starts with: "expected"
Exception.tests.cpp:<line number>: passed: thisThrows(), EndsWith( "exception" ) for: "expected exception" ends with: "exception"
Exception.tests.cpp:<line number>: passed: thisThrows(), ContainsSubstring( "except" ) for: "expected exception" contains: "except"
Exception.tests.cpp:<line number>: passed: thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) for: "expected exception" contains: "except" (case insensitive)
Exception.tests.cpp:<line number>: passed: thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) for: "expected exception" contains: "exCept" (case insensitive)
ToString.tests.cpp:<line number>: passed: tos == tos for: { stringification failed with an exception: "Invalid" }
==
{ stringification failed with an exception: "Invalid" }
@@ -1529,9 +1529,9 @@ CmdLine.tests.cpp:<line number>: passed: config.noThrow == false for: false == f
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications.empty() for: true
CmdLine.tests.cpp:<line number>: passed: !(cfg.hasTestFilters()) for: !false
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs().size() == 1 for: 1 == 1
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs().size() == 1 for: 1 == 1
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: result for: {?}
CmdLine.tests.cpp:<line number>: passed: cfg.hasTestFilters() for: true
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("notIncluded")) == false for: false == false
@@ -1545,21 +1545,21 @@ CmdLine.tests.cpp:<line number>: passed: cfg.hasTestFilters() for: true
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("test1")) == false for: false == false
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("alwaysIncluded")) for: true
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: !result for: true
CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Unrecognized reporter") for: "Unrecognized reporter, 'unsupported'. Check available with --list-reporters" contains: "Unrecognized reporter"
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }) for: {?}
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }) for: {?}
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: !result for: true
CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Only one reporter may have unspecified output file.") for: "Only one reporter may have unspecified output file." contains: "Only one reporter may have unspecified output file."
CmdLine.tests.cpp:<line number>: passed: cli.parse({"test", "-b"}) for: {?}
@@ -1694,7 +1694,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fa
" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: console'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fakeTag"s) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1711,7 +1711,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fak
]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fake reporter"s) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1726,7 +1726,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fak
]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1736,14 +1736,9 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fa
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
@@ -1910,11 +1905,11 @@ ToStringGeneral.tests.cpp:<line number>: passed: Catch::Detail::stringify(arr) =
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "string" ) for: "this string contains 'abc' as a substring" contains: "string"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "string", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "string" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "abc" ) for: "this string contains 'abc' as a substring" contains: "abc"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "abc" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "aBC" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), StartsWith( "this" ) for: "this string contains 'abc' as a substring" starts with: "this"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" starts with: "this" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" starts with: "THIS" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), EndsWith( "substring" ) for: "this string contains 'abc' as a substring" ends with: "substring"
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: " substring" (case insensitive)
Matchers.tests.cpp:<line number>: passed: testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: " SuBsTrInG" (case insensitive)
String.tests.cpp:<line number>: passed: empty.empty() for: true
String.tests.cpp:<line number>: passed: empty.size() == 0 for: 0 == 0
String.tests.cpp:<line number>: passed: std::strcmp( empty.data(), "" ) == 0 for: 0 == 0
@@ -4521,7 +4521,7 @@ Matchers.tests.cpp:<line number>: PASSED:
CHECK_THAT( testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) )
with expansion:
"this string contains 'abc' as a substring" equals: "this string contains
'abc' as a substring" (case insensitive)
'ABC' as a substring" (case insensitive)
-------------------------------------------------------------------------------
Equals string matcher
@@ -4679,7 +4679,7 @@ Exception.tests.cpp:<line number>
Exception.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_WITH( thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) )
with expansion:
"expected exception" equals: "expected exception" (case insensitive)
"expected exception" equals: "expecteD Exception" (case insensitive)
-------------------------------------------------------------------------------
Exception messages can be tested for
@@ -4706,7 +4706,7 @@ with expansion:
Exception.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_WITH( thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) )
with expansion:
"expected exception" contains: "except" (case insensitive)
"expected exception" contains: "exCept" (case insensitive)
-------------------------------------------------------------------------------
Exception thrown inside stringify does not fail the test
@@ -9977,7 +9977,7 @@ with expansion:
1 == 1
CmdLine.tests.cpp:<line number>: PASSED:
CHECK( cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } )
CHECK( cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } )
with expansion:
{?} == {?}
@@ -9987,7 +9987,7 @@ with expansion:
1 == 1
CmdLine.tests.cpp:<line number>: PASSED:
CHECK( cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } )
CHECK( cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } )
with expansion:
{?} == {?}
@@ -10091,7 +10091,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10113,7 +10113,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10135,7 +10135,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10176,7 +10176,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10198,7 +10198,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10219,7 +10219,7 @@ with expansion:
{?}
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } )
with expansion:
{ {?}, {?} } == { {?}, {?} }
@@ -10238,7 +10238,7 @@ with expansion:
{?}
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } )
with expansion:
{ {?}, {?} } == { {?}, {?} }
@@ -11233,7 +11233,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring("fakeTag"s) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11273,7 +11273,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring("fake reporter"s) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11311,7 +11311,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11321,14 +11321,9 @@ with expansion:
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" )
with message:
@@ -12200,7 +12195,7 @@ with expansion:
Matchers.tests.cpp:<line number>: PASSED:
CHECK_THAT( testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) )
with expansion:
"this string contains 'abc' as a substring" contains: "abc" (case
"this string contains 'abc' as a substring" contains: "aBC" (case
insensitive)
Matchers.tests.cpp:<line number>: PASSED:
@@ -12211,7 +12206,7 @@ with expansion:
Matchers.tests.cpp:<line number>: PASSED:
CHECK_THAT( testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) )
with expansion:
"this string contains 'abc' as a substring" starts with: "this" (case
"this string contains 'abc' as a substring" starts with: "THIS" (case
insensitive)
Matchers.tests.cpp:<line number>: PASSED:
@@ -12222,7 +12217,7 @@ with expansion:
Matchers.tests.cpp:<line number>: PASSED:
CHECK_THAT( testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) )
with expansion:
"this string contains 'abc' as a substring" ends with: " substring" (case
"this string contains 'abc' as a substring" ends with: " SuBsTrInG" (case
insensitive)
-------------------------------------------------------------------------------
@@ -4519,7 +4519,7 @@ Matchers.tests.cpp:<line number>: PASSED:
CHECK_THAT( testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) )
with expansion:
"this string contains 'abc' as a substring" equals: "this string contains
'abc' as a substring" (case insensitive)
'ABC' as a substring" (case insensitive)
-------------------------------------------------------------------------------
Equals string matcher
@@ -4677,7 +4677,7 @@ Exception.tests.cpp:<line number>
Exception.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_WITH( thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) )
with expansion:
"expected exception" equals: "expected exception" (case insensitive)
"expected exception" equals: "expecteD Exception" (case insensitive)
-------------------------------------------------------------------------------
Exception messages can be tested for
@@ -4704,7 +4704,7 @@ with expansion:
Exception.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_WITH( thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) )
with expansion:
"expected exception" contains: "except" (case insensitive)
"expected exception" contains: "exCept" (case insensitive)
-------------------------------------------------------------------------------
Exception thrown inside stringify does not fail the test
@@ -9975,7 +9975,7 @@ with expansion:
1 == 1
CmdLine.tests.cpp:<line number>: PASSED:
CHECK( cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } )
CHECK( cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } )
with expansion:
{?} == {?}
@@ -9985,7 +9985,7 @@ with expansion:
1 == 1
CmdLine.tests.cpp:<line number>: PASSED:
CHECK( cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } )
CHECK( cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } )
with expansion:
{?} == {?}
@@ -10089,7 +10089,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10111,7 +10111,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10133,7 +10133,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10174,7 +10174,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10196,7 +10196,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10217,7 +10217,7 @@ with expansion:
{?}
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } )
with expansion:
{ {?}, {?} } == { {?}, {?} }
@@ -10236,7 +10236,7 @@ with expansion:
{?}
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } )
with expansion:
{ {?}, {?} } == { {?}, {?} }
@@ -11231,7 +11231,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring("fakeTag"s) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11271,7 +11271,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring("fake reporter"s) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11309,7 +11309,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11319,14 +11319,9 @@ with expansion:
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" )
with message:
@@ -12193,7 +12188,7 @@ with expansion:
Matchers.tests.cpp:<line number>: PASSED:
CHECK_THAT( testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) )
with expansion:
"this string contains 'abc' as a substring" contains: "abc" (case
"this string contains 'abc' as a substring" contains: "aBC" (case
insensitive)
Matchers.tests.cpp:<line number>: PASSED:
@@ -12204,7 +12199,7 @@ with expansion:
Matchers.tests.cpp:<line number>: PASSED:
CHECK_THAT( testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) )
with expansion:
"this string contains 'abc' as a substring" starts with: "this" (case
"this string contains 'abc' as a substring" starts with: "THIS" (case
insensitive)
Matchers.tests.cpp:<line number>: PASSED:
@@ -12215,7 +12210,7 @@ with expansion:
Matchers.tests.cpp:<line number>: PASSED:
CHECK_THAT( testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) )
with expansion:
"this string contains 'abc' as a substring" ends with: " substring" (case
"this string contains 'abc' as a substring" ends with: " SuBsTrInG" (case
insensitive)
-------------------------------------------------------------------------------
+18 -18
View File
@@ -1131,7 +1131,7 @@ ok {test-number} - x == Approx( 1.3 ) for: 1.30000000000000027 == Approx( 1.3000
# Equals
ok {test-number} - testStringForMatching(), Equals( "this string contains 'abc' as a substring" ) for: "this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring"
# Equals
ok {test-number} - testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring" (case insensitive)
ok {test-number} - testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring" (case insensitive)
# Equals string matcher
not ok {test-number} - testStringForMatching(), Equals( "this string contains 'ABC' as a substring" ) for: "this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring"
# Equals string matcher
@@ -1169,7 +1169,7 @@ ok {test-number} - throwsSpecialException( 2 ), SpecialException, MessageMatches
# Exception messages can be tested for
ok {test-number} - thisThrows(), "expected exception" for: "expected exception" equals: "expected exception"
# Exception messages can be tested for
ok {test-number} - thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) for: "expected exception" equals: "expected exception" (case insensitive)
ok {test-number} - thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) for: "expected exception" equals: "expecteD Exception" (case insensitive)
# Exception messages can be tested for
ok {test-number} - thisThrows(), StartsWith( "expected" ) for: "expected exception" starts with: "expected"
# Exception messages can be tested for
@@ -1177,7 +1177,7 @@ ok {test-number} - thisThrows(), EndsWith( "exception" ) for: "expected exceptio
# Exception messages can be tested for
ok {test-number} - thisThrows(), ContainsSubstring( "except" ) for: "expected exception" contains: "except"
# Exception messages can be tested for
ok {test-number} - thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) for: "expected exception" contains: "except" (case insensitive)
ok {test-number} - thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) for: "expected exception" contains: "exCept" (case insensitive)
# Exception thrown inside stringify does not fail the test
ok {test-number} - tos == tos for: { stringification failed with an exception: "Invalid" } == { stringification failed with an exception: "Invalid" }
# Exceptions matchers
@@ -2515,11 +2515,11 @@ ok {test-number} - !(cfg.hasTestFilters()) for: !false
# Process can be configured on command line
ok {test-number} - cfg.getReporterSpecs().size() == 1 for: 1 == 1
# Process can be configured on command line
ok {test-number} - cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } for: {?} == {?}
ok {test-number} - cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } for: {?} == {?}
# Process can be configured on command line
ok {test-number} - cfg.getProcessedReporterSpecs().size() == 1 for: 1 == 1
# Process can be configured on command line
ok {test-number} - cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } for: {?} == {?}
ok {test-number} - cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } for: {?} == {?}
# Process can be configured on command line
ok {test-number} - result for: {?}
# Process can be configured on command line
@@ -2547,15 +2547,15 @@ ok {test-number} - cfg.testSpec().matches(*fakeTestCase("alwaysIncluded")) for:
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - !result for: true
# Process can be configured on command line
@@ -2563,19 +2563,19 @@ ok {test-number} - result.errorMessage(), ContainsSubstring("Unrecognized report
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }) for: {?}
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
# Process can be configured on command line
ok {test-number} - cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }) for: {?}
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
# Process can be configured on command line
ok {test-number} - !result for: true
# Process can be configured on command line
@@ -2767,15 +2767,15 @@ ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && Cont
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring("fakeTag"s) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tags": [ { "aliases": [ "fakeTag" ], "count": 1 } ]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring("fakeTag"s) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tags": [ { "aliases": [ "fakeTag" ], "count": 1 } ]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring("fake reporter"s) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "reporters": [ { "name": "fake reporter", "description": "fake description" } ]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring("fake reporter"s) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "reporters": [ { "name": "fake reporter", "description": "fake description" } ]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tests": [ { "name": "fake test name", "class-name": "", "tags": [ "fakeTestTag" ], "source-location": { "filename": "fake-file.cpp", "line": 123456789 } } ]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tests": [ { "name": "fake test name", "tags": [ "fakeTestTag" ] } ]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
@@ -2916,15 +2916,15 @@ ok {test-number} - testStringForMatching(), ContainsSubstring( "string", Catch::
# String matchers
ok {test-number} - testStringForMatching(), ContainsSubstring( "abc" ) for: "this string contains 'abc' as a substring" contains: "abc"
# String matchers
ok {test-number} - testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "abc" (case insensitive)
ok {test-number} - testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "aBC" (case insensitive)
# String matchers
ok {test-number} - testStringForMatching(), StartsWith( "this" ) for: "this string contains 'abc' as a substring" starts with: "this"
# String matchers
ok {test-number} - testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" starts with: "this" (case insensitive)
ok {test-number} - testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" starts with: "THIS" (case insensitive)
# String matchers
ok {test-number} - testStringForMatching(), EndsWith( "substring" ) for: "this string contains 'abc' as a substring" ends with: "substring"
# String matchers
ok {test-number} - testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: " substring" (case insensitive)
ok {test-number} - testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: " SuBsTrInG" (case insensitive)
# StringRef
ok {test-number} - empty.empty() for: true
# StringRef
@@ -1129,7 +1129,7 @@ ok {test-number} - x == Approx( 1.3 ) for: 1.30000000000000027 == Approx( 1.3000
# Equals
ok {test-number} - testStringForMatching(), Equals( "this string contains 'abc' as a substring" ) for: "this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring"
# Equals
ok {test-number} - testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring" (case insensitive)
ok {test-number} - testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring" (case insensitive)
# Equals string matcher
not ok {test-number} - testStringForMatching(), Equals( "this string contains 'ABC' as a substring" ) for: "this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring"
# Equals string matcher
@@ -1167,7 +1167,7 @@ ok {test-number} - throwsSpecialException( 2 ), SpecialException, MessageMatches
# Exception messages can be tested for
ok {test-number} - thisThrows(), "expected exception" for: "expected exception" equals: "expected exception"
# Exception messages can be tested for
ok {test-number} - thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) for: "expected exception" equals: "expected exception" (case insensitive)
ok {test-number} - thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No ) for: "expected exception" equals: "expecteD Exception" (case insensitive)
# Exception messages can be tested for
ok {test-number} - thisThrows(), StartsWith( "expected" ) for: "expected exception" starts with: "expected"
# Exception messages can be tested for
@@ -1175,7 +1175,7 @@ ok {test-number} - thisThrows(), EndsWith( "exception" ) for: "expected exceptio
# Exception messages can be tested for
ok {test-number} - thisThrows(), ContainsSubstring( "except" ) for: "expected exception" contains: "except"
# Exception messages can be tested for
ok {test-number} - thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) for: "expected exception" contains: "except" (case insensitive)
ok {test-number} - thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No ) for: "expected exception" contains: "exCept" (case insensitive)
# Exception thrown inside stringify does not fail the test
ok {test-number} - tos == tos for: { stringification failed with an exception: "Invalid" } == { stringification failed with an exception: "Invalid" }
# Exceptions matchers
@@ -2513,11 +2513,11 @@ ok {test-number} - !(cfg.hasTestFilters()) for: !false
# Process can be configured on command line
ok {test-number} - cfg.getReporterSpecs().size() == 1 for: 1 == 1
# Process can be configured on command line
ok {test-number} - cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } for: {?} == {?}
ok {test-number} - cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } for: {?} == {?}
# Process can be configured on command line
ok {test-number} - cfg.getProcessedReporterSpecs().size() == 1 for: 1 == 1
# Process can be configured on command line
ok {test-number} - cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } for: {?} == {?}
ok {test-number} - cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } for: {?} == {?}
# Process can be configured on command line
ok {test-number} - result for: {?}
# Process can be configured on command line
@@ -2545,15 +2545,15 @@ ok {test-number} - cfg.testSpec().matches(*fakeTestCase("alwaysIncluded")) for:
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - !result for: true
# Process can be configured on command line
@@ -2561,19 +2561,19 @@ ok {test-number} - result.errorMessage(), ContainsSubstring("Unrecognized report
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }) for: {?}
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
# Process can be configured on command line
ok {test-number} - cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }) for: {?}
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
# Process can be configured on command line
ok {test-number} - !result for: true
# Process can be configured on command line
@@ -2765,15 +2765,15 @@ ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && Cont
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring("fakeTag"s) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tags": [ { "aliases": [ "fakeTag" ], "count": 1 } ]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring("fakeTag"s) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tags": [ { "aliases": [ "fakeTag" ], "count": 1 } ]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring("fake reporter"s) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "reporters": [ { "name": "fake reporter", "description": "fake description" } ]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring("fake reporter"s) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "reporters": [ { "name": "fake reporter", "description": "fake description" } ]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tests": [ { "name": "fake test name", "class-name": "", "tags": [ "fakeTestTag" ], "source-location": { "filename": "fake-file.cpp", "line": 123456789 } } ]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tests": [ { "name": "fake test name", "tags": [ "fakeTestTag" ] } ]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
@@ -2909,15 +2909,15 @@ ok {test-number} - testStringForMatching(), ContainsSubstring( "string", Catch::
# String matchers
ok {test-number} - testStringForMatching(), ContainsSubstring( "abc" ) for: "this string contains 'abc' as a substring" contains: "abc"
# String matchers
ok {test-number} - testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "abc" (case insensitive)
ok {test-number} - testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" contains: "aBC" (case insensitive)
# String matchers
ok {test-number} - testStringForMatching(), StartsWith( "this" ) for: "this string contains 'abc' as a substring" starts with: "this"
# String matchers
ok {test-number} - testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" starts with: "this" (case insensitive)
ok {test-number} - testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" starts with: "THIS" (case insensitive)
# String matchers
ok {test-number} - testStringForMatching(), EndsWith( "substring" ) for: "this string contains 'abc' as a substring" ends with: "substring"
# String matchers
ok {test-number} - testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: " substring" (case insensitive)
ok {test-number} - testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: " SuBsTrInG" (case insensitive)
# StringRef
ok {test-number} - empty.empty() for: true
# StringRef
+19 -24
View File
@@ -5102,7 +5102,7 @@ Approx( 1.30000000000000004 )
testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No )
</Original>
<Expanded>
"this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring" (case insensitive)
"this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring" (case insensitive)
</Expanded>
</Expression>
<OverallResult success="true" skips="0"/>
@@ -5295,7 +5295,7 @@ Approx( 1.30000000000000004 )
thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No )
</Original>
<Expanded>
"expected exception" equals: "expected exception" (case insensitive)
"expected exception" equals: "expecteD Exception" (case insensitive)
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
@@ -5330,7 +5330,7 @@ Approx( 1.30000000000000004 )
thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No )
</Original>
<Expanded>
"expected exception" contains: "except" (case insensitive)
"expected exception" contains: "exCept" (case insensitive)
</Expanded>
</Expression>
<OverallResults successes="4" failures="0" expectedFailures="0" skipped="false"/>
@@ -11976,7 +11976,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} }
cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} }
</Original>
<Expanded>
{?} == {?}
@@ -11992,7 +11992,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} }
cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} }
</Original>
<Expanded>
{?} == {?}
@@ -12132,7 +12132,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12160,7 +12160,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12188,7 +12188,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12238,7 +12238,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12266,7 +12266,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12289,7 +12289,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?}, {?} } == { {?}, {?} }
@@ -12314,7 +12314,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?}, {?} } == { {?}, {?} }
@@ -13419,7 +13419,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13456,7 +13456,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13491,7 +13491,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13501,14 +13501,9 @@ Approx( 0.98999999999999999 )
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" )
</Expanded>
@@ -14362,7 +14357,7 @@ Message from section two
testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No )
</Original>
<Expanded>
"this string contains 'abc' as a substring" contains: "abc" (case insensitive)
"this string contains 'abc' as a substring" contains: "aBC" (case insensitive)
</Expanded>
</Expression>
<Expression success="true" type="CHECK_THAT" filename="tests/<exe-name>/UsageTests/Matchers.tests.cpp" >
@@ -14378,7 +14373,7 @@ Message from section two
testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No )
</Original>
<Expanded>
"this string contains 'abc' as a substring" starts with: "this" (case insensitive)
"this string contains 'abc' as a substring" starts with: "THIS" (case insensitive)
</Expanded>
</Expression>
<Expression success="true" type="CHECK_THAT" filename="tests/<exe-name>/UsageTests/Matchers.tests.cpp" >
@@ -14394,7 +14389,7 @@ Message from section two
testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No )
</Original>
<Expanded>
"this string contains 'abc' as a substring" ends with: " substring" (case insensitive)
"this string contains 'abc' as a substring" ends with: " SuBsTrInG" (case insensitive)
</Expanded>
</Expression>
<OverallResult success="true" skips="0"/>
@@ -5102,7 +5102,7 @@ Approx( 1.30000000000000004 )
testStringForMatching(), Equals( "this string contains 'ABC' as a substring", Catch::CaseSensitive::No )
</Original>
<Expanded>
"this string contains 'abc' as a substring" equals: "this string contains 'abc' as a substring" (case insensitive)
"this string contains 'abc' as a substring" equals: "this string contains 'ABC' as a substring" (case insensitive)
</Expanded>
</Expression>
<OverallResult success="true" skips="0"/>
@@ -5295,7 +5295,7 @@ Approx( 1.30000000000000004 )
thisThrows(), Equals( "expecteD Exception", Catch::CaseSensitive::No )
</Original>
<Expanded>
"expected exception" equals: "expected exception" (case insensitive)
"expected exception" equals: "expecteD Exception" (case insensitive)
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
@@ -5330,7 +5330,7 @@ Approx( 1.30000000000000004 )
thisThrows(), ContainsSubstring( "exCept", Catch::CaseSensitive::No )
</Original>
<Expanded>
"expected exception" contains: "except" (case insensitive)
"expected exception" contains: "exCept" (case insensitive)
</Expanded>
</Expression>
<OverallResults successes="4" failures="0" expectedFailures="0" skipped="false"/>
@@ -11976,7 +11976,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} }
cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} }
</Original>
<Expanded>
{?} == {?}
@@ -11992,7 +11992,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} }
cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} }
</Original>
<Expanded>
{?} == {?}
@@ -12132,7 +12132,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12160,7 +12160,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12188,7 +12188,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12238,7 +12238,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12266,7 +12266,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12289,7 +12289,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?}, {?} } == { {?}, {?} }
@@ -12314,7 +12314,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?}, {?} } == { {?}, {?} }
@@ -13419,7 +13419,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13456,7 +13456,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13491,7 +13491,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13501,14 +13501,9 @@ Approx( 0.98999999999999999 )
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" )
</Expanded>
@@ -14362,7 +14357,7 @@ Message from section two
testStringForMatching(), ContainsSubstring( "aBC", Catch::CaseSensitive::No )
</Original>
<Expanded>
"this string contains 'abc' as a substring" contains: "abc" (case insensitive)
"this string contains 'abc' as a substring" contains: "aBC" (case insensitive)
</Expanded>
</Expression>
<Expression success="true" type="CHECK_THAT" filename="tests/<exe-name>/UsageTests/Matchers.tests.cpp" >
@@ -14378,7 +14373,7 @@ Message from section two
testStringForMatching(), StartsWith( "THIS", Catch::CaseSensitive::No )
</Original>
<Expanded>
"this string contains 'abc' as a substring" starts with: "this" (case insensitive)
"this string contains 'abc' as a substring" starts with: "THIS" (case insensitive)
</Expanded>
</Expression>
<Expression success="true" type="CHECK_THAT" filename="tests/<exe-name>/UsageTests/Matchers.tests.cpp" >
@@ -14394,7 +14389,7 @@ Message from section two
testStringForMatching(), EndsWith( " SuBsTrInG", Catch::CaseSensitive::No )
</Original>
<Expanded>
"this string contains 'abc' as a substring" ends with: " substring" (case insensitive)
"this string contains 'abc' as a substring" ends with: " SuBsTrInG" (case insensitive)
</Expanded>
</Expression>
<OverallResult success="true" skips="0"/>
@@ -58,12 +58,13 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CHECK( cfg.getReporterSpecs().size() == 1 );
CHECK( cfg.getReporterSpecs()[0] ==
Catch::ReporterSpec{ expectedReporter, {}, {}, {} } );
Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } );
CHECK( cfg.getProcessedReporterSpecs().size() == 1 );
CHECK( cfg.getProcessedReporterSpecs()[0] ==
Catch::ProcessedReporterSpec{ expectedReporter,
std::string{},
Catch::ColourMode::PlatformDefault,
Catch::Verbosity::Normal,
{} } );
}
@@ -108,7 +109,7 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "console", {}, {}, {} } } );
vec_Specs{ { "console", {}, {}, {}, {} } } );
}
SECTION("-r/xml") {
auto result = cli.parse({"test", "-r", "xml"});
@@ -116,7 +117,7 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "xml", {}, {}, {} } } );
vec_Specs{ { "xml", {}, {}, {}, {} } } );
}
SECTION("--reporter/junit") {
auto result = cli.parse({"test", "--reporter", "junit"});
@@ -124,7 +125,7 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "junit", {}, {}, {} } } );
vec_Specs{ { "junit", {}, {}, {}, {} } } );
}
SECTION("must match one of the available ones") {
auto result = cli.parse({"test", "--reporter", "unsupported"});
@@ -137,27 +138,27 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CAPTURE(result.errorMessage());
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "console", "out.txt"s, {}, {} } } );
vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } );
}
SECTION("With Windows-like absolute path as output file") {
auto result = cli.parse({ "test", "-r", "console::out=C:\\Temp\\out.txt" });
CAPTURE(result.errorMessage());
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } );
vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } );
}
SECTION("Multiple reporters") {
SECTION("All with output files") {
CHECK(cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }));
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "xml", "output.xml"s, {}, {} },
{ "junit", "output-junit.xml"s, {}, {} } } );
vec_Specs{ { "xml", "output.xml"s, {}, {}, {} },
{ "junit", "output-junit.xml"s, {}, {}, {} } } );
}
SECTION("Mixed output files and default output") {
CHECK(cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }));
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "xml", "output.xml"s, {}, {} },
{ "console", {}, {}, {} } } );
vec_Specs{ { "xml", "output.xml"s, {}, {}, {} },
{ "console", {}, {}, {}, {} } } );
}
SECTION("cannot have multiple reporters with default output") {
auto result = cli.parse({ "test", "-r", "console", "-r", "xml::out=output.xml", "-r", "junit" });
@@ -472,15 +473,34 @@ TEST_CASE( "Parse rng seed in different formats", "[approvals][cli][rng-seed]" )
CAPTURE( seed_string );
auto result = cli.parse( { "tests", "--rng-seed", seed_string } );
REQUIRE( result );
REQUIRE( config.rngSeed == seed_value );
Catch::Config cfg{config};
REQUIRE( cfg.rngSeed() == seed_value );
REQUIRE( cfg.rngSeedWasFixed() );
}
SECTION( "time seed is not considered fixed" ) {
auto result = cli.parse( { "tests", "--rng-seed", "time" } );
REQUIRE( result );
Catch::Config cfg{config};
REQUIRE_FALSE( cfg.rngSeedWasFixed() );
}
SECTION( "random-device seed is not considered fixed" ) {
auto result = cli.parse( { "tests", "--rng-seed", "random-device" } );
REQUIRE( result );
Catch::Config cfg{config};
REQUIRE_FALSE( cfg.rngSeedWasFixed() );
}
SECTION( "Error cases" ) {
auto seed_string =
GENERATE( "0xSEED", "999999999999", "08888", "BEEF", "123 456" );
CAPTURE( seed_string );
REQUIRE_FALSE( cli.parse( { "tests", "--rng-seed", seed_string } ) );
Catch::Config cfg{config};
REQUIRE_FALSE( cfg.rngSeedWasFixed() );
}
}
@@ -63,6 +63,20 @@ TEST_CASE( "Parsing colour mode", "[cli][colour][approvals]" ) {
}
}
TEST_CASE( "Parsing verbosity", "[cli][colour][approvals]" ) {
using Catch::Detail::stringToVerbosity;
using Catch::Verbosity;
SECTION( "Valid strings" ) {
REQUIRE( stringToVerbosity( "quiet" ) == Verbosity::Quiet );
REQUIRE( stringToVerbosity( "normal" ) == Verbosity::Normal );
REQUIRE( stringToVerbosity( "high" ) == Verbosity::High );
}
SECTION( "Wrong strings" ) {
REQUIRE_FALSE( stringToVerbosity( "QUIET" ) );
REQUIRE_FALSE( stringToVerbosity( "medium" ) );
REQUIRE_FALSE( stringToVerbosity( "vvv" ) );
}
}
TEST_CASE("Parsing reporter specs", "[cli][reporter-spec][approvals]") {
using Catch::parseReporterSpec;
@@ -71,12 +85,13 @@ TEST_CASE("Parsing reporter specs", "[cli][reporter-spec][approvals]") {
SECTION( "Correct specs" ) {
REQUIRE( parseReporterSpec( "someReporter" ) ==
ReporterSpec( "someReporter"s, {}, {}, {} ) );
ReporterSpec( "someReporter"s, {}, {}, {}, {} ) );
REQUIRE( parseReporterSpec( "otherReporter::Xk=v::out=c:\\blah" ) ==
ReporterSpec(
"otherReporter"s, "c:\\blah"s, {}, { { "Xk"s, "v"s } } ) );
"otherReporter"s, "c:\\blah"s, {}, {}, { { "Xk"s, "v"s } } ) );
REQUIRE( parseReporterSpec( "diffReporter::Xk1=v1::Xk2==v2" ) ==
ReporterSpec( "diffReporter",
{},
{},
{},
{ { "Xk1"s, "v1"s }, { "Xk2"s, "=v2"s } } ) );
@@ -85,7 +100,15 @@ TEST_CASE("Parsing reporter specs", "[cli][reporter-spec][approvals]") {
ReporterSpec( "Foo:bar:reporter",
{},
Catch::ColourMode::ANSI,
{},
{ { "Xk 1"s, "v 1"s }, { "Xk2"s, "v:3"s } } ) );
REQUIRE(
parseReporterSpec( "my:reporter::X1=V1::verbosity=high::X2=V2" ) ==
ReporterSpec( "my:reporter",
{},
{},
Catch::Verbosity::High,
{ { "X1"s, "V1"s }, { "X2"s, "V2"s } } ) );
}
SECTION( "Bad specs" ) {
@@ -107,5 +130,9 @@ TEST_CASE("Parsing reporter specs", "[cli][reporter-spec][approvals]") {
REQUIRE_FALSE( parseReporterSpec( "reporter::Xa=" ) );
// non-key value later field
REQUIRE_FALSE( parseReporterSpec( "reporter::Xab" ) );
// Invalid verbosity value
REQUIRE_FALSE( parseReporterSpec( "reporter::verbosity=medium" ) );
// Duplicated verbosity
REQUIRE_FALSE( parseReporterSpec( "reporter::verbosity=high::X1=X2::verbosity=high" ) );
}
}
@@ -429,9 +429,6 @@ TEST_CASE("run benchmark", "[benchmark][approvals]") {
}
TEST_CASE("Failing benchmarks", "[!benchmark][.approvals]") {
SECTION("empty", "Benchmark that has been optimized away (because it is empty)") {
BENCHMARK("Empty benchmark") {};
}
SECTION("throw", "Benchmark that throws an exception") {
BENCHMARK("Throwing benchmark") {
throw "just a plain literal, bleh";
@@ -42,6 +42,7 @@ namespace {
&config,
Catch::Detail::make_unique<StringIStream>(),
Catch::ColourMode::None,
Catch::Verbosity::Normal,
{} };
}
}
@@ -114,7 +115,7 @@ TEST_CASE( "Reporter's write listings to provided stream", "[reporters]" ) {
cfg_data.rngSeed = 1234;
Catch::Config config( cfg_data );
auto reporter = factory.second->create( Catch::ReporterConfig{
&config, CATCH_MOVE( sstream ), Catch::ColourMode::None, {} } );
&config, CATCH_MOVE( sstream ), Catch::ColourMode::None, Catch::Verbosity::Normal, {} } );
DYNAMIC_SECTION( factory.first << " reporter lists tags" ) {
std::vector<Catch::TagInfo> tags(1);
@@ -15,6 +15,8 @@ if(CMAKE_VERSION GREATER_EQUAL 3.27)
endif()
catch_discover_tests(
tests
TEST_PREFIX " prefix "
TEST_SUFFIX " suffix "
ADD_TAGS_AS_LABELS
DISCOVERY_MODE PRE_TEST
${extra_args}
@@ -1,12 +1,12 @@
# SPDX-License-Identifier: BSL-1.0
# Unit tests for `prepare_command` helper in `extras/CatchAddTests.cmake`.
# Unit tests for `prepare_command_fragment` helper in `extras/CatchAddTests.cmake`.
#
# Yes, we are at the stage where the script helpers need unit tests.
#
# Run as
# cmake -DCATCH_ADD_TESTS_SCRIPT=/path/to/extras/CatchAddTests.cmake \
# -P TestPrepareCommand.cmake
# -P TestPrepareCommandFragment.cmake
cmake_minimum_required(VERSION 3.19)
@@ -37,25 +37,25 @@ function(expect_equal description actual expected)
endif()
endfunction()
prepare_command(add_test SimpleName /path/to/tests)
expect_equal("Simple arg, no quotes" "${_Command}" "add_test( SimpleName /path/to/tests)\n")
prepare_command_fragment(test_fragment SimpleName /path/to/tests)
expect_equal("Simple arg, no quotes" "${test_fragment}" " SimpleName /path/to/tests")
prepare_command(add_test "Name with spaces")
expect_equal("Spaces in arg, needs quotes" "${_Command}" "add_test( [==[Name with spaces]==])\n")
prepare_command_fragment(test_fragment "Name with spaces")
expect_equal("Spaces in arg, needs quotes" "${test_fragment}" " [==[Name with spaces]==]")
prepare_command(set_tests_properties Foo PROPERTIES LABELS "tagA\;tagB\;tagC")
prepare_command_fragment(test_fragment Foo PROPERTIES LABELS "tagA\;tagB\;tagC")
expect_equal("semicolons in argument are kept and quoted"
"${_Command}" "set_tests_properties( Foo PROPERTIES LABELS [==[tagA\;tagB\;tagC]==])\n")
"${test_fragment}" " Foo PROPERTIES LABELS [==[tagA\;tagB\;tagC]==]")
set(_Command "PRE-EXISTING")
prepare_command(set_tests_properties Foo PROPERTIES BAR baz)
expect_equal("_Command var does no accumulate commands"
"${_Command}" "set_tests_properties( Foo PROPERTIES BAR baz)\n")
set(test_fragment "PRE-EXISTING")
prepare_command_fragment(test_fragment Foo PROPERTIES BAR baz)
expect_equal("out var does no accumulate commands"
"${test_fragment}" " Foo PROPERTIES BAR baz")
if(_failures GREATER 0)
message(FATAL_ERROR "${_failures} prepare_command test(s) failed")
message(FATAL_ERROR "${_failures} prepare_command_fragment test(s) failed")
else()
message(STATUS "All prepare_command tests passed")
message(STATUS "All prepare_command_fragment tests passed")
endif()
@@ -7,6 +7,7 @@
# SPDX-License-Identifier: BSL-1.0
import glob
import os
import subprocess
import sys
@@ -20,6 +21,11 @@ TestInfo = namedtuple('TestInfo', ['name', 'tags'])
cmake_version_regex = re.compile(r'cmake version (\d+)\.(\d+)\.(\d+)')
# Note that these both intentionally include preceding/trailing space,
# which should not get stripped.
CTEST_NAME_PREFIX = ' prefix '
CTEST_NAME_SUFFIX = ' suffix '
def get_cmake_version():
result = subprocess.run(['cmake', '--version'],
capture_output = True,
@@ -65,7 +71,6 @@ def build_project(sources_dir, output_base_path, catch2_path):
return build_dir
def get_test_names(build_path: str) -> List[TestInfo]:
# For now we assume that Windows builds are done using MSBuild under
# Debug configuration. This means that we need to add "Debug" folder
@@ -87,7 +92,7 @@ def get_test_names(build_path: str) -> List[TestInfo]:
with open(fname, mode='r', encoding='utf-8') as file:
test_listing = json.load(file)
assert test_listing['version'] == 1
assert test_listing['version'] == 2
tests = []
for test in test_listing['listings']['tests']:
@@ -97,6 +102,7 @@ def get_test_names(build_path: str) -> List[TestInfo]:
return tests
def get_ctest_listing(build_path):
old_path = os.getcwd()
os.chdir(build_path)
@@ -137,6 +143,7 @@ def extract_tests_from_ctest(ctest_output) -> List[TestInfo]:
return test_infos
def check_DL_PATHS(ctest_output):
ctest_response = json.loads(ctest_output)
tests = ctest_response['tests']
@@ -146,6 +153,98 @@ def check_DL_PATHS(ctest_output):
if property['name'] == 'ENVIRONMENT_MODIFICATION':
assert len(property['value']) == 2, f"The test provides 2 arguments to DL_PATHS, but instead found {len(property['value'])}"
def add_test_list_extractor(build_path: str) -> str:
# The actual CTest script file has one of two names:
# * `<target>-<short-hash>_tests.cmake` on single-config generators
# * `<target>-<short-hash>_tests-<Config>.cmake` on multi-config generators
#
# We know the target name (`tests`), so we glob for the hash part
patterns = [
os.path.join(build_path, 'tests-*_tests.cmake'),
os.path.join(build_path, 'tests-*_tests-Debug.cmake'),
]
matches = []
for pattern in patterns:
matches.extend(glob.glob(pattern))
if len(matches) != 1:
print(f"Found {len(matches)} CTest files in '{build_path}'. Expected only 1.")
exit(5)
test_script_file = matches[0]
basename = os.path.basename(test_script_file)
extractor_fname = os.path.join(build_path, 'extractor.cmake')
with open(extractor_fname, 'w') as f:
f.write(fr"""
cmake_minimum_required(VERSION 3.19)
cmake_policy(VERSION 3.19...4.4)
# This dummies out the `add_test` and `set_tests_properties` commands
# inside the CTest script, so we can include it during CMake script call.
macro(add_test)
endmacro()
macro(set_tests_properties)
endmacro()
include(${{CMAKE_CURRENT_LIST_DIR}}/{basename})
list(LENGTH tests_TESTS num_tests)
message(STATUS "NUM TESTS: ${{num_tests}}")
# '[' and ']' were escaped into ASCII 2 and 3 respectively, we have to
# unescape them back here. Note that this has to be done per-element,
# or CMake's list parsing breaks (which is why they were escaped).
string(ASCII 2 _LeftBracketListingEscape)
string(ASCII 3 _RightBracketListingEscape)
foreach(test IN LISTS tests_TESTS)
string(REPLACE "${{_LeftBracketListingEscape}}" "[" test "${{test}}")
string(REPLACE "${{_RightBracketListingEscape}}" "]" test "${{test}}")
string(REPLACE "\\" "\\\\" test "${{test}}")
string(REPLACE "\r" "\\r" test "${{test}}")
string(REPLACE "\n" "\\n" test "${{test}}")
message(STATUS "TEST_NAME: ${{test}}")
endforeach()
""")
return extractor_fname
def extract_tests_list_from_ctest_script(build_path: str) -> List[str]:
extractor = add_test_list_extractor(build_path)
cmd = ['cmake', '-P', extractor]
try:
result = subprocess.run(cmd,
capture_output = True,
check = True,
text = True)
except subprocess.CalledProcessError as err:
print('Error when calling CTest test extractor')
print(f'cmd: {err.cmd}')
print(f'stderr: {err.stderr}')
print(f'stdout: {err.stdout}')
exit(4)
lines = [x for x in result.stdout.split('\n') if x.strip()]
test_num_line = lines[0]
test_lines = lines[1:]
test_num_prefix = '-- NUM TESTS: '
assert test_num_prefix in test_num_line, test_num_line
test_num_line = test_num_line[len(test_num_prefix):]
num_tests = int(test_num_line)
assert num_tests == len(test_lines), len(test_lines)
test_name_prefix = '-- TEST_NAME: '
assert all(test_name_prefix in x for x in test_lines)
test_names = [x[len(test_name_prefix):] for x in test_lines]
# Unescape the names, so that names with literal newlines have newlines in them again
test_names = [x.encode('utf-8').decode('unicode-escape') for x in test_names]
return test_names
def escape_catch2_test_names(infos: List[TestInfo]):
escaped = []
for info in infos:
@@ -155,6 +254,7 @@ def escape_catch2_test_names(infos: List[TestInfo]):
escaped.append(TestInfo(name, info.tags))
return escaped
if __name__ == '__main__':
if len(sys.argv) != 3:
print(f'Usage: {sys.argv[0]} path-to-catch2-cml output-path')
@@ -165,7 +265,8 @@ if __name__ == '__main__':
build_path = build_project(sources_dir, output_base_path, catch2_path)
catch_test_names = escape_catch2_test_names(get_test_names(build_path))
raw_catch_test_names = get_test_names(build_path)
catch_test_names = escape_catch2_test_names(raw_catch_test_names)
ctest_output = get_ctest_listing(build_path)
ctest_test_names = extract_tests_from_ctest(ctest_output)
@@ -182,7 +283,20 @@ if __name__ == '__main__':
if mismatched:
print(f"Found {mismatched} mismatched tests catch test names and ctest test commands!")
exit(1)
print(f"{len(catch_test_names)} tests matched")
print(f"{len(catch_test_names)} tests matched in CTest listing")
test_list_names = sorted(extract_tests_list_from_ctest_script(build_path))
expected_names = sorted(CTEST_NAME_PREFIX + info.name + CTEST_NAME_SUFFIX for info in raw_catch_test_names)
if test_list_names != expected_names:
print("TEST_LIST variable (tests_TESTS) does not match Catch2 test listing!")
for name in test_list_names:
if name not in expected_names:
print(f" TEST_LIST name '{name}' not in Catch2 listing")
for name in expected_names:
if name not in test_list_names:
print(f" Catch2 name '{name}' not in TEST_LIST")
exit(1)
print(f"{len(test_list_names)} tests matched in TEST_LIST variable")
cmake_version = get_cmake_version()
if cmake_version >= (3, 27):
@@ -40,7 +40,11 @@ public:
TEST_CASE_METHOD(TestCaseFixture, "A test case as method", "[tagstagstags]") {}
TEST_CASE("Unclosed right ) parenthesis") {}
TEST_CASE("Unclosed left ( parenthesis") {}
TEST_CASE( "Newlines\nAnd\rOther\n\tWhitespace", "[whitespace-going-wild]" ) {}
TEST_CASE( "Escaped \\n newline and \\r other whitespace", "[whitespace-going-wild]") {}
// Some JSON-like and JSON-adjacent characters and substrings in the test names/tags
// This serves to test that the parse-json-via-string-splitting hack in