Compare commits

..
35 Commits
Author SHA1 Message Date
Martin Hořeňovský 97ec4e8e2e catch_discover_tests: Escape test-invariant parts of CTest script only once
Previously, the `catch_discover_tests` would prepare the entire CTest
command (e.g. `add_test(...)` or `set_tests_properties(...)`) first, and
then escape it when finished. However, this caused lot of the command args
to be escaped over and over again (e.g. executable name or Catch2's reporter
args), for no reason, as they were always the same, and thus their escaping
was always the same.

Until recently, the performance overhead didn't matter as there were many
spots which had quadratic runtime in number of tests. However, the recent
refactorings fixed these, and this commit now improves the throughput by
10-20%.

Also extended the benchmarked COUNTS in `benchmark_discovery.py`,
because the performance is now good enough that it is reasonable
to benchmark 16k tests.
2026-08-10 11:53:22 +02:00
Martin Hořeňovský 0136276e15 Avoid n**2 runtime when creating list of tests in catch_discover_tests
This was another place where the script triggered the quadratic runtime
from calling `string(APPEND` (or `list(APPEND`) repeatedly. As in
60c8b87, we avoid this by flushing the test list into a file every
50kB of text input.

As the string with the test list never grew quite as much as the
string that contains all the test definitions, this only provides
significant savings for high number of tests. It starts being
properly measurable around 4k tests at ~100ms, but grows to ~4s at
32k tests.
2026-08-10 10:36:04 +02:00
Martin Hořeňovský 630840c500 Fix <target>_TESTS variable generated by catch_discover_tests
There were 2 separate bugs, one recently added and one ancient
(since roughly the first version of the registration script).

1) The recent refactoring of how the JSON output from Catch2 is
   parsed caused shadowing between the variable containing the
   per-test JSON fragments and the accumulator of test names for
   <target>_TESTS variable. This led to the variable containing
   both the names and the JSON fragments, and thus being completely
   wrong.

2) The test name accumulation has never accounted for characters
   that need escaping to be present in a CMake list. This means
   that e.g. test names with semicolon in them would end up with
   two elements in the test list.

Both of these are now fixed, at the cost of extra complexity and
my sanity as I had to learn more about CMake escaping rules.

(CMake escaping rules are dumb)
2026-08-10 00:17:26 +02:00
Martin Hořeňovský 4cde128517 JSONReporter considers verbosity when listing tests
* Quiet verbosity provides just the test names
* Normal verbosity adds tags
* High verbosity add the source location of the tests

Listing tests with the quiet verbosity results in about 1/4 of the
previous output, which leads to measurably faster execution of
`catch_discover_tests` when it does not need tags for labels.
(If it does need tags, the output is about 1/2 of previous).
2026-08-08 00:05:18 +02:00
Martin Hořeňovský 0b4d7a5a16 JSONReporter's listTests only lists class-name if it is non-empty 2026-08-07 23:07:46 +02:00
Martin Hořeňovský a0eba50e9d Make verbosity per-reporter 2026-08-07 20:57:42 +02:00
Martin Hořeňovský 8b08d4d795 v3.15.3 2026-07-26 22:19:31 +02:00
Martin Hořeňovský 1079da4c5f Avoid quadratic JSON array parse behaviour in catch_discover_tests
Using CMake's `string(JSON` to parse JSON array leads to quadratic
running time in number of tests, see https://gitlab.kitware.com/cmake/cmake/-/work_items/27985

This leads to _terrible_ runtime for `catch_discover_tests` when called
on binaries with lot of tests (1k+). To get reasonable runtimes, we have
to avoid using `string(JSON` to parse out the individual test objects
from the array with all tests.

This commit replaces the sane approach of using real JSON parser with
a set of terrible hacks, where we use CMake's string APIs to split the
JSON array on what looks like object boundary (`}<ws>*,<ws>*{`), and then
checking whether the resulting thing can be parsed as JSON object. If not,
we append the next piece and check again. And again, and again, until we
get a proper JSON object.

This is all around a hilariously terrible idea, however:

1) It works in practice for all tested inputs.
2) It improves the time it takes to run `catch_discover_tests` on binary
   with 1k tests from 4.2s to 1.1s and 2k tests from 16s to 3.9s.
2026-07-26 19:51:58 +02:00
Martin Hořeňovský 60c8b87829 Avoid n^2 behaviour when appending script commands in catch_discover_tests
Repeatedly calling `string(APPEND` on the same destination string
leads to quadratic running time. We avoid this by flushing the CTest
script into a file periodically, currently every 50kB of text.

This can cause _slight_ slowdown just around the flush boundary (each
file write is quite expensive, so if we flush just before the last test
is written, it hurts), but it avoids terrible performance for large
test suites.

The performance is roughly equal at 500 tests to discover, and clearly
wins at more; for 1/2/4/8 k tests, the improvements are 0.3/1.6/6.4/27 s.
2026-07-26 15:50:33 +02:00
Martin Hořeňovský 64a551e2e7 Optimize adding test commands in catch_discover_tests
The previous approach was for `add_command` to behave as append function
via concatenating the command string internally and then saving it into
`PARENT_SCOPE`. Because this in practice ended up meaning concatenating
a copy of the string inside the function and then overwriting the string
outside the function, the performance was lacking.

The new approach is for `prepare_command` to only escape & return the
command from single call, and the caller is responsible for concatenating
the result. Since the actual concatenation no longer crosses function
scope boundaries, the performance is much better, even though the scaling
is still quadratic.

The new approach only takes 1/4-1/5 of the time, saving ~1s at 1k tests,
4.5s at 2k tests and 19s at 4k tests.
2026-07-26 15:50:31 +02:00
Martin Hořeňovský 2b971368dd Determinize the test registration order in catch_discover_tests 2026-07-26 15:23:39 +02:00
Martin Hořeňovský f5db82a5c4 Add Python script for benchmarking catch_discover_tests 2026-07-24 23:53:50 +02:00
Martin Hořeňovský dcbb2d5d84 Fix bad ToC link in docs/command-line.md 2026-07-24 23:52:44 +02:00
dvir aradandMartin Hořeňovský 1a9625a6e7 Fix typo in generator exception message ("Coud" -> "Could")
The "Could not jump to Nth element: not enough elements" message thrown
by throw_generator_exception() had "Coud" misspelled in all three places
it appears (catch_generators.hpp, catch_generators_adapters.hpp, and
catch_interfaces_generatortracker.cpp).
2026-07-24 17:12:50 +02:00
Martin Hořeňovský bb8873ccd7 JSONWriter uses LUT for faster checking if a char needs escaping
This provides nice speedup when writing strings that don't need
escaping, at about 6% in Debug build and ~40% in Release build.
If the strings do need escaping the speedup is much smaller, at
~1% in Debug and ~6% in Release build, as the cost of actually
escaping the strings dwarves the cost of checking.
2026-07-23 19:10:14 +02:00
Martin Hořeňovský 46bdf1a04c Optimize writing of non-string values to JSON
In practice, the majority of values the JSON writer handles are
strings, but this makes the code more obvious and can still provide
a small speed-up.
2026-07-23 14:34:43 +02:00
Martin Hořeňovský c267251e70 Better support for infs and NaNs in JSON writer
As the types and values sent into the writer are determined by Catch2,
I do not expect to actually need this support, but it is better to have
it and not be surprised in the future.
2026-07-23 14:26:26 +02:00
Martin Hořeňovský 0cc833ea95 Make JSONWriter's number handling locale-independent
Closes #3176
2026-07-23 00:10:28 +02:00
Martin Hořeňovský 8494e2dce4 Add TEST_CASE with newlines & stuff to the catch_discover_tests's tests 2026-07-22 21:11:58 +02:00
Phil NashandMartin Hořeňovský eb431764b4 Document that DISCOVERY_MODE PRE_TEST should be used with catch_discover_tests when used with XCode.
This addresses https://github.com/catchorg/Catch2/issues/2411 by making the "workaround" the official approach and is consistent with how gtest and cmake deal with the same situation.
2026-07-22 20:50:17 +02:00
Manuel KnörleandMartin Hořeňovský ae5d271da2 Remove debug message for environment variable check 2026-07-15 11:19:22 +02:00
Nikolay BaklicharovandMartin Hořeňovský a15f718c82 Optimize JSON parsing in catch_discover_tests() 2026-07-08 10:29:59 +02:00
Martin Hořeňovský 191fa38c9b v3.15.2 2026-07-07 20:44:33 +02:00
Martin Hořeňovský dd94b9a780 Fix --warn InfiniteGenerators firing even if the generator was filtered 2026-07-07 14:09:11 +02:00
Martin Hořeňovský 9f7c9f6872 Add extra test binary & test for CATCH_CONFIG_FAST_COMPILE
Closes #3100
2026-07-06 11:03:47 +02:00
Martin Hořeňovský 919385f704 Fix some test binaries in ExtraTests not having warnings enabled 2026-07-06 10:22:55 +02:00
Martin Hořeňovský 15d52830ee Fix -Wunused-parameter warning with exceptions disabled
Closes #3114
2026-07-05 12:08:06 +02:00
Martin Hořeňovský 9ec44dd62b catch_discover_tests uses tempfile to retrieve JSON from the binary
This allows it to deal with badly behaved code, where 3rd party
dependencies write into stdout during global construction.

Closes #3162
Closes #3166
2026-07-04 16:36:05 +02:00
Martin Hořeňovský 675f9eaeb1 Add LLM policy to contributing.md 2026-06-14 20:25:57 +02:00
Martin Hořeňovský bcfb10e498 v3.15.1 2026-06-14 10:57:30 +02:00
Dominic KoepkeandMartin Hořeňovský b7e0310fbe fix: move struct TestName of INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD_2 into the anonymous namespace 2026-06-09 19:17:14 +02:00
Dominic KoepkeandMartin Hořeňovský 6d4ea62200 fix: move struct TestName of INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD_2 into the anonymous namespace 2026-06-09 19:17:14 +02:00
offaandMartin Hořeňovský 195231c59a docs: Set missing syntax language 2026-05-31 17:17:46 +02:00
offaandMartin Hořeňovský 3a865e8c0f docs: Enable syntax coloring for code example 2026-05-31 17:17:46 +02:00
Dirk MüllerandMartin Hořeňovský 69e0473f6e Avoid a potential underflow when iterating over invalid inputs
The forward iteration logic already bounds-check for m_it != m_string->end(),
do the same for the backward iteration. The issue with the assert is
that the assert() might not be compiled in, and it is happening after
the dereference, so it was too late.
2026-05-14 10:58:09 +02:00
72 changed files with 2689 additions and 493 deletions
+1 -1
View File
@@ -35,7 +35,7 @@ if(CMAKE_BINARY_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
endif()
project(Catch2
VERSION 3.15.0 # CML version placeholder, don't delete
VERSION 3.15.3 # CML version placeholder, don't delete
LANGUAGES CXX
HOMEPAGE_URL "https://github.com/catchorg/Catch2"
DESCRIPTION "A modern, C++-native, unit test framework."
@@ -0,0 +1,206 @@
#!/usr/bin/env python3
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
import argparse
import copy
import json
import os
import re
import statistics
import subprocess
import sys
import tempfile
import time
HERE = os.path.dirname(os.path.abspath(__file__))
TEMPLATE = os.path.join(HERE, "listing_template.json")
SHIM = os.path.join(HERE, "copy_shim.cmake")
COUNTS = [1, 10, 500, 1000, 2000, 4000, 8000, 16000]
#COUNTS = [1, 10, 100]
REPEATS = 5
def load_template(path: str) -> dict:
with open(path, encoding="utf-8") as f:
catch_out = json.load(f)
if 'listings' not in catch_out or 'tests' not in catch_out['listings']:
sys.exit(f"Template '{path}' does not contain the expected 'listings.tests' data")
return catch_out
def synthesize_listing(template_doc: dict, count: int, file, with_tags: bool) -> str:
"""Writes JSON listing with exactly `count` test cases into `file`.
The template entries are cycled through, and each name is made unique by
appending an index, so that every registered CTest test has a distinct name.
"""
# To avoid messing up the template tests by deletion, we have
# to take a deepcopy before the changes.
doc_copy = copy.deepcopy(template_doc)
template_tests = doc_copy['listings']['tests']
if not with_tags:
for test in template_tests:
del test['tags']
num_original = len(template_tests)
out_tests = []
for i in range(count):
entry = template_tests[i % num_original].copy()
entry["name"] = f"{entry['name']} #{i // num_original}"
out_tests.append(entry)
doc_copy['listings']['tests'] = out_tests
# indent=2 matches Catch2's pretty printing
json.dump(doc_copy, file, indent=2)
file.write('\n')
def build_command(args, listing_path: str, ctest_file: str, add_tags: bool) -> list[str]:
# The executor needs to be a CMake list, so it has to be separated by ';'
executor = ";".join(['cmake', f"-DBENCH_LISTING={listing_path}", "-P", SHIM, "--"])
return [
'cmake',
"-DTEST_TARGET=benchmark",
f"-DTEST_EXECUTABLE={listing_path}",
f"-DTEST_EXECUTOR={executor}",
f"-DTEST_WORKING_DIR={os.path.dirname(ctest_file)}",
f"-DCTEST_FILE={ctest_file}",
f"-DADD_TAGS_AS_LABELS={'TRUE' if add_tags else 'FALSE'}",
"-P",
args.script,
]
def run_once(cmd: list[str]) -> float:
start = time.monotonic()
result = subprocess.run(cmd, capture_output=True, text=True)
elapsed = time.monotonic() - start
if result.returncode != 0:
sys.stderr.write("Benchmark invocation failed:\n")
sys.stderr.write(f" cmd: {' '.join(cmd)}\n")
sys.stderr.write(f" stdout: {result.stdout}\n")
sys.stderr.write(f" stderr: {result.stderr}\n")
sys.exit(1)
return elapsed
def verify_output(ctest_file: str, count: int, add_tags: bool):
"""Sanity-check the generated CTest script and return its add_test count."""
with open(ctest_file, encoding="utf-8") as f:
content = f.read()
add_test_lines = sum(1 for line in content.splitlines() if line.startswith("add_test("))
if add_test_lines != count:
sys.exit(
f"Sanity check failed: expected {count} add_test() lines, "
f"found {add_test_lines} in {ctest_file}"
)
if add_tags and count > 0 and "LABELS" not in content:
sys.exit(
f"Sanity check failed: ADD_TAGS_AS_LABELS was on but no LABELS found "
f"in {ctest_file}"
)
def cmake_version() -> str:
stdout = subprocess.run(
['cmake', "--version"], capture_output=True, text=True, check=True
).stdout
match = re.match("cmake version (.+)", stdout)
return match.group(1)
def print_table(counts, tag_modes, results):
timing_by_ct = {(count, tags): median_time for (count, tags, median_time) in results}
headers = ["N"]
if False in tag_modes:
headers.append("tags:off")
if True in tag_modes:
headers.append("tags:on")
def fmt_cell(count, tags_on):
median_time = timing_by_ct[(count, tags_on)]
return f"{median_time:8.1f} ms"
# Header
row = f"{headers[0]:>6}"
if False in tag_modes:
row += f" {headers[1]:>14}"
if True in tag_modes:
row += f" {headers[2]:>14}"
print()
print("-" * len(row))
print(row)
print("-" * len(row))
for count in counts:
row = f"{count:>6}"
if False in tag_modes:
row += f" {fmt_cell(count, False):>14}"
if True in tag_modes:
row += f" {fmt_cell(count, True):>14}"
print(row)
def main():
parser = argparse.ArgumentParser(
description="Benchmark the CMake-side cost of catch_discover_tests."
)
parser.add_argument(
"--script",
required=True,
help="Path to the CatchAddTests.cmake to benchmark.",
)
args = parser.parse_args()
if not os.path.exists(args.script):
sys.exit(f"Script to benchmark '{args.script}' does not exist")
if not os.path.exists(SHIM):
sys.exit(f"Copy-shim '{SHIM}' does not exist")
tag_modes = [False, True]
template_doc = load_template(TEMPLATE)
print(f"Benchmarking: {os.path.abspath(args.script)}")
print(f"CMake version: {cmake_version()}")
print(f"Ns: {COUNTS}")
print(f"Repeats: {REPEATS}")
print()
results = [] # (count, tags_on, median_ms)
with tempfile.TemporaryDirectory() as workdir:
os.makedirs(workdir, exist_ok=True)
for count in COUNTS:
listing_path = os.path.join(workdir, f"listing-{count}.json")
for add_tags in tag_modes:
with open(listing_path, "w", encoding="utf-8") as f:
synthesize_listing(template_doc, count, f, add_tags)
mode = "on" if add_tags else "off"
ctest_file = os.path.join(workdir, f"ctest-{count}-{mode}.cmake")
cmd = build_command(args, listing_path, ctest_file, add_tags)
# Untimed warmup run (populates any filesystem/OS caches).
run_once(cmd)
verify_output(ctest_file, count, add_tags)
timings = [run_once(cmd) for _ in range(REPEATS)]
median_ms = statistics.median(timings) * 1000.0
results.append((count, add_tags, median_ms))
print(f'N = {count} done in ~{len(tag_modes) * sum(timings):.2f} s')
print_table(COUNTS, tag_modes, results)
if __name__ == "__main__":
main()
+40
View File
@@ -0,0 +1,40 @@
# SPDX-License-Identifier: BSL-1.0
# Fake Catch2 binary used by benchmark_discovery.py.
#
# catch_discover_tests_impl discovers tests by running
#
# ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ... --list-tests --reporter json --out <file>
#
# and then reading the JSON listing back from <file>. This script fakes
# being Catch2 binary by taking the fake JSON listing prepared by
# `benchmark_discovery.py` and copying it over to the file path specified
# after the `--out` argument.
#
# The fake JSON listing is passed as BENCH_LISTING CMake argument.
if(NOT DEFINED BENCH_LISTING)
message(FATAL_ERROR "copy_shim: BENCH_LISTING is not set")
endif()
if(NOT EXISTS "${BENCH_LISTING}")
message(FATAL_ERROR "copy_shim: BENCH_LISTING '${BENCH_LISTING}' does not exist")
endif()
# FIXME: better handling of bad --out
# Find the "--out <file>" argument pair among the forwarded arguments.
set(_out_file "")
math(EXPR _last "${CMAKE_ARGC} - 1")
foreach(_i RANGE 0 ${_last})
if("${CMAKE_ARGV${_i}}" STREQUAL "--out")
math(EXPR _next "${_i} + 1")
set(_out_file "${CMAKE_ARGV${_next}}")
break()
endif()
endforeach()
if(_out_file STREQUAL "")
message(FATAL_ERROR "copy_shim: could not find '--out <file>' in arguments")
endif()
file(COPY_FILE "${BENCH_LISTING}" "${_out_file}")
@@ -0,0 +1,85 @@
{
"version": 2,
"metadata": {
"name": "benchmark-template",
"catch2-version": "3.15.2"
},
"listings": {
"tests": [
{
"name": "Comparing function pointers",
"tags": [
"function pointer",
"Tricky"
]
},
{
"name": "Testing checked-if 4",
"tags": [
"!shouldfail",
"checked-if"
]
},
{
"name": "count_equidistant_floats - double",
"tags": [
"approvals",
"distance",
"floating-point"
]
},
{
"name": "Usage of AllTrue range matcher",
"tags": [
"matchers",
"quantifiers",
"templated"
]
},
{
"name": "Exception matchers that succeed",
"tags": [
"!throws",
"exceptions",
"matchers"
]
},
{
"name": "A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float",
"class-name": "Template_Fixture",
"tags": [
"class",
"template"
]
},
{
"name": "Approximate PI",
"tags": [
"Approx",
"PI"
]
},
{
"name": "TextFlow::Column respects width setting",
"tags": [
"approvals",
"column",
"TextFlow"
]
},
{
"name": "Generators internals",
"tags": [
"generators",
"internals"
]
},
{
"name": "Mayfail test case with nested sections",
"tags": [
"!mayfail"
]
}
]
}
}
+37 -1
View File
@@ -56,9 +56,9 @@ _Note that we redirect the output to `/dev/null` to reduce the overhead of the a
TODO:
* Start empty binary (set up cost base)
* Start binary with X (100/1k/10k) tests (test registration cost)
* Section tracking
## Compilation benchmarks
As tests are often iterated upon and relinked, the compilation cost of
@@ -79,3 +79,39 @@ hyperfine --warmup 2 --parameter-list version old,vas --prepare 'find ~/benches/
TODO:
* Link-only recipe
## Misc. benchmarks
### `catch_discover_tests`
The first JSON-based implementation of `catch_discover_tests` turned out
to have quadratic complexity in number of tests, which meant that registering
say 1k test cases took ~4s , and it became unusably slow with larger
test suites.
To prevent backsliding, and enable future optimizations, there is now
a benchmarking script for `catch_discover_tests` in the `discover_tests`
directory.
`discover_tests/benchmark_discovery.py` runs the real `catch_discover_tests`
implementation (through CMake script-mode) on a synthesized JSON listing
(generated from `discover_tests/listing_template.json`) through an executor
shim (`discover_tests/copy_shim.cmake`). This means it can run even without
real test binary.
If the JSON output from Catch2 changes, the synthesized JSON listing
can be updated by taking a real listing from `SelfTest` binary, and pruning
it down to keep only ~10 entries.
#### Examples
**Benchmark the current `catch_discover_tests`, with and without tags-as-labels**
```text
./benchmarks/discover_tests/benchmark_discovery.py --script ./extras/CatchAddTests.cmake
```
Note that 8k test cases is unrealistically high, but still useful to see scaling.
Benchmark for single test case is useful to provide estimate of the flat
overhead from using `catch_discover_tests` at all.
+1 -1
View File
@@ -26,7 +26,7 @@ The ```CHECK``` family are equivalent but execution continues in the same test c
Evaluates the expression and records the result. If an exception is thrown, it is caught, reported, and counted as a failure. These are the macros you will use most of the time.
Examples:
```
```cpp
CHECK( str == "string value" );
CHECK( thisReturnsTrue() );
REQUIRE( i == 42 );
+11
View File
@@ -212,6 +212,17 @@ execution (useful e.g. in cross-compilation environments).
calling ``catch_discover_tests``. This provides a mechanism for globally
selecting a preferred test discovery behavior.
_Note that on Apple Silicon with the Xcode generator you must use `PRE_TEST`,
e.g. `catch_discover_tests(tests DISCOVERY_MODE PRE_TEST)`. With the default
`POST_BUILD` mode the build fails with `Result: Subprocess killed`, because
macOS on Apple Silicon refuses to run unsigned binaries and Xcode code-signs
the test executable only **after** the post-build script that `POST_BUILD`
mode uses to run it for test discovery. `PRE_TEST` avoids this by delaying
discovery until test time, when the executable is already signed. The same
limitation affects CMake's `gtest_discover_tests`; see
[Catch2 #2411](https://github.com/catchorg/Catch2/issues/2411) and
[CMake #21845](https://gitlab.kitware.com/cmake/cmake/-/issues/21845)._
* `SKIP_IS_FAILURE`
Skipped tests will be marked as failed instead.
+8 -3
View File
@@ -26,7 +26,7 @@
[Disable statistical analysis of collected benchmark samples](#disable-statistical-analysis-of-collected-benchmark-samples)<br>
[Specify the amount of time in milliseconds spent on warming up each test](#specify-the-amount-of-time-in-milliseconds-spent-on-warming-up-each-test)<br>
[Usage](#usage)<br>
[Specify the section to run](#specify-the-section-to-run)<br>
[Specify the section/generator element to run](#specify-the-sectiongenerator-element-to-run)<br>
[Filenames as tags](#filenames-as-tags)<br>
[Override output colouring](#override-output-colouring)<br>
[Test Sharding](#test-sharding)<br>
@@ -202,8 +202,13 @@ as many times as you want, e.g. `--reporter xml::out=someFile.xml` or
The keys must either be prefixed by "X", in which case they are not parsed
by Catch2 and are only passed down to the reporter, or one of options
hardcoded into Catch2. Currently there are only 2,
["out"](#sending-output-to-a-file), and ["colour-mode"](#colour-mode).
hardcoded into Catch2. Currently there are 3 supported options:
* ["out"](#sending-output-to-a-file)
* ["colour-mode"](#colour-mode)
* ["verbosity"](#output-verbosity)
> Support for per-reporter verbosity option was added in Catch2 vX.Y.Z
_Note that the reporter might still check the X-prefixed options for
validity, and throw an error if they are wrong._
+15
View File
@@ -7,6 +7,8 @@
[Writing documentation](#writing-documentation)<br>
[Writing code](#writing-code)<br>
[CoC](#coc)<br>
[Using LLMs when contributing](#using-llms-when-contributing)<br>
So you want to contribute something to Catch2? That's great! Whether it's
a bug fix, a new feature, support for additional compilers - or just
@@ -332,6 +334,19 @@ When adding new `CATCH_CONFIG` option, there are multiple places to edit:
This project has a [CoC](../CODE_OF_CONDUCT.md). Please adhere to it
while contributing to Catch2.
## Using LLMs when contributing
I do not care whether you used LLM for your contribution. What I care
about is the quality of the contribution, not whether you made it through
prompting LLM, letting GAs run wild, dictated it into the computer, wrote
it using your nose or used butterflies to flip the correct bits.
The flipside of this is that I am also not going to iterate your LLM for
you. If a PR looks LLM generated and does not pass the muster to be merged
as-is, I am going to close it.
-----------
_This documentation will always be in-progress as new information comes
+42
View File
@@ -2,6 +2,9 @@
# Release notes
**Contents**<br>
[3.15.3](#3153)<br>
[3.15.2](#3152)<br>
[3.15.1](#3151)<br>
[3.15.0](#3150)<br>
[3.14.0](#3140)<br>
[3.13.0](#3130)<br>
@@ -75,6 +78,45 @@
[Even Older versions](#even-older-versions)<br>
## 3.15.3
### Fixes
* The JSON reporter's number handling is locale-independent. (#3176)
* Removed leftover debug message from `catch_discover_tests` (#3177)
* Fixed typo in the "Could not jump to the Nth element" exception message (#3181)
### Improvements
* `catch_discover_tests` registers tests in deterministic (alphabetical) order.
* `catch_discover_tests` has been rewritten to be massively faster.
* Preparing the actual CTest script is significantly faster.
* Parsing the JSON test array is significantly faster.
* Running without `ADD_TAGS_AS_LABELS` set is 10-15% faster (as opposed to being the same speed) (#3169)
* The new implementation is about 4x-5x faster, so registering 500 tests now takes ~350ms (down from 1.1s).
* JSON writing is faster
* Small improvement in writing non-string values
* ~6-40% improvement in writing string values that do not need escaping
* ~1-6% improvement in writing string values that do need escaping
* Better support for infs and NaNs in JSON output
## 3.15.2
### Fixes
* Fixed `--warn InfiniteGenerators` triggering even if the generator was limited to specific element via path filtering.
* Fixed `-Wunused-parameter` triggering in `-fnoexceptions` builds.
### Improvements
* `catch_discover_tests` can handle cases where the binary prints out non-Catch2 output due to global constructors (#3162)
## 3.15.1
### Fixes
* Fixed potential OOB access when looking for start of broken UTF-8 sequence during linebreaking (#3096)
* Fixed `TEMPLATE_LIST_TEST_CASE_METHOD` and `CATCH_TEMPLATE_PRODUCT_TEST_CASE` potentially causing ODR violations (#3161)
## 3.15.0
### Fixes
+4 -4
View File
@@ -43,13 +43,13 @@ them write into different destinations. The two main uses of this are
Specifying multiple reporter looks like this:
```
--reporter JUnit::out=result-junit.xml --reporter console::out=-::colour-mode=ansi
--reporter JUnit::out=result-junit.xml --reporter console::out=-::colour-mode=ansi::verbosity=quiet
```
This tells Catch2 to use two reporters, `JUnit` reporter that writes
its machine-readable XML output to file `result-junit.xml`, and the
`console` reporter that writes its user-friendly output to stdout and
uses ANSI colour codes for colouring the output.
`console` reporter that writes its user-friendly output to stdout, uses
ANSI colour codes for colouring the output and is set to "quiet" verbosity.
Using multiple reporters (or one reporter and one-or-more [event
listeners](event-listeners.md#top)) can have surprisingly complex semantics
@@ -110,7 +110,7 @@ passing and failing assertions.
_Generally we recommend that if you override a member function from either
of the bases, you call into the base's implementation first. This is not
necessarily in all cases, but it is safer and easier._
necessary in all cases, but it is safer and easier._
Writing your own reporter then looks like this:
+6 -4
View File
@@ -36,10 +36,12 @@ Tags allow an arbitrary number of additional strings to be associated with a tes
As an example - given the following test cases:
TEST_CASE( "A", "[widget]" ) { /* ... */ }
TEST_CASE( "B", "[widget]" ) { /* ... */ }
TEST_CASE( "C", "[gadget]" ) { /* ... */ }
TEST_CASE( "D", "[widget][gadget]" ) { /* ... */ }
```cpp
TEST_CASE( "A", "[widget]" ) { /* ... */ }
TEST_CASE( "B", "[widget]" ) { /* ... */ }
TEST_CASE( "C", "[gadget]" ) { /* ... */ }
TEST_CASE( "D", "[widget][gadget]" ) { /* ... */ }
```
The tag expression, ```"[widget]"``` selects A, B & D. ```"[gadget]"``` selects C & D. ```"[widget][gadget]"``` selects just D and ```"[widget],[gadget]"``` selects all four test cases.
+13 -2
View File
@@ -44,7 +44,7 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
``catch_discover_tests`` sets up a post-build command on the test executable
that generates the list of tests by parsing the output from running the test
with the ``--list-test-names-only`` argument. This ensures that the full
with the ``--list-tests --reporter json`` argument. This ensures that the full
list of tests is obtained. Since test discovery occurs at build time, it is
not necessary to re-run CMake when the list of tests changes.
However, it requires that :prop_tgt:`CROSSCOMPILING_EMULATOR` is properly set
@@ -58,6 +58,11 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
directory property. The set of discovered tests is made accessible to such a
script via the ``<target>_TESTS`` variable.
Note that ``<target>_TESTS`` variable contains test names with brackets
("[", "]") escaped into ASCII char 2, 3 respectively, to work around CMake's
list parsing rules. You have to unescape them back for each element to get
the original names.
The options are:
``target``
@@ -67,7 +72,7 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
``TEST_SPEC arg1...``
Specifies test cases, wildcarded test cases, tags and tag expressions to
pass to the Catch executable with the ``--list-test-names-only`` argument.
pass to the Catch executable when listing the tests.
``EXTRA_ARGS arg1...``
Any extra arguments to pass on the command line to each test case.
@@ -146,6 +151,12 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
calling ``catch_discover_tests``. This provides a mechanism for globally selecting
a preferred test discovery behavior without having to modify each call site.
On Apple Silicon with the Xcode generator you must use ``PRE_TEST``. With the
default ``POST_BUILD`` mode the build fails with ``Result: Subprocess killed``,
because macOS on Apple Silicon refuses to run unsigned binaries and Xcode
code-signs the test executable only after the post-build script that
``POST_BUILD`` mode uses to run it for test discovery. See Catch2 issue #2411.
``SKIP_IS_FAILURE``
Disables skipped test detection.
+346 -53
View File
@@ -1,23 +1,220 @@
# Distributed under the OSI-approved BSD 3-Clause License. See accompanying
# file Copyright.txt or https://cmake.org/licensing for details.
function(add_command NAME)
set(_args "")
# use ARGV* instead of ARGN, because ARGN splits arrays into multiple arguments
math(EXPR _last_arg ${ARGC}-1)
foreach(_n RANGE 1 ${_last_arg})
set(_arg "${ARGV${_n}}")
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]") # form a bracket_argument
else()
set(_args "${_args} ${_arg}")
# Because natively using CMake's JSON processing for arrays leads to quadratic
# running times, we do terrible hack and split JSON array by object boundary
# + commas and try to reconstruct valid JSON objects. During this, we need
# to replace CMake characters that could be in the test name/tags with
# placeholder, so it doesn't affect CMake's processing of the strings/lists
# we create during the parsing.
#
# We use 0x01, 0x02, 0x03, and 0x04 as placeholder bytes, as those cannot
# exist in JSON unescaped.
#
# 0x01 <=> ';' (CMake list separator)
# 0x02 <-> '[' (opens a CMake bracket-argument context)
# 0x03 <-> ']' (closes a CMake bracket-argument context)
# 0x04 == element boundary marker used while splitting the tests array
#
string(ASCII 1 _SemicolonEscape)
string(ASCII 2 _OpenBracketEscape)
string(ASCII 3 _CloseBracketEscape)
string(ASCII 4 _BoundaryEscape)
# As far as I can tell, the only way of having unclosed '[' or ']' in the
# list of test names we define for CTest, is to escape them into completely
# different characters. Otherwise, they will break parsing of unrelated
# semicolons (inserted by CMake after parsing the list from the user-facing
# space separated format). This means that the consumers of the list have
# to unescape them back, but that's CMake :v
#
# 0x02 <-> '['
# 0x03 <-> ']'
#
string(ASCII 2 _OpenBracketListingEscape)
string(ASCII 3 _CloseBracketListingEscape)
# Placeholder bytes in the listing would break our parsing hack, so
# we check they don't exist. They shouldn't exist in valid JSON, but
# the reporter might not be escaping them properly.
function(validate_input_noescapes listing_var)
foreach(byte "${_SemicolonEscape}" "${_BoundaryEscape}" "${_OpenBracketEscape}" "${_CloseBracketEscape}")
string(FIND "${${listing_var}}" "${byte}" found)
if(NOT found EQUAL -1)
message(FATAL_ERROR
"The test listing contains raw control byte (0x01-0x04) which should not "
"be there. This means either bad escaping in JSON reporter, or corrupted file. "
)
endif()
endforeach()
set(script "${script}${NAME}(${_args})\n" PARENT_SCOPE)
endfunction()
# Replaces relevant characters with their placeholders, see the top of this file.
# Modifies argument `var` in place.
function(magic_escape_chars var)
set(value "${${var}}")
string(REPLACE ";" "${_SemicolonEscape}" value "${value}")
string(REPLACE "[" "${_OpenBracketEscape}" value "${value}")
string(REPLACE "]" "${_CloseBracketEscape}" value "${value}")
set(${var} "${value}" PARENT_SCOPE)
endfunction()
# Turns placeholders back into original characters, see the top of this file.
# Modifies argument `var` in place.
function(magic_unescape_chars var)
set(value "${${var}}")
string(REPLACE "${_SemicolonEscape}" ";" value "${value}")
string(REPLACE "${_OpenBracketEscape}" "[" value "${value}")
string(REPLACE "${_CloseBracketEscape}" "]" value "${value}")
set(${var} "${value}" PARENT_SCOPE)
endfunction()
# Abuses knowledge of Catch2's JSON reporter output for listing tests to
# split JSON array of the test listings into a CMake list of strings,
# with each element being the JSON string of one array entry.
#
# This avoids the terrible quadratic running time of using CMake's JSON
# support to parse the JSON reporter output "properly", where the whole
# JSON array of tests is parsed again for every element. Instead, we can
# use the CMake's API to only parse the individual test's objects, which
# are usually small and only have to be reparsed fixed number of times
# (once for test names, once for labels).
#
# We process the string representing the JSON array by splitting it on
# `}<ws>*,<ws>*{` and then checking for each chunk whether it is a valid
# JSON object representing Catch2's test. If not (e.g. because we split
# on the presence of `}<ws>*,<ws>*{` inside a test name), then we append
# the next chunk to the current one and check again. And again, until
# we get back to a valid JSON.
#
# Note that to support passing the object strings back from the function,
# they will still contain the placeholders and need to be unescaped before
# further processing (e.g. sending them into CMake's JSON parsing API).
function(split_json_array json_array_var out_var)
# We have to pass the input by var name to avoid CMake processing
# the input as an arg.
set(json_in "${${json_array_var}}")
# Strip the array brackets at the start and end of the JSON array.
# Must happen before we escape the other [] instances below from the
# actual array data.
string(REGEX REPLACE "^[ \t\r\n]*\\[" "" json_in "${json_in}")
string(REGEX REPLACE "\\][ \t\r\n]*$" "" json_in "${json_in}")
magic_escape_chars(json_in)
# We need to keep the whitespace around comma around, so that if we
# split inside the test object, we can reconstruct it losslessly.
string(REGEX REPLACE "(}[ \t\r\n]*)[,]([ \t\r\n]*{)" "\\1${_BoundaryEscape}\\2" json_in "${json_in}")
# We escaped all list separators above, so now we can turn the JSON
# string into a CMake list of fragments in single pass.
string(REPLACE "${_BoundaryEscape}" ";" fragments "${json_in}")
# And now we have to reconstruct the actual JSON structure from fragments.
set(array_elements "")
set(accumulator "")
foreach(next_fragment IN LISTS fragments)
if(accumulator)
set(accumulator "${accumulator},${next_fragment}")
else()
set(accumulator "${next_fragment}")
endif()
# Because the fragments (might) contain invalid JSON characters due
# to escaping, we have to unescape it before checking if we can parse it.
set(maybe_json "${accumulator}")
magic_unescape_chars(maybe_json)
string(JSON unused ERROR_VARIABLE err GET "${maybe_json}" "name")
if(err STREQUAL "NOTFOUND")
list(APPEND array_elements "${accumulator}")
set(accumulator "")
endif()
endforeach()
set(${out_var} "${array_elements}" PARENT_SCOPE)
endfunction()
# Prepare (a part of) command with bracketed arguments and return it via `out_var`.
#
# This allows the registration script to escape parts of the test script
# only once, instead of escaping the unchanged arguments over and over again.
function(prepare_command_fragment out_var)
set(_args "")
math(EXPR _last_arg ${ARGC}-1)
if(_last_arg GREATER_EQUAL 1)
foreach(_n RANGE 1 ${_last_arg})
set(_arg "${ARGV${_n}}")
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]") # form a bracket_argument
else()
set(_args "${_args} ${_arg}")
endif()
endforeach()
endif()
set(${out_var} "${_args}" PARENT_SCOPE)
endfunction()
# Generates random filename in the temp folder.
# Temp folder is retrieved by checking env vars from various platforms.
function(make_temp_file_path OUT_VARIABLE FALLBACK_PATH)
set(TEMP_DIR "")
set(ENV_VARS
# From XDG base dir specification
XDG_RUNTIME_DIR
# From POSIX standard
TMPDIR
# From Windows
TMP
TEMP
)
foreach(var ${ENV_VARS})
if(DEFINED ENV{${var}} AND NOT "$ENV{${var}}" STREQUAL "")
set(TEMP_DIR "$ENV{${var}}")
break()
endif()
endforeach()
# If all checks fail, we use the fallback path
if(TEMP_DIR STREQUAL "")
set(TEMP_DIR "${FALLBACK_PATH}")
endif()
file(TO_CMAKE_PATH "${TEMP_DIR}" TEMP_DIR)
# Generate the random file name
string(RANDOM LENGTH 8 RAND_ID)
set(FINAL_TEMP_PATH "${TEMP_DIR}/Catch2-test-listing.${RAND_ID}.json")
set(${OUT_VARIABLE} "${FINAL_TEMP_PATH}" PARENT_SCOPE)
endfunction()
# Computes the path of the test-list file based on the path for the main
# CTest script file (passed in `CTEST_FILE`).
#
# It works by replacing the `_tests` part of the original name with
# `_test-list`, or failing that, it appends `-list.cmake` instead.
#
# <base>_tests.cmake -> <base>_test-list.cmake
# <base>_tests-Debug.cmake -> <base>_test-list-Debug.cmake
function(make_test_list_file_path CTEST_FILE OUT_VARIABLE)
if("${CTEST_FILE}" MATCHES "_tests(.*)\\.cmake$")
string(REGEX REPLACE "_tests(.*)\\.cmake$" "_test-list\\1.cmake" list_file "${CTEST_FILE}")
else()
set(list_file "${CTEST_FILE}-list.cmake")
endif()
set(${OUT_VARIABLE} "${list_file}" PARENT_SCOPE)
endfunction()
function(catch_discover_tests_impl)
cmake_parse_arguments(
""
""
@@ -26,6 +223,17 @@ function(catch_discover_tests_impl)
${ARGN}
)
# We write the list of all discovered tests into a separate file
make_test_list_file_path("${_CTEST_FILE}" _CTEST_LIST_FILE)
# We periodically append to the output files below, so we have to ensure
# that it is empty at the start, or we get duplicated test scripts.
file(REMOVE "${_CTEST_FILE}")
file(REMOVE "${_CTEST_LIST_FILE}")
# Size (in Bytes) at which the intermediate `script` var is dumped to file.
set(_WriteToFileThreshold 50000)
set(add_tags "${_ADD_TAGS_AS_LABELS}")
set(prefix "${_TEST_PREFIX}")
set(suffix "${_TEST_SUFFIX}")
@@ -42,6 +250,10 @@ function(catch_discover_tests_impl)
set(script)
set(suite)
set(tests)
# Holds the list of all registered test names **as a string**, not list.
# This avoids issue of CMake removing semicolon escapes even inside
# bracket-quoted strings.
set(_test_names)
if(WIN32)
set(dl_paths_variable_name PATH)
@@ -72,8 +284,19 @@ function(catch_discover_tests_impl)
set(ENV{DYLD_FRAMEWORK_PATH} "${paths}")
endif()
make_temp_file_path(listing_output_path "${_TEST_WORKING_DIR}")
set(_Verbosity "quiet")
if (add_tags)
set(_Verbosity "normal")
endif()
execute_process(
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" ${spec} --list-tests --reporter json
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" ${spec}
--list-tests
--reporter "json::verbosity=${_Verbosity}"
--out "${listing_output_path}"
--order lex # Make sure the output order, and thus test registration order, is consistent across runs.
OUTPUT_VARIABLE listing_output
RESULT_VARIABLE result
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
@@ -86,6 +309,11 @@ function(catch_discover_tests_impl)
)
endif()
# Read the JSON output back from the output file and validate it.
file(READ ${listing_output_path} listing_output)
file(REMOVE ${listing_output_path})
validate_input_noescapes(listing_output)
# Prepare reporter
if(reporter)
set(reporter_arg "--reporter ${reporter}")
@@ -136,26 +364,87 @@ function(catch_discover_tests_impl)
# Parse JSON output for list of tests/class names/tags
string(JSON version GET "${listing_output}" "version")
if(NOT version STREQUAL "1")
if(NOT version STREQUAL "2")
message(FATAL_ERROR "Unsupported catch output version: '${version}'")
endif()
# Speed-up reparsing by cutting away unneeded parts of JSON.
# Extract just the JSON array with tests and then split them into
# individual objects.
string(JSON test_listing GET "${listing_output}" "listings" "tests")
string(JSON num_tests LENGTH "${test_listing}")
split_json_array(test_listing tests)
# Exit early if no tests are detected
if(num_tests STREQUAL "0")
file(WRITE "${_CTEST_FILE}" "")
if(NOT tests)
# Still emit an (empty) test list file and have the main script include
# it, so that consumers relying on the `${_TEST_LIST}` variable and on the
# include structure get consistent behavior regardless of test count.
file(WRITE "${_CTEST_LIST_FILE}" "set(${_TEST_LIST})\n")
file(WRITE "${_CTEST_FILE}" "include(\"${_CTEST_LIST_FILE}\")\n")
return()
endif()
# CMake's foreach-RANGE is inclusive, so we have to subtract 1
math(EXPR num_tests "${num_tests} - 1")
foreach(idx RANGE ${num_tests})
string(JSON single_test GET ${test_listing} ${idx})
string(JSON test_tags GET "${single_test}" "tags")
# The 'set(VAR` header for the test list has to be written separately,
# so that each test name can be appended file without further processing.
set(test_names "set(${_TEST_LIST}")
# Most of the commands/arguments in the CTest script are identical
# for every test registered with one `catch_discover_tests` call.
# To avoid repeating the work in escaping them, we escape them before
# the per-test loop and reuse the escaped fragments.
#
# `add_test` calls are
# add_test(<name><exec><exe><escaped_name><extra_args><reporter><out_dir>)
# Of these, <exec>,<exe>,<extra_args>, and <reporter> are the same between
# all tests.
prepare_command_fragment(_exec_exe_fragment
${_TEST_EXECUTOR}
"${_TEST_EXECUTABLE}"
)
prepare_command_fragment(_args_reporter_fragment
${extra_args}
"${reporter_arg}"
)
# `set_tests_properties` calls are
# set_tests_properties(<name> PROPERTIES WORKING_DIRECTORY <dir> <properties>)
# set_tests_properties(<name> PROPERTIES ENVIRONMENT_MODIFICATION <env_mod>)
# Of these, only the <name> changes between tests.
prepare_command_fragment(_properties_fragment
PROPERTIES
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
${properties}
)
# Env modification is optional, so we prepare it in a separate command
if(environment_modifications)
prepare_command_fragment(_env_modification_fragment
PROPERTIES
ENVIRONMENT_MODIFICATION "${environment_modifications}"
)
endif()
# Each element in the tests is JSON-string representing one test object.
# We have to parse it and then turn it into CTest script commands.
foreach(single_test IN LISTS tests)
# Because appending to the same string in CMake has quadratic runtime,
# we flush the script into the file periodically to avoid the worst case.
string(LENGTH "${script}" script_len)
if (script_len GREATER _WriteToFileThreshold)
file(APPEND "${_CTEST_FILE}" "${script}")
set(script "")
endif()
string(LENGTH "${test_names}" names_len)
if (names_len GREATER _WriteToFileThreshold)
file(APPEND "${_CTEST_LIST_FILE}" "${test_names}")
set(test_names "")
endif()
# The elements are still escaped and contain JSON-invalid characters,
# they have to be unescaped before parsing them as JSON.
magic_unescape_chars(single_test)
if(add_tags)
string(JSON test_tags GET "${single_test}" "tags")
endif()
string(JSON plain_name GET "${single_test}" "name")
# Escape characters in test case names that would be parsed by Catch2
@@ -170,22 +459,15 @@ function(catch_discover_tests_impl)
set(output_dir_arg "--out ${output_dir}/${output_prefix}${escaped_name_clean}${output_suffix}")
endif()
# ...and add to script
add_command(add_test
"${prefix}${plain_name}${suffix}"
${_TEST_EXECUTOR}
"${_TEST_EXECUTABLE}"
"${escaped_name}"
${extra_args}
"${reporter_arg}"
"${output_dir_arg}"
)
add_command(set_tests_properties
"${prefix}${plain_name}${suffix}"
PROPERTIES
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
${properties}
)
set(full_name "${prefix}${plain_name}${suffix}")
prepare_command_fragment(_full_name_fragment "${full_name}")
prepare_command_fragment(_escaped_name_fragment "${escaped_name}")
prepare_command_fragment(_outdir_fragment "${output_dir_arg}")
string(APPEND script
"add_test(${_full_name_fragment}${_exec_exe_fragment}${_escaped_name_fragment}${_args_reporter_fragment}${_outdir_fragment})\n")
string(APPEND script
"set_tests_properties(${_full_name_fragment}${_properties_fragment})\n")
if(add_tags)
string(JSON num_tags LENGTH "${test_tags}")
@@ -205,33 +487,44 @@ function(catch_discover_tests_impl)
list(APPEND tag_list "${a_tag}")
endforeach()
add_command(set_tests_properties
"${prefix}${plain_name}${suffix}"
prepare_command_fragment(_labels_fragment
PROPERTIES
LABELS "${tag_list}"
)
string(APPEND script "set_tests_properties(${_full_name_fragment}${_labels_fragment})\n")
endif()
endif(add_tags)
if(environment_modifications)
add_command(set_tests_properties
"${prefix}${plain_name}${suffix}"
PROPERTIES
ENVIRONMENT_MODIFICATION "${environment_modifications}")
string(APPEND script "set_tests_properties(${_full_name_fragment}${_env_modification_fragment})\n")
endif()
list(APPEND tests "${prefix}${plain_name}${suffix}")
# The test name has to be escaped using the same rules as prepare_command
# uses for the arguments, so that it keeps being single element in list
# even with weird characters and semicolons.
set(full_list_name "${prefix}${plain_name}${suffix}")
string(REPLACE ";" "\\;" full_list_name "${full_list_name}")
string(REPLACE "[" "${_OpenBracketListingEscape}" full_list_name "${full_list_name}")
string(REPLACE "]" "${_CloseBracketListingEscape}" full_list_name "${full_list_name}")
if(full_list_name MATCHES "[^-./:a-zA-Z0-9_]")
# The space before the start of quote is important, so that we get
# space-separated list in the final file.
string(APPEND test_names " [==[${full_list_name}]==]")
else()
string(APPEND test_names " ${full_list_name}")
endif()
endforeach()
# Create a list of all discovered tests, which users may use to e.g. set
# properties on the tests
add_command(set ${_TEST_LIST} ${tests})
# Write CTest script
file(WRITE "${_CTEST_FILE}" "${script}")
# Write any test names leftovers we have
file(APPEND "${_CTEST_LIST_FILE}" "${test_names})\n")
# Write any main script leftovers we have, and append the include of test names
file(APPEND "${_CTEST_FILE}" "${script}")
file(APPEND "${_CTEST_FILE}" "include(\"${_CTEST_LIST_FILE}\")\n")
endfunction()
if(CMAKE_SCRIPT_MODE_FILE)
# To enable `include`ing this file in the unit test scripts, we only run
# the impl if an actual `TEST_EXECUTABLE` is provided.
if(CMAKE_SCRIPT_MODE_FILE AND DEFINED TEST_EXECUTABLE)
catch_discover_tests_impl(
TEST_EXECUTABLE ${TEST_EXECUTABLE}
TEST_EXECUTOR ${TEST_EXECUTOR}
+110 -36
View File
@@ -6,8 +6,8 @@
// SPDX-License-Identifier: BSL-1.0
// Catch v3.15.0
// Generated: 2026-05-12 13:08:21.086523
// Catch v3.15.3
// Generated: 2026-07-26 22:17:52.418168
// ----------------------------------------------------------
// This file is an amalgamation of multiple different files.
// You probably shouldn't edit it directly.
@@ -2394,7 +2394,7 @@ namespace Catch {
}
Version const& libraryVersion() {
static Version version( 3, 15, 0, "", 0 );
static Version version( 3, 15, 3, "", 0 );
return version;
}
@@ -2541,7 +2541,7 @@ namespace Catch {
bool isValid = next();
if ( !isValid ) {
Detail::throw_generator_exception(
"Coud not jump to Nth element: not enough elements" );
"Could not jump to Nth element: not enough elements" );
}
}
}
@@ -4678,12 +4678,28 @@ namespace Detail {
#include <cmath>
#include <locale>
namespace Catch {
namespace {
static bool needsEscape( char c ) {
return c == '"' || c == '\\' || c == '\b' || c == '\f' ||
c == '\n' || c == '\r' || c == '\t';
struct EscapeLUT {
bool escape[256] = {};
constexpr EscapeLUT() {
escape[static_cast<unsigned char>( '"' )] = true;
escape[static_cast<unsigned char>( '\\' )] = true;
escape[static_cast<unsigned char>( '\b' )] = true;
escape[static_cast<unsigned char>( '\f' )] = true;
escape[static_cast<unsigned char>( '\n' )] = true;
escape[static_cast<unsigned char>( '\r' )] = true;
escape[static_cast<unsigned char>( '\t' )] = true;
}
};
static constexpr EscapeLUT escapeLUT{};
static constexpr bool needsEscape( char c ) {
return escapeLUT.escape[static_cast<unsigned char>( c )];
}
static Catch::StringRef makeEscapeStringRef( char c ) {
@@ -4795,7 +4811,10 @@ namespace Catch {
JsonValueWriter::JsonValueWriter( std::ostream& os,
std::uint64_t indent_level ):
m_os{ os }, m_indent_level{ indent_level } {}
m_os{ os }, m_indent_level{ indent_level } {
// We use C locale so that writing of numerical values is locale-independent.
m_sstream.imbue( std::locale::classic() );
}
JsonObjectWriter JsonValueWriter::writeObject() && {
return JsonObjectWriter{ m_os, m_indent_level };
@@ -4809,26 +4828,55 @@ namespace Catch {
writeImpl( value, true );
}
void JsonValueWriter::write( float value ) && {
writeFloatingPoint( value );
}
void JsonValueWriter::write( double value ) && {
writeFloatingPoint( value );
}
void JsonValueWriter::write( bool value ) && {
writeImpl( value ? "true"_sr : "false"_sr, false );
}
template <typename T>
void JsonValueWriter::writeFloatingPoint( T value ) {
if ( Catch::isnan( value ) ) {
writeImpl( "NaN"_sr, false );
} else if ( std::isinf( value ) ) {
writeImpl( value < 0 ? "-Infinity"_sr : "Infinity"_sr, false );
} else {
m_sstream << value;
writeImpl( m_sstream.str(), false );
}
}
void JsonValueWriter::writeImpl( Catch::StringRef value, bool quote ) {
if ( quote ) { m_os << '"'; }
size_t current_start = 0;
for ( size_t i = 0; i < value.size(); ++i ) {
if ( needsEscape( value[i] ) ) {
if ( current_start < i ) {
m_os << value.substr( current_start, i - current_start );
// Escaping only makes sense for actual strings, which are passed
// with `quote == true`. Non-quoted values are things like bools
// and numbers, which cannot create inputs that need escaping.
if ( !quote ) {
m_os << value;
} else {
m_os << '"';
size_t current_start = 0;
for ( size_t i = 0; i < value.size(); ++i ) {
if ( needsEscape( value[i] ) ) {
if ( current_start < i ) {
m_os
<< value.substr( current_start, i - current_start );
}
m_os << makeEscapeStringRef( value[i] );
current_start = i + 1;
}
m_os << makeEscapeStringRef( value[i] );
current_start = i + 1;
}
if ( current_start < value.size() ) {
m_os << value.substr( current_start,
value.size() - current_start );
}
m_os << '"';
}
if ( current_start < value.size() ) {
m_os << value.substr( current_start, value.size() - current_start );
}
if ( quote ) { m_os << '"'; }
}
} // namespace Catch
@@ -6016,6 +6064,8 @@ namespace Catch {
auto getGenerator() const -> GeneratorBasePtr const& override {
return m_generator;
}
bool isFilteredImpl() const override { return m_isFiltered; }
};
} // namespace
}
@@ -6360,17 +6410,6 @@ namespace Catch {
SourceLineInfo lineInfo,
Generators::GeneratorBasePtr&& generator ) {
// TBD: Do we want to avoid the warning if the generator is filtered?
if ( m_config->warnAboutInfiniteGenerators() &&
!generator->isFinite() ) {
// We want the semantics of `FAIL()`, but we inline it
// to avoid issues with conditionally prefixed macros
INTERNAL_CATCH_MSG( "FAIL",
Catch::ResultWas::ExplicitFailure,
Catch::ResultDisposition::Normal,
"GENERATE() would run infinitely" );
}
auto nameAndLoc = TestCaseTracking::NameAndLocation( static_cast<std::string>( generatorName ), lineInfo );
auto& currentTracker = m_trackerContext.currentTracker();
assert(
@@ -6383,11 +6422,24 @@ namespace Catch {
m_trackerContext,
&currentTracker,
CATCH_MOVE( generator ) );
auto ret = newTracker.get();
// The warning shouldn't fire if the generator is infinite, **but** filtered down.
if ( m_config->warnAboutInfiniteGenerators() &&
!newTracker->m_generator->isFinite() &&
!newTracker->isFiltered() ) {
// We want the semantics of `FAIL()`, but we inline it
// to avoid issues with conditionally prefixed macros
INTERNAL_CATCH_MSG( "FAIL",
Catch::ResultWas::ExplicitFailure,
Catch::ResultDisposition::Normal,
"GENERATE() would run infinitely" );
}
auto returnPtr = newTracker.get();
currentTracker.addChild( CATCH_MOVE( newTracker ) );
ret->open();
return ret;
returnPtr->open();
return returnPtr;
}
bool RunContext::testForMissingAssertions(Counts& assertions) {
@@ -7470,6 +7522,28 @@ namespace TestCaseTracking {
m_ctx.setCurrentTracker( this );
}
bool SectionTracker::isFilteredImpl() const {
// TBD: This is currently _very_ similar to the block in `isComplete`.
// Is this neccessarily that way, or just accident of current semantics?
const size_t filterIndex =
m_newStyleFilters ? m_allTrackerDepth : m_sectionOnlyDepth;
if ( filterIndex < m_filterRef->size() ) {
// 1) New style filter must explicitly target section
if ( m_newStyleFilters && ( *m_filterRef )[filterIndex].type !=
PathFilter::For::Section ) {
return true;
}
// 2) Both style filters must match the trimmed name exactly
if ( m_trimmed_name !=
StringRef( ( *m_filterRef )[filterIndex].filter ) ) {
return true;
}
}
return false;
}
SectionTracker::SectionTracker( NameAndLocation&& nameAndLocation, TrackerContext& ctx, ITracker* parent )
: TrackerBase( CATCH_MOVE(nameAndLocation), ctx, parent ),
m_trimmed_name(trim(StringRef(ITracker::nameAndLocation().name)))
@@ -8016,8 +8090,8 @@ namespace Catch {
m_it--;
}
// Skip back over UTF-8 continuation bytes to the leading byte
while ( isUtf8ContinuationByte( *m_it ) ) {
assert( m_it != m_string->begin() );
while ( m_it != m_string->begin() &&
isUtf8ContinuationByte( *m_it ) ) {
m_it--;
}
}
+96 -69
View File
@@ -6,8 +6,8 @@
// SPDX-License-Identifier: BSL-1.0
// Catch v3.15.0
// Generated: 2026-05-12 13:08:20.543985
// Catch v3.15.3
// Generated: 2026-07-26 22:17:52.004020
// ----------------------------------------------------------
// This file is an amalgamation of multiple different files.
// You probably shouldn't edit it directly.
@@ -7079,42 +7079,42 @@ namespace Catch {
INTERNAL_CATCH_EXPAND_VARGS( INTERNAL_CATCH_TEMPLATE_TEST_CASE_METHOD_2( INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_CLASS_ ), INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ) , ClassName, Name, Tags, Signature, __VA_ARGS__ ) )
#endif
#define INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD_2(TestNameClass, TestName, ClassName, Name, Tags, Signature, TmplTypes, TypesList)\
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS \
CATCH_INTERNAL_SUPPRESS_ZERO_VARIADIC_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_TEMPLATE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS \
#define INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD_2(TestNameClass, TestName, ClassName, Name, Tags, Signature, TmplTypes, TypesList)\
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS \
CATCH_INTERNAL_SUPPRESS_ZERO_VARIADIC_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_TEMPLATE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS \
namespace {\
template<typename TestType> \
struct TestName : INTERNAL_CATCH_REMOVE_PARENS(ClassName <TestType>) { \
void test();\
};\
namespace {\
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestNameClass) {\
INTERNAL_CATCH_TYPE_GEN \
INTERNAL_CATCH_NTTP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature))\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
std::size_t index = 0;\
using expander = std::size_t[];\
constexpr char const* tmpl_types[] = {CATCH_REC_LIST(INTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS, INTERNAL_CATCH_REMOVE_PARENS(TmplTypes))};\
constexpr char const* types_list[] = {CATCH_REC_LIST(INTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS, INTERNAL_CATCH_REMOVE_PARENS(TypesList))};\
constexpr auto num_types = sizeof(types_list) / sizeof(types_list[0]);\
(void)expander{(Catch::AutoReg( Catch::makeTestInvoker( &TestName<Types>::test ), CATCH_INTERNAL_LINEINFO, #ClassName, Catch::NameAndTags{ Name " - " + std::string(tmpl_types[index / num_types]) + '<' + types_list[index % num_types] + '>', Tags } ), index++)... };/* NOLINT */ \
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename create<TestNameClass, decltype(get_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
}(); \
}\
}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
template<typename TestType> \
void TestName<TestType>::test()
struct TestName : INTERNAL_CATCH_REMOVE_PARENS(ClassName <TestType>) { \
void test();\
};\
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestNameClass) {\
INTERNAL_CATCH_TYPE_GEN \
INTERNAL_CATCH_NTTP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature))\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
std::size_t index = 0;\
using expander = std::size_t[];\
constexpr char const* tmpl_types[] = {CATCH_REC_LIST(INTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS, INTERNAL_CATCH_REMOVE_PARENS(TmplTypes))};\
constexpr char const* types_list[] = {CATCH_REC_LIST(INTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS, INTERNAL_CATCH_REMOVE_PARENS(TypesList))};\
constexpr auto num_types = sizeof(types_list) / sizeof(types_list[0]);\
(void)expander{(Catch::AutoReg( Catch::makeTestInvoker( &TestName<Types>::test ), CATCH_INTERNAL_LINEINFO, #ClassName, Catch::NameAndTags{ Name " - " + std::string(tmpl_types[index / num_types]) + '<' + types_list[index % num_types] + '>', Tags } ), index++)... };/* NOLINT */ \
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename create<TestNameClass, decltype(get_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
}(); \
}\
}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
template<typename TestType> \
void TestName<TestType>::test()
#ifndef CATCH_CONFIG_TRADITIONAL_MSVC_PREPROCESSOR
#define INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD( ClassName, Name, Tags, ... )\
@@ -7132,37 +7132,37 @@ namespace Catch {
INTERNAL_CATCH_EXPAND_VARGS( INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD_2( INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ), INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ), ClassName, Name, Tags, Signature,__VA_ARGS__ ) )
#endif
#define INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD_2( TestNameClass, TestName, ClassName, Name, Tags, TmplList) \
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_TEMPLATE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_COMMA_WARNINGS \
#define INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD_2( TestNameClass, TestName, ClassName, Name, Tags, TmplList) \
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_TEMPLATE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_COMMA_WARNINGS \
namespace {\
template<typename TestType> \
struct TestName : INTERNAL_CATCH_REMOVE_PARENS(ClassName <TestType>) { \
void test();\
};\
namespace {\
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestName){ \
INTERNAL_CATCH_TYPE_GEN\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
size_t index = 0;\
using expander = size_t[];\
(void)expander{(Catch::AutoReg( Catch::makeTestInvoker( &TestName<Types>::test ), CATCH_INTERNAL_LINEINFO, #ClassName##_catch_sr, Catch::NameAndTags{ Name " - " INTERNAL_CATCH_STRINGIZE(TmplList) " - " + std::to_string(index), Tags } ), index++)... };/* NOLINT */ \
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename convert<TestNameClass, TmplList>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
}(); \
}}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
template<typename TestType> \
void TestName<TestType>::test()
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestName){ \
INTERNAL_CATCH_TYPE_GEN\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
size_t index = 0;\
using expander = size_t[];\
(void)expander{(Catch::AutoReg( Catch::makeTestInvoker( &TestName<Types>::test ), CATCH_INTERNAL_LINEINFO, #ClassName##_catch_sr, Catch::NameAndTags{ Name " - " INTERNAL_CATCH_STRINGIZE(TmplList) " - " + std::to_string(index), Tags } ), index++)... };/* NOLINT */ \
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename convert<TestNameClass, TmplList>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
}(); \
}}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
template<typename TestType> \
void TestName<TestType>::test()
#define INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD(ClassName, Name, Tags, TmplList) \
INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD_2( INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ), INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ), ClassName, Name, Tags, TmplList )
@@ -7484,6 +7484,8 @@ namespace Catch {
return m_translateFunction( ex );
}
#else
(void)it;
(void)itEnd;
return "You should never get here!";
#endif
}
@@ -7570,7 +7572,7 @@ namespace Catch {
#define CATCH_VERSION_MAJOR 3
#define CATCH_VERSION_MINOR 15
#define CATCH_VERSION_PATCH 0
#define CATCH_VERSION_PATCH 3
#endif // CATCH_VERSION_MACROS_HPP_INCLUDED
@@ -7826,7 +7828,7 @@ namespace Generators {
void skipToNthElementImpl( std::size_t n ) override {
if ( n >= m_values.size() ) {
Detail::throw_generator_exception(
"Coud not jump to Nth element: not enough elements" );
"Could not jump to Nth element: not enough elements" );
}
m_idx = n;
}
@@ -7910,8 +7912,7 @@ namespace Generators {
bool isFinite() const override {
for (auto const& gen : m_generators) {
if (!gen.isFinite()) { return false;
}
if (!gen.isFinite()) { return false; }
}
return true;
}
@@ -8010,7 +8011,7 @@ namespace Generators {
void skipToNthElementImpl( std::size_t n ) override {
if ( n >= m_target ) {
Detail::throw_generator_exception(
"Coud not jump to Nth element: not enough elements" );
"Could not jump to Nth element: not enough elements" );
}
m_generator.skipToNthElement( n );
@@ -10308,11 +10309,19 @@ namespace Catch {
writeImpl( value, !std::is_arithmetic<T>::value );
}
void write( StringRef value ) &&;
void write( float value ) &&;
void write( double value ) &&;
void write( bool value ) &&;
private:
void writeImpl( StringRef value, bool quote );
// Helper to deal with non-finite floating point values, which
// are not standard JSON, but we use JS/Python/etc. approach of
// emitting `NaN`, `Infinity`, `-Infinity` as number(like).
template <typename T>
void writeFloatingPoint( T value );
// Without this SFINAE, this overload is a better match
// for `std::string`, `char const*`, `char const[N]` args.
// While it would still work, it would cause code bloat
@@ -10657,6 +10666,8 @@ namespace TestCaseTracking {
using Children = std::vector<ITrackerPtr>;
virtual bool isFilteredImpl() const = 0;
protected:
enum CycleState {
NotStarted,
@@ -10754,6 +10765,20 @@ namespace TestCaseTracking {
* for internal debug checks.
*/
virtual bool isGeneratorTracker() const;
/**
* Returns true if the concrete tracker instance has a filter that applies to it.
*/
bool isFiltered() const {
// Fast path: are there even filters for tracker in this position?
const size_t filter_depth =
m_newStyleFilters ? m_allTrackerDepth : m_sectionOnlyDepth;
if ( m_filterRef->size() <= filter_depth ) { return false; }
// Slow path: If there are filters, ask the concrete tracker.
// This handles things like match-all filters for that tracker.
return isFilteredImpl();
}
};
class TrackerContext {
@@ -10809,6 +10834,8 @@ namespace TestCaseTracking {
// to not own the name, the name still has to outlive the `ITracker` parent, so
// this should still be safe.
StringRef m_trimmed_name;
bool isFilteredImpl() const override;
public:
SectionTracker( NameAndLocation&& nameAndLocation, TrackerContext& ctx, ITracker* parent );
+1 -1
View File
@@ -8,7 +8,7 @@
project(
'catch2',
'cpp',
version: '3.15.0', # CML version placeholder, don't delete
version: '3.15.3', # CML version placeholder, don't delete
license: 'BSL-1.0',
meson_version: '>=0.54.1',
)
+3 -1
View File
@@ -89,6 +89,7 @@ namespace Catch {
return lhs.name == rhs.name &&
lhs.outputFilename == rhs.outputFilename &&
lhs.colourMode == rhs.colourMode &&
lhs.verbosity == rhs.verbosity &&
lhs.customOptions == rhs.customOptions;
}
@@ -157,6 +158,7 @@ namespace Catch {
reporterSpec.outputFile() ? *reporterSpec.outputFile()
: data.defaultOutputFilename,
reporterSpec.colourMode().valueOr( data.defaultColourMode ),
reporterSpec.verbosity().valueOr( data.verbosity ),
reporterSpec.customOptions() } );
}
}
@@ -232,7 +234,7 @@ namespace Catch {
if ( bazelOutputFile ) {
m_data.reporterSpecifications.push_back(
{ "junit", std::string( bazelOutputFile ), {}, {} } );
{ "junit", std::string( bazelOutputFile ), {}, {}, {} } );
}
const auto bazelTestSpec = Detail::getEnv( "TESTBRIDGE_TEST_ONLY" );
+1
View File
@@ -35,6 +35,7 @@ namespace Catch {
std::string name;
std::string outputFilename;
ColourMode colourMode;
Verbosity verbosity;
std::map<std::string, std::string> customOptions;
friend bool operator==( ProcessedReporterSpec const& lhs,
ProcessedReporterSpec const& rhs );
+2
View File
@@ -52,6 +52,7 @@ namespace Catch {
ReporterConfig( config,
makeStream( spec.outputFilename ),
spec.colourMode,
spec.verbosity,
spec.customOptions ) );
}
@@ -68,6 +69,7 @@ namespace Catch {
ReporterConfig( config,
makeStream( reporterSpec.outputFilename ),
reporterSpec.colourMode,
reporterSpec.verbosity,
reporterSpec.customOptions ) ) );
}
+2
View File
@@ -41,6 +41,8 @@ namespace Catch {
return m_translateFunction( ex );
}
#else
(void)it;
(void)itEnd;
return "You should never get here!";
#endif
}
+1 -1
View File
@@ -36,7 +36,7 @@ namespace Catch {
}
Version const& libraryVersion() {
static Version version( 3, 15, 0, "", 0 );
static Version version( 3, 15, 3, "", 0 );
return version;
}
+1 -1
View File
@@ -10,6 +10,6 @@
#define CATCH_VERSION_MAJOR 3
#define CATCH_VERSION_MINOR 15
#define CATCH_VERSION_PATCH 0
#define CATCH_VERSION_PATCH 3
#endif // CATCH_VERSION_MACROS_HPP_INCLUDED
+2 -3
View File
@@ -95,7 +95,7 @@ namespace Generators {
void skipToNthElementImpl( std::size_t n ) override {
if ( n >= m_values.size() ) {
Detail::throw_generator_exception(
"Coud not jump to Nth element: not enough elements" );
"Could not jump to Nth element: not enough elements" );
}
m_idx = n;
}
@@ -179,8 +179,7 @@ namespace Generators {
bool isFinite() const override {
for (auto const& gen : m_generators) {
if (!gen.isFinite()) { return false;
}
if (!gen.isFinite()) { return false; }
}
return true;
}
@@ -27,7 +27,7 @@ namespace Generators {
void skipToNthElementImpl( std::size_t n ) override {
if ( n >= m_target ) {
Detail::throw_generator_exception(
"Coud not jump to Nth element: not enough elements" );
"Could not jump to Nth element: not enough elements" );
}
m_generator.skipToNthElement( n );
@@ -28,7 +28,7 @@ namespace Catch {
bool isValid = next();
if ( !isValid ) {
Detail::throw_generator_exception(
"Coud not jump to Nth element: not enough elements" );
"Could not jump to Nth element: not enough elements" );
}
}
}
@@ -19,10 +19,12 @@ namespace Catch {
IConfig const* _fullConfig,
Detail::unique_ptr<IStream> _stream,
ColourMode colourMode,
Verbosity verbosity,
std::map<std::string, std::string> customOptions ):
m_stream( CATCH_MOVE(_stream) ),
m_fullConfig( _fullConfig ),
m_colourMode( colourMode ),
m_verbosity( verbosity ),
m_customOptions( CATCH_MOVE( customOptions ) ) {}
Detail::unique_ptr<IStream> ReporterConfig::takeStream() && {
@@ -31,6 +33,7 @@ namespace Catch {
}
IConfig const * ReporterConfig::fullConfig() const { return m_fullConfig; }
ColourMode ReporterConfig::colourMode() const { return m_colourMode; }
Verbosity ReporterConfig::verbosity() const { return m_verbosity; }
std::map<std::string, std::string> const&
ReporterConfig::customOptions() const {
@@ -12,6 +12,7 @@
#include <catch2/catch_test_run_info.hpp>
#include <catch2/catch_totals.hpp>
#include <catch2/catch_assertion_result.hpp>
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/internal/catch_message_info.hpp>
#include <catch2/internal/catch_stringref.hpp>
#include <catch2/internal/catch_unique_ptr.hpp>
@@ -36,6 +37,7 @@ namespace Catch {
ReporterConfig( IConfig const* _fullConfig,
Detail::unique_ptr<IStream> _stream,
ColourMode colourMode,
Verbosity verbosity,
std::map<std::string, std::string> customOptions );
ReporterConfig( ReporterConfig&& ) = default;
@@ -45,12 +47,14 @@ namespace Catch {
Detail::unique_ptr<IStream> takeStream() &&;
IConfig const* fullConfig() const;
ColourMode colourMode() const;
Verbosity verbosity() const;
std::map<std::string, std::string> const& customOptions() const;
private:
Detail::unique_ptr<IStream> m_stream;
IConfig const* m_fullConfig;
ColourMode m_colourMode;
Verbosity m_verbosity;
std::map<std::string, std::string> m_customOptions;
};
+65 -16
View File
@@ -7,14 +7,31 @@
// SPDX-License-Identifier: BSL-1.0
#include <catch2/internal/catch_enforce.hpp>
#include <catch2/internal/catch_jsonwriter.hpp>
#include <catch2/internal/catch_polyfills.hpp>
#include <catch2/internal/catch_unreachable.hpp>
#include <cmath>
#include <locale>
namespace Catch {
namespace {
static bool needsEscape( char c ) {
return c == '"' || c == '\\' || c == '\b' || c == '\f' ||
c == '\n' || c == '\r' || c == '\t';
struct EscapeLUT {
bool escape[256] = {};
constexpr EscapeLUT() {
escape[static_cast<unsigned char>( '"' )] = true;
escape[static_cast<unsigned char>( '\\' )] = true;
escape[static_cast<unsigned char>( '\b' )] = true;
escape[static_cast<unsigned char>( '\f' )] = true;
escape[static_cast<unsigned char>( '\n' )] = true;
escape[static_cast<unsigned char>( '\r' )] = true;
escape[static_cast<unsigned char>( '\t' )] = true;
}
};
static constexpr EscapeLUT escapeLUT{};
static constexpr bool needsEscape( char c ) {
return escapeLUT.escape[static_cast<unsigned char>( c )];
}
static Catch::StringRef makeEscapeStringRef( char c ) {
@@ -126,7 +143,10 @@ namespace Catch {
JsonValueWriter::JsonValueWriter( std::ostream& os,
std::uint64_t indent_level ):
m_os{ os }, m_indent_level{ indent_level } {}
m_os{ os }, m_indent_level{ indent_level } {
// We use C locale so that writing of numerical values is locale-independent.
m_sstream.imbue( std::locale::classic() );
}
JsonObjectWriter JsonValueWriter::writeObject() && {
return JsonObjectWriter{ m_os, m_indent_level };
@@ -140,26 +160,55 @@ namespace Catch {
writeImpl( value, true );
}
void JsonValueWriter::write( float value ) && {
writeFloatingPoint( value );
}
void JsonValueWriter::write( double value ) && {
writeFloatingPoint( value );
}
void JsonValueWriter::write( bool value ) && {
writeImpl( value ? "true"_sr : "false"_sr, false );
}
template <typename T>
void JsonValueWriter::writeFloatingPoint( T value ) {
if ( Catch::isnan( value ) ) {
writeImpl( "NaN"_sr, false );
} else if ( std::isinf( value ) ) {
writeImpl( value < 0 ? "-Infinity"_sr : "Infinity"_sr, false );
} else {
m_sstream << value;
writeImpl( m_sstream.str(), false );
}
}
void JsonValueWriter::writeImpl( Catch::StringRef value, bool quote ) {
if ( quote ) { m_os << '"'; }
size_t current_start = 0;
for ( size_t i = 0; i < value.size(); ++i ) {
if ( needsEscape( value[i] ) ) {
if ( current_start < i ) {
m_os << value.substr( current_start, i - current_start );
// Escaping only makes sense for actual strings, which are passed
// with `quote == true`. Non-quoted values are things like bools
// and numbers, which cannot create inputs that need escaping.
if ( !quote ) {
m_os << value;
} else {
m_os << '"';
size_t current_start = 0;
for ( size_t i = 0; i < value.size(); ++i ) {
if ( needsEscape( value[i] ) ) {
if ( current_start < i ) {
m_os
<< value.substr( current_start, i - current_start );
}
m_os << makeEscapeStringRef( value[i] );
current_start = i + 1;
}
m_os << makeEscapeStringRef( value[i] );
current_start = i + 1;
}
if ( current_start < value.size() ) {
m_os << value.substr( current_start,
value.size() - current_start );
}
m_os << '"';
}
if ( current_start < value.size() ) {
m_os << value.substr( current_start, value.size() - current_start );
}
if ( quote ) { m_os << '"'; }
}
} // namespace Catch
+8
View File
@@ -39,11 +39,19 @@ namespace Catch {
writeImpl( value, !std::is_arithmetic<T>::value );
}
void write( StringRef value ) &&;
void write( float value ) &&;
void write( double value ) &&;
void write( bool value ) &&;
private:
void writeImpl( StringRef value, bool quote );
// Helper to deal with non-finite floating point values, which
// are not standard JSON, but we use JS/Python/etc. approach of
// emitting `NaN`, `Infinity`, `-Infinity` as number(like).
template <typename T>
void writeFloatingPoint( T value );
// Without this SFINAE, this overload is a better match
// for `std::string`, `char const*`, `char const[N]` args.
// While it would still work, it would cause code bloat
@@ -93,6 +93,18 @@ namespace Catch {
return {};
}
}
Optional<Verbosity> stringToVerbosity( StringRef verbosity ) {
if (verbosity == "quiet") { return Verbosity::Quiet;
} else if ( verbosity == "normal" ) {
return Verbosity::Normal;
} else if ( verbosity == "high" ) {
return Verbosity::High;
} else {
return {};
}
}
} // namespace Detail
@@ -100,6 +112,7 @@ namespace Catch {
return lhs.m_name == rhs.m_name &&
lhs.m_outputFileName == rhs.m_outputFileName &&
lhs.m_colourMode == rhs.m_colourMode &&
lhs.m_verbosity == rhs.m_verbosity &&
lhs.m_customOptions == rhs.m_customOptions;
}
@@ -111,6 +124,7 @@ namespace Catch {
std::map<std::string, std::string> kvPairs;
Optional<std::string> outputFileName;
Optional<ColourMode> colourMode;
Optional<Verbosity> verbosity;
// First part is always reporter name, so we skip it
for ( size_t i = 1; i < parts.size(); ++i ) {
@@ -148,6 +162,12 @@ namespace Catch {
if ( !colourMode ) {
return {};
}
} else if ( key == "verbosity" ) {
// Duplicated key
if ( verbosity ) { return {}; }
verbosity = Detail::stringToVerbosity( value );
// Parsing failed
if ( !verbosity ) { return {}; }
} else {
// Unrecognized option
return {};
@@ -157,6 +177,7 @@ namespace Catch {
return ReporterSpec{ CATCH_MOVE( parts[0] ),
CATCH_MOVE( outputFileName ),
CATCH_MOVE( colourMode ),
CATCH_MOVE( verbosity),
CATCH_MOVE( kvPairs ) };
}
@@ -164,10 +185,12 @@ ReporterSpec::ReporterSpec(
std::string name,
Optional<std::string> outputFileName,
Optional<ColourMode> colourMode,
Optional<Verbosity> verbosity,
std::map<std::string, std::string> customOptions ):
m_name( CATCH_MOVE( name ) ),
m_outputFileName( CATCH_MOVE( outputFileName ) ),
m_colourMode( CATCH_MOVE( colourMode ) ),
m_verbosity( CATCH_MOVE( verbosity ) ),
m_customOptions( CATCH_MOVE( customOptions ) ) {}
} // namespace Catch
@@ -25,6 +25,7 @@ namespace Catch {
std::vector<std::string> splitReporterSpec( StringRef reporterSpec );
Optional<ColourMode> stringToColourMode( StringRef colourMode );
Optional<Verbosity> stringToVerbosity( StringRef verbosity );
}
/**
@@ -39,6 +40,7 @@ namespace Catch {
std::string m_name;
Optional<std::string> m_outputFileName;
Optional<ColourMode> m_colourMode;
Optional<Verbosity> m_verbosity;
std::map<std::string, std::string> m_customOptions;
friend bool operator==( ReporterSpec const& lhs,
@@ -53,6 +55,7 @@ namespace Catch {
std::string name,
Optional<std::string> outputFileName,
Optional<ColourMode> colourMode,
Optional<Verbosity> verbosity,
std::map<std::string, std::string> customOptions );
std::string const& name() const { return m_name; }
@@ -63,13 +66,15 @@ namespace Catch {
Optional<ColourMode> const& colourMode() const { return m_colourMode; }
Optional<Verbosity> const& verbosity() const { return m_verbosity; }
std::map<std::string, std::string> const& customOptions() const {
return m_customOptions;
}
};
/**
* Parses provided reporter spec string into
* Parses provided reporter spec string into actual `ReporterSpec`
*
* Returns empty optional on errors, e.g.
* * field that is not first and not a key+value pair
+18 -14
View File
@@ -196,6 +196,8 @@ namespace Catch {
auto getGenerator() const -> GeneratorBasePtr const& override {
return m_generator;
}
bool isFilteredImpl() const override { return m_isFiltered; }
};
} // namespace
}
@@ -540,17 +542,6 @@ namespace Catch {
SourceLineInfo lineInfo,
Generators::GeneratorBasePtr&& generator ) {
// TBD: Do we want to avoid the warning if the generator is filtered?
if ( m_config->warnAboutInfiniteGenerators() &&
!generator->isFinite() ) {
// We want the semantics of `FAIL()`, but we inline it
// to avoid issues with conditionally prefixed macros
INTERNAL_CATCH_MSG( "FAIL",
Catch::ResultWas::ExplicitFailure,
Catch::ResultDisposition::Normal,
"GENERATE() would run infinitely" );
}
auto nameAndLoc = TestCaseTracking::NameAndLocation( static_cast<std::string>( generatorName ), lineInfo );
auto& currentTracker = m_trackerContext.currentTracker();
assert(
@@ -563,11 +554,24 @@ namespace Catch {
m_trackerContext,
&currentTracker,
CATCH_MOVE( generator ) );
auto ret = newTracker.get();
// The warning shouldn't fire if the generator is infinite, **but** filtered down.
if ( m_config->warnAboutInfiniteGenerators() &&
!newTracker->m_generator->isFinite() &&
!newTracker->isFiltered() ) {
// We want the semantics of `FAIL()`, but we inline it
// to avoid issues with conditionally prefixed macros
INTERNAL_CATCH_MSG( "FAIL",
Catch::ResultWas::ExplicitFailure,
Catch::ResultDisposition::Normal,
"GENERATE() would run infinitely" );
}
auto returnPtr = newTracker.get();
currentTracker.addChild( CATCH_MOVE( newTracker ) );
ret->open();
return ret;
returnPtr->open();
return returnPtr;
}
bool RunContext::testForMissingAssertions(Counts& assertions) {
@@ -245,42 +245,42 @@
INTERNAL_CATCH_EXPAND_VARGS( INTERNAL_CATCH_TEMPLATE_TEST_CASE_METHOD_2( INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_CLASS_ ), INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ) , ClassName, Name, Tags, Signature, __VA_ARGS__ ) )
#endif
#define INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD_2(TestNameClass, TestName, ClassName, Name, Tags, Signature, TmplTypes, TypesList)\
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS \
CATCH_INTERNAL_SUPPRESS_ZERO_VARIADIC_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_TEMPLATE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS \
#define INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD_2(TestNameClass, TestName, ClassName, Name, Tags, Signature, TmplTypes, TypesList)\
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS \
CATCH_INTERNAL_SUPPRESS_ZERO_VARIADIC_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_TEMPLATE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS \
namespace {\
template<typename TestType> \
struct TestName : INTERNAL_CATCH_REMOVE_PARENS(ClassName <TestType>) { \
void test();\
};\
namespace {\
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestNameClass) {\
INTERNAL_CATCH_TYPE_GEN \
INTERNAL_CATCH_NTTP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature))\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
std::size_t index = 0;\
using expander = std::size_t[];\
constexpr char const* tmpl_types[] = {CATCH_REC_LIST(INTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS, INTERNAL_CATCH_REMOVE_PARENS(TmplTypes))};\
constexpr char const* types_list[] = {CATCH_REC_LIST(INTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS, INTERNAL_CATCH_REMOVE_PARENS(TypesList))};\
constexpr auto num_types = sizeof(types_list) / sizeof(types_list[0]);\
(void)expander{(Catch::AutoReg( Catch::makeTestInvoker( &TestName<Types>::test ), CATCH_INTERNAL_LINEINFO, #ClassName, Catch::NameAndTags{ Name " - " + std::string(tmpl_types[index / num_types]) + '<' + types_list[index % num_types] + '>', Tags } ), index++)... };/* NOLINT */ \
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename create<TestNameClass, decltype(get_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
}(); \
}\
}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
template<typename TestType> \
void TestName<TestType>::test()
struct TestName : INTERNAL_CATCH_REMOVE_PARENS(ClassName <TestType>) { \
void test();\
};\
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestNameClass) {\
INTERNAL_CATCH_TYPE_GEN \
INTERNAL_CATCH_NTTP_GEN(INTERNAL_CATCH_REMOVE_PARENS(Signature))\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
std::size_t index = 0;\
using expander = std::size_t[];\
constexpr char const* tmpl_types[] = {CATCH_REC_LIST(INTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS, INTERNAL_CATCH_REMOVE_PARENS(TmplTypes))};\
constexpr char const* types_list[] = {CATCH_REC_LIST(INTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS, INTERNAL_CATCH_REMOVE_PARENS(TypesList))};\
constexpr auto num_types = sizeof(types_list) / sizeof(types_list[0]);\
(void)expander{(Catch::AutoReg( Catch::makeTestInvoker( &TestName<Types>::test ), CATCH_INTERNAL_LINEINFO, #ClassName, Catch::NameAndTags{ Name " - " + std::string(tmpl_types[index / num_types]) + '<' + types_list[index % num_types] + '>', Tags } ), index++)... };/* NOLINT */ \
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename create<TestNameClass, decltype(get_wrapper<INTERNAL_CATCH_REMOVE_PARENS(TmplTypes)>(Catch::Detail::priority_tag<1>{})), TypeList<INTERNAL_CATCH_MAKE_TYPE_LISTS_FROM_TYPES(INTERNAL_CATCH_REMOVE_PARENS(TypesList))>>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
}(); \
}\
}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
template<typename TestType> \
void TestName<TestType>::test()
#ifndef CATCH_CONFIG_TRADITIONAL_MSVC_PREPROCESSOR
#define INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD( ClassName, Name, Tags, ... )\
@@ -298,37 +298,37 @@
INTERNAL_CATCH_EXPAND_VARGS( INTERNAL_CATCH_TEMPLATE_PRODUCT_TEST_CASE_METHOD_2( INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ), INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ), ClassName, Name, Tags, Signature,__VA_ARGS__ ) )
#endif
#define INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD_2( TestNameClass, TestName, ClassName, Name, Tags, TmplList) \
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_TEMPLATE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_COMMA_WARNINGS \
#define INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD_2( TestNameClass, TestName, ClassName, Name, Tags, TmplList) \
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_TEMPLATE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS \
CATCH_INTERNAL_SUPPRESS_COMMA_WARNINGS \
namespace {\
template<typename TestType> \
struct TestName : INTERNAL_CATCH_REMOVE_PARENS(ClassName <TestType>) { \
void test();\
};\
namespace {\
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestName){ \
INTERNAL_CATCH_TYPE_GEN\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
size_t index = 0;\
using expander = size_t[];\
(void)expander{(Catch::AutoReg( Catch::makeTestInvoker( &TestName<Types>::test ), CATCH_INTERNAL_LINEINFO, #ClassName##_catch_sr, Catch::NameAndTags{ Name " - " INTERNAL_CATCH_STRINGIZE(TmplList) " - " + std::to_string(index), Tags } ), index++)... };/* NOLINT */ \
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename convert<TestNameClass, TmplList>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
}(); \
}}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
template<typename TestType> \
void TestName<TestType>::test()
namespace INTERNAL_CATCH_MAKE_NAMESPACE(TestName){ \
INTERNAL_CATCH_TYPE_GEN\
template<typename...Types>\
struct TestNameClass{\
void reg_tests(){\
size_t index = 0;\
using expander = size_t[];\
(void)expander{(Catch::AutoReg( Catch::makeTestInvoker( &TestName<Types>::test ), CATCH_INTERNAL_LINEINFO, #ClassName##_catch_sr, Catch::NameAndTags{ Name " - " INTERNAL_CATCH_STRINGIZE(TmplList) " - " + std::to_string(index), Tags } ), index++)... };/* NOLINT */ \
}\
};\
static const int INTERNAL_CATCH_UNIQUE_NAME( globalRegistrar ) = [](){\
using TestInit = typename convert<TestNameClass, TmplList>::type;\
TestInit t;\
t.reg_tests();\
return 0;\
}(); \
}}\
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
template<typename TestType> \
void TestName<TestType>::test()
#define INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD(ClassName, Name, Tags, TmplList) \
INTERNAL_CATCH_TEMPLATE_LIST_TEST_CASE_METHOD_2( INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ), INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEMPLATE_TEST_ ), ClassName, Name, Tags, TmplList )
@@ -167,6 +167,28 @@ namespace TestCaseTracking {
m_ctx.setCurrentTracker( this );
}
bool SectionTracker::isFilteredImpl() const {
// TBD: This is currently _very_ similar to the block in `isComplete`.
// Is this neccessarily that way, or just accident of current semantics?
const size_t filterIndex =
m_newStyleFilters ? m_allTrackerDepth : m_sectionOnlyDepth;
if ( filterIndex < m_filterRef->size() ) {
// 1) New style filter must explicitly target section
if ( m_newStyleFilters && ( *m_filterRef )[filterIndex].type !=
PathFilter::For::Section ) {
return true;
}
// 2) Both style filters must match the trimmed name exactly
if ( m_trimmed_name !=
StringRef( ( *m_filterRef )[filterIndex].filter ) ) {
return true;
}
}
return false;
}
SectionTracker::SectionTracker( NameAndLocation&& nameAndLocation, TrackerContext& ctx, ITracker* parent )
: TrackerBase( CATCH_MOVE(nameAndLocation), ctx, parent ),
m_trimmed_name(trim(StringRef(ITracker::nameAndLocation().name)))
@@ -12,6 +12,7 @@
#include <catch2/internal/catch_source_line_info.hpp>
#include <catch2/internal/catch_unique_ptr.hpp>
#include <catch2/internal/catch_stringref.hpp>
#include <catch2/internal/catch_path_filter.hpp>
#include <string>
#include <vector>
@@ -81,6 +82,8 @@ namespace TestCaseTracking {
using Children = std::vector<ITrackerPtr>;
virtual bool isFilteredImpl() const = 0;
protected:
enum CycleState {
NotStarted,
@@ -178,6 +181,20 @@ namespace TestCaseTracking {
* for internal debug checks.
*/
virtual bool isGeneratorTracker() const;
/**
* Returns true if the concrete tracker instance has a filter that applies to it.
*/
bool isFiltered() const {
// Fast path: are there even filters for tracker in this position?
const size_t filter_depth =
m_newStyleFilters ? m_allTrackerDepth : m_sectionOnlyDepth;
if ( m_filterRef->size() <= filter_depth ) { return false; }
// Slow path: If there are filters, ask the concrete tracker.
// This handles things like match-all filters for that tracker.
return isFilteredImpl();
}
};
class TrackerContext {
@@ -233,6 +250,8 @@ namespace TestCaseTracking {
// to not own the name, the name still has to outlive the `ITracker` parent, so
// this should still be safe.
StringRef m_trimmed_name;
bool isFilteredImpl() const override;
public:
SectionTracker( NameAndLocation&& nameAndLocation, TrackerContext& ctx, ITracker* parent );
+2 -2
View File
@@ -148,8 +148,8 @@ namespace Catch {
m_it--;
}
// Skip back over UTF-8 continuation bytes to the leading byte
while ( isUtf8ContinuationByte( *m_it ) ) {
assert( m_it != m_string->begin() );
while ( m_it != m_string->begin() &&
isUtf8ContinuationByte( *m_it ) ) {
m_it--;
}
}
@@ -19,6 +19,7 @@ namespace Catch {
m_wrapped_stream( CATCH_MOVE(config).takeStream() ),
m_stream( m_wrapped_stream->stream() ),
m_colour( makeColourImpl( config.colourMode(), m_wrapped_stream.get() ) ),
m_verbosity( config.verbosity() ),
m_customOptions( config.customOptions() )
{}
@@ -26,12 +27,12 @@ namespace Catch {
void ReporterBase::listReporters(
std::vector<ReporterDescription> const& descriptions ) {
defaultListReporters( m_stream, descriptions, m_config->verbosity() );
defaultListReporters( m_stream, descriptions, m_verbosity );
}
void ReporterBase::listListeners(
std::vector<ListenerDescription> const& descriptions ) {
defaultListListeners( m_stream, descriptions, m_config->verbosity() );
defaultListListeners( m_stream, descriptions, m_verbosity );
}
void ReporterBase::listTests(std::vector<TestCaseHandle> const& tests) {
@@ -39,11 +40,11 @@ namespace Catch {
m_colour.get(),
tests,
m_config->hasTestFilters(),
m_config->verbosity());
m_verbosity);
}
void ReporterBase::listTags(std::vector<TagInfo> const& tags) {
defaultListTags( m_stream, tags, m_config->hasTestFilters(), m_config->verbosity() );
defaultListTags( m_stream, tags, m_config->hasTestFilters(), m_verbosity );
}
} // namespace Catch
@@ -35,6 +35,8 @@ namespace Catch {
std::ostream& m_stream;
//! Colour implementation this reporter was configured for
Detail::unique_ptr<ColourImpl> m_colour;
//! Verbosity configured for this reporter
Verbosity m_verbosity;
//! The custom reporter options user passed down to the reporter
std::map<std::string, std::string> m_customOptions;
+10 -4
View File
@@ -16,6 +16,8 @@
namespace Catch {
namespace {
static size_t kJsonOutputVersion = 2;
void writeSourceInfo( JsonObjectWriter& writer,
SourceLineInfo const& sourceInfo ) {
auto source_location_writer =
@@ -58,7 +60,7 @@ namespace Catch {
m_writers.emplace( Writer::Object );
auto& writer = m_objectWriters.top();
writer.write( "version"_sr ).write( 1 );
writer.write( "version"_sr ).write( kJsonOutputVersion );
{
auto metadata_writer = writer.write( "metadata"_sr ).writeObject();
@@ -344,14 +346,18 @@ namespace Catch {
auto const& info = test.getTestCaseInfo();
desc_writer.write( "name"_sr ).write( info.name );
desc_writer.write( "class-name"_sr ).write( info.className );
{
if (!info.className.empty()) {
desc_writer.write( "class-name"_sr ).write( info.className );
}
if ( m_verbosity >= Verbosity::Normal ) {
auto tag_writer = desc_writer.write( "tags"_sr ).writeArray();
for ( auto const& tag : info.tags ) {
tag_writer.write( tag.original );
}
}
writeSourceInfo( desc_writer, info.lineInfo );
if ( m_verbosity >= Verbosity::High) {
writeSourceInfo( desc_writer, info.lineInfo );
}
}
}
void JsonReporter::listTags( std::vector<TagInfo> const& tags ) {
@@ -85,9 +85,6 @@ namespace Catch {
std::stack<Writer> m_writers{};
bool m_startedListing = false;
// std::size_t m_sectionDepth = 0;
// std::size_t m_sectionStarted = 0;
};
} // namespace Catch
+60
View File
@@ -184,6 +184,13 @@ set_tests_properties(List::Tests::Output PROPERTIES
PASS_REGULAR_EXPRESSION "[0-9]+ test cases"
FAIL_REGULAR_EXPRESSION "Hidden Test"
)
add_test(NAME VerbosityIsPerReporter COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high --reporter console::verbosity=quiet)
set_tests_properties(VerbosityIsPerReporter
PROPERTIES
FAIL_REGULAR_EXPRESSION "\.cpp"
)
# This should be equivalent to the old --list-test-names-only and be usable
# with --input-file.
add_test(NAME List::Tests::Quiet COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity quiet)
@@ -193,6 +200,7 @@ set_tests_properties(List::Tests::Quiet PROPERTIES
PASS_REGULAR_EXPRESSION "\"#1905 -- test spec parser properly clears internal state between compound tests\"[\r\n]"
FAIL_REGULAR_EXPRESSION "[ \t]\"#1905 -- test spec parser properly clears internal state between compound tests\""
)
add_test(NAME List::Tests::ExitCode COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high)
add_test(NAME List::Tests::XmlOutput COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high -r xml)
set_tests_properties(List::Tests::XmlOutput PROPERTIES
@@ -200,6 +208,21 @@ set_tests_properties(List::Tests::XmlOutput PROPERTIES
FAIL_REGULAR_EXPRESSION "[0-9]+ test cases"
)
add_test(NAME List::Tests::Json::Normal COMMAND $<TARGET_FILE:SelfTest> --list-tests -r json)
set_tests_properties(List::Tests::Json::Normal PROPERTIES
PASS_REGULAR_EXPRESSION "\"tags\": \\["
FAIL_REGULAR_EXPRESSION "\"source-location\": {"
)
add_test(NAME List::Tests::Json::Quiet COMMAND $<TARGET_FILE:SelfTest> --list-tests -r json::verbosity=quiet)
set_tests_properties(List::Tests::Json::Quiet PROPERTIES
PASS_REGULAR_EXPRESSION "\"name\": \""
FAIL_REGULAR_EXPRESSION "\"tags\": \\["
)
add_test(NAME List::Tests::Json::High COMMAND $<TARGET_FILE:SelfTest> --list-tests -r json::verbosity=high)
set_tests_properties(List::Tests::Json::High PROPERTIES
PASS_REGULAR_EXPRESSION "\"source-location\": {"
)
add_test(NAME List::Tags::Output COMMAND $<TARGET_FILE:SelfTest> --list-tags)
set_tests_properties(List::Tags::Output PROPERTIES
PASS_REGULAR_EXPRESSION "[0-9]+ tags"
@@ -210,6 +233,11 @@ set_tests_properties(List::Tags::XmlOutput PROPERTIES
PASS_REGULAR_EXPRESSION "<Count>18</Count>"
FAIL_REGULAR_EXPRESSION "[0-9]+ tags"
)
add_test(NAME List::Tags::JsonOutput COMMAND $<TARGET_FILE:SelfTest> --list-tags -r json)
set_tests_properties(List::Tags::JsonOutput PROPERTIES
PASS_REGULAR_EXPRESSION "\"count\": 18"
FAIL_REGULAR_EXPRESSION "[0-9]+ tags"
)
add_test(NAME List::Reporters::Output COMMAND $<TARGET_FILE:SelfTest> --list-reporters)
set_tests_properties(List::Reporters::Output PROPERTIES PASS_REGULAR_EXPRESSION "Available reporters:")
@@ -219,6 +247,12 @@ set_tests_properties(List::Reporters::XmlOutput PROPERTIES
PASS_REGULAR_EXPRESSION "<Name>compact</Name>"
FAIL_REGULAR_EXPRESSION "Available reporters:"
)
add_test(NAME List::Reporters::JsonOutput COMMAND $<TARGET_FILE:SelfTest> --list-reporters -r json)
set_tests_properties(List::Reporters::JsonOutput PROPERTIES
PASS_REGULAR_EXPRESSION "\"name\": \"compact\""
FAIL_REGULAR_EXPRESSION "Available reporters:"
)
add_test(NAME List::Listeners::Output
COMMAND
@@ -243,6 +277,18 @@ set_tests_properties(List::Listeners::XmlOutput
PASS_REGULAR_EXPRESSION "<RegisteredListeners>"
FAIL_REGULAR_EXPRESSION "Registered listeners:"
)
add_test(NAME List::Listeners::JsonOutput
COMMAND
$<TARGET_FILE:SelfTest>
--list-listeners
--reporter json
)
set_tests_properties(List::Listeners::JsonOutput
PROPERTIES
PASS_REGULAR_EXPRESSION "\"listeners\": \\["
FAIL_REGULAR_EXPRESSION "Registered listeners:"
)
add_test(NAME NoAssertions COMMAND $<TARGET_FILE:SelfTest> -w NoAssertions "An empty test with no assertions")
set_tests_properties(NoAssertions PROPERTIES PASS_REGULAR_EXPRESSION "No assertions in test case")
@@ -625,6 +671,20 @@ if(CATCH_ENABLE_CMAKE_HELPER_TESTS)
COST 240
LABELS "uses-python"
)
add_test(NAME "CMakeHelper::PrepareCommandFragment"
COMMAND
"${CMAKE_COMMAND}"
"-DCATCH_ADD_TESTS_SCRIPT=${CATCH_DIR}/extras/CatchAddTests.cmake"
-P "${CMAKE_CURRENT_LIST_DIR}/TestScripts/DiscoverTests/TestPrepareCommandFragment.cmake"
)
add_test(NAME "CMakeHelper::DecomposeJsonArray"
COMMAND
"${CMAKE_COMMAND}"
"-DCATCH_ADD_TESTS_SCRIPT=${CATCH_DIR}/extras/CatchAddTests.cmake"
-P "${CMAKE_CURRENT_LIST_DIR}/TestScripts/DiscoverTests/TestDecomposeJsonArray.cmake"
)
endif()
foreach(reporterName # "Automake" - the simple .trs format does not support any kind of comments/metadata
+80 -24
View File
@@ -526,29 +526,18 @@ set_tests_properties(TestSpecs::SkippingAllTestsFails
WILL_FAIL ON
)
set(EXTRA_TEST_BINARIES
AllSkipped
PrefixedMacros
DisabledMacros
DisabledExceptions-DefaultHandler
DisabledExceptions-CustomHandler
FallbackStringifier
DisableStringification
PartialTestCaseEvents
DuplicatedTestCases-SameNameAndTags
DuplicatedTestCases-SameNameDifferentTags
DuplicatedTestCases-DuplicatedTestCaseMethods
NoTests
ListenersGetEventsBeforeReporters
MixingClearedAndUnclearedMessages
# DebugBreakMacros
)
add_executable(FastCompileMacros ${TESTS_DIR}/X07-FastCompileMacros.cpp)
target_link_libraries(FastCompileMacros PRIVATE Catch2_buildall_interface)
target_compile_definitions(FastCompileMacros PRIVATE CATCH_CONFIG_FAST_COMPILE)
# Notice that we are modifying EXTRA_TEST_BINARIES destructively, do not
# use it after this point!
list(FILTER EXTRA_TEST_BINARIES EXCLUDE REGEX "DisabledExceptions.*")
list(APPEND CATCH_TEST_TARGETS ${EXTRA_TEST_BINARIES})
set(CATCH_TEST_TARGETS ${CATCH_TEST_TARGETS} PARENT_SCOPE)
add_test(
NAME CompileConfiguration::FastCompile
COMMAND $<TARGET_FILE:FastCompileMacros>
)
set_tests_properties(CompileConfiguration::FastCompile
PROPERTIES
PASS_REGULAR_EXPRESSION "test cases: 6 \\| 1 passed \\| 1 failed \\| 4 failed as expected\nassertions: 13 \\| 6 passed \\| 2 failed \\| 5 failed as expected"
)
# This sets up a one-off executable that compiles against the amalgamated
# files, and then runs it for a super simple check that the amalgamated
@@ -594,12 +583,79 @@ add_executable(InfiniteGenerators ${TESTS_DIR}/X95-InfiniteGenerators.cpp)
target_link_libraries(InfiniteGenerators PRIVATE Catch2::Catch2WithMain)
add_test(
NAME Warnings::InfiniteGenerators
NAME Warnings::InfiniteGenerators::NoFilterWarns
COMMAND $<TARGET_FILE:InfiniteGenerators> --warn InfiniteGenerators
)
set_tests_properties(Warnings::InfiniteGenerators
set_tests_properties(Warnings::InfiniteGenerators::NoFilterWarns
PROPERTIES
# One test case fails with infinite generator, but the other one runs
PASS_REGULAR_EXPRESSION "test cases: 2 \\| 1 passed \\| 1 failed"
TIMEOUT 5
)
add_test(
NAME Warnings::InfiniteGenerators::MatchAllFilterWarns
COMMAND $<TARGET_FILE:InfiniteGenerators>
--warn InfiniteGenerators
--path-filter g:*
)
set_tests_properties(Warnings::InfiniteGenerators::MatchAllFilterWarns
PROPERTIES
# One test case fails with infinite generator, but the other one runs
PASS_REGULAR_EXPRESSION "test cases: 2 \\| 1 passed \\| 1 failed"
TIMEOUT 5
)
add_test(
NAME Warnings::InfiniteGenerators::MatchOneFilterDoesntWarn
COMMAND $<TARGET_FILE:InfiniteGenerators>
--warn InfiniteGenerators
--path-filter g:1
)
set_tests_properties(Warnings::InfiniteGenerators::MatchOneFilterDoesntWarn
PROPERTIES
# One test case fails with infinite generator, but the other one runs
PASS_REGULAR_EXPRESSION "All tests passed \\(1 assertion in 2 test cases\\)"
TIMEOUT 5
)
add_test(
NAME Warnings::InfiniteGenerators::SectionFilterWarns
COMMAND $<TARGET_FILE:InfiniteGenerators>
--warn InfiniteGenerators
--section FooBarBaz
)
set_tests_properties(Warnings::InfiniteGenerators::SectionFilterWarns
PROPERTIES
# One test case fails with infinite generator, but the other one runs
PASS_REGULAR_EXPRESSION "test cases: 2 \\| 1 passed \\| 1 failed"
TIMEOUT 5
)
set(EXTRA_TEST_BINARIES
AllSkipped
PrefixedMacros
DisabledMacros
DisabledExceptions-DefaultHandler
DisabledExceptions-CustomHandler
FallbackStringifier
DisableStringification
PartialTestCaseEvents
DuplicatedTestCases-SameNameAndTags
DuplicatedTestCases-SameNameDifferentTags
DuplicatedTestCases-DuplicatedTestCaseMethods
NoTests
ListenersGetEventsBeforeReporters
MixingClearedAndUnclearedMessages
FastCompileMacros
InfiniteGenerators
ThreadSafetyTests
# DebugBreakMacros
)
# Notice that we are modifying EXTRA_TEST_BINARIES destructively, do not
# use it after this point!
list(FILTER EXTRA_TEST_BINARIES EXCLUDE REGEX "DisabledExceptions.*")
list(APPEND CATCH_TEST_TARGETS ${EXTRA_TEST_BINARIES})
set(CATCH_TEST_TARGETS ${CATCH_TEST_TARGETS} PARENT_SCOPE)
-1
View File
@@ -3,7 +3,6 @@ yet:
CATCH_CONFIG_COUNTER // Use __COUNTER__ to generate unique names for test cases
CATCH_CONFIG_WINDOWS_SEH // Enable SEH handling on Windows
CATCH_CONFIG_FAST_COMPILE // Sacrifices some (rather minor) features for compilation speed
CATCH_CONFIG_POSIX_SIGNALS // Enable handling POSIX signals
CATCH_CONFIG_WINDOWS_CRTDBG // Enable leak checking using Windows's CRT Debug Heap
CATCH_CONFIG_DEFAULT_REPORTER
@@ -0,0 +1,63 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
/**\file
* Test that various basic macros work with CATCH_CONFIG_FAST_COMPILE.
*
* Note that the current checking is rather loose. We check that the
* macros compile, and that the test cases (don't) fail as they are
* supposed to.
*/
#include <catch2/catch_test_macros.hpp>
#include <stdexcept>
namespace {
[[noreturn]]
static void throws() {
throw std::runtime_error{ "sup" };
}
static void doesnt_throw() {}
[[noreturn]]
static int throws_i() {
throw std::runtime_error{ "sup" };
}
} // namespace
TEST_CASE( "Passing macros work" ) {
REQUIRE( 1 != 2 );
CHECK( 2 == 2 );
REQUIRE_THROWS( throws() );
REQUIRE_NOTHROW( doesnt_throw() );
}
TEST_CASE( "Failing macros work", "[!shouldfail]" ) {
CHECK( 1 != 2 );
CHECK( 2 == 2 );
CHECK( 3 == 2 );
}
TEST_CASE( "Failing NOTHROW works", "[!shouldfail]" ) {
REQUIRE_NOTHROW( throws() );
}
TEST_CASE( "Failing THROW works", "[!shouldfail]" ) {
REQUIRE_THROWS( doesnt_throw() );
}
TEST_CASE( "Unexpected exception in REQUIRE gets inverted properly",
"[!shouldfail]" ) {
REQUIRE( throws_i() == 1 );
}
TEST_CASE( "Unexpected exception in REQUIRE fails properly" ) {
REQUIRE( throws_i() == 2 );
}
@@ -33,6 +33,7 @@ namespace {
TEST_CASE() {
auto _ = GENERATE( make_infinite_generator() );
(void)_;
}
TEST_CASE() {
@@ -190,6 +190,9 @@ Nor would this
:test-result: PASS Inequality checks that should succeed
:test-result: PASS JsonWriter
:test-result: PASS JsonWriter escapes characters in strings properly
:test-result: PASS JsonWriter serializes non-finite FP using Python spelling - double
:test-result: PASS JsonWriter serializes non-finite FP using Python spelling - float
:test-result: PASS JsonWriter serializes numbers independently of the global locale
:test-result: PASS Lambdas in assertions
:test-result: PASS Less-than inequalities with different epsilons
:test-result: PASS ManuallyRegistered
@@ -188,6 +188,9 @@
:test-result: PASS Inequality checks that should succeed
:test-result: PASS JsonWriter
:test-result: PASS JsonWriter escapes characters in strings properly
:test-result: PASS JsonWriter serializes non-finite FP using Python spelling - double
:test-result: PASS JsonWriter serializes non-finite FP using Python spelling - float
:test-result: PASS JsonWriter serializes numbers independently of the global locale
:test-result: PASS Lambdas in assertions
:test-result: PASS Less-than inequalities with different epsilons
:test-result: PASS ManuallyRegistered
@@ -1248,6 +1248,26 @@ Json.tests.cpp:<line number>: passed: sstream.str() == "\"\\n\"" for: ""\n"" ==
Json.tests.cpp:<line number>: passed: sstream.str() == "\"\\r\"" for: ""\r"" == ""\r""
Json.tests.cpp:<line number>: passed: sstream.str() == "\"\\t\"" for: ""\t"" == ""\t""
Json.tests.cpp:<line number>: passed: sstream.str() == "\"\\\\/\\t\\r\\n\"" for: ""\\/\t\r\n"" == ""\\/\t\r\n""
Json.tests.cpp:<line number>: passed: sstream.str() == "NaN" for: "NaN" == "NaN"
Json.tests.cpp:<line number>: passed: sstream.str() == "Infinity" for: "Infinity" == "Infinity"
Json.tests.cpp:<line number>: passed: sstream.str() == "-Infinity" for: "-Infinity" == "-Infinity"
Json.tests.cpp:<line number>: passed: sstream.str() == "NaN" for: "NaN" == "NaN"
Json.tests.cpp:<line number>: passed: sstream.str() == "Infinity" for: "Infinity" == "Infinity"
Json.tests.cpp:<line number>: passed: sstream.str() == "-Infinity" for: "-Infinity" == "-Infinity"
Json.tests.cpp:<line number>: passed: sstream.str(), ContainsSubstring( "\"double\": 1.5," ) && ContainsSubstring( "\"int\": 1234567," ) && ContainsSubstring( "\"bool-1\": true," ) && ContainsSubstring( "\"bool-2\": false," ) && ContainsSubstring( "\"array\": [\n 2.5,\n 1234567\n ]\n}" ) for: "{
"double": 1.5,
"int": 1234567,
"bool-1": true,
"bool-2": false,
"array": [
2.5,
1234567
]
}" ( contains: ""double": 1.5," and contains: ""int": 1234567," and contains: ""bool-1": true," and contains: ""bool-2": false," and contains: ""array": [
2.5,
1234567
]
}" )
Compilation.tests.cpp:<line number>: passed: []() { return true; }() for: true
Approx.tests.cpp:<line number>: passed: d <= Approx( 1.24 ) for: 1.22999999999999998
<=
@@ -1511,9 +1531,9 @@ CmdLine.tests.cpp:<line number>: passed: config.noThrow == false for: false == f
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications.empty() for: true
CmdLine.tests.cpp:<line number>: passed: !(cfg.hasTestFilters()) for: !false
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs().size() == 1 for: 1 == 1
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs().size() == 1 for: 1 == 1
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: result for: {?}
CmdLine.tests.cpp:<line number>: passed: cfg.hasTestFilters() for: true
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("notIncluded")) == false for: false == false
@@ -1527,21 +1547,21 @@ CmdLine.tests.cpp:<line number>: passed: cfg.hasTestFilters() for: true
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("test1")) == false for: false == false
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("alwaysIncluded")) for: true
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: !result for: true
CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Unrecognized reporter") for: "Unrecognized reporter, 'unsupported'. Check available with --list-reporters" contains: "Unrecognized reporter"
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }) for: {?}
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }) for: {?}
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: !result for: true
CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Only one reporter may have unspecified output file.") for: "Only one reporter may have unspecified output file." contains: "Only one reporter may have unspecified output file."
CmdLine.tests.cpp:<line number>: passed: cli.parse({"test", "-b"}) for: {?}
@@ -1676,7 +1696,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fa
" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: console'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fakeTag"s) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1693,7 +1713,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fak
]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fake reporter"s) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1708,7 +1728,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fak
]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1718,14 +1738,9 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fa
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
@@ -3000,7 +3015,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
Misc.tests.cpp:<line number>: passed:
Misc.tests.cpp:<line number>: passed:
test cases: 451 | 331 passed | 96 failed | 6 skipped | 18 failed as expected
assertions: 2416 | 2215 passed | 158 failed | 43 failed as expected
test cases: 454 | 334 passed | 96 failed | 6 skipped | 18 failed as expected
assertions: 2423 | 2222 passed | 158 failed | 43 failed as expected
@@ -1246,6 +1246,26 @@ Json.tests.cpp:<line number>: passed: sstream.str() == "\"\\n\"" for: ""\n"" ==
Json.tests.cpp:<line number>: passed: sstream.str() == "\"\\r\"" for: ""\r"" == ""\r""
Json.tests.cpp:<line number>: passed: sstream.str() == "\"\\t\"" for: ""\t"" == ""\t""
Json.tests.cpp:<line number>: passed: sstream.str() == "\"\\\\/\\t\\r\\n\"" for: ""\\/\t\r\n"" == ""\\/\t\r\n""
Json.tests.cpp:<line number>: passed: sstream.str() == "NaN" for: "NaN" == "NaN"
Json.tests.cpp:<line number>: passed: sstream.str() == "Infinity" for: "Infinity" == "Infinity"
Json.tests.cpp:<line number>: passed: sstream.str() == "-Infinity" for: "-Infinity" == "-Infinity"
Json.tests.cpp:<line number>: passed: sstream.str() == "NaN" for: "NaN" == "NaN"
Json.tests.cpp:<line number>: passed: sstream.str() == "Infinity" for: "Infinity" == "Infinity"
Json.tests.cpp:<line number>: passed: sstream.str() == "-Infinity" for: "-Infinity" == "-Infinity"
Json.tests.cpp:<line number>: passed: sstream.str(), ContainsSubstring( "\"double\": 1.5," ) && ContainsSubstring( "\"int\": 1234567," ) && ContainsSubstring( "\"bool-1\": true," ) && ContainsSubstring( "\"bool-2\": false," ) && ContainsSubstring( "\"array\": [\n 2.5,\n 1234567\n ]\n}" ) for: "{
"double": 1.5,
"int": 1234567,
"bool-1": true,
"bool-2": false,
"array": [
2.5,
1234567
]
}" ( contains: ""double": 1.5," and contains: ""int": 1234567," and contains: ""bool-1": true," and contains: ""bool-2": false," and contains: ""array": [
2.5,
1234567
]
}" )
Compilation.tests.cpp:<line number>: passed: []() { return true; }() for: true
Approx.tests.cpp:<line number>: passed: d <= Approx( 1.24 ) for: 1.22999999999999998
<=
@@ -1509,9 +1529,9 @@ CmdLine.tests.cpp:<line number>: passed: config.noThrow == false for: false == f
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications.empty() for: true
CmdLine.tests.cpp:<line number>: passed: !(cfg.hasTestFilters()) for: !false
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs().size() == 1 for: 1 == 1
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs().size() == 1 for: 1 == 1
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } for: {?} == {?}
CmdLine.tests.cpp:<line number>: passed: result for: {?}
CmdLine.tests.cpp:<line number>: passed: cfg.hasTestFilters() for: true
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("notIncluded")) == false for: false == false
@@ -1525,21 +1545,21 @@ CmdLine.tests.cpp:<line number>: passed: cfg.hasTestFilters() for: true
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("test1")) == false for: false == false
CmdLine.tests.cpp:<line number>: passed: cfg.testSpec().matches(*fakeTestCase("alwaysIncluded")) for: true
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: !result for: true
CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Unrecognized reporter") for: "Unrecognized reporter, 'unsupported'. Check available with --list-reporters" contains: "Unrecognized reporter"
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: result for: {?} with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }) for: {?}
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }) for: {?}
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
CmdLine.tests.cpp:<line number>: passed: !result for: true
CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Only one reporter may have unspecified output file.") for: "Only one reporter may have unspecified output file." contains: "Only one reporter may have unspecified output file."
CmdLine.tests.cpp:<line number>: passed: cli.parse({"test", "-b"}) for: {?}
@@ -1674,7 +1694,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fa
" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: console'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fakeTag"s) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1691,7 +1711,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fak
]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fake reporter"s) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1706,7 +1726,7 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring("fak
]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -1716,14 +1736,9 @@ Reporters.tests.cpp:<line number>: passed: listingString, ContainsSubstring( "fa
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
Reporters.tests.cpp:<line number>: passed: !(factories.empty()) for: !false
@@ -2989,7 +3004,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
Misc.tests.cpp:<line number>: passed:
Misc.tests.cpp:<line number>: passed:
test cases: 451 | 331 passed | 96 failed | 6 skipped | 18 failed as expected
assertions: 2416 | 2215 passed | 158 failed | 43 failed as expected
test cases: 454 | 334 passed | 96 failed | 6 skipped | 18 failed as expected
assertions: 2423 | 2222 passed | 158 failed | 43 failed as expected
@@ -1743,6 +1743,6 @@ due to unexpected exception with message:
Why would you throw a std::string?
===============================================================================
test cases: 451 | 349 passed | 76 failed | 7 skipped | 19 failed as expected
assertions: 2394 | 2215 passed | 136 failed | 43 failed as expected
test cases: 454 | 352 passed | 76 failed | 7 skipped | 19 failed as expected
assertions: 2401 | 2222 passed | 136 failed | 43 failed as expected
+112 -20
View File
@@ -8219,6 +8219,103 @@ Json.tests.cpp:<line number>: PASSED:
with expansion:
""\\/\t\r\n"" == ""\\/\t\r\n""
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - double
NaN
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "NaN" )
with expansion:
"NaN" == "NaN"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - double
Pos nf
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "Infinity" )
with expansion:
"Infinity" == "Infinity"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - double
Neg inf
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "-Infinity" )
with expansion:
"-Infinity" == "-Infinity"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - float
NaN
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "NaN" )
with expansion:
"NaN" == "NaN"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - float
Pos nf
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "Infinity" )
with expansion:
"Infinity" == "Infinity"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - float
Neg inf
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "-Infinity" )
with expansion:
"-Infinity" == "-Infinity"
-------------------------------------------------------------------------------
JsonWriter serializes numbers independently of the global locale
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( sstream.str(), ContainsSubstring( "\"double\": 1.5," ) && ContainsSubstring( "\"int\": 1234567," ) && ContainsSubstring( "\"bool-1\": true," ) && ContainsSubstring( "\"bool-2\": false," ) && ContainsSubstring( "\"array\": [\n 2.5,\n 1234567\n ]\n}" ) )
with expansion:
"{
"double": 1.5,
"int": 1234567,
"bool-1": true,
"bool-2": false,
"array": [
2.5,
1234567
]
}" ( contains: ""double": 1.5," and contains: ""int": 1234567," and contains:
""bool-1": true," and contains: ""bool-2": false," and contains: ""array": [
2.5,
1234567
]
}" )
-------------------------------------------------------------------------------
Lambdas in assertions
-------------------------------------------------------------------------------
@@ -9880,7 +9977,7 @@ with expansion:
1 == 1
CmdLine.tests.cpp:<line number>: PASSED:
CHECK( cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } )
CHECK( cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } )
with expansion:
{?} == {?}
@@ -9890,7 +9987,7 @@ with expansion:
1 == 1
CmdLine.tests.cpp:<line number>: PASSED:
CHECK( cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } )
CHECK( cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } )
with expansion:
{?} == {?}
@@ -9994,7 +10091,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10016,7 +10113,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10038,7 +10135,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10079,7 +10176,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10101,7 +10198,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10122,7 +10219,7 @@ with expansion:
{?}
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } )
with expansion:
{ {?}, {?} } == { {?}, {?} }
@@ -10141,7 +10238,7 @@ with expansion:
{?}
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } )
with expansion:
{ {?}, {?} } == { {?}, {?} }
@@ -11136,7 +11233,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring("fakeTag"s) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11176,7 +11273,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring("fake reporter"s) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11214,7 +11311,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11224,14 +11321,9 @@ with expansion:
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" )
with message:
@@ -20134,6 +20226,6 @@ Misc.tests.cpp:<line number>
Misc.tests.cpp:<line number>: PASSED:
===============================================================================
test cases: 451 | 331 passed | 96 failed | 6 skipped | 18 failed as expected
assertions: 2416 | 2215 passed | 158 failed | 43 failed as expected
test cases: 454 | 334 passed | 96 failed | 6 skipped | 18 failed as expected
assertions: 2423 | 2222 passed | 158 failed | 43 failed as expected
@@ -8217,6 +8217,103 @@ Json.tests.cpp:<line number>: PASSED:
with expansion:
""\\/\t\r\n"" == ""\\/\t\r\n""
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - double
NaN
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "NaN" )
with expansion:
"NaN" == "NaN"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - double
Pos nf
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "Infinity" )
with expansion:
"Infinity" == "Infinity"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - double
Neg inf
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "-Infinity" )
with expansion:
"-Infinity" == "-Infinity"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - float
NaN
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "NaN" )
with expansion:
"NaN" == "NaN"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - float
Pos nf
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "Infinity" )
with expansion:
"Infinity" == "Infinity"
-------------------------------------------------------------------------------
JsonWriter serializes non-finite FP using Python spelling - float
Neg inf
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE( sstream.str() == "-Infinity" )
with expansion:
"-Infinity" == "-Infinity"
-------------------------------------------------------------------------------
JsonWriter serializes numbers independently of the global locale
-------------------------------------------------------------------------------
Json.tests.cpp:<line number>
...............................................................................
Json.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( sstream.str(), ContainsSubstring( "\"double\": 1.5," ) && ContainsSubstring( "\"int\": 1234567," ) && ContainsSubstring( "\"bool-1\": true," ) && ContainsSubstring( "\"bool-2\": false," ) && ContainsSubstring( "\"array\": [\n 2.5,\n 1234567\n ]\n}" ) )
with expansion:
"{
"double": 1.5,
"int": 1234567,
"bool-1": true,
"bool-2": false,
"array": [
2.5,
1234567
]
}" ( contains: ""double": 1.5," and contains: ""int": 1234567," and contains:
""bool-1": true," and contains: ""bool-2": false," and contains: ""array": [
2.5,
1234567
]
}" )
-------------------------------------------------------------------------------
Lambdas in assertions
-------------------------------------------------------------------------------
@@ -9878,7 +9975,7 @@ with expansion:
1 == 1
CmdLine.tests.cpp:<line number>: PASSED:
CHECK( cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } )
CHECK( cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } )
with expansion:
{?} == {?}
@@ -9888,7 +9985,7 @@ with expansion:
1 == 1
CmdLine.tests.cpp:<line number>: PASSED:
CHECK( cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } )
CHECK( cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } )
with expansion:
{?} == {?}
@@ -9992,7 +10089,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10014,7 +10111,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10036,7 +10133,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10077,7 +10174,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10099,7 +10196,7 @@ with message:
result.errorMessage() := ""
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } )
with expansion:
{ {?} } == { {?} }
with message:
@@ -10120,7 +10217,7 @@ with expansion:
{?}
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } )
with expansion:
{ {?}, {?} } == { {?}, {?} }
@@ -10139,7 +10236,7 @@ with expansion:
{?}
CmdLine.tests.cpp:<line number>: PASSED:
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } )
REQUIRE( config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } )
with expansion:
{ {?}, {?} } == { {?}, {?} }
@@ -11134,7 +11231,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring("fakeTag"s) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11174,7 +11271,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring("fake reporter"s) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11212,7 +11309,7 @@ Reporters.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) )
with expansion:
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -11222,14 +11319,9 @@ with expansion:
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" )
with message:
@@ -20123,6 +20215,6 @@ Misc.tests.cpp:<line number>
Misc.tests.cpp:<line number>: PASSED:
===============================================================================
test cases: 451 | 331 passed | 96 failed | 6 skipped | 18 failed as expected
assertions: 2416 | 2215 passed | 158 failed | 43 failed as expected
test cases: 454 | 334 passed | 96 failed | 6 skipped | 18 failed as expected
assertions: 2423 | 2222 passed | 158 failed | 43 failed as expected
+10 -1
View File
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuitesloose text artifact
>
<testsuite name="<exe-name>" errors="17" failures="141" skipped="12" tests="2428" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<testsuite name="<exe-name>" errors="17" failures="141" skipped="12" tests="2435" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<properties>
<property name="random-seed" value="1"/>
<property name="filters" value="&quot;*&quot; ~[!nonportable] ~[!benchmark] ~[approvals]"/>
@@ -774,6 +774,15 @@ at Condition.tests.cpp:<line number>
<testcase classname="<exe-name>.global" name="JsonWriter escapes characters in strings properly/carriage return in a string is escaped" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter escapes characters in strings properly/tab in a string is escaped" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter escapes characters in strings properly/combination of characters is escaped" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - double" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - double/NaN" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - double/Pos nf" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - double/Neg inf" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - float" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - float/NaN" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - float/Pos nf" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - float/Neg inf" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes numbers independently of the global locale" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Lambdas in assertions" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Less-than inequalities with different epsilons" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="ManuallyRegistered" time="{duration}" status="run"/>
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="<exe-name>" errors="17" failures="141" skipped="12" tests="2428" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<testsuite name="<exe-name>" errors="17" failures="141" skipped="12" tests="2435" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<properties>
<property name="random-seed" value="1"/>
<property name="filters" value="&quot;*&quot; ~[!nonportable] ~[!benchmark] ~[approvals]"/>
@@ -773,6 +773,15 @@ at Condition.tests.cpp:<line number>
<testcase classname="<exe-name>.global" name="JsonWriter escapes characters in strings properly/carriage return in a string is escaped" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter escapes characters in strings properly/tab in a string is escaped" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter escapes characters in strings properly/combination of characters is escaped" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - double" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - double/NaN" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - double/Pos nf" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - double/Neg inf" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - float" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - float/NaN" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - float/Pos nf" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes non-finite FP using Python spelling - float/Neg inf" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="JsonWriter serializes numbers independently of the global locale" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Lambdas in assertions" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Less-than inequalities with different epsilons" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="ManuallyRegistered" time="{duration}" status="run"/>
@@ -262,6 +262,15 @@ at AssertionHandler.tests.cpp:<line number>
<testCase name="JsonWriter escapes characters in strings properly/carriage return in a string is escaped" duration="{duration}"/>
<testCase name="JsonWriter escapes characters in strings properly/tab in a string is escaped" duration="{duration}"/>
<testCase name="JsonWriter escapes characters in strings properly/combination of characters is escaped" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - double" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - double/NaN" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - double/Pos nf" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - double/Neg inf" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - float" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - float/NaN" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - float/Pos nf" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - float/Neg inf" duration="{duration}"/>
<testCase name="JsonWriter serializes numbers independently of the global locale" duration="{duration}"/>
</file>
<file path="tests/<exe-name>/IntrospectiveTests/Parse.tests.cpp">
<testCase name="Parse uints" duration="{duration}"/>
@@ -261,6 +261,15 @@ at AssertionHandler.tests.cpp:<line number>
<testCase name="JsonWriter escapes characters in strings properly/carriage return in a string is escaped" duration="{duration}"/>
<testCase name="JsonWriter escapes characters in strings properly/tab in a string is escaped" duration="{duration}"/>
<testCase name="JsonWriter escapes characters in strings properly/combination of characters is escaped" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - double" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - double/NaN" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - double/Pos nf" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - double/Neg inf" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - float" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - float/NaN" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - float/Pos nf" duration="{duration}"/>
<testCase name="JsonWriter serializes non-finite FP using Python spelling - float/Neg inf" duration="{duration}"/>
<testCase name="JsonWriter serializes numbers independently of the global locale" duration="{duration}"/>
</file>
<file path="tests/<exe-name>/IntrospectiveTests/Parse.tests.cpp">
<testCase name="Parse uints" duration="{duration}"/>
+27 -13
View File
@@ -2054,6 +2054,20 @@ ok {test-number} - sstream.str() == "\"\\r\"" for: ""\r"" == ""\r""
ok {test-number} - sstream.str() == "\"\\t\"" for: ""\t"" == ""\t""
# JsonWriter escapes characters in strings properly
ok {test-number} - sstream.str() == "\"\\\\/\\t\\r\\n\"" for: ""\\/\t\r\n"" == ""\\/\t\r\n""
# JsonWriter serializes non-finite FP using Python spelling - double
ok {test-number} - sstream.str() == "NaN" for: "NaN" == "NaN"
# JsonWriter serializes non-finite FP using Python spelling - double
ok {test-number} - sstream.str() == "Infinity" for: "Infinity" == "Infinity"
# JsonWriter serializes non-finite FP using Python spelling - double
ok {test-number} - sstream.str() == "-Infinity" for: "-Infinity" == "-Infinity"
# JsonWriter serializes non-finite FP using Python spelling - float
ok {test-number} - sstream.str() == "NaN" for: "NaN" == "NaN"
# JsonWriter serializes non-finite FP using Python spelling - float
ok {test-number} - sstream.str() == "Infinity" for: "Infinity" == "Infinity"
# JsonWriter serializes non-finite FP using Python spelling - float
ok {test-number} - sstream.str() == "-Infinity" for: "-Infinity" == "-Infinity"
# JsonWriter serializes numbers independently of the global locale
ok {test-number} - sstream.str(), ContainsSubstring( "\"double\": 1.5," ) && ContainsSubstring( "\"int\": 1234567," ) && ContainsSubstring( "\"bool-1\": true," ) && ContainsSubstring( "\"bool-2\": false," ) && ContainsSubstring( "\"array\": [\n 2.5,\n 1234567\n ]\n}" ) for: "{ "double": 1.5, "int": 1234567, "bool-1": true, "bool-2": false, "array": [ 2.5, 1234567 ] }" ( contains: ""double": 1.5," and contains: ""int": 1234567," and contains: ""bool-1": true," and contains: ""bool-2": false," and contains: ""array": [ 2.5, 1234567 ] }" )
# Lambdas in assertions
ok {test-number} - []() { return true; }() for: true
# Less-than inequalities with different epsilons
@@ -2501,11 +2515,11 @@ ok {test-number} - !(cfg.hasTestFilters()) for: !false
# Process can be configured on command line
ok {test-number} - cfg.getReporterSpecs().size() == 1 for: 1 == 1
# Process can be configured on command line
ok {test-number} - cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } for: {?} == {?}
ok {test-number} - cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } for: {?} == {?}
# Process can be configured on command line
ok {test-number} - cfg.getProcessedReporterSpecs().size() == 1 for: 1 == 1
# Process can be configured on command line
ok {test-number} - cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } for: {?} == {?}
ok {test-number} - cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } for: {?} == {?}
# Process can be configured on command line
ok {test-number} - result for: {?}
# Process can be configured on command line
@@ -2533,15 +2547,15 @@ ok {test-number} - cfg.testSpec().matches(*fakeTestCase("alwaysIncluded")) for:
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - !result for: true
# Process can be configured on command line
@@ -2549,19 +2563,19 @@ ok {test-number} - result.errorMessage(), ContainsSubstring("Unrecognized report
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }) for: {?}
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
# Process can be configured on command line
ok {test-number} - cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }) for: {?}
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
# Process can be configured on command line
ok {test-number} - !result for: true
# Process can be configured on command line
@@ -2753,15 +2767,15 @@ ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && Cont
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring("fakeTag"s) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tags": [ { "aliases": [ "fakeTag" ], "count": 1 } ]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring("fakeTag"s) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tags": [ { "aliases": [ "fakeTag" ], "count": 1 } ]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring("fake reporter"s) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "reporters": [ { "name": "fake reporter", "description": "fake description" } ]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring("fake reporter"s) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "reporters": [ { "name": "fake reporter", "description": "fake description" } ]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tests": [ { "name": "fake test name", "class-name": "", "tags": [ "fakeTestTag" ], "source-location": { "filename": "fake-file.cpp", "line": 123456789 } } ]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tests": [ { "name": "fake test name", "tags": [ "fakeTestTag" ] } ]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
@@ -4851,5 +4865,5 @@ ok {test-number} - q3 == 23. for: 23.0 == 23.0
ok {test-number} -
# xmlentitycheck
ok {test-number} -
1..2428
1..2435
@@ -2052,6 +2052,20 @@ ok {test-number} - sstream.str() == "\"\\r\"" for: ""\r"" == ""\r""
ok {test-number} - sstream.str() == "\"\\t\"" for: ""\t"" == ""\t""
# JsonWriter escapes characters in strings properly
ok {test-number} - sstream.str() == "\"\\\\/\\t\\r\\n\"" for: ""\\/\t\r\n"" == ""\\/\t\r\n""
# JsonWriter serializes non-finite FP using Python spelling - double
ok {test-number} - sstream.str() == "NaN" for: "NaN" == "NaN"
# JsonWriter serializes non-finite FP using Python spelling - double
ok {test-number} - sstream.str() == "Infinity" for: "Infinity" == "Infinity"
# JsonWriter serializes non-finite FP using Python spelling - double
ok {test-number} - sstream.str() == "-Infinity" for: "-Infinity" == "-Infinity"
# JsonWriter serializes non-finite FP using Python spelling - float
ok {test-number} - sstream.str() == "NaN" for: "NaN" == "NaN"
# JsonWriter serializes non-finite FP using Python spelling - float
ok {test-number} - sstream.str() == "Infinity" for: "Infinity" == "Infinity"
# JsonWriter serializes non-finite FP using Python spelling - float
ok {test-number} - sstream.str() == "-Infinity" for: "-Infinity" == "-Infinity"
# JsonWriter serializes numbers independently of the global locale
ok {test-number} - sstream.str(), ContainsSubstring( "\"double\": 1.5," ) && ContainsSubstring( "\"int\": 1234567," ) && ContainsSubstring( "\"bool-1\": true," ) && ContainsSubstring( "\"bool-2\": false," ) && ContainsSubstring( "\"array\": [\n 2.5,\n 1234567\n ]\n}" ) for: "{ "double": 1.5, "int": 1234567, "bool-1": true, "bool-2": false, "array": [ 2.5, 1234567 ] }" ( contains: ""double": 1.5," and contains: ""int": 1234567," and contains: ""bool-1": true," and contains: ""bool-2": false," and contains: ""array": [ 2.5, 1234567 ] }" )
# Lambdas in assertions
ok {test-number} - []() { return true; }() for: true
# Less-than inequalities with different epsilons
@@ -2499,11 +2513,11 @@ ok {test-number} - !(cfg.hasTestFilters()) for: !false
# Process can be configured on command line
ok {test-number} - cfg.getReporterSpecs().size() == 1 for: 1 == 1
# Process can be configured on command line
ok {test-number} - cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} } for: {?} == {?}
ok {test-number} - cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } for: {?} == {?}
# Process can be configured on command line
ok {test-number} - cfg.getProcessedReporterSpecs().size() == 1 for: 1 == 1
# Process can be configured on command line
ok {test-number} - cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} } for: {?} == {?}
ok {test-number} - cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} } for: {?} == {?}
# Process can be configured on command line
ok {test-number} - result for: {?}
# Process can be configured on command line
@@ -2531,15 +2545,15 @@ ok {test-number} - cfg.testSpec().matches(*fakeTestCase("alwaysIncluded")) for:
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - !result for: true
# Process can be configured on command line
@@ -2547,19 +2561,19 @@ ok {test-number} - result.errorMessage(), ContainsSubstring("Unrecognized report
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - result for: {?} with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } for: { {?} } == { {?} } with 1 message: 'result.errorMessage() := ""'
# Process can be configured on command line
ok {test-number} - cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }) for: {?}
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
# Process can be configured on command line
ok {test-number} - cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }) for: {?}
# Process can be configured on command line
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
ok {test-number} - config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } } for: { {?}, {?} } == { {?}, {?} }
# Process can be configured on command line
ok {test-number} - !result for: true
# Process can be configured on command line
@@ -2751,15 +2765,15 @@ ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && Cont
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring("fakeTag"s) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tags": [ { "aliases": [ "fakeTag" ], "count": 1 } ]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring("fakeTag"s) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tags": [ { "aliases": [ "fakeTag" ], "count": 1 } ]" contains: "fakeTag" with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring("fake reporter"s) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "reporters": [ { "name": "fake reporter", "description": "fake description" } ]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring("fake reporter"s) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "reporters": [ { "name": "fake reporter", "description": "fake description" } ]" contains: "fake reporter" with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{ "version": 1, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tests": [ { "name": "fake test name", "class-name": "", "tags": [ "fakeTestTag" ], "source-location": { "filename": "fake-file.cpp", "line": 123456789 } } ]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
ok {test-number} - listingString, ContainsSubstring( "fake test name"s ) && ContainsSubstring( "fakeTestTag"s ) for: "{ "version": 2, "metadata": { "name": "", "rng-seed": 1234, "catch2-version": "<version>" }, "listings": { "tests": [ { "name": "fake test name", "tags": [ "fakeTestTag" ] } ]" ( contains: "fake test name" and contains: "fakeTestTag" ) with 1 message: 'Tested reporter: JSON'
# Reporter's write listings to provided stream
ok {test-number} - !(factories.empty()) for: !false
# Reporter's write listings to provided stream
@@ -4840,5 +4854,5 @@ ok {test-number} - q3 == 23. for: 23.0 == 23.0
ok {test-number} -
# xmlentitycheck
ok {test-number} -
1..2428
1..2435
@@ -470,6 +470,12 @@
##teamcity[testFinished name='JsonWriter' duration="{duration}"]
##teamcity[testStarted name='JsonWriter escapes characters in strings properly']
##teamcity[testFinished name='JsonWriter escapes characters in strings properly' duration="{duration}"]
##teamcity[testStarted name='JsonWriter serializes non-finite FP using Python spelling - double']
##teamcity[testFinished name='JsonWriter serializes non-finite FP using Python spelling - double' duration="{duration}"]
##teamcity[testStarted name='JsonWriter serializes non-finite FP using Python spelling - float']
##teamcity[testFinished name='JsonWriter serializes non-finite FP using Python spelling - float' duration="{duration}"]
##teamcity[testStarted name='JsonWriter serializes numbers independently of the global locale']
##teamcity[testFinished name='JsonWriter serializes numbers independently of the global locale' duration="{duration}"]
##teamcity[testStarted name='Lambdas in assertions']
##teamcity[testFinished name='Lambdas in assertions' duration="{duration}"]
##teamcity[testStarted name='Less-than inequalities with different epsilons']
@@ -470,6 +470,12 @@
##teamcity[testFinished name='JsonWriter' duration="{duration}"]
##teamcity[testStarted name='JsonWriter escapes characters in strings properly']
##teamcity[testFinished name='JsonWriter escapes characters in strings properly' duration="{duration}"]
##teamcity[testStarted name='JsonWriter serializes non-finite FP using Python spelling - double']
##teamcity[testFinished name='JsonWriter serializes non-finite FP using Python spelling - double' duration="{duration}"]
##teamcity[testStarted name='JsonWriter serializes non-finite FP using Python spelling - float']
##teamcity[testFinished name='JsonWriter serializes non-finite FP using Python spelling - float' duration="{duration}"]
##teamcity[testStarted name='JsonWriter serializes numbers independently of the global locale']
##teamcity[testFinished name='JsonWriter serializes numbers independently of the global locale' duration="{duration}"]
##teamcity[testStarted name='Lambdas in assertions']
##teamcity[testFinished name='Lambdas in assertions' duration="{duration}"]
##teamcity[testStarted name='Less-than inequalities with different epsilons']
+111 -20
View File
@@ -9894,6 +9894,102 @@ Approx( 3.14150000000000018 )
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="JsonWriter serializes non-finite FP using Python spelling - double" tags="[floating-point][JsonWriter]" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Section name="NaN" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "NaN"
</Original>
<Expanded>
"NaN" == "NaN"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<Section name="Pos nf" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "Infinity"
</Original>
<Expanded>
"Infinity" == "Infinity"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<Section name="Neg inf" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "-Infinity"
</Original>
<Expanded>
"-Infinity" == "-Infinity"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="JsonWriter serializes non-finite FP using Python spelling - float" tags="[floating-point][JsonWriter]" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Section name="NaN" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "NaN"
</Original>
<Expanded>
"NaN" == "NaN"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<Section name="Pos nf" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "Infinity"
</Original>
<Expanded>
"Infinity" == "Infinity"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<Section name="Neg inf" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "-Infinity"
</Original>
<Expanded>
"-Infinity" == "-Infinity"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="JsonWriter serializes numbers independently of the global locale" tags="[JsonWriter]" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE_THAT" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str(), ContainsSubstring( "\"double\": 1.5," ) &amp;&amp; ContainsSubstring( "\"int\": 1234567," ) &amp;&amp; ContainsSubstring( "\"bool-1\": true," ) &amp;&amp; ContainsSubstring( "\"bool-2\": false," ) &amp;&amp; ContainsSubstring( "\"array\": [\n 2.5,\n 1234567\n ]\n}" )
</Original>
<Expanded>
"{
"double": 1.5,
"int": 1234567,
"bool-1": true,
"bool-2": false,
"array": [
2.5,
1234567
]
}" ( contains: ""double": 1.5," and contains: ""int": 1234567," and contains: ""bool-1": true," and contains: ""bool-2": false," and contains: ""array": [
2.5,
1234567
]
}" )
</Expanded>
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="Lambdas in assertions" filename="tests/<exe-name>/UsageTests/Compilation.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/UsageTests/Compilation.tests.cpp" >
<Original>
@@ -11880,7 +11976,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} }
cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} }
</Original>
<Expanded>
{?} == {?}
@@ -11896,7 +11992,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} }
cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} }
</Original>
<Expanded>
{?} == {?}
@@ -12036,7 +12132,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12064,7 +12160,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12092,7 +12188,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12142,7 +12238,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12170,7 +12266,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12193,7 +12289,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?}, {?} } == { {?}, {?} }
@@ -12218,7 +12314,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?}, {?} } == { {?}, {?} }
@@ -13323,7 +13419,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13360,7 +13456,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13395,7 +13491,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13405,14 +13501,9 @@ Approx( 0.98999999999999999 )
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" )
</Expanded>
@@ -23385,6 +23476,6 @@ Approx( -1.95996398454005449 )
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<OverallResults successes="2215" failures="158" expectedFailures="43" skips="12"/>
<OverallResultsCases successes="331" failures="96" expectedFailures="18" skips="6"/>
<OverallResults successes="2222" failures="158" expectedFailures="43" skips="12"/>
<OverallResultsCases successes="334" failures="96" expectedFailures="18" skips="6"/>
</Catch2TestRun>
@@ -9894,6 +9894,102 @@ Approx( 3.14150000000000018 )
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="JsonWriter serializes non-finite FP using Python spelling - double" tags="[floating-point][JsonWriter]" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Section name="NaN" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "NaN"
</Original>
<Expanded>
"NaN" == "NaN"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<Section name="Pos nf" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "Infinity"
</Original>
<Expanded>
"Infinity" == "Infinity"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<Section name="Neg inf" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "-Infinity"
</Original>
<Expanded>
"-Infinity" == "-Infinity"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="JsonWriter serializes non-finite FP using Python spelling - float" tags="[floating-point][JsonWriter]" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Section name="NaN" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "NaN"
</Original>
<Expanded>
"NaN" == "NaN"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<Section name="Pos nf" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "Infinity"
</Original>
<Expanded>
"Infinity" == "Infinity"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<Section name="Neg inf" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str() == "-Infinity"
</Original>
<Expanded>
"-Infinity" == "-Infinity"
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0" skipped="false"/>
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="JsonWriter serializes numbers independently of the global locale" tags="[JsonWriter]" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Expression success="true" type="REQUIRE_THAT" filename="tests/<exe-name>/IntrospectiveTests/Json.tests.cpp" >
<Original>
sstream.str(), ContainsSubstring( "\"double\": 1.5," ) &amp;&amp; ContainsSubstring( "\"int\": 1234567," ) &amp;&amp; ContainsSubstring( "\"bool-1\": true," ) &amp;&amp; ContainsSubstring( "\"bool-2\": false," ) &amp;&amp; ContainsSubstring( "\"array\": [\n 2.5,\n 1234567\n ]\n}" )
</Original>
<Expanded>
"{
"double": 1.5,
"int": 1234567,
"bool-1": true,
"bool-2": false,
"array": [
2.5,
1234567
]
}" ( contains: ""double": 1.5," and contains: ""int": 1234567," and contains: ""bool-1": true," and contains: ""bool-2": false," and contains: ""array": [
2.5,
1234567
]
}" )
</Expanded>
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="Lambdas in assertions" filename="tests/<exe-name>/UsageTests/Compilation.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/UsageTests/Compilation.tests.cpp" >
<Original>
@@ -11880,7 +11976,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {} }
cfg.getReporterSpecs()[0] == Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} }
</Original>
<Expanded>
{?} == {?}
@@ -11896,7 +11992,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, {} }
cfg.getProcessedReporterSpecs()[0] == Catch::ProcessedReporterSpec{ expectedReporter, std::string{}, Catch::ColourMode::PlatformDefault, Catch::Verbosity::Normal, {} }
</Original>
<Expanded>
{?} == {?}
@@ -12036,7 +12132,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12064,7 +12160,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12092,7 +12188,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "junit", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12142,7 +12238,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", "out.txt"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12170,7 +12266,7 @@ Approx( 1.21999999999999997 )
</Info>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?} } == { {?} }
@@ -12193,7 +12289,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "junit", "output-junit.xml"s, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "junit", "output-junit.xml"s, {}, {}, {} } }
</Original>
<Expanded>
{ {?}, {?} } == { {?}, {?} }
@@ -12218,7 +12314,7 @@ Approx( 1.21999999999999997 )
</Expression>
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp" >
<Original>
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {} }, { "console", {}, {}, {} } }
config.reporterSpecifications == vec_Specs{ { "xml", "output.xml"s, {}, {}, {} }, { "console", {}, {}, {}, {} } }
</Original>
<Expanded>
{ {?}, {?} } == { {?}, {?} }
@@ -13323,7 +13419,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13360,7 +13456,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13395,7 +13491,7 @@ Approx( 0.98999999999999999 )
</Original>
<Expanded>
"{
"version": 1,
"version": 2,
"metadata": {
"name": "",
"rng-seed": 1234,
@@ -13405,14 +13501,9 @@ Approx( 0.98999999999999999 )
"tests": [
{
"name": "fake test name",
"class-name": "",
"tags": [
"fakeTestTag"
],
"source-location": {
"filename": "fake-file.cpp",
"line": 123456789
}
]
}
]" ( contains: "fake test name" and contains: "fakeTestTag" )
</Expanded>
@@ -23384,6 +23475,6 @@ Approx( -1.95996398454005449 )
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<OverallResults successes="2215" failures="158" expectedFailures="43" skips="12"/>
<OverallResultsCases successes="331" failures="96" expectedFailures="18" skips="6"/>
<OverallResults successes="2222" failures="158" expectedFailures="43" skips="12"/>
<OverallResultsCases successes="334" failures="96" expectedFailures="18" skips="6"/>
</Catch2TestRun>
@@ -58,12 +58,13 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CHECK( cfg.getReporterSpecs().size() == 1 );
CHECK( cfg.getReporterSpecs()[0] ==
Catch::ReporterSpec{ expectedReporter, {}, {}, {} } );
Catch::ReporterSpec{ expectedReporter, {}, {}, {}, {} } );
CHECK( cfg.getProcessedReporterSpecs().size() == 1 );
CHECK( cfg.getProcessedReporterSpecs()[0] ==
Catch::ProcessedReporterSpec{ expectedReporter,
std::string{},
Catch::ColourMode::PlatformDefault,
Catch::Verbosity::Normal,
{} } );
}
@@ -108,7 +109,7 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "console", {}, {}, {} } } );
vec_Specs{ { "console", {}, {}, {}, {} } } );
}
SECTION("-r/xml") {
auto result = cli.parse({"test", "-r", "xml"});
@@ -116,7 +117,7 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "xml", {}, {}, {} } } );
vec_Specs{ { "xml", {}, {}, {}, {} } } );
}
SECTION("--reporter/junit") {
auto result = cli.parse({"test", "--reporter", "junit"});
@@ -124,7 +125,7 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "junit", {}, {}, {} } } );
vec_Specs{ { "junit", {}, {}, {}, {} } } );
}
SECTION("must match one of the available ones") {
auto result = cli.parse({"test", "--reporter", "unsupported"});
@@ -137,27 +138,27 @@ TEST_CASE( "Process can be configured on command line", "[config][command-line]"
CAPTURE(result.errorMessage());
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "console", "out.txt"s, {}, {} } } );
vec_Specs{ { "console", "out.txt"s, {}, {}, {} } } );
}
SECTION("With Windows-like absolute path as output file") {
auto result = cli.parse({ "test", "-r", "console::out=C:\\Temp\\out.txt" });
CAPTURE(result.errorMessage());
CHECK(result);
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {} } } );
vec_Specs{ { "console", "C:\\Temp\\out.txt"s, {}, {}, {} } } );
}
SECTION("Multiple reporters") {
SECTION("All with output files") {
CHECK(cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "junit::out=output-junit.xml" }));
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "xml", "output.xml"s, {}, {} },
{ "junit", "output-junit.xml"s, {}, {} } } );
vec_Specs{ { "xml", "output.xml"s, {}, {}, {} },
{ "junit", "output-junit.xml"s, {}, {}, {} } } );
}
SECTION("Mixed output files and default output") {
CHECK(cli.parse({ "test", "-r", "xml::out=output.xml", "-r", "console" }));
REQUIRE( config.reporterSpecifications ==
vec_Specs{ { "xml", "output.xml"s, {}, {} },
{ "console", {}, {}, {} } } );
vec_Specs{ { "xml", "output.xml"s, {}, {}, {} },
{ "console", {}, {}, {}, {} } } );
}
SECTION("cannot have multiple reporters with default output") {
auto result = cli.parse({ "test", "-r", "console", "-r", "xml::out=output.xml", "-r", "junit" });
@@ -63,6 +63,20 @@ TEST_CASE( "Parsing colour mode", "[cli][colour][approvals]" ) {
}
}
TEST_CASE( "Parsing verbosity", "[cli][colour][approvals]" ) {
using Catch::Detail::stringToVerbosity;
using Catch::Verbosity;
SECTION( "Valid strings" ) {
REQUIRE( stringToVerbosity( "quiet" ) == Verbosity::Quiet );
REQUIRE( stringToVerbosity( "normal" ) == Verbosity::Normal );
REQUIRE( stringToVerbosity( "high" ) == Verbosity::High );
}
SECTION( "Wrong strings" ) {
REQUIRE_FALSE( stringToVerbosity( "QUIET" ) );
REQUIRE_FALSE( stringToVerbosity( "medium" ) );
REQUIRE_FALSE( stringToVerbosity( "vvv" ) );
}
}
TEST_CASE("Parsing reporter specs", "[cli][reporter-spec][approvals]") {
using Catch::parseReporterSpec;
@@ -71,12 +85,13 @@ TEST_CASE("Parsing reporter specs", "[cli][reporter-spec][approvals]") {
SECTION( "Correct specs" ) {
REQUIRE( parseReporterSpec( "someReporter" ) ==
ReporterSpec( "someReporter"s, {}, {}, {} ) );
ReporterSpec( "someReporter"s, {}, {}, {}, {} ) );
REQUIRE( parseReporterSpec( "otherReporter::Xk=v::out=c:\\blah" ) ==
ReporterSpec(
"otherReporter"s, "c:\\blah"s, {}, { { "Xk"s, "v"s } } ) );
"otherReporter"s, "c:\\blah"s, {}, {}, { { "Xk"s, "v"s } } ) );
REQUIRE( parseReporterSpec( "diffReporter::Xk1=v1::Xk2==v2" ) ==
ReporterSpec( "diffReporter",
{},
{},
{},
{ { "Xk1"s, "v1"s }, { "Xk2"s, "=v2"s } } ) );
@@ -85,7 +100,15 @@ TEST_CASE("Parsing reporter specs", "[cli][reporter-spec][approvals]") {
ReporterSpec( "Foo:bar:reporter",
{},
Catch::ColourMode::ANSI,
{},
{ { "Xk 1"s, "v 1"s }, { "Xk2"s, "v:3"s } } ) );
REQUIRE(
parseReporterSpec( "my:reporter::X1=V1::verbosity=high::X2=V2" ) ==
ReporterSpec( "my:reporter",
{},
{},
Catch::Verbosity::High,
{ { "X1"s, "V1"s }, { "X2"s, "V2"s } } ) );
}
SECTION( "Bad specs" ) {
@@ -107,5 +130,9 @@ TEST_CASE("Parsing reporter specs", "[cli][reporter-spec][approvals]") {
REQUIRE_FALSE( parseReporterSpec( "reporter::Xa=" ) );
// non-key value later field
REQUIRE_FALSE( parseReporterSpec( "reporter::Xab" ) );
// Invalid verbosity value
REQUIRE_FALSE( parseReporterSpec( "reporter::verbosity=medium" ) );
// Duplicated verbosity
REQUIRE_FALSE( parseReporterSpec( "reporter::verbosity=high::X1=X2::verbosity=high" ) );
}
}
@@ -7,11 +7,14 @@
// SPDX-License-Identifier: BSL-1.0
#include <catch2/catch_test_macros.hpp>
#include <catch2/catch_template_test_macros.hpp>
#include <catch2/benchmark/catch_benchmark.hpp>
#include <catch2/generators/catch_generators.hpp>
#include <catch2/internal/catch_jsonwriter.hpp>
#include <catch2/matchers/catch_matchers_string.hpp>
#include <limits>
#include <locale>
#include <sstream>
namespace {
@@ -20,6 +23,27 @@ namespace {
return os << "custom";
}
// Obviously wrong numpunct if it is actually used by the JSON writer.
class TestNumpunct : public std::numpunct<char> {
char do_decimal_point() const override { return '?'; }
char do_thousands_sep() const override { return '!'; }
std::string do_grouping() const override { return "\1"; }
std::string do_truename() const override { return "real"; }
std::string do_falsename() const override { return "fake"; }
};
class LocaleGuard {
std::locale m_previous_locale;
public:
explicit LocaleGuard( std::locale const& locale ):
m_previous_locale{ std::locale() } {
std::locale::global( locale );
}
~LocaleGuard() { std::locale::global( m_previous_locale ); }
};
TEST_CASE( "JsonWriter", "[JSON][JsonWriter]" ) {
std::stringstream stream;
@@ -152,6 +176,54 @@ TEST_CASE( "JsonWriter escapes characters in strings properly", "[JsonWriter]" )
}
}
TEST_CASE( "JsonWriter serializes numbers independently of the global locale",
"[JsonWriter]" ) {
using Catch::Matchers::ContainsSubstring;
LocaleGuard locale_guard{ std::locale{ std::locale(), new TestNumpunct } };
std::stringstream sstream;
{
auto writer = Catch::JsonValueWriter{ sstream }.writeObject();
writer.write( "double" ).write( 1.5 );
writer.write( "int" ).write( 1234567 );
writer.write( "bool-1" ).write( true );
writer.write( "bool-2" ).write( false );
writer.write( "array" ).writeArray().write( 2.5 ).write( 1234567 );
}
REQUIRE_THAT( sstream.str(),
ContainsSubstring( "\"double\": 1.5," ) &&
ContainsSubstring( "\"int\": 1234567," ) &&
ContainsSubstring( "\"bool-1\": true," ) &&
ContainsSubstring( "\"bool-2\": false," ) &&
ContainsSubstring(
"\"array\": [\n 2.5,\n 1234567\n ]\n}" ) );
}
TEMPLATE_TEST_CASE( "JsonWriter serializes non-finite FP using Python spelling",
"[JsonWriter][floating-point]",
float,
double ) {
std::stringstream sstream;
SECTION( "NaN" ) {
Catch::JsonValueWriter{ sstream }.write(
std::numeric_limits<TestType>::quiet_NaN() );
REQUIRE( sstream.str() == "NaN" );
}
SECTION( "Pos nf" ) {
Catch::JsonValueWriter{ sstream }.write(
std::numeric_limits<TestType>::infinity() );
REQUIRE( sstream.str() == "Infinity" );
}
SECTION( "Neg inf" ) {
Catch::JsonValueWriter{ sstream }.write(
-std::numeric_limits<TestType>::infinity() );
REQUIRE( sstream.str() == "-Infinity" );
}
}
TEST_CASE( "JsonWriter benchmarks", "[JsonWriter][!benchmark]" ) {
const auto input_length = GENERATE( as<size_t>{}, 10, 100, 10'000 );
std::string test_input( input_length, 'a' );
@@ -42,6 +42,7 @@ namespace {
&config,
Catch::Detail::make_unique<StringIStream>(),
Catch::ColourMode::None,
Catch::Verbosity::Normal,
{} };
}
}
@@ -114,7 +115,7 @@ TEST_CASE( "Reporter's write listings to provided stream", "[reporters]" ) {
cfg_data.rngSeed = 1234;
Catch::Config config( cfg_data );
auto reporter = factory.second->create( Catch::ReporterConfig{
&config, CATCH_MOVE( sstream ), Catch::ColourMode::None, {} } );
&config, CATCH_MOVE( sstream ), Catch::ColourMode::None, Catch::Verbosity::Normal, {} } );
DYNAMIC_SECTION( factory.first << " reporter lists tags" ) {
std::vector<Catch::TagInfo> tags(1);
@@ -437,6 +437,25 @@ TEST_CASE( "TextFlow::AnsiSkippingString iterates UTF-8 codepoints",
}
}
TEST_CASE( "TextFlow::AnsiSkippingString handles invalid UTF-8",
"[TextFlow][ansiskippingstring][approvals]" ) {
SECTION( "Continuation byte at the start" ) {
// 0x80 is a continuation byte
AnsiSkippingString str( "\x80" );
auto it = str.end();
--it;
CHECK( it == str.begin() );
CHECK( *it == static_cast<char>( 0x80 ) );
}
SECTION( "Multiple continuation bytes at the start" ) {
AnsiSkippingString str( "\x80\x80\x80" );
auto it = str.end();
--it;
CHECK( it == str.begin() );
CHECK( *it == static_cast<char>( 0x80 ) );
}
}
TEST_CASE( "TextFlow::Column wraps UTF-8 text correctly",
"[TextFlow][column][approvals]" ) {
// "äöü äöü äöü" = 11 codepoints, 17 bytes
@@ -0,0 +1,194 @@
# SPDX-License-Identifier: BSL-1.0
# Unit tests for `split_json_array` helper in `extras/CatchAddTests.cmake`.
#
# Yes, we are at the stage where the script helpers need unit tests.
#
# Run as
# cmake -DCATCH_ADD_TESTS_SCRIPT=/path/to/extras/CatchAddTests.cmake \
# -P TestDecomposeJsonArray.cmake
cmake_minimum_required(VERSION 3.19)
if(NOT DEFINED CATCH_ADD_TESTS_SCRIPT)
message(FATAL_ERROR "Missing argument `CATCH_ADD_TESTS_SCRIPT`")
endif()
if(NOT EXISTS "${CATCH_ADD_TESTS_SCRIPT}")
message(FATAL_ERROR "Cannot find CatchAddTests.cmake at '${CATCH_ADD_TESTS_SCRIPT}'")
endif()
# Pull in the helper functions. Without `TEST_EXECUTABLE` being defined,
# `catch_discover_tests_impl` is not called.
include("${CATCH_ADD_TESTS_SCRIPT}")
set(_failures 0)
# Parses out test names from provided listings and returns them through `out_var`.
# Semicolons in test names are escaped as `@SEMI@`.
function(decomposed_names listing_var out_var)
split_json_array(${listing_var} elements)
set(names "")
foreach(element IN LISTS elements)
magic_unescape_chars(element)
string(JSON name ERROR_VARIABLE err GET "${element}" "name")
if(NOT err STREQUAL "NOTFOUND")
set(${out_var} "PARSE-ERROR" PARENT_SCOPE)
return()
endif()
string(REPLACE ";" "@SEMI@" name "${name}")
list(APPEND names "${name}")
endforeach()
set(${out_var} "${names}" PARENT_SCOPE)
endfunction()
# Assert that decomposing the provided listing returns the expected list
# of (test) names.
# Semicolons inside expected names must be escaped as `@SEMI@`.
function(expect_names description listing_var expected_names)
decomposed_names(${listing_var} actual_names)
if(actual_names STREQUAL expected_names)
message(" [PASS] ${description}")
else()
message(" [FAIL] ${description}")
message(" expected: ${expected_names}")
message(" actual: ${actual_names}")
math(EXPR _n "${_failures} + 1")
set(_failures "${_n}" PARENT_SCOPE)
endif()
endfunction()
# Assert that the tags in one decomposed element are preserved exactly.
# Semicolons in expected tags have to be escaped as `@SEMI@`.
function(expect_tags description listing_var element_index expected_tags)
split_json_array(${listing_var} elements)
list(GET elements ${element_index} element)
magic_unescape_chars(element)
string(JSON tags ERROR_VARIABLE err GET "${element}" "tags")
if(NOT err STREQUAL "NOTFOUND")
set(actual_tags "PARSE-ERROR")
else()
string(JSON tag_count LENGTH "${tags}")
set(actual_tags "")
if(tag_count GREATER 0)
math(EXPR last_tag "${tag_count} - 1")
foreach(tag_index RANGE ${last_tag})
string(JSON tag GET "${tags}" ${tag_index})
string(REPLACE ";" "@SEMI@" tag "${tag}")
list(APPEND actual_tags "${tag}")
endforeach()
endif()
endif()
if(actual_tags STREQUAL expected_tags)
message(" [PASS] ${description}")
else()
message(" [FAIL] ${description}")
message(" expected: ${expected_tags}")
message(" actual: ${actual_tags}")
math(EXPR _n "${_failures} + 1")
set(_failures "${_n}" PARENT_SCOPE)
endif()
endfunction()
# Convenience for building a minimal-but-realistic (pretty-printed) listing.
function(make_listing out_var)
set(objects "")
foreach(name IN LISTS ARGN)
# Build each object by hand; the names passed in are already JSON-safe.
string(APPEND objects
" {\n"
" \"class-name\" : \"\",\n"
" \"name\" : \"${name}\",\n"
" \"tags\" : [ \"[tag]\" ]\n"
" },\n")
endforeach()
string(REGEX REPLACE ",\n$" "\n" objects "${objects}")
set(${out_var} "[\n${objects}]" PARENT_SCOPE)
endfunction()
message(STATUS "Running split_json_array correctness tests")
# There are 2 main difficulties in the array decomposition that we need
# to check for:
# 1) Names/tags that contain the expected element boundary (`}<ws>*,<ws>*{`)
# inside them, and thus are split into invalid JSON.
# 2) Names/tags that contain CMake-relevant characters (e.g. semicolon,
# which is list separator) and thus cause issues when processing the
# string splits.
make_listing(listing "")
expect_names("No tests" listing "")
make_listing(listing "plain")
expect_names("Single test" listing "plain")
make_listing(listing "n1" "n2" "n3")
expect_names("Multiple tests" listing "n1;n2;n3")
make_listing(listing "before },{ after" "second")
expect_names("The element boundary in a test name" listing "before },{ after;second")
make_listing(listing "},{" "next")
expect_names("Test name is just the boundary" listing "},{;next")
make_listing(listing "a},{b},{c" "x" "y},{z")
expect_names("Test name has multiple boundaries" listing "a},{b},{c;x;y},{z")
# Listings with semicolons have to be built by hand, or CMake would mess
# them up before decomposition.
set(listing "[{\"name\":\"has;semicolon\",\"tags\":[]},{\"name\":\"and;another;one\",\"tags\":[]}]")
expect_names("Test names with semicolons" listing "has@SEMI@semicolon;and@SEMI@another@SEMI@one")
set(listing "[{\"name\":\"C:\\\\path\\\\file\",\"tags\":[]},{\"name\":\"plain\",\"tags\":[]}]")
expect_names("Test names with backslashes" listing "C:\\path\\file;plain")
set(listing "[{\"name\":\"compact1\",\"tags\":[]},{\"name\":\"compact2\",\"tags\":[]}]")
expect_names("compact json" listing "compact1;compact2")
make_listing(listing "Then } , { we }\t,\t{ concatenate } , { them } ,{back},{" "second")
expect_names("Whitespace around commas in test names survive split"
listing "Then } , { we }\t,\t{ concatenate } , { them } ,{back},{;second")
make_listing(listing "}},{{" "second")
expect_names("Doubled up boundary braces in names" listing "}},{{;second")
# Square brackets and array likes in the test names.
set(listing "[{\"name\":\"Arrays [{},{}] wheee\",\"tags\":[\"also},{tags\",\"tag;with;semicolons\"]},{\"name\":\"n\",\"tags\":[]}]")
expect_names("array-like substring in name" listing "Arrays [{},{}] wheee;n")
expect_tags("boundary-like and semicolon tags are preserved" listing 0
"also},{tags;tag@SEMI@with@SEMI@semicolons")
expect_tags("empty tags are preserved" listing 1 "")
# Listings with square brackets have to be built by hand, or CMake would
# mess them up before decomposition.
set(listing "[{\"name\":\"[\",\"tags\":[]},{\"name\":\"middle\",\"tags\":[]},{\"name\":\"]\",\"tags\":[]}]")
expect_names("unmatched square brackets" listing "[;middle;]")
set(listing "[{\"name\":\"a[b]c\",\"tags\":[]},{\"name\":\"[open\",\"tags\":[]},{\"name\":\"close]\",\"tags\":[]},{\"name\":\"[]\",\"tags\":[]}]")
expect_names("Test names with mess of brackets" listing "a[b]c;[open;close];[]")
# Without careful handling, these could be evaluated as variables.
make_listing(listing
"curly \${NOT_A_VAR}"
"env \$ENV{HOME}"
"cache \$CACHE{FOO}"
"genex \$<CONFIG>"
"bare \$ and \$\$ and \${ unterminated")
expect_names("Test names with dollars and various brackets (CMake vars)" listing
"curly \${NOT_A_VAR};env \$ENV{HOME};cache \$CACHE{FOO};genex \$<CONFIG>;bare \$ and \$\$ and \${ unterminated")
# Listings with semicolons have to be built by hand, or CMake would mess
# them up before decomposition.
# This is just a huge mess of everything to see if anything shakes loose.
set(listing "[{\"name\":\"\$ENV{X};[weird]{},{ mix \$<0:no> \${VAR} };,{ }},{{\",\"tags\":[]},{\"name\":\"after\",\"tags\":[]}]")
expect_names("combined mega-case" listing
"\$ENV{X}@SEMI@[weird]{},{ mix \$<0:no> \${VAR} }@SEMI@,{ }},{{;after")
if(_failures GREATER 0)
message(FATAL_ERROR "${_failures} decomposition test(s) failed")
endif()
message(STATUS "All split_json_array correctness tests passed")
@@ -0,0 +1,61 @@
# SPDX-License-Identifier: BSL-1.0
# Unit tests for `prepare_command_fragment` helper in `extras/CatchAddTests.cmake`.
#
# Yes, we are at the stage where the script helpers need unit tests.
#
# Run as
# cmake -DCATCH_ADD_TESTS_SCRIPT=/path/to/extras/CatchAddTests.cmake \
# -P TestPrepareCommandFragment.cmake
cmake_minimum_required(VERSION 3.19)
if(NOT DEFINED CATCH_ADD_TESTS_SCRIPT)
message(FATAL_ERROR "Missing argument `CATCH_ADD_TESTS_SCRIPT`")
endif()
if(NOT EXISTS "${CATCH_ADD_TESTS_SCRIPT}")
message(FATAL_ERROR "Cannot find CatchAddTests.cmake at '${CATCH_ADD_TESTS_SCRIPT}'")
endif()
# Pull in the helper functions. Without `TEST_EXECUTABLE` being defined,
# `catch_discover_tests_impl` is not called.
include("${CATCH_ADD_TESTS_SCRIPT}")
set(_failures 0)
function(expect_equal description actual expected)
if(actual STREQUAL expected)
message(" [PASS] ${description}")
else()
message(" [FAIL] ${description}")
message(" expected: [${expected}]")
message(" actual: [${actual}]")
math(EXPR _n "${_failures} + 1")
set(_failures "${_n}" PARENT_SCOPE)
endif()
endfunction()
prepare_command_fragment(test_fragment SimpleName /path/to/tests)
expect_equal("Simple arg, no quotes" "${test_fragment}" " SimpleName /path/to/tests")
prepare_command_fragment(test_fragment "Name with spaces")
expect_equal("Spaces in arg, needs quotes" "${test_fragment}" " [==[Name with spaces]==]")
prepare_command_fragment(test_fragment Foo PROPERTIES LABELS "tagA\;tagB\;tagC")
expect_equal("semicolons in argument are kept and quoted"
"${test_fragment}" " Foo PROPERTIES LABELS [==[tagA\;tagB\;tagC]==]")
set(test_fragment "PRE-EXISTING")
prepare_command_fragment(test_fragment Foo PROPERTIES BAR baz)
expect_equal("out var does no accumulate commands"
"${test_fragment}" " Foo PROPERTIES BAR baz")
if(_failures GREATER 0)
message(FATAL_ERROR "${_failures} prepare_command_fragment test(s) failed")
else()
message(STATUS "All prepare_command_fragment tests passed")
endif()
@@ -7,11 +7,13 @@
# SPDX-License-Identifier: BSL-1.0
import glob
import os
import subprocess
import sys
import re
import json
import tempfile
from collections import namedtuple
from typing import List
@@ -64,7 +66,6 @@ def build_project(sources_dir, output_base_path, catch2_path):
return build_dir
def get_test_names(build_path: str) -> List[TestInfo]:
# For now we assume that Windows builds are done using MSBuild under
# Debug configuration. This means that we need to add "Debug" folder
@@ -72,16 +73,21 @@ def get_test_names(build_path: str) -> List[TestInfo]:
config_path = "Debug" if os.name == 'nt' else ""
full_path = os.path.join(build_path, config_path, 'tests')
with tempfile.TemporaryDirectory() as tmpdir:
fname = f'{tmpdir}/listing-output.json'
cmd = [full_path,
'--list-tests',
'--reporter', 'json',
'--out', fname
]
result = subprocess.run(cmd,
capture_output = False,
check = True,
text = True)
with open(fname, mode='r', encoding='utf-8') as file:
test_listing = json.load(file)
cmd = [full_path, '--reporter', 'json', '--list-tests']
result = subprocess.run(cmd,
capture_output = True,
check = True,
text = True)
test_listing = json.loads(result.stdout)
assert test_listing['version'] == 1
assert test_listing['version'] == 2
tests = []
for test in test_listing['listings']['tests']:
@@ -91,15 +97,24 @@ def get_test_names(build_path: str) -> List[TestInfo]:
return tests
def get_ctest_listing(build_path):
old_path = os.getcwd()
os.chdir(build_path)
cmd = ['ctest', '-C', 'debug', '--show-only=json-v1']
result = subprocess.run(cmd,
capture_output = True,
check = True,
text = True)
try:
result = subprocess.run(cmd,
capture_output = True,
check = True,
text = True)
except subprocess.CalledProcessError as err:
print('Error when getting output from CTest')
print(f'cmd: {err.cmd}')
print(f'stderr: {err.stderr}')
print(f'stdout: {err.stdout}')
exit(4)
os.chdir(old_path)
return result.stdout
@@ -123,6 +138,7 @@ def extract_tests_from_ctest(ctest_output) -> List[TestInfo]:
return test_infos
def check_DL_PATHS(ctest_output):
ctest_response = json.loads(ctest_output)
tests = ctest_response['tests']
@@ -132,6 +148,98 @@ def check_DL_PATHS(ctest_output):
if property['name'] == 'ENVIRONMENT_MODIFICATION':
assert len(property['value']) == 2, f"The test provides 2 arguments to DL_PATHS, but instead found {len(property['value'])}"
def add_test_list_extractor(build_path: str) -> str:
# The actual CTest script file has one of two names:
# * `<target>-<short-hash>_tests.cmake` on single-config generators
# * `<target>-<short-hash>_tests-<Config>.cmake` on multi-config generators
#
# We know the target name (`tests`), so we glob for the hash part
patterns = [
os.path.join(build_path, 'tests-*_tests.cmake'),
os.path.join(build_path, 'tests-*_tests-Debug.cmake'),
]
matches = []
for pattern in patterns:
matches.extend(glob.glob(pattern))
if len(matches) != 1:
print(f"Found {len(matches)} CTest files in '{build_path}'. Expected only 1.")
exit(5)
test_script_file = matches[0]
basename = os.path.basename(test_script_file)
extractor_fname = os.path.join(build_path, 'extractor.cmake')
with open(extractor_fname, 'w') as f:
f.write(fr"""
cmake_minimum_required(VERSION 3.19)
cmake_policy(VERSION 3.19...4.4)
# This dummies out the `add_test` and `set_tests_properties` commands
# inside the CTest script, so we can include it during CMake script call.
macro(add_test)
endmacro()
macro(set_tests_properties)
endmacro()
include(${{CMAKE_CURRENT_LIST_DIR}}/{basename})
list(LENGTH tests_TESTS num_tests)
message(STATUS "NUM TESTS: ${{num_tests}}")
# '[' and ']' were escaped into ASCII 2 and 3 respectively, we have to
# unescape them back here. Note that this has to be done per-element,
# or CMake's list parsing breaks (which is why they were escaped).
string(ASCII 2 _LeftBracketListingEscape)
string(ASCII 3 _RightBracketListingEscape)
foreach(test IN LISTS tests_TESTS)
string(REPLACE "${{_LeftBracketListingEscape}}" "[" test "${{test}}")
string(REPLACE "${{_RightBracketListingEscape}}" "]" test "${{test}}")
string(REPLACE "\\" "\\\\" test "${{test}}")
string(REPLACE "\r" "\\r" test "${{test}}")
string(REPLACE "\n" "\\n" test "${{test}}")
message(STATUS "TEST_NAME: ${{test}}")
endforeach()
""")
return extractor_fname
def extract_tests_list_from_ctest_script(build_path: str) -> List[str]:
extractor = add_test_list_extractor(build_path)
cmd = ['cmake', '-P', extractor]
try:
result = subprocess.run(cmd,
capture_output = True,
check = True,
text = True)
except subprocess.CalledProcessError as err:
print('Error when calling CTest test extractor')
print(f'cmd: {err.cmd}')
print(f'stderr: {err.stderr}')
print(f'stdout: {err.stdout}')
exit(4)
lines = result.stdout.strip().split('\n')
test_num_line = lines[0]
test_lines = lines[1:]
test_num_prefix = '-- NUM TESTS: '
assert test_num_prefix in test_num_line, test_num_line
test_num_line = test_num_line[len(test_num_prefix):]
num_tests = int(test_num_line)
assert num_tests == len(test_lines), len(test_lines)
test_name_prefix = '-- TEST_NAME: '
assert all(test_name_prefix in x for x in test_lines)
test_names = [x[len(test_name_prefix):] for x in test_lines]
# Unescape the names, so that names with literal newlines have newlines in them again
test_names = [x.encode('utf-8').decode('unicode-escape') for x in test_names]
return test_names
def escape_catch2_test_names(infos: List[TestInfo]):
escaped = []
for info in infos:
@@ -141,6 +249,7 @@ def escape_catch2_test_names(infos: List[TestInfo]):
escaped.append(TestInfo(name, info.tags))
return escaped
if __name__ == '__main__':
if len(sys.argv) != 3:
print(f'Usage: {sys.argv[0]} path-to-catch2-cml output-path')
@@ -151,7 +260,8 @@ if __name__ == '__main__':
build_path = build_project(sources_dir, output_base_path, catch2_path)
catch_test_names = escape_catch2_test_names(get_test_names(build_path))
raw_catch_test_names = get_test_names(build_path)
catch_test_names = escape_catch2_test_names(raw_catch_test_names)
ctest_output = get_ctest_listing(build_path)
ctest_test_names = extract_tests_from_ctest(ctest_output)
@@ -168,7 +278,20 @@ if __name__ == '__main__':
if mismatched:
print(f"Found {mismatched} mismatched tests catch test names and ctest test commands!")
exit(1)
print(f"{len(catch_test_names)} tests matched")
print(f"{len(catch_test_names)} tests matched in CTest listing")
test_list_names = sorted(extract_tests_list_from_ctest_script(build_path))
expected_names = sorted(info.name for info in raw_catch_test_names)
if test_list_names != expected_names:
print("TEST_LIST variable (tests_TESTS) does not match Catch2 test listing!")
for name in test_list_names:
if name not in expected_names:
print(f" TEST_LIST name '{name}' not in Catch2 listing")
for name in expected_names:
if name not in test_list_names:
print(f" Catch2 name '{name}' not in TEST_LIST")
exit(1)
print(f"{len(test_list_names)} tests matched in TEST_LIST variable")
cmake_version = get_cmake_version()
if cmake_version >= (3, 27):
@@ -8,6 +8,24 @@
#include <catch2/catch_test_macros.hpp>
#include <cstdio>
#include <iostream>
namespace {
struct PrintsWhenConstructed {
PrintsWhenConstructed() {
std::cout << "Hello\n";
std::cerr << "Holla\n";
std::fprintf(stdout, "Hullo\n");
std::fprintf(stderr, "Hillo\n");
}
};
static PrintsWhenConstructed instance;
}
TEST_CASE("@Script[C:\\EPM1A]=x;\"SCALA_ZERO:\"", "[script regressions]"){}
TEST_CASE("Some test") {}
TEST_CASE( "Let's have a test case with a long name. Longer. No, even longer. "
@@ -21,3 +39,30 @@ public:
};
TEST_CASE_METHOD(TestCaseFixture, "A test case as method", "[tagstagstags]") {}
TEST_CASE("Unclosed right ) parenthesis") {}
TEST_CASE("Unclosed left ( parenthesis") {}
TEST_CASE( "Newlines\nAnd\rOther\n\tWhitespace", "[whitespace-going-wild]" ) {}
TEST_CASE( "Escaped \\n newline and \\r other whitespace", "[whitespace-going-wild]") {}
// Some JSON-like and JSON-adjacent characters and substrings in the test names/tags
// This serves to test that the parse-json-via-string-splitting hack in
// catch_discover_tests works properly.
TEST_CASE( "We split on variants of },{ in name" ) {}
TEST_CASE( "Then }\t, { we }\t,\t{ concatenate } , { them } ,{back},{" ) {}
TEST_CASE( "}},{{" ) {}
TEST_CASE( "Arrays [{},{}] wheee", "[also},{tags]" ) {}
TEST_CASE( "Let's add semicolon into the mix ;},{;},{};,{}" ) {}
TEST_CASE( "[", "[unmatched-square-bracket]" ) {}
TEST_CASE( "]", "[unmatched-square-bracket]" ) {}
// Some CMake-like special strings ($ as dereference) strings in test names.
// This serves to test that the names of test cases are not evaluated
// inside the catch_discover_tests.
TEST_CASE( "Plain ${NOT_A_VAR} variable" ) {}
TEST_CASE( "Env variable access $ENV{HOME}" ) {}
TEST_CASE( "Cache check $CACHE{FOO}" ) {}
TEST_CASE( "Also some generator exprs $<CONFIG> in $<1:yes> name" ) {}
TEST_CASE( "Mess of bare $ $$ $$$ and unterminated $} ${ exprs" ) {}
TEST_CASE( "$ENV{X};[weird]{},{ mix $<0:no> ${VAR} };,{ }},{{" ) {}