The old internals reached into the global hub to stash the allocation(s)
for enum value -> string value there, then kept around a potentially
invalid (in case the hub was cleaned up) reference into it, going through
whole bunch of virtual dispatch in the process.
The new internals just store the data in a static variable inside the
`StringMaker` specialization. This avoids potential lifetime issues,
avoids all virtual dispatch and (almost) reduces the include bloat in
the main header path.
The reason for (almost) there is that for full include correctness,
`EnumInfo` needs `<utility>` include for `std::pair`. However, this brings
in things like `std::relops`, because the std headers in C++ are dumb.
As this was not included before, and instead we relied on `std::pair`
existing in an internal stdlib header that we transitively included, the
full include size ends up bigger than before.
Previously the mapping was eager on construction and calls to
`next()`. This was fine when generators were always exhausted fully,
but with the new generator filtering, we might not want to call
the map function eagerly; rather we want to call it only once,
after the generator is moved to the target element.
Not being able to filter generators to specific element has been regularly
causing problems. It was possible to use a dynamic section to run tests
for specific element in a generator, at least if the element had a nice
string representation, but the test case would still run once per element
in the generator.
With this change, it is possible to have the generator return only one
specific element, and do so based on the index, rather than the string
representation of the element. This enables simple debugging of tests
that fail for specific generator element.
* Add validation for --benchmark-samples to prevent crash with zero samples
* Add automated test for benchmark samples validation
* Update baselines for benchmark-samples validation
* Add missing test for benchmark samples validation
---------
Co-authored-by: Chan Aung <chan@Thinkpad.localdomain>
Forbid deducing reference types for m_predicate in FilterGenerator to prevent dangling references.
This is needed for out-of-line predicates to work correctly instead of undefined behavior or crashes.
---------
Co-authored-by: Tek Mate <mate.tek@evosoft.com>
This stops tests failing falsely if the assertion passed, but the
stringification itself failed as the assertion was sent to the reporter.
I don't think that stringification should be fallible, but the
overhead in compilation ended up being small enough (<0.5% on `SelfTest`)
that it might be worth implementing, in case there is more users
with weird `StringMaker`s than just MongoDB.
Closes#2980
As it turns out, enums can be used to declare bitfields, and we
cannot form a reference to a bitfield in the cpature. Thus, we add
`std::is_enum` as a criteria to the default for `capture_by_value`,
so that enum-based bitfields are also captured by value and thus
decomposable.
Closes#3001
Bad handling of `TestFailureException` when translating unexpected
exceptions inside assertion macros led to the unexpected exceptions
handling erroring out through throwing the same exception again.
This was then backstopped by the machinery for handling uncaught
exceptions from assertions, which is normally used by the
`CATCH_CONFIG_FAST_COMPILE` machinery, where we assume that it can
only be invoked because the assertion macros are not configured to
catch assertions.
Closes#1292
This means that:
1) Scoped messages are always removed at the end of their scope,
even if the scope ended due to an exception.
2) Scoped messages outlive section end, if that section's scope is
enclosed in their own.
Previously neither of these were true, which has led to a number
of surprising behaviour, where e.g. this:
```cpp
TEST_CASE() {
try {
INFO( "some info" );
throw std::runtime_error( "ex" );
} catch (std::exception const&) {}
REQUIRE( false );
}
```
would print "some info" as the message for the assertion, while this:
```cpp
TEST_CASE() {
INFO("Hello");
SECTION("dummy") {}
REQUIRE(false);
}
```
would not print out "Hello" as the message for the assertion.
This had an underlying reason, in that it was trying to helpfully
keep the messages around in case of unexpected exceptions, so that
code like this:
```cpp
TEST_CASE() {
auto [input, expected] = GENERATE(...);
CAPTURE(input);
auto result = transform(input); // throws
REQUIRE(result == expected);
}
```
would report the value of `input` when `transform` throws. However,
it was surprising in practice and was causing various issues around
handling of messages in other cases.
Closes#1759Closes#2019Closes#2959
There were only two places where we used the full `AssertionInfo`
instance in `m_lastAssertionInfo`:
1) when reporting unexpected exception from running a test case
2) when reporting fatal error
because in those two places we do not have access to a real
instance of `AssertionInfo`, but we still need to send one to the
reporters. As a bonus, in both of these places we were already
constructing a fake-ish assertion info, by using the last encountered
source location, but dummying out the other information.
Instead, we only keep track of the last encountered source location,
and construct the dummy `AssertionInfo` on-demand.
This finishes the set of refactoring around `m_lastAssertionInfo`
in `RunContext` and improves the performance of running assertions
by ~5% in both Debug and Release mode.
--------------
Note that this change also causes small difference in output. It
could be avoided by having an invalidation flag and tracking where
the information would be invalidated before, but the difference
includes more precise line location for unexpected errors (both
exceptions and fatals), so I prefer the new output.
Right now `TEMPLATE_TEST_CASE_SIG` fails to compile when the signature contains only types:
```
TEMPLATE_TEST_CASE_SIG(
"TemplateTestSig: compiles with two type parameters",
"[template][onlytypes]",
((typename U, typename V), U, V), (int,int)) {}
```
The trick is to resolve the ambiguity between the two overloads of
`get_wrapper` (`TypeList` and `Nttp`) by making one match more strongly.
We also need to allow `reg_test` to register more than one type.
Add unit tests.
Fixes#2680
---------
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
This is done by no longer requiring all assertions to be seen
by the JUnit reporter. Since the JUnit reporter never outputs
all assertions, even with `-s`, the only difference from storing
passing assertions in the `CumulativeReporterBase` was some
bookkeeping in deciding which sections are relevant to the output.
Given `TEST_CASE` that just runs many passing assertions, e.g.
```
TEST_CASE( "PERFORMANCE_TEST_CASE", "[.]" ) {
for (size_t i = 0; i < 1000'000'000; ++i) {
REQUIRE( i == i );
}
}
```
the new JUnit reporter will finish in 5:47, using up ~7.7 MB of RAM.
The old JUnit reporter would finish in 0:30, due to bad_alloc
after using up 14.5 GB of RAM (the system has 16 GB total).
If the total number of assertions is lowered to 10M, the old
implementation uses up ~7.1 GB of RAM and finishes in 12 minutes.
The new implementation still needs only ~7.7 MB of RAM, and finishes
in 4 minutes.
There is a slight downside in that the output is slightly different;
the new implementation will include the `TEST_CASE` level section
in output, even if it does not have its own assertion. In other words,
given a `TEST_CASE` like this
```
TEST_CASE( "JsonWriter", "[JSON][JsonWriter]" ) {
std::stringstream stream;
SECTION( "Newly constructed JsonWriter does nothing" ) {
Catch::JsonValueWriter writer{ stream };
REQUIRE( stream.str() == "" );
}
SECTION( "Calling writeObject will create an empty pair of braces" ) {
{ auto writer = Catch::JsonValueWriter{ stream }.writeObject(); }
REQUIRE( stream.str() == "{\n}" );
}
}
```
the new implementation will output 3 `testcase` tags, 2 for the explicit
`SECTION`s with tests, and 1 for the top level section.
However, this can be worked-around if required, and the performance
improvement is such that it is worth changing the current output,
even if it needs to be fixed in the future.
Closes#2897
This PR introduces a new `TEST_CASE` macro called `TEST_CASE_PERSISTENT_FIXTURE`. `TEST_CASE_PERSISTENT_FIXTURE` offers the same functionality as `TEST_CASE_METHOD` except for one difference. The object on which the test method is invoked is only created once for all invocations of the test case. The object is created just after the `testCaseStarting` event is broadcast and the object is destroyed just before the `testCaseEnding` event is broadcast.
The main motivation for this new functionality is to allow `TEST_CASE`s to do expensive setup and teardown once per `TEST_CASE`, without having to resort to abusing event listeners or static function variables with manual initialization.
Implements #1602
---------
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
`max_digits10` is the number of decimal digits that the type _can_
represent, in other words, the number of decimal digits needed to
disambiguate any two floating point numbers when serialized to
a string.
The WithinRelMatcher does not listen to the precision specification; it just naively pipes to stringstream. If you specify the precision, that has no impact on the outputted values when the match fails.
This fix makes the WithinRelMatcher listen to the precision (https://github.com/catchorg/Catch2/blob/devel/docs/tostring.md#floating-point-precision) specified by: Catch::StringMaker<float>::precision
* AssertionEnd does not reset the assertion info yet. That is done after populateReaction. And reset assertion info would also reset the result disposition to normal, so that any uncaught exception would be reported as failure
* Approving test output changes due to added unit tests
* Unit tests to throw std::runtime_error instead of std::exception
* Add a unit test to test incomplete assertion handler
---------
Co-authored-by: Ross <ross.tang@gfo-x.com>
Martin Hořeňovský
qerased
Szapi
Clement Courbet
Chris Thrasher
Thomas Braun
Stefan Haller
Mark Jansen
Keith Stockdale
Ian Bell
Martin Jeřábek
Beartama
rosstang
Vertexwahn