Compare commits

...

1307 Commits

Author SHA1 Message Date
181265e9a6 Add missing cstdint includes to avoid compiler errors because of unknown symbols uint32_t and so on 2023-06-26 16:20:53 +02:00
97c48e0c34 v3.1.0 2022-07-17 20:18:44 +02:00
9c9f35068e Normalize C++ namespace in JUnit's reporter classname field
Closes #2468
2022-07-17 19:15:20 +02:00
1bd233866c Add AllTrue, AnyTrue, NoneTrue matchers 2022-07-16 16:16:05 +02:00
f993b702c6 extend gitignore 2022-07-16 16:16:05 +02:00
caf1264588 Fixes for matcher testing helpers when testing bools 2022-07-16 16:16:05 +02:00
a6d59b62b2 Remove obsolete link to benchmarking tests
The benchmarking feature is now documented, so there is no need
to direct our users towards our test file to understand how it
works.

Closes #2471
2022-07-15 19:49:24 +02:00
cc0e91472a Different way of checking for support of local warning suppression
Closes #2474
2022-07-15 19:34:35 +02:00
3bd0c58878 Fix Wunused-variable warning for MinGW
Closes #2132
2022-07-11 22:28:15 +02:00
a63ad74554 Added new DL_PATHS option to catch_discover_tests() (#2467)
This enables setting the required PATH/LD_LIBRARY_PATH environment variables both when retrieving the list of text cases and when executing the tests.

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-07-11 18:31:32 +02:00
5f9109a8dc Fix nvcc compile error (#2477)
Co-authored-by: Niels Kristian Kjærgård Madsen <nkm@kvantify.dk>
2022-07-07 00:03:02 +02:00
5a1ef7e4a6 Redo visibility fallback 2022-06-27 13:10:22 +02:00
bea58bf8bb Allow building Catch2 as dynamic library
Also have a check that warns users if they try to combined dynamic
library with hidden visibility, which we do not support.

Closes #2397
Closes #2398
2022-06-24 16:26:12 +02:00
34d9724058 Add experimental CMake script for sharding tests in binaries 2022-06-24 14:12:55 +02:00
5d269045b2 Add section on running tests in parallel to the FAQ 2022-06-24 11:20:31 +02:00
95a1206805 Add doc page with best practices and other usage tips 2022-06-24 11:20:22 +02:00
6f9f1465c3 Shorten lines in sharding docs 2022-06-22 00:12:16 +02:00
8730260457 Split apart combined TUs
The compile time improvements from using combined TUs mostly isn't
worth the annoyance they cause with various IDE shortcuts, like
when switching between header and its impl. file.

Splitting them apart also fixes the issue of empty subdirs being
installed due to `foo/internal` folders that only contained the
combined TUs and no headers.

Closes #2457
Closes #2463
2022-06-21 18:48:44 +02:00
bdfa920f93 Use binary path in testBazelReporter reporter output path 2022-06-21 14:21:12 +02:00
a369267874 test-fixtures.md: Line-wrap code examples (#2464)
* test-fixtures.md: Line-wrap code examples

* relinebreak

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-06-19 16:24:59 +02:00
1f381a1f62 Update commercial-users.md (#2465)
Using Catch2 and now Catch3 for our client's as well as our own projects and loving it. Especially the easy C++ BDD setup and implementation.
2022-06-18 23:47:49 +02:00
165647abbc Change provideBazelReporterOutput to local linkage 2022-06-17 18:23:06 +02:00
7e4ec432d0 Change Bazel XML support to depend upon BAZEL_TEST 2022-06-17 16:36:14 +02:00
078201fcf4 Remove HOMEPAGE_URL from project call
Closes #2428
2022-06-16 22:16:44 +02:00
8110ee9206 Add lib/cmake/Catch2 to conan builddirs in package recipe so extras/*.cmake files are packaged 2022-06-15 10:50:24 +02:00
fa9416426a Refactor shared args in approvalTests.py 2022-06-14 23:42:00 +02:00
338e4ec1f8 Force disable console colours for tests that regex check output
Fixes #2458
2022-06-14 23:24:06 +02:00
372b7575f6 Remove deprecation notice for console colour CLI
The required changes were one of the last commits before the full
v3 release.
2022-06-13 23:52:31 +02:00
d32fca4a49 Merge pull request #2452 from lizzyd710/devel
Added Cytopia to opensource-users.md
2022-06-08 15:59:57 +02:00
a0ece7b252 Compatility fixes for GCC5 (#2448)
* GCC5 compat: work around inherited constructor issues

Don't use inherited constructors, forward manually instead. This
basically reverts 61f803126d.

I believe that GCC5 does not implement P0136, a C++17 change that made
inherited constructors actually usable and was backported as a DR all
the way to C++11.

* GCC5 compat: bypass std::pair construction issue

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-06-07 18:46:46 +02:00
0a810c5e59 Added Cytopia to opensource-users.md 2022-06-07 09:06:10 -04:00
d0177ee686 Fix implicit long to double conversion
Raises a compiler warning when compiled with `-Wimplicit-int-float-conversion` using clang.
2022-06-07 00:27:37 +02:00
173539ab9e Update the 'Try online' badge link for 3.0.1 2022-06-06 00:29:05 +02:00
8822e28772 Update CE link in BDD documentation for 3.0.1
The old compiler was no longer built-for, so it couldn't link
against new versions, and also didn't properly provide the
user-config header.

Closes #2396
2022-06-06 00:29:02 +02:00
ff9506cedd Only test SEH handling with MSVC
MinGW doesn't support `__try` and friends at all, while Clang
only supports it partially, and the test would require some
changes to make it work there. Since this is only a test, we can
afford to keep it MSVC-only.

Closes #2447
2022-06-06 00:29:01 +02:00
0c13d021da Update documentation - add pkg-config examples.
Use -std=c++14 (instead of c++11). Pointers how to integrate with
pkg-config for non-CMake projects.
2022-06-05 15:58:41 +02:00
3644b4135d Doc: correct the address of the link to the command-line doc in tutorial.md 2022-06-04 10:20:01 +02:00
1c4f52b24a Suppress NVCC unused variable warnings (#2427)
Closes #2306
2022-06-03 16:17:14 +02:00
231c58a048 Add table with verbosity levels (missing --list-listeners) (#2443)
Co-authored-by: Sergio Losilla <sergio.losilla@nt-bnct.com>
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-06-02 21:50:51 +02:00
5efd327dd4 Fix crash when running with --list-listeners and no registered listeners (#2442)
Co-authored-by: Sergio Losilla <sergio.losilla@nt-bnct.com>
2022-06-02 16:11:12 +02:00
40dd9dd3f4 Fix copy paste error in the new random gen constraint 2022-06-02 15:51:21 +02:00
4142e699c2 Add full set of constraints to random integral generator
Related to #2433
2022-06-02 12:15:35 +02:00
9e445930cc Remove obsolete todo 2022-06-02 11:54:15 +02:00
2dc657cd1f Make default colour selection fall-through
Closes #2426
2022-06-02 09:16:10 +02:00
cca5923502 Leak full Wparentheses suppression for GCC 9
Reported as an issue on Discord. I thought that by GCC 9, the
C++ frontend was fixed enough to support `_Pragma`-based
suppression correctly, but apparently I was wrong.
2022-05-31 23:57:09 +02:00
8c952bd076 Point to 3.0.1 in FetchContent example
Closes #2326
Closes #2438
2022-05-31 11:49:24 +02:00
85c00eb946 Ensure that console_width.hpp includes user_config.hpp first
Otherwise we can get macro redefinition warnings when another
file includes console_width.hpp first, before user_config.hpp.

Closes #2431
2022-05-23 23:57:42 +02:00
3a18a688a0 Mention CMake's integration with CATCH_CONFIG options 2022-05-21 13:04:09 +02:00
605a34765a v3.0.1 2022-05-17 22:13:36 +02:00
abb669d4fd Extend project call for Catch2
This provides some extra metadata for CPack and related tools.
2022-05-17 22:12:44 +02:00
dcafc605f3 Only reseed the internal RNG when a test is first entered
This fixes multiple issues with random generators, with the most
important one being that multiple nested generators could return
values from the same sequence, due to internal implementation
details of `GENERATE`, and how they interact with test case
paths.

The cost of doing this is that given this simple `TEST_CASE`,
```cpp
TEST_CASE("foo") {
    auto i = GENERATE(take(10, random(0, 100));
    SECTION("A") {
        auto j = GENERATE(take(10, random(0, 100));
    }
    SECTION("B") {
        auto k = GENERATE(take(10, random(0, 100));
    }
}
```

`k` will have different values between running the test as
a whole, e.g. with `./tests "foo"`, and running only the "B"
section with `./tests "foo" -c "B"`.

I consider this an acceptable cost, because the only alternative
would be very messy to implement, and add a lot of brittle and
complex code for relatively little benefit.

If this calculation changes, we will need to instead walk
the current tracker tree whenever a random generator is being
constructed, check for random generators on the path to root,
and take a seed from them.
2022-05-17 21:49:26 +02:00
7a2a6c632f Make Random*Generators be seedable
This is a necessary step to fix issue with combining multiple
random generators in one test case, with different section nesting.
2022-05-17 17:57:21 +02:00
359cd6187d Rename global rng() to sharedRng() 2022-05-17 15:43:23 +02:00
9c72b303d9 Reseed global rng instance unconditionally 2022-05-17 12:35:34 +02:00
6044f021cf Don't reseed std::srand when resetting rng state
We no longer use `std::rand` anywhere, so there is no point,
and it might even interfere with our users use of `std::(s)rand`.
2022-05-17 11:54:19 +02:00
5d7883b551 Remove leftover comment in tests 2022-05-16 21:50:52 +02:00
04a54b0e87 Fix release automation script 2022-05-16 18:39:01 +02:00
48f3226974 Add support for retrieving generator's element as string 2022-05-16 16:34:06 +02:00
af8b54ecd5 Generators now keep track of the index of current element
This might become potentially useful in the future, when we want
to provide the ability to forward jump generators, to be able to
simply reproduce specific input to a test.

I am not yet sure how it would work, but the basic idea is that
it could be similar to the `-c` switch for selecting specific
`SECTION` in a test case.
2022-05-16 11:56:21 +02:00
07bec74096 Use better instruction for CATCH_TRAP on ARM Macs
The old instruction would cause the debugger to be stuck at the
triggering source line forever, while the new one should have the
expected semantics, where the debugger can then single-step,
continue. or generally do things, afterwards.

Closes #2422
2022-05-15 23:22:00 +02:00
316025a0d8 Properly handle startup reporter registration errors 2022-05-15 10:35:12 +02:00
33aeb603fe Add check for registering multiple reporters under same name 2022-05-15 09:54:27 +02:00
fc3d11b1d1 Add Wfloat-equal to default-enabled warnings
This is kinda messy, because there is no good way to signal to
the compiler that some code uses direct comparison of floating
point numbers intentionally, so instead we have to use diagnostic
pragmas.

We also have to over-suppress the test files, because Clang (and
possibly GCC) still issue warnings from template instantiation
even if the instantion site is under warning suppression, so the
template definition has to be under warning suppression as well.

Closes #2406
2022-05-14 15:47:20 +02:00
1ef65d60f1 Tiny fixup in Misc.tests.cpp 2022-05-11 22:56:30 +02:00
ed6b38b096 Specialize XML reporter's listener listing 2022-05-10 20:58:02 +02:00
5a49285e9c Add --list-listeners option 2022-05-10 20:57:59 +02:00
ae475a3c19 Optional support for listener descriptions 2022-05-10 20:57:53 +02:00
d60fbe49be Slight improvement for reporter test 2022-05-10 20:57:50 +02:00
a733b58cd2 Replace C++23-deprecated aligned_storage_t 2022-05-10 00:07:22 +02:00
d9b0a38f81 Report Catch2's version in the xml reporter output 2022-05-03 19:46:35 +02:00
40c8909a49 Rename useColourOnPlatform => useImplementationForStream 2022-05-03 19:04:43 +02:00
91ea25e51a Fix use of u8-literal that would break in C++20 2022-05-03 13:54:07 +02:00
e2d07d35f4 Add -Wmissing-prototypes to common warnings 2022-05-03 13:54:05 +02:00
d2cb934d28 Add -Wmissing-variable-declarations to common warnings 2022-05-03 13:54:03 +02:00
7752229105 Add -Wsuggest-destructor-override to enabled warnings 2022-05-03 13:54:02 +02:00
722c197855 Enable a bunch of extra warnings 2022-05-03 13:54:00 +02:00
198808a24e Misc cleanups from exploring Clang warnings 2022-05-03 13:53:55 +02:00
198713e5dc Cleanup of unused functions
The cleanup also found out that custom translation for std-derived
exceptions test wasn't running properly, and fixed that.
We cannot enable the warning globally, because the tests contain
some functions that are unused by design -- e.g. when checking
stringification priority of StringMaker vs range fallback and so
on.
2022-05-02 21:32:24 +02:00
b84067ea6f Clean up .gitignore 2022-05-01 23:10:37 +02:00
332de39cd4 Remove some useless casts 2022-04-28 14:56:33 +02:00
c410e2596c Add some extra warnings 2022-04-28 14:56:31 +02:00
4c1cf4aa67 Sorted and removed duplicates from the compiler warning list
The duplicates were '-Wreturn-std-move' and '-Wunreachable-code'.
2022-04-28 11:06:53 +02:00
745cc82cd3 Remove some old commented out CMake code 2022-04-28 10:53:03 +02:00
07dfb4b070 Update PARENT_SCOPE cmake module path with path to extras/
This allows projects including us directly to do e.g.
`include(Catch)` and get access to the test autoregistration
CMake scripts. Note that this inclusion can be done directly
through `add_subdirectory`, or indirectly via `FetchContent`.

Closes #1805
Closes #2026
Closes #2130
2022-04-28 10:52:16 +02:00
5e86ead366 Exclude doxygen docs from documentation install step
We might want to keep them in at some point, but currently the
doxygen docs are very incomplete and unfinished.
2022-04-27 14:57:59 +02:00
9dc229693d Document the stability guarantees of '--order rand' option 2022-04-25 21:45:50 +02:00
db57a4956f Fixup ToC in command-line.md 2022-04-25 21:39:48 +02:00
48177831ee Add version placeholder to the example doc in contributing.md 2022-04-25 21:36:03 +02:00
ee3bbecf51 Improve include guard docs in contributing.md 2022-04-25 21:33:05 +02:00
431dcf36ea Document Catch2's API/ABI stability policies 2022-04-25 18:58:50 +02:00
e882cb8eb1 Document the new --skip-benchmarks flag 2022-04-25 18:36:40 +02:00
c2bc321607 Add pipeline running tests under Valgrind to GitHub Action 2022-04-24 13:34:50 +02:00
ea9029c478 Fix pkg list name in CMake config workflow 2022-04-24 13:34:36 +02:00
c65e5b6514 Add 'uses-python' label to tests using Python 2022-04-24 13:31:28 +02:00
880285b433 Tiny cleanup in catch_clara.cpp 2022-04-24 13:31:25 +02:00
07cdef2096 Fix uninit memory in Clara tests 2022-04-24 13:31:19 +02:00
5baa29b6b9 Add test for %debug stream name 2022-04-23 23:34:27 +02:00
291b35b389 Added --skip-benchmarks command-line option. (#2408)
Closes #2392

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-04-23 23:14:49 +02:00
f526ff0fc3 Bump version to v3.0.0 preview 5 2022-04-20 23:46:07 +02:00
17a04f88d9 Always report rng seed from builtin reporters
Not all reporters use a format that supports this, so TeamCity
and Automake reporters still do not report it. The console
reporter now reports it even on successful runs, where before
it only reported the rng seed in the header, which was showed
either for failed run, or for run with `-s`.

CLoses #2065
2022-04-19 10:32:13 +02:00
90e6905050 Remove no longer used CATCH_USE_VALGRIND cmake option 2022-04-18 12:14:21 +02:00
6bdc7e1a65 Remove unused function in Clara 2022-04-17 23:30:40 +02:00
7b93a2014c Rename catch_stream -> catch_reusable_string_stream
After everything else was split out, this name much reflects the
actual contents of the file(s).
2022-04-16 16:34:08 +02:00
98bb638fb2 Split out IStream out of catch_stream.hpp 2022-04-16 16:33:57 +02:00
05e85c5652 Split out Catch::cout/cerr/clog into their own file 2022-04-16 16:33:50 +02:00
b520257676 Split REMOVE_PARENS from catch_preprocessor.hpp
This way, some headers can include just the new header, which
transitively decreases the total include size of test_macros.hpp.
2022-04-15 17:14:28 +02:00
574d042821 Push catch_preprocessor include down from test_macros header 2022-04-15 16:40:06 +02:00
c742ea9ad9 Some IWYU include cleanups 2022-04-14 17:16:01 +02:00
a243cbae52 Introduce type to handle reporter specs with defaults filled-in 2022-04-13 15:02:25 +02:00
79d1e82381 Store IStream instance owning ptrs in reporter instances
This ended up being a surprisingly large refactoring, motivated
by removing a `const_cast` from `Config`'s handling of reporter
streams, forced by previous commit.
2022-04-13 15:00:50 +02:00
4f09f1120b Make IStream::stream non-const
This way it makes much more sense from logically-const point
of view, and also means that concrete implementations don't
have to always have a `mutable` keyword on the stream member.
2022-04-13 14:59:39 +02:00
9934b7de13 Remove pointless mutable from ScopedElement's XmlWriter ptr 2022-04-12 22:31:51 +02:00
7a89916198 Remove superfluous default argument from XMLWriter's constructor 2022-04-12 22:28:40 +02:00
61f803126d Use parent constructor where applicable in reporters 2022-04-12 22:27:12 +02:00
d698776ec5 Remove unused RDBufStream type 2022-04-12 22:22:04 +02:00
f25236ff43 Small cleanup for pluralise 2022-04-11 20:37:54 +02:00
8cdaebe964 struct -> class normalization for various interface types 2022-04-11 18:32:35 +02:00
1a56ba851b struct -> class normalization for matchers 2022-04-11 13:06:54 +02:00
9abe49ec53 struct -> class normalization for reporters 2022-04-11 12:58:58 +02:00
195a6ac941 Use StringRefs for operation name in StringMatcherBase 2022-04-11 10:58:45 +02:00
be948f1fd0 Add COST property to the CMakeConfig tests 2022-04-10 21:55:52 +02:00
4e006a93ff Rename IStreamingReporterPtr => IEventListenerPtr
This reflects the renaming of the reporter/listener base class.
2022-04-10 21:50:51 +02:00
73d8fb5bca Update release-notes.md 2022-04-10 21:43:28 +02:00
0a33405983 Mention Bazel support in release notes 2022-04-09 20:55:13 +02:00
cb551b4f6d Add default reporter for Bazel integration (#2399)
When the added Bazel configuration flag is enabled,
a default JUnit reporter will be added if the XML
envrioment variable is defined.
Fix include paths for generated config header.
Enable Bazel config by default when building with
Bazel.


Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-04-08 18:01:59 +02:00
4b78157981 Remove Config::openStream function
It has become completely vestigial, as it only ever passed-through
the argument down to a different function, and being private
member function, it didn't even introduce a useful compilation
firewall.
2022-04-07 12:50:20 +02:00
46b3f7ee5f Document magic %-prefixed streams 2022-04-07 12:50:18 +02:00
f9f4e58dfb Add missing version placeholder to --colour-mode cli option docs 2022-04-07 12:50:14 +02:00
d5bfce4d84 Rename config::colourMode to config::defaultConfigMode
This describes the actual intention much better.
2022-04-07 12:50:13 +02:00
c43947eb47 Fix CLI hints for colour and reporter options 2022-04-07 12:50:10 +02:00
423e1d2ebb Integrate the new reporter specs into Catch2
This means that the CLI interface now uses the new key-value oriented
reporter spec, the common reporter base creates the colour implementation
based on the reporter-specific configuration, and it also stores the
custom configuration options for each reporter instance.

Closes #339 as it allows per-reporter forcing of ansi colour codes.
2022-04-07 12:50:08 +02:00
3c06bcb374 Fix reporter parser spec to use the right key for colour-mode 2022-04-06 21:41:45 +02:00
a51fd07bd0 Add helper for parsing the new reporter specs
The new reporter spec generalizes key-value options that can be
passed to the reporter, looking like this
`reporterName[::key=value]*`. A key can be either Catch2-recognized,
which currently means either `out` or `colour`, or reporter-specific
which is anything prefixed with `X`, e.g. `Xfoo`.
2022-04-05 15:45:54 +02:00
8ac86495de Cleanup ColourMode parsing in CLI 2022-04-05 13:42:21 +02:00
d750da13a8 Split out colour mode string parsing into its own function 2022-04-05 13:42:06 +02:00
c045733d05 Add splitReporterSpec helper 2022-04-05 12:57:33 +02:00
9fea3d253f Add missing include to catch_optional.hpp 2022-04-05 12:55:30 +02:00
797c3e7318 Refactor tests 2022-03-31 22:30:38 +02:00
6206db5a73 Fix missing licence headers 2022-03-31 20:04:08 +02:00
78e33ce51f Added TestCaseInfoHasher and tests. (#2394)
Test case hashing includes tags and class name

As the hasher involves more code now, it was split out into its own file
and it got its own set of tests.

Closes #2304
2022-03-31 16:46:41 +02:00
1a8a793178 Refactor colour handling to prepare for per-reporter colour modes
This includes always compiling the ANSI and None colour
implementations, as they don't need to touch any platform
specific APIs, and removing their respective compile-time
configuration options.

Because the Win32 colour implementation requires Win32-specific
APIs, it is still hidden behind a compile-time toggle,
`CATCH_CONFIG_COLOUR_WIN32` (renamed from `..._COLOUR_WINDOWS`).

The commandline options for colours were also changed. The
option now uses different name, and allows to select between
different implementations, rather than changing whether
the compiled-in colour implementation is used through
"yes/no/default" options.
2022-03-28 13:10:13 +02:00
a4e4e82474 Always indent test names at 2 spaces with default listing
This makes the output slightly more parsable in cases where
using the xml output from xml reporter is impractical, e.g. from
CMake scripts.
2022-03-24 16:03:46 +01:00
6227ca317e Add tests for ColourGuard 2022-03-24 16:02:56 +01:00
081a1e9aba ColourGuard is no longer constructed engaged
Forcing it to be engaged explicitly, either via `op<<`, or by
`ColourGuard::engage`, fixes an issue with multiple `ColourGuard`s
being constructed in a single expression. Because the construction
of the `ColourGuard` instances can happen in arbitrary order,
colours would be applied in arbitrary order too. However, a chain
of `op<<`s has strict call orders, fixing this issue.
2022-03-24 16:02:54 +01:00
4d8acafecb Rename isStdout => isConsole and add doccomment 2022-03-24 16:02:51 +01:00
cf6dd937ab Add test for FatalConditionHandler and stack unwinding
Signed-off-by: Kochetkov, Yuriy <yuriyx.kochetkov@intel.com>
2022-03-24 15:49:02 +01:00
2ce64d1d8f Fix disengage failure logs for FatalConditionHandlerGuard
FatalConditionHandlerGuard is used within RunContext::invokeActiveTestCase().
The intent of this guard is to avoid binary crash without failed test being
reported.
Still in case FatalConditionHandlerGuard destructor being called during stack
unwinding AND finds unexpected top-level filter for SEH unhandled exception,
the binary may still crash. As result of such crash the original exception
details are being hidden.

As the Catch2 provides only `CATCH_CATCH_ANON` macro, with no access to
exception details by design, looks like the best way to handle issue is to:
 - state requirements explicitly by `noexcept` specifier
 - use `Catch::cerr()` to print out possible issue notification

Signed-off-by: Kochetkov, Yuriy <yuriyx.kochetkov@intel.com>
2022-03-24 15:49:02 +01:00
7882f7359e Remove unnecessary casts
May break the logic in case of WinAPI changes with no warnings

Signed-off-by: Kochetkov, Yuriy <yuriyx.kochetkov@intel.com>
2022-03-24 15:49:02 +01:00
0e176c318b Refactored colour handling
* POSIX colour impl is now compiled for all platforms.
* Deciding whether a colour impl should be picked is now stream
  dependent, and thus incompatible implementations can be removed
  immediately, rather than checking when the colour is being used.
2022-03-18 14:04:07 +01:00
c1c72c7e05 Better doccomment for IEventListener 2022-03-18 10:30:48 +01:00
06092f727d Remove special handling of default stream for reporters
This fixes an issue where reporter with default-output to stdout
would think that it was given a stream _not_ backed by console,
thus not using colour.
2022-03-18 00:41:47 +01:00
4acc520f76 Event listeners no longer take reporter config in constructor
This also required splitting out Listener factory from
the reporter factory hierarchy. In return, the listener
factories only need to take in `IConfig const*`, which
opens up further refactorings down the road in the colour
selection and implementation.
2022-03-18 00:36:18 +01:00
18c58667d7 Generate Visual Studio project files with constant project guids
By default, CMake derives a Visual Studio project GUID from the
file path but the GUID can be overridden via a property
(see https://gitlab.kitware.com/cmake/cmake/-/commit/c85367f4).

Using a non-constant GUID can cause problems if other projects/repos
want to reference the catch2 vcxproj files, so we force a constant GUID here.

Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Resolves: #2388
2022-03-18 00:21:12 +01:00
634cdb4efe Fix borken Bazel build 2022-03-16 14:01:18 +01:00
38879296a7 Add eventpp and accessorpp to Catch2 users 2022-03-13 16:56:07 +01:00
81f612c96c Refactor MatchersRanges test utils (#2386)
- do not hardcode content of containers
- prefix members with m_
- add const and non-const iterators to `with_mocked_iterator_access`
- remove `m_touched` as it wasn't filled properly and isn't used anyway
2022-03-09 23:13:37 +01:00
913f79a661 Each reporter keeps its own colour implementation
This opens path to per-reporter colour output customization,
and fixes multiple issues with the old colour implementation.

Under the old implementation, using Win32-backed colouring
would always change the colour used by the console, even if the
actual output was written elsewhere, such as a file passed by
the `--out` flag. This will no longer happen, as the reporter's
colour impl will check that the reporter's stream is pointed
to console before trying to change the colours.

POSIX/ANSI colour implementation suffered a similar-ish issue,
in that it only wrote the colour escape codes into the default
output stream, even if the reporter asking for colouring was
actually writing to a completely different output stream.
2022-03-08 12:51:13 +01:00
06f74a0f8e Add %stderr and %stdout as recognized magic stream names 2022-03-07 10:59:27 +01:00
61d0f7a9af Pass the whole IStream wrapper into reporter
This will become useful when reworking colour support, because
Win32 colour support requires checking whether the output is
stdout, which is done through the `IStream` wrapper.
2022-03-06 16:34:13 +01:00
9b01c404f5 Fix cli help tooltip for --use-colour 2022-03-06 16:28:30 +01:00
f206162b2d Fix some messy formatting in catch_stats 2022-03-06 16:28:28 +01:00
05d4ec62c8 Add common ReporterBase as parent of the helper bases
This is useful as a centralized place for handling common reporter
problems like handling output streams, and soon also colour impl
handling.
2022-03-06 16:28:22 +01:00
4dd5e2eece Rename base reporter iface IStreamingReporter -> IEventListener 2022-03-06 16:28:21 +01:00
f9facc1881 Add variadic CAPTURE tests
This makes sure that CAPTURE works when called with variadic arguments,
and also works that way when disabled.

The underlying fix to #2316 is not applicable (CAPTURE is already
variadic when disabled).

This is a port of 5e94498ed0 to the
devel branch.
2022-03-05 16:06:54 +01:00
2ebc041903 Fix name of with-main library. 2022-03-04 00:40:55 +01:00
529eec97bb Mention CMake configure options in release notes 2022-02-23 14:16:36 +01:00
ff5b311898 Remove superfluous warning suppression in tests 2022-02-22 20:13:17 +01:00
4a2eb90302 Add basic CMake presets 2022-02-22 15:47:11 +01:00
715cd25081 Add IStream::isStdOut 2022-02-22 15:45:51 +01:00
1d4b42ad7b Minor whitespace fix 2022-02-21 20:04:43 +01:00
72f0372664 Add tests for CMake configure toggles passing to Catch2 build 2022-02-20 19:32:30 +01:00
4396a9119f Add some missing configuration includes 2022-02-20 19:32:25 +01:00
bda4b7df84 Fix fallback stringifier config 2022-02-20 19:32:22 +01:00
0c722564c3 Examples: remove references to catch_default_main
catch_default_main.hpp was removed in db32550898

The example 000-CatchMain.cpp is no longer compiled, but is still
present in the examples and is still references by other example
files. Remove the file and references to it, as they are confusing.
2022-02-20 13:10:42 +01:00
33ffc3b6fc Allow passing compile-time configuration options through CMake 2022-02-18 11:15:08 +01:00
fc5552d27b Push down handling of default reporter to Config's constructor
This simplifies the handling of default reporter in console
parsing, at the cost of making `Config`'s constructor responsible
for more things.
2022-02-17 23:02:30 +01:00
7cf2f88e50 Add small missing things to release notes 2022-02-14 18:22:11 +01:00
a9ed2c235d Rename MiscFunctions.cmake -> CatchMiscFunctions.cmake 2022-02-07 00:05:11 +01:00
a1e5934aa9 Add licence header to MiscFunctions.cmake 2022-02-06 23:59:15 +01:00
190f71792a proper support for lib suffix in conan 2022-02-05 20:53:43 +01:00
c912f62fc4 Remove Windows.h related defines 2022-02-05 16:21:06 +01:00
aa3c7be434 Fix ToC in CLI documentation 2022-02-05 16:14:47 +01:00
b0279e0c14 Remove mention of CATCH_CONFIG_MAIN from docs/configuration.md 2022-02-05 14:55:01 +01:00
9afb6ce138 feature: double -> const auto when static_cast in right
</subject>

Branch: devel

<type>:
- [ ] Bug fix
- [ ] Bug fix (Test)
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] This change requires a documentation update

<body>

<footer>

Signed-off-by: Certseeds <51754303+Certseeds@users.noreply.github.com>
2022-02-04 00:09:01 +01:00
efb54926ee Don't cache stdout handle in Windows console colour impl
The cached handle would become invalid if some other code, say
a user-provided test code, redirects stdout through `freopen`
or `_dup2`, which would then cause AppVerifier to complain.

Fixes #2345
2022-01-29 23:12:55 +01:00
7a2f9f4633 Extend checkLicense.py to also check tests 2022-01-29 13:10:54 +01:00
79e4cd1af4 Cleanup summary comments in ExtraTests 2022-01-29 00:40:03 +01:00
635db2785f Use the new licence header in ExtraTests's cpp files 2022-01-29 00:07:44 +01:00
51888d360a Use the new licence header in TestScripts 2022-01-29 00:04:49 +01:00
f83332d89b Use the new licence header in SelfTest's cpp files 2022-01-29 00:03:43 +01:00
b5dbdc858d Link to the v2-v3 migration docs from release notes 2022-01-28 23:55:27 +01:00
e53a75b425 Add integration test for listener-reporter event ordering 2022-01-28 00:00:52 +01:00
4ff8b27bb6 Cleanups output normalization in ApprovalTests
Removed:
 * NaN normalization
 * INFINITY normalization
 * errno normalization
 * Completely unused duration regex

Tests using these macros should be tagged `[approvals]`
so they are not run as part of approval tests.

Also simplified regex for the test's executable filename,
and hidden some tests relying on nullptr normalization.
2022-01-27 21:11:51 +01:00
d861e73f86 Handle full unique test identifiers in testRandomOrder.py 2022-01-27 21:11:37 +01:00
dc86d51af2 Update version string to mention Catch2 2022-01-27 20:15:05 +01:00
5121660e7f Add OpenALpp as an open source library that uses Catch2 2022-01-22 21:33:19 +01:00
ce556fd646 Ignoring VSCode settings dir 2022-01-15 15:45:56 +01:00
b6ff2c3dda Error out reporter registration for reporter names with '::' 2022-01-09 19:01:14 +01:00
875299cff0 Remove unused member from Totals
At one point it was inserted there as the simplest way to smuggle
around an extra return value for specific errors in executing
tests. Since then, the error has been changed to be handled
differently, and the member became unused.
2022-01-07 10:34:25 +01:00
39d3de17f3 Fix mentions of CATCH_INSTALL_HELPERS
Fixes #2347
2022-01-06 23:37:44 +01:00
fff494c10a Tear out trivial_abi on unique_ptr
Turns out people are bad at not combining code compiled with GCC
and Clang, and the improvement from `trivial_abi` on `unique_ptr`
is not worth the maintenance cost of having this be an opt-in
change.

Closes #2344
2022-01-05 00:04:25 +01:00
103cb16696 Rename listening reporter to multi reporter 2022-01-05 00:00:28 +01:00
244680d512 Update release-process.md for v3 2022-01-03 23:30:39 +01:00
f4af9f6926 Bump version to v3.0.0 preview 4 2022-01-03 23:16:39 +01:00
57c9c935ee Update release notes 2022-01-03 23:11:55 +01:00
98a6c69e1e Switch from AddVectoredExceptionHandler to SetUnhandledExceptionFilter
This avoids issues with Catch2's handler firing too early, on
structured exceptions that would be handled later. This issue
meant that the old attempts at structured exception handling
were incompatible with Windows's ASan, because it throws
continuable `C0000005` exception, which it then handles.

With the new handling, Catch2 is only notified if nothing else,
including the debugger, has handled the exception.

Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Closes #2332
Closes #2286
Closes #898
2022-01-03 22:50:32 +01:00
d3199c42c2 Redo reporter CLI and reporter-specific docs
Closes #2259
2022-01-03 20:38:13 +01:00
eeee4a49af Remove short flags from --list-tests and --list-tags 2022-01-02 21:23:29 +01:00
0d1bdea69f Remove all mentions of the removed --list-test-names-only flag 2022-01-02 21:23:26 +01:00
3ab981fa21 Add documentation for the --verbosity option 2022-01-02 21:02:43 +01:00
54e89e8364 Update release notes to account for recent changes 2022-01-01 23:50:12 +01:00
88b28ab592 Extra compilation checks for compiling without exceptions 2022-01-01 23:21:37 +01:00
ef3374ed81 Add more macros to the DisabledMacros test 2022-01-01 16:45:28 +01:00
f2f585b478 Merge pull request #2183 from mjerabek/dev-multiple-reporters
Add support for multiple reporters
2022-01-01 16:19:31 +01:00
b5547f2ef6 Passthrough stdout/err in multireporter's partial test end
This should provide the same overall stdout/err, but the new
output should feel "faster" for test cases that are entered
and exited multiple times (e.g. due to generators).
2022-01-01 14:20:23 +01:00
93882f7fab Support '-' as output path for stdout 2022-01-01 14:16:12 +01:00
4752545a69 Add version placeholders to the reporter docs 2022-01-01 14:16:12 +01:00
fae0be25b3 Improve error handling in parsing reporter specs 2022-01-01 14:16:11 +01:00
899554bff2 Add test for handling of reporter's assertion preferences 2022-01-01 14:15:44 +01:00
b4efa4751a Add test for handling of Listeners requesting stdout capture 2022-01-01 14:13:48 +01:00
22547a3c5f Add test for stdout pass-through behaviour in MultiReporter 2022-01-01 14:11:49 +01:00
8baf9c05a3 Add more unit tests
* Call order of listeners/reporters in multireporter
* Adding listeners/reporters properly updates reporter preferences
2022-01-01 14:10:52 +01:00
ccd67b293d Add support for multiple parallel reporters
This requires a bunch of different changes across the reporter
subsystem.

* We need to handle multiple reporters and their differing
  preferences in `ListeningReporter`, e.g. what to do when
  we mix reporters that capture and don't capture stdout.
* We need to change how the reporter is given output and
  how we parse reporter's output destination from CLI.
* Approval tests need to handle multireporter option
2022-01-01 14:02:23 +01:00
6b55f5d780 Don't send full stdout so far to testCasePartialEnded event 2022-01-01 12:21:41 +01:00
c9c3b74805 Remove some obsolete tests 2021-12-31 15:12:53 +01:00
8711b63a0a Add tests for StringRef's copy operations being shallow 2021-12-29 20:20:29 +01:00
72a09de236 Add tests for Optional's op== and != 2021-12-29 16:58:22 +01:00
f0a89b7345 Add ops == and != for Optional 2021-12-29 16:58:16 +01:00
f00b6e2019 Add more tests for various less common tag-related cases 2021-12-28 00:22:53 +01:00
45577a1f4c Refactor implementation of case-insensitivity in tags
By not materializing the lower cased tags ahead of time, we
save allocations at the cost of worsened performance when comparing
two tags.

Since there are rarely many tags, and commonly they are not
compared even if present, this is almost always a win. The new
implementation also improves the robustness of the code
responsible for handling tags in a case-insensitive manner.
2021-12-28 00:20:55 +01:00
cbb6764fb1 Add CaseInsensitiveEqualTo comparison type 2021-12-26 18:54:47 +01:00
156e6fdfa9 CaseInsensitiveLess takes args as StringRefs 2021-12-26 00:52:52 +01:00
187bf6db2b Split out CaseInsensitiveCmp into its own file 2021-12-26 00:35:46 +01:00
cde26de803 Add buildAndTest.cmd for Windows users (#2338)
* Add buildAndTest.cmd for Windows users

Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-12-18 23:52:35 +01:00
3cc0c033e4 Allow enabling multiple warnings in one invocation 2021-12-18 20:50:05 +01:00
840acedf62 Implement warning for unmatched test specs 2021-12-18 20:50:03 +01:00
9f2dca5384 Exit with non-0 return code if no tests were run
A new flag, `--allow-running-no-tests` was added to override this
behaviour if exit code of 0 was desired.

This change also made `-w NoTests` obsolete, and so it has been
removed.
2021-12-18 20:50:00 +01:00
602e484f02 Rename reportInvalidArguments -> reportInvalidTestSpec
This is has been what it actually does for a long time, but it
has not been renamed earlier due to API compatibility.
2021-12-18 20:49:50 +01:00
08939cc8bb Error out early if invalid test specs are provided 2021-12-18 20:49:47 +01:00
3bfe900bbc Move shard logic check after help flag check 2021-12-18 20:49:44 +01:00
d30d0c01a7 Disable JIT debug for ctest.ext
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
2021-12-16 12:17:52 +01:00
dcf9479c85 Counts internally use uint64_t instead of size_t
This ensures that even for 32 bit platforms, the assertion count
should not plausibly overflow.
2021-12-06 20:42:51 +01:00
c49faa62dd Let users of amalgamated distribution override the default main
Closes #2324
2021-12-02 13:49:16 +01:00
c097609115 Cleanup release scripts a bit 2021-12-02 11:48:50 +01:00
9d6fffb922 Remove no longer used Python scripts 2021-12-02 11:48:28 +01:00
153965a655 Delete useless MatcherMethod base class
All of its functionality can be moved into the `MatcherBase` class,
simplifying the code a bit and removing a warning about class with
virtual member functions but no virtual destructor.

Closes #2182 as it is no longer relevant.
2021-11-26 00:38:13 +01:00
0ac9f44985 Add SECURITY.md 2021-11-26 00:10:01 +01:00
b9baae6d93 Cleanup StringRef impl a bit 2021-11-23 23:16:46 +01:00
c95072408f Add catch_sharding.hpp to CML 2021-11-23 23:16:17 +01:00
8cb8f0b08b Sweep out some Wsign-conversion warnings 2021-11-23 23:16:08 +01:00
9952f29f01 Fix link in event-listeners.md 2021-11-22 11:44:36 +01:00
2db1cf3404 Fix template args to avoid conversion warnings
Fix avoids a warning about sign conversion when included from a file compiled with -Werror=sign-conversion.
2021-11-20 21:38:01 +01:00
fabe614ba8 Rewrite own-main.md for v3 2021-11-19 15:52:32 +01:00
acdb85c398 Add FAQ page to the documentation 2021-11-19 15:23:50 +01:00
726fdd7f8e Update known-limitations.md
Removed 3rd party bugs for no-longer supported compiler versions,
reworded section on parallel test execution to take into account
sharding.
2021-11-19 00:32:59 +01:00
0ccb1c30c6 Rewrite event listener docs and add event documentation 2021-11-18 23:05:45 +01:00
dd12ce8141 Rewrite tutorial to shorten BDD section and mention generators 2021-11-16 23:46:23 +01:00
d32e89eb84 Reorganize the main readme 2021-11-16 23:46:22 +01:00
ce6aca81ad Remove Google Groups link from README.md 2021-11-16 23:46:21 +01:00
61489e863e Fixup ToC link to 3.0.1 release notes 2021-11-16 23:46:19 +01:00
2287d225e5 Mention that RangeGenerator works on half open ranges
Closes #2145
2021-11-15 23:52:26 +01:00
4eb00afe69 Add allowing tests with same name to release notes 2021-11-15 23:18:55 +01:00
e86f84b8ef Mention that run == iteration in benchmarking docs 2021-11-15 23:09:18 +01:00
d012735c6e Add test for runtime behaviour of STATIC_CHECK 2021-11-15 20:06:29 +01:00
67caef6f45 Mention STATIC_CHECK in release notes 2021-11-15 13:04:08 +01:00
f41d761674 Add STATIC_CHECK and STATIC_CHECK_FALSE (#2318) 2021-11-15 00:28:27 +01:00
edc2f6e8a3 Further refactoring of StreamingReporterBase 2021-11-14 11:41:29 +01:00
b2ac27423a Cleanup visibility in StreamingReporterBase 2021-11-14 11:41:27 +01:00
a754cb9062 Make TestRunInfo constexpr 2021-11-14 11:41:26 +01:00
5f38cc39fa JUnit/SonarQube reporters use base's stream member 2021-11-14 11:41:25 +01:00
b892ab133c Cleanup visibility in CumulativeReporterBase 2021-11-14 11:41:24 +01:00
0c9fe16537 Opt out JUnit/SonarQube reporter from expanding passing assertions
Closes #1966
2021-11-14 11:27:45 +01:00
d02ea5adee Cumulative reporter base can be customized to not expand assertions 2021-11-14 11:27:44 +01:00
9b4e69333f Small cleanup in tests 2021-11-14 11:27:43 +01:00
4d9bfb2951 Avoid static analysis warning in Approx 2021-11-14 11:27:42 +01:00
c4df47c246 Approvals normalize paths with both backward and forward slash 2021-11-13 22:10:25 +01:00
9200b4078b Move reporter_registrars.hpp to reporters/ 2021-11-10 23:52:59 +01:00
6603f1d972 Use case in names of default reporters 2021-11-10 23:32:01 +01:00
62d8913d67 Cumulative reporter base records benchmark results 2021-11-09 11:52:50 +01:00
8780425385 Make reporter lookup case insensitive, registration case preserving
Previously registration was case preserving, but lookup used
lowercased reporter name, so a reporter whose name contained
upper case character could not be requested by the user.
2021-11-09 11:50:03 +01:00
7800fe9708 Lift toLower(char) to header 2021-11-09 11:44:54 +01:00
141e384c60 Fix missing include in reporter_registrars.hpp 2021-11-08 11:32:27 +01:00
f1239b2045 Add doxygen doccomments to IStreamingReporter 2021-11-05 14:24:22 +01:00
912df7df35 Fix quadratic runtime when linebreaking strings without newlines
The problem was that every line would iterate from current line
start position to the end of the string, looking for a newline
to break on, leading to accidentally quadratic runtime. With this
change, the code only ever searches up to the current line's
length and not more.

Credit to @jorgenpt for the fix suggestion.

Closes #2315
2021-11-04 00:23:56 +01:00
931f41b4d6 Add some tests for TextFlow::Column 2021-11-01 22:51:17 +01:00
70c4ec78fb Improve comments and names in TextFlow::Column 2021-11-01 19:14:37 +01:00
455ae0c561 Typedef Column::iterator as Column::const_iterator not vice versa 2021-10-31 13:01:41 +01:00
2520ad4b6e Return const_iterator from Column::begin/end const
This is what should normally happen, even if it does not change
anything given that `Column::const_iterator` is currently a typedef
for `Column::iterator`.
2021-10-31 12:59:00 +01:00
e539e1cb52 Move strings in Clara's result type 2021-10-29 23:04:24 +02:00
3c5c86a4e4 Add test for filtering out multiple initial values in filter gen 2021-10-28 11:26:53 +02:00
514206df36 Add accept-many Clara lambdas to release notes 2021-10-27 20:54:49 +02:00
becab0cf74 Add test sharding to the release notes 2021-10-27 20:49:29 +02:00
12d14a3c63 Add support for multiply calling lambda parsers in Clara
Previously a lambda parser in Clara could only be invoked once,
even if it internally was ok with being invoked multiple times.

With this change, a lambda parser can mark itself as `accept_many`,
in which case it will be invoked multiple times if the appropriate
flag was supplied multiple times by the user.
2021-10-27 20:15:28 +02:00
f17725a186 Split void_type into its own header and rename it to void_t 2021-10-27 20:01:13 +02:00
ec2d5013fb Make testSharding.py test script executable 2021-10-27 17:24:30 +02:00
342ef5ca7e Cleanup the shard integration test script 2021-10-27 17:24:30 +02:00
5ac1ffe9ee Improve shardIndex/Count cli argument parsing 2021-10-27 17:24:30 +02:00
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
6456ee8b01 Return Clara parsing error message by const-ref 2021-10-27 15:10:52 +02:00
905bf438ae Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
0fdee1c273 Stop declaring compiled Catch2 artifacts as arch independent 2021-10-25 14:46:54 +02:00
22750cde0e Disable false positive from clang-tidy
Clang-tidy is smart enough to understand that the conditional is never
updated in the loop body. It will let you get away with it if it can
prove that the conditional is always false, but that is not always
possible.

Here is an example where it's not able to prove it, and thus gives a
false positive. This is a minimal reproduction of an actual case I hit
in production, where `function` is picking the function based on some
`constexpr` logic related to which type argument is currently being
tested.

```
int f();

TEMPLATE_TEST_CASE("reproduction", "", int) {
    const auto function = []() {
        return f;
    }();
    const int error = function();
    REQUIRE(error == 0); // clang-tidy complains: bugprone-infinite-loop
}
```

I did not choose to add this test to the test suite, since we're not
running `clang-tidy` in CI afaik. To run it manually, simply add the
snippet above somewhere and run clang-tidy with
`--checks=bugprone-infinite-loop`. Or see an example at
https://godbolt.org/z/4v8b8WexP.

The reason we get the infinite loop warning in the first place is the
conditional at the end of this `do`-loop. Ideally, this conditional
would just be `while(false)`, but the actual content of the
`REQUIRE`-statement has been included here too in order to not loose
warnings from signed/unsigned comparisons. In short, if you do
`REQUIRE(i < j)`, where `i` is a negative signed integer and `j` is an
unsigned integer, you're supposed to get a warning from
`-Wsign-compare`. Due to the decomposition in Catch2, you lose this
warning, which is why the content of the `REQUIRE` statement has been
added to the conditional to force the compiler to evaluate the actual
comparison as well.

This was discussed on Discord today, and an alternative approach (which
I don't have time to implement) would be to in the decomposition replace
the comparison operators with `cmp_less` and friends. These are C++20
though, and would have to be implemented manually. I am also not sure
it's a good idea to "magically" change the semantics of `<` when it's
used inside a `REQUIRE` macro.

Another alternative approach would be to trigger this warning in a
different way, by including the content of the `REQUIRE` macro in a
different way which doesn't affect the for loop. But I don't have enough
of an overview here to know where would be a good place and how to test
that I didn't break anything.
2021-10-21 22:45:19 +02:00
bf5c58adf6 The limit on TEMPLATE* test cases is actually reachable 2021-10-21 15:52:07 +02:00
06cf2a4724 Apply PR #2297 to devel branch (#2300)
* Apply PR #2297 to devel branch

It turns out that Issue #2272 partially affected the devel branch. When
building tests with C++20, the compiler emits a warning that top-level
comma expressions in array subscripts are depricated. Warnings are
treated as errors, so this caused the build to fail.

This commit adds localized warning suppression
in accordance with this recommendation here:
https://github.com/catchorg/Catch2/pull/2297#discussion_r720848392

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>

* Fixed unknown pragma warning on old versions of gcc & clang

This commit fixes an unkwown pragma warning on older versions of GCC
and Clang. These older versions don't have a warning for depricated use
of the comma subscript. Because warning suppression is localized, and
only refers to the comma subscript warning, it doesn't affect compiler
warnings in other parts of the code.

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>

* More #warning backwards compatibility fixes

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>
2021-10-21 15:47:21 +02:00
4436a60456 Add myself to FUNDING 2021-10-11 10:44:29 +02:00
36b4a71ff0 Pick release notes for 2.13.7 2021-10-10 22:23:14 +02:00
b406ad52a7 Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-10-10 22:21:39 +02:00
de67278e14 JUnit reporter uses only 3 decimal places when reporting durations
We used to use whatever precision we ended up having from C++'s
stdlib. However, some relatively popular tools, like Jenkins,
use Maven SureFire XML schema to validate JUnit test reports, and
Maven SureFire schema requires the duration to have at most 3
decimal places.

For compatibility, the JUnit reporter will now respect this
limitation.

Closes #2221
2021-10-10 22:10:48 +02:00
1d9696d22d Cleanup reporting of rng seed in existing reporters 2021-10-09 00:02:30 +02:00
ed1f343a41 Provide random-device option for --rng-seed and make it default 2021-10-08 21:35:45 +02:00
200a487cf2 Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
fce42b62ad Generate random rng seed if user did not specify one
Closes #2161
2021-10-08 21:35:29 +02:00
4e6d306742 Rename Catch::Option to Optional 2021-10-05 20:39:28 +02:00
c6c46a168f Add non-empty assertion to Option's deref op 2021-10-05 19:22:22 +02:00
48a889859b Simplify how ratio symbol is returned when stringifying ratios 2021-10-03 20:23:10 +02:00
d65ee04b74 Remove double serialization when stringifying std::optional 2021-10-03 20:19:36 +02:00
928e198ef2 Demote some single character strings into plain chars 2021-10-03 10:37:30 +02:00
3e9c6fec22 Remove XmlWriter::writeBlankLine
There is no good reason to provide a "add empty line" primitive
for writing XML documents, and the fact that it remains unused
after all the time it was provided only confirms this further.
2021-10-02 14:39:54 +02:00
13670f535f Add more tests for XmlWrite::write* members 2021-10-02 14:39:52 +02:00
c6640e4f47 Improve documentation of CATCH_MOVE and CATCH_FORWARD 2021-09-30 20:40:24 +02:00
23f0d94b4f Use CATCH_FORWARD in Detail::make_unique 2021-09-30 20:27:38 +02:00
77c7e9803e Improve documentation on Detail::unique_ptr 2021-09-30 20:17:59 +02:00
1d79683ea8 Add [[trivial_abi]] to Detail::unique_ptr when compiled with Clang
This decreases code size and improves performance of passing around
`unique_ptr` instances by value somewhat. It virtually guarantees
problems when combining code compiled with Clang and GCC, but that
was never supported anyway.
2021-09-30 20:11:22 +02:00
eb452e9b35 Bring back useful comment
In b7b346c3e5 this conditional was simplified to just
`while( false)` rather than the current one. Then in 3a33315ff8, that
change was reverted. I found it hard to understand this
complicated conditional, but after some digging in history I found this
comment which used to be here. It was removed in b7b346c3e5, but not
restored together with the revert in 3a33315ff8. Let's revive it.
2021-09-29 18:19:31 +02:00
2deafc33e9 Demote getCurrentNanosecondsSinceEpoch to internal linkage 2021-09-29 11:26:38 +02:00
5250cf6d58 Remove getEstimatedClockResolution from Timer
It used to be part of the experimental benchmarking support, but
since that was replaced with proper benchmarking support with its
own timer facilities, it is now a dead code and useless.
2021-09-29 11:24:18 +02:00
426954032f Rename Contains string matcher builder to ContainsSubstring
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.

Closes #2131
2021-09-27 21:28:33 +02:00
f02c2678a1 Use StringRef for className in TestCaseInfo 2021-09-27 19:07:45 +02:00
21b99d6f58 Add StringRef::compare for three way comparison 2021-09-27 18:41:17 +02:00
d42e7a23a0 Change startsWith(char) to take StringRef as argument 2021-09-27 14:55:28 +02:00
7bb00a42be Remove stub of obsolete test 2021-09-27 14:45:39 +02:00
fb4153e05e Multiple tests can have same name as long as their tags differ
This change also changes it so that test case macros using a
class name can have same name **and** tags as long as the
used class name differs.

Closes #1915
Closes #1999
2021-09-25 21:37:03 +02:00
e8e28ba401 Remove obsolete documentation page 2021-09-23 23:42:16 +02:00
ee1435793e add alpaka and LLAMA to Catch2 users 2021-09-23 12:39:56 +02:00
3f8cae8025 Add consistency-checking event listener to SelfTest
This event listener performs basic consistency checks (akin to
matching braces) on events that are passed to the listeners
when the `SelfTest` test binary is run.

The current checks are about nesting events (e.g. `testCaseStarting`
cannot be received before `testRunStarting`, `sectionStarting`
can only be received when a test case is active, etc), and matching
up counts of starting/ended events.

The simplicity means that it could be confused by starting/ended
events matching up but being out of order, e.g.
```
* test case A starting
* test case B ended
* test case B starting
* test case A ended
```
would be accepted, even though it is wrong. However, doing full
order checking would be much more implementation work, for relatively
little benefit, so it is left out for now.
2021-09-15 23:38:43 +02:00
2c82f82ee2 Mark first party reporters final
They are not intended to be derived from, so forbidding it
explicitly improves our ability to refactor them later.
2021-09-15 21:29:18 +02:00
785436cd74 Make benchmark* and fatalError reporter events pure virtual 2021-09-15 21:17:39 +02:00
f314fa1f8c assertionEnded event in reporters no longer returns bool 2021-09-15 18:57:56 +02:00
013edc208b Refactor noMatchingTestCases and skipTests reporter methods
They now take `StringRef` as the argument, and are virtual only
in the basic interface.

Also cleaned out the various reporters and their overrides
of these members which were often empty or delegating up.
2021-09-15 18:08:31 +02:00
10fb93cce8 Add new reporter event called for each test case enter/exit
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.

Closes #2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
2021-09-14 13:19:45 +02:00
4dcf8382c7 Default-implement assertion* events in StreamingReporterBase
This means that a reporter deriving from `StreamingReporterBase`
does not have to override them if it isn't interested in the individual
assertions.
2021-09-14 12:41:24 +02:00
efd8cc8777 pkgconfig: Add missing entries
This adds prefix and exec_prefix fields
2021-09-11 19:02:21 +02:00
12bca890b7 Use StringRefs in SectionTracker's filter bookkeeping 2021-09-09 11:04:34 +02:00
317db82396 Use StringRef in more places in reporters 2021-09-09 10:41:09 +02:00
cf5ccaa9df AssertionResult::getMessage returns StringRef to internal string 2021-09-08 10:08:07 +02:00
b3a84c7983 Use StringRef in pluralise implementation
This means that it can no longer be safely made ahead of time,
but nothing in our existing code used it like that. Normally it
is constructed and used in the same expression, which is now
more efficient.
2021-09-08 00:24:40 +02:00
c0f866c7cf Make CumulativeReporterBase's members protected 2021-09-08 00:23:37 +02:00
29caae5ce5 Pass around the test suite name by StringRef 2021-09-08 00:01:31 +02:00
ea49210eae Remove Config::getProcessName because it was unused 2021-09-08 00:00:16 +02:00
e4719fb51c Update release notes with new fixes, improvements and breaking changes 2021-09-07 22:00:57 +02:00
290c1b60e6 Store singular TestRunNode in CumulativeReporterBase
There can never be more than one, so there is no reason to provide
genericity over multiple nodes.
2021-09-07 21:18:48 +02:00
e5938007f7 Completely remove the testGroup events
This was done because they were 1:1 mapping to testRun events, and
I could not think of a reasonable way to make them their own thing.
2021-09-07 20:00:22 +02:00
ab3fe0053d Remove superfluous warning suppression from console reporter 2021-09-07 19:57:55 +02:00
432d03d1aa Use -Wextra-semi instead of -Wextra-semi-stmt for stricter warnings 2021-09-02 11:04:55 +02:00
Roc
9ac9fb164e Suffix the library name with 'd' for debug builds 2021-08-28 00:47:54 +02:00
07018e2fba Force __cdecl calling convention for wmain
Fixes #2219
2021-08-26 23:13:27 +02:00
ff0a5227ca Make Approx::operator() const
Closes #2273
2021-08-26 21:27:43 +02:00
54edab53bf Add missing include to benchmark/catch_execution_plan.hpp
Fixes #2275
2021-08-26 20:19:25 +02:00
0a8516aeea Use make_unique when registering an exception translator 2021-08-21 21:32:15 +02:00
928ecbaccf Change callable/model::clone to return a unique_ptr 2021-08-21 21:00:00 +02:00
1cbbc5d2cb Add test for !shouldfail and failed benchmarks 2021-08-21 00:06:31 +02:00
7f3297f7e8 Sweep out unused <utility> includes 2021-08-19 21:44:17 +02:00
7ff54ebc06 Add _t suffix typedef for internal traits 2021-08-19 21:29:41 +02:00
ca8546efc6 Mention C++14 baseline in contributing documentation 2021-08-19 19:16:12 +02:00
4113a12c69 Fix Wreserved-identifier for UDLs in Catch2
See #578
2021-08-19 19:15:46 +02:00
edad4d0af7 Prefer _t over ::type form for std type traits 2021-08-19 00:21:13 +02:00
88c27ffaf2 Mention CATCH_MOVE and CATCH_FORWARD in contributing docs 2021-08-16 16:22:21 +02:00
d2ee7100d2 Replace uses of std::move and std::forward with macros
This improves the SelfTest build times by about 3% (measured
with Clang 10 on random Linux box I had lying around).
2021-08-16 16:22:19 +02:00
03ce304102 Add std::move/std::forward replacement macros
Using the `CATCH_MOVE` and `CATCH_FORWARD` macros instead of the
`std::move` and `std::forward<T>` utility functions can improve
compilation times and debug build's performance, and thus will
be preferred going forward.
2021-08-16 16:22:16 +02:00
7040f03b54 Fix typo in docs/matchers.md
There were a few instances in matchers.md where header files erroneously
had a .h file extension instead of .hpp
2021-08-16 15:19:49 +02:00
1554251f97 Fix typo in docs/cmake-integration.md 2021-08-11 22:57:36 +02:00
2b54f1e7a6 Remove superfluous catch_enforce include from complete_invoke
Also added missing transitive include to `catch_benchmark.hpp`.
2021-08-11 22:52:04 +02:00
2c84854b90 Use unique_ptr to hold Environment measurements in benchmarking 2021-08-11 22:52:01 +02:00
3579c055c8 Use Detail::make_unique<T> in more places 2021-08-11 22:51:59 +02:00
3ec63324a8 Replace owning naked IStream pointers with unique_ptrs 2021-08-11 22:51:57 +02:00
7d0770adf2 Merge pull request #2269 from numaru/feature/rename-contrib-in-docs
Rename contrib/ to extras/ in the docs
2021-08-11 00:26:38 +02:00
74db06199b Rename contrib/ to extras/ in the docs
The directory has been renamed in 918aa32 but the doc was not updated
yet.
2021-08-10 10:41:16 +02:00
52a3144145 Update users docs 2021-08-08 23:42:24 +02:00
a62974eb6a Update why-catch.md 2021-08-08 23:42:17 +02:00
a0d84654dd Add GitHub Actions badges to README 2021-08-07 21:18:00 +02:00
557e5118f1 Remove TravisCI badge 2021-08-07 16:25:22 +02:00
0a3f511cfe Fix inconsistencies in reporting benchmarking failures
With these changes, all these benchmarks
```cpp
BENCHMARK("Empty benchmark") {};
BENCHMARK("Throwing benchmark") {
    throw "just a plain literal, bleh";
};
BENCHMARK("Asserting benchmark") {
    REQUIRE(1 == 2);
};
BENCHMARK("FAIL'd benchmark") {
    FAIL("This benchmark only fails, nothing else");
};
```

report the respective failure and mark the outer `TEST_CASE` as
failed. Previously, the first two would not fail the `TEST_CASE`,
and the latter two would break xml reporter's formatting, because
`benchmarkFailed`, `benchmarkEnded` etc would not be be called
properly in failure cases.
2021-08-07 15:16:11 +02:00
9ef510b769 Remove unused travis yaml 2021-08-07 15:09:50 +02:00
1b1f3a88bc Don't debug log empty strings 2021-08-06 11:38:54 +02:00
02ab64da2e Cleanup ExceptionTranslatorRegistry::translateActiveException 2021-08-06 00:56:09 +02:00
77df08b44d Better arg name for run_for_at_least 2021-08-06 00:55:31 +02:00
79c2daa4a0 Update tutorial docs 2021-08-03 13:53:44 +02:00
1e0dc61d16 Update CMake integration documentation 2021-07-31 21:46:11 +02:00
02e5951f11 Add major changes and their motivation to v2 -> v3 migration docs
Closes #2135
2021-07-29 23:19:48 +02:00
1ecc79bb56 Fix superfluous uppercase C in matcher docs 2021-07-28 23:38:27 +02:00
73cae40a90 Fix WithinULP matcher factory name in docs 2021-07-28 23:37:23 +02:00
6c4c961207 Update migrate-v2-to-v3.md
#2135 Adding "Changes overview" to "Migrating from v2 to v3" doc
2021-07-28 23:18:50 +02:00
340a61af50 Always use 'Detail' for the detail namespace 2021-07-27 21:45:27 +02:00
3d1cf95b32 Fix ulp distance calculation for numbers with different signs
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.

Closes #2152
2021-07-27 21:04:41 +02:00
6f21a3609c Update CMake doc for automatic test registration
`FetchContent` doesn't include `contrib` directory as part of `CMAKE_MODULE_PATH`. This results into `include(Catch)` to fail. This patch just updates the documentation describing how to do include the path, so the new users don't have to figure this out themselves.

Source: https://github.com/catchorg/Catch2/issues/2103#issuecomment-730626324
2021-06-30 00:12:36 +02:00
bf61a418cb Remove the ill-conceived compilation perf tests using real tests 2021-06-20 19:15:02 +02:00
849002aec0 Move specialization for global op<< to the proper place
In v2 it was placed in a very central header due to the way it was
stitched together. Now that we don't do that, we can move it to the
proper place, removing the potential for confusion given that the
original header was split apart and renamed.
2021-06-20 19:14:56 +02:00
4eb9af51af Use non-reserved identifiers for test autoregistration objects 2021-06-19 23:15:11 +02:00
78e4fbdb12 Use non-reserved identifiers for BENCHMARK macro implementation 2021-06-19 23:14:20 +02:00
a7533707ff Use unique_ptr for passing around exception translators 2021-06-19 20:57:06 +02:00
28a33497be Move extractClassName to the single calling TU and make it static 2021-06-18 23:39:07 +02:00
70f5392210 Devirtualize more things in trackers 2021-06-18 10:35:02 +02:00
61461dfd1d Move default impls of isFooTracker to the interface 2021-06-18 10:35:01 +02:00
c064322a9d Devirtualize ITracker::parent 2021-06-18 10:34:58 +02:00
a14d67cace ITracker::parent returns pointer not reference
We were already using it mostly to get pointers rather than
references, so this makes it make more sense.
2021-06-18 10:34:55 +02:00
4ce8a23edd Tiny cleanups in tracking 2021-06-18 10:34:43 +02:00
c1b59b7071 Mark JUnit reporter's internal members as private 2021-06-18 00:00:24 +02:00
c77ba5314a Fix decomposing in presence of universal ADL-found operators
Closes #2121
2021-06-08 23:36:06 +02:00
65c9a1d31a Add test for comparing immovable types 2021-06-07 20:05:03 +02:00
fa31d58934 Don't use removed-in-C++20 allocator parts in CustomAllocator tests 2021-06-07 17:44:39 +02:00
9ac8cad2d1 Merge pull request #2244 from jbadwaik/devel
Disable CATCH_INTERNAL_IGNORE_BUT_WARN for NVHPC Compilers
2021-06-07 15:22:53 +02:00
c71f42cc29 Disable CATCH_INTERNAL_IGNORE_BUT_WARN for NVHPC Compilers
- NVHPC's implementation of `__builtin_constant_p` has a bug which
    results in calls to the immediately evaluated lambda expressions to be
    reported as unevaluated lambdas.

    https://developer.nvidia.com/nvidia_bug/3321845.

  - Hence, we disable CATCH_INTERNAL_IGNORE_BUT_WARN for NVHPC Compilers
2021-06-07 12:28:16 +02:00
e6da4e10ae Use make_unique instead of unique_ptr(new T) 2021-06-01 00:17:22 +02:00
816f69416b Use TestCaseInfo in tests directly 2021-06-01 00:17:19 +02:00
aee31d0620 Disallow empty tags in test case specification 2021-06-01 00:17:13 +02:00
c9371865d4 Use StringRef as the argument to benchmark{Failed,Preparing} 2021-05-31 08:42:03 +02:00
5741de9ccd Add explicit test for serialization of boolean attributes in XML 2021-05-31 08:41:28 +02:00
0e2895934c Use precomputed string sizes when constructing std::strings 2021-05-31 08:41:26 +02:00
a01073d871 Write single characters directly instead of as C-strings in Sonarqube 2021-05-31 08:41:20 +02:00
02839ba934 Remove superfluous usings in the Contains matcher 2021-05-31 08:40:09 +02:00
8d6a1c27ef Use StringRef literals where possible in XML using reporters
This let's us avoid running `strlen` at runtime to convert the
plain string literals to `StringRef`s, by guaranteeing that we
instead have the size available after compilation.

In optimized builds the performance improvement should be even
greater, as the `StringRef` UDL and the related constructor
are both `constexpr`, and thus can be baked completely during
compilation.
2021-05-30 13:59:47 +02:00
41ad0fda11 Improve XmlWriter::writeAttribute overload set for string-like types
Previously, string literals and `std::string`s would match the
template variant, which would serialize them into a stream and then
call the `StringRef` overload for resulting string. This caused
bunch of codebloat and unnecessary pessimization for common usage.
2021-05-30 13:06:03 +02:00
d9f72868b2 Use StringRef instead of std::string in XmlWriter::ScopedElement 2021-05-29 21:47:12 +02:00
c7241bb76e Use StringRef in XmLWriter when the text is not stored 2021-05-29 21:23:24 +02:00
1d04427fcd Use StringRef through XmlEncode
This introduces a potential lifetime risk when using the API, but
the intended way to use the `XmlEncode` class is to use it directly,
e.g. `out << XmlEncode(some-text-argument)`, not to store it around.

The benefit is that we avoid allocations for strings that do not fit
into SSO for given platform.
2021-05-29 16:41:57 +02:00
aba114d6fe Don't include CTest in non-development builds.
When Catch2 is used as a CMake subproject (via add_subdirectory) and is not built in development mode (CATCH_DEVELOPMENT_BUILD) CTest is not required for the build, as testing is off. When CTest is included it creates various targets (around 20) which are shown in IDEs such as Clion and pollute the list of configurations.

This patch conditionally includes CTest only if Catch2 is built in development mode.
2021-05-29 16:24:44 +02:00
0221148ac3 Cleanup string escaping in XmlWriter with raw string literals 2021-05-29 14:11:50 +02:00
9f091cbe9d Remove some code duplication from default test listing impl 2021-05-29 14:06:40 +02:00
4c1e896d47 Flush less in default test listing impl 2021-05-29 13:52:23 +02:00
2c04850f88 Add some constexpr in compact reporter impl 2021-05-29 13:49:33 +02:00
c9027375a3 Replace std::endl with \n and flush
In some places the `std::flush` was not added, as it was sufficiently
obvious that the flush semantics are not intended. There are likely
other places where the flush semantics aren't intended, but that
is a cleanup for later.
2021-05-29 13:15:46 +02:00
2ae28fc852 Use make_unique instead of unique_ptr(new T) in more places 2021-05-28 23:43:15 +02:00
f9ec34ce01 Make operator BenchmarkStats<Duration2> explicit 2021-05-28 23:19:07 +02:00
96790b1d23 Always install the used compiler in Github Actions
This avoids the CI breaking again when the base images remove
an older compiler again.
2021-05-28 11:30:06 +02:00
86f86c4c23 Fix compiling with C++17 + Clang 5 + libstdc++ v5
This basically tests the combination where the compiler supports most
of C++17 but the library does not.
2021-05-26 00:34:23 +02:00
023b5306b4 Add deep const to unique_ptr::operator-> 2021-05-19 11:00:53 +02:00
9137e591fa Cleanup and optimize stringifying of string-like types
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.

This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.

The actual string stringification routine is now also better about
reserving sufficient space.
2021-05-18 23:15:22 +02:00
f50a06affa Small optimization for stringifying chrono::time_point 2021-05-18 00:17:34 +02:00
4cc247cc70 Use constexpr StringRef instance for unprintable strings 2021-05-18 00:17:32 +02:00
8ee422d6bf Remove pointless branch from stringifying matcher assertions 2021-05-15 11:47:02 +02:00
0c0f73a48d Constexprify NameAndTags constructor 2021-05-15 00:02:09 +02:00
61e16416a9 Pass other StringRef arguments by value instead of by-ref
Apart from being clearer, it also improves the overall codesize
of the implementation library, and should improve the performance
as well, by removing one level of indirection.
2021-05-14 23:45:59 +02:00
074017f5ad Inline MessageBuilder's constructor 2021-05-14 16:00:15 +02:00
0a89e7f0c4 Pass StringRef by value instead of by const-ref 2021-05-14 15:59:42 +02:00
28f6698ec8 Remove unneeded interface IRunner 2021-05-13 09:14:26 +02:00
b36f8daaad Clarify bunch of default, out of line destructors 2021-05-12 23:55:46 +02:00
d86cb5f95d Add override to virtual destructor of derived class 2021-05-12 23:23:02 +02:00
5eb7aa4f90 Add test for dots in tag names 2021-05-12 21:43:14 +02:00
35cba5486d Remove mention of ANON_TEST_CASE from documentation
It was removed from v3, because the standard `TEST_CASE` can now be
anonymous as well.
2021-05-10 21:49:45 +02:00
eb911aa995 Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187)
Resolves #1390

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
313071e8fe Pick release notes for v2.13.6 2021-05-09 23:45:00 +02:00
f9bb2668e4 Pick release notes for v2.13.5 2021-05-09 23:43:05 +02:00
baf0cd0be4 Fixed [dis]engage_platform declarations mismatch 2021-05-09 23:34:17 +02:00
c0d0a50bdb Significantly refactor fatal error handling
Because new glibc has changed `MINSIGSTKSZ` to be a syscall instead
of being constant, the signal posix handling needed changes, as it
used the value in constexpr context, for deciding size of an array.
It would be simple to fix it by having the handler determine the
signal handling stack size and allocate the memory every time the
handler is being installed, but that would add another allocation
and a syscall every time a test case is entered.

Instead, I split apart the idea of preparing fatal error handlers,
and engaging them, so that the memory can be allocated only once
and still be guarded by RAII.

Also turns out that Catch2's use of `MINSIGSTKSZ` was wrong, and
we should've been using `SIGSTKSZ` the whole time, which we use now.

Closes #2178
2021-05-09 23:34:15 +02:00
cbcab2dbcd Add lcc to the list of unwanted compilers that mimic gcc 2021-05-09 18:12:58 +02:00
ea44e73961 Wrap all std::min and std::max calls in parentheses 2021-05-09 18:10:50 +02:00
d61fe3ecc3 [Issue 2154] Correct error when building with IBM's latest XLC (#2155)
* [Issue 2154] Correct error when building with IBM's latest XLC compiler with xlclang++ front-end.

On AIX, the XLC 16.1.0.1 compiler considers the call to `std::abs` ambigious, so it needs help with a static_cast to the type of the template argument.

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-09 18:05:39 +02:00
b325c6d81e Making target detection on Mac more robust 2021-05-09 18:05:37 +02:00
d4a3cd9992 Make the static library build reproducible with supported compilers
This is based on bed285af07 from
`v2.x` branch, but done properly for the build in v3 branch,
effectively porting #2141 to devel.
2021-05-09 18:05:34 +02:00
342dd3445c Pick documentation changes for 2.13.4 2021-05-09 18:05:31 +02:00
23760327ae Fix updateVersionPlaceholder when the placeholder starts the line 2021-05-09 18:05:27 +02:00
48f220b68a Add deprecation warning in ParseAndCatchTests
Parsing C++ with regex in CMake is error prone and regularly leads to silently
dropped (not run) test cases.

Going forward the function `catch_discover_tests` from `contrib/CMake.cmake`
should be used.

For more information see https://github.com/catchorg/Catch2/issues/2092#issuecomment-747342765
2021-05-09 18:05:25 +02:00
031a57e7b7 Fix Catch.cmake helper by setting variable globally
Set `_CATCH_DISCOVER_TESTS_SCRIPT` helper variable globally. Otherwise in a
scoped call (like `add_subdirectory()`) the variable gets lost. This lost
variable results in a post build error with not much information to lead to the
root of the problem.

This enables the usage of the helper script with the following example structure

- CMakeLists.txt (project root with `add_subdirectory(external/catch2)`
- external/catch2
  - CMakeLists.txt (contents listed below)
  - contrib/Catch.cmake
  - contrib/CatchAddTests.cmake
  - catch2/catch.hpp
- tests
  - CMakeLists.txt (add tests with `catch_discover_tests(${PROJECT_NAME})`)

contents of project specific helper `external/catch2/CMakeLists.txt`
```cmake
cmake_minimum_required (VERSION 3.1...${CMAKE_VERSION})
project(Catch2 LANGUAGES CXX VERSION 2.13.3)
add_library(Catch2 INTERFACE)
target_include_directories(Catch2
  INTERFACE
    $<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}>
)
 # provide a namespaced alias for clients to 'link' against if catch is included as a sub-project
add_library(Catch2::Catch2 ALIAS Catch2)
include(contrib/Catch.cmake)
```
2021-05-09 18:05:18 +02:00
cdf4748d1c Modified hash to make it more independent of the choice of test names. 2021-05-09 18:05:14 +02:00
520b6dace9 Remove conan to bintray packaging task from travis
As bintray is dead, it is no longer relevant and in fact, it will
fail during build.
2021-05-09 17:45:29 +02:00
2cb5210caf Fix and improve GitHub Actions CI
* Fixed problem with older compilers by explicitly installing the
  right compiler version.
* Split apart simple builds that can be well described by a build
  matrix, and builds that need special CMake defines or similar
  special snowflake handling.
* Ported some extras + examples builds from TravisCI to GitHub
  Actions.
2021-04-30 23:26:28 +02:00
2dc5a5f402 Prepare to prevent Catch2 v3's tests from being run, and CTest targets from being added, if Catch2 is configured as a sub-project. (#2205)
This commit fixes issue that happens if the project above us uses the same variable name, thus confusing our script which see the variable scoped from the project including Catch2, rather than ours

See #2202
2021-04-16 21:19:58 +02:00
04166514fe fixed inconsistent semicolon expansion in catch_discover_tests (Bug #2214) 2021-04-16 17:22:14 +02:00
e8cdfdca87 Fix typos in the code base (#2206)
Note that only documentation and comments are impacted by this change.
2021-04-07 20:43:50 +02:00
a5abec9cb5 Remove superfluous virtual from ExceptionTranslatorRegistry 2021-03-26 21:11:49 +01:00
f1d7a10e06 Remove StringRef::isNullTerminated and StringRef::c_str
Both of them were fundamentally unsafe to use and shouldn't be used
at all.
2021-03-26 21:11:44 +01:00
e50e10ef8f Use std::string instead of StringRef as argument to the makeStream
This is done because `makeStream` was the only place using
`StringRef::c_str()`, which is an error-prone and unsafe API that
I want to remove.
2021-03-26 21:11:42 +01:00
2c269eb633 Fixed compilation under MinGW - use gmtime_s() instead of gmtime_r(). 2021-03-23 23:46:04 +01:00
4b5812e932 Remove trailing whitespace from test-cases-and-sections.md 2021-03-23 13:02:24 +01:00
9f44bd57f1 Add more documentation to BDD macros 2021-03-23 13:02:07 +01:00
6734c0aa64 Expand documentation for BDD test macros (#2179) 2021-03-23 12:20:33 +01:00
037ddbc75c Fix introduced in version placeholders
Resolves #2082
2021-03-13 10:26:12 +01:00
6d803cba5d Add example of negated tag selection to the CLI test spec docs
Close #2180
2021-02-25 15:54:27 +01:00
551946c45b Split out INTERNAL_CATCH_UNIQUE_NAME into its own header
Also small cleanup of includes of related headers.
2021-02-24 23:14:18 +01:00
653764d53b Simplify workaround for MSVC issue with XmlWriter 2021-02-21 11:19:02 +01:00
3afea8128a Increase timing window for min duration tests to 1s
This test tends to be brittle on Mac CI machines, which are
heavily loaded and bursty. Since the tests are only run as part
of the "extra tests" test set, this increase should not have
a significant impact on the total duration of CI runs.
2021-02-20 23:09:02 +01:00
749d953712 Add Clang 10 + C++17 build to linux CI 2021-02-20 22:26:22 +01:00
4b50b13970 Disable C++ extensions for linux CI builds 2021-02-20 20:39:23 +01:00
1ee0940427 Merge pull request #2177 from catchorg/devel-generating-surrogates
Add surrogate TUs option to CMake
2021-02-20 10:29:01 +01:00
29050daec0 Add Linux and Windows CI builds with surrogate TUs 2021-02-19 23:23:00 +01:00
e5e9afad16 Fix headers whose surrogate TUs did not compile 2021-02-19 23:22:43 +01:00
8b27041fbe Add CMake option to generate surrogate TUs in development build
A surrogate TU is TU that includes 1 specific header, and does
nothing else. This is useful to verify that the header is
self-sufficient and does not require other headers to be included
for compilation to succeed.

Closes #2106
Closes #2166 (this is a better solution)
2021-02-19 23:22:39 +01:00
c12170ff69 Extend release notes
Included some changes that weren't updated at the time of merging,
and also results of a new runtime benchmark.
2021-02-13 15:34:38 +01:00
3eade52fc0 users: Add d-SEAMS 2021-02-04 22:53:00 +01:00
2dbe63a6ba Merge pull request #2165 from catchorg/devel-gmtime-fixup
Use gmtime_r instead of gmtime when compiling for posixy platforms
2021-01-27 19:55:41 +01:00
477540760a Use gmtime_r instead of gmtime when compiling for posixy platforms 2021-01-27 11:49:52 +01:00
b435e391c4 Merge pull request #2163 from catchorg/devel-list-outputs-to-out-flag
The output of --list-* flags obeys -o flag
2021-01-27 09:07:15 +01:00
971b1fc32a --list-* flags write to target specified by the -o flag
Also added tests for the default implementations of list* reporter
helpers.

Closes #2061
2021-01-26 18:36:54 +01:00
6798c139a6 Extract the default listing implementation out of reporter interface
Also generalized the implementations to write to the provided
output stream, which will be required for the follow up changes,
where the listings should happen to the location user asked for
by specifying the `-o` flag.
2021-01-26 11:23:37 +01:00
7111b2a8e2 Cleanup IStreamingReporter::listTests 2021-01-16 15:50:52 +01:00
5509ceff60 Remove superfluous IConfig argument from IStreamingReporter::list*
The previous commit made it so that the `IConfig` is stored in
`IStreamingReporter` and thus always available.
2021-01-16 15:48:29 +01:00
74f2f4ba5e Hold reporter's IConfig instance in the interface
Previously, every base derived from the IStreamingReporter had
its own `IConfig const* m_config` member, so this just centralizes
the handling thereof.

Part of #2061
2021-01-16 15:27:56 +01:00
ba81505168 Do not match exact amount of spaces for errno macro expansion in approvalTests.py
E.g. musl libc expands errno() to __errno_location() without a space between, glibc has 1 space.
2021-01-08 11:31:28 +01:00
f5b413aa58 Merge pull request #2138 from catchorg/devel-config-splitting
Split some configuration options out of catch_compiler_capabilities
2020-12-29 12:37:27 +01:00
4e8832fc88 Split CATCH_CONFIG_COUNTER into its own header
Part of #2041
2020-12-29 11:07:45 +01:00
bdd1e5c613 Remove some superfluous includes 2020-12-28 21:11:54 +01:00
1d269211bd Split CATCH_CONFIG_WCHAR into its own header
Part of #2041
2020-12-28 20:51:49 +01:00
0acb371b92 Fix Wold-style-cast error (#2125)
* Add Wold-style-cast to cmake flags
* Fix old style cast in catch_stats.hpp
* Fix old style cast in catch_stats.cpp
2020-12-28 14:00:19 +01:00
045feff834 Update cmake-integration.md (#2115)
* Update cmake-integration.md

CMake related, mainly more modern and provide an executable to be correct

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-12-28 13:41:55 +01:00
965afc4b2e Split CATCH_CONFIG_ANDROID_LOGWRITE into its own header
Part of #2041
2020-12-28 12:53:52 +01:00
77643ce2e5 Add more comprehensive tests for the quantifier matchers
This includes
* Testing both positive and negative path through the matchers
* Testing them with types whose `begin` and `end` member functions
require ADL
* Testing them with types that return different types from `begin`
and `end`
2020-12-27 20:20:55 +01:00
552af8920d Added AnyMatch, AllMatch and NoneMatch 2020-12-27 20:20:55 +01:00
ce54ec185f Add the HIP CPU Runtime as a Catch2 user (#2123) 2020-12-18 13:23:17 +01:00
c787b1edc9 Make Linux builds print stdout on ctest test failure 2020-12-15 09:58:19 +01:00
a091853f4a Rename file catch_matchers_floating -> catch_matchers_floating_point
The old name was a legacy of v2 era, where all headers were stitched
into one. With v3 using separate headers, it is better when they have
proper name.
2020-12-13 18:12:20 +01:00
be813faaa0 Fix typo 2020-12-12 15:31:20 +01:00
4b51d0dd3b Remove rest of std::shared_ptr usage in Trackers
Closes #2088
2020-12-05 15:19:23 +01:00
6350851f9a Return plain ptr from ITracker::findChild instead of shared_ptr
This is part of wider rework of trackers to avoid `std::shared_ptr`
everywhere.

Related to #2088
2020-12-04 21:37:24 +01:00
21c97f2fad Use snprintf instead of sprintf (#2111) 2020-12-01 18:59:27 +01:00
5b1a6ae00a Merge pull request #2112 from Emil-Jarosz/patch-1
Update cmake-integration.md
2020-11-30 21:18:25 +01:00
b9fe8a208f Update cmake-integration.md
The "contrib" folder was renamed to "extras", but the name wasn't updated in the documentation.
2020-11-30 13:10:26 +00:00
c19b8ec5d7 Remove TravisCI builds subsumed by GitHub Actions builds 2020-11-29 23:18:57 +01:00
230f23e6ee Add more Clang builds to github actions 2020-11-29 10:22:21 +01:00
88504e5332 Use Ubuntu-20.04 runner for running header guard validation
Using `ubuntu-latest` is triggering warning that it will soon migrate
from 18.04 to 20.04, so I am migrating preventively.
2020-11-28 23:09:10 +01:00
4da0c18526 C++17 Mac builds 2020-11-28 21:13:00 +01:00
1d746a15ac Move Examples and ExtraTests Mac builds to Github Actions 2020-11-28 21:03:38 +01:00
19cbdebb0e Lenghten timing threshold in --min-duration integration test 2020-11-28 21:00:09 +01:00
f30a9e3feb Move some Mac builds from Travis to GitHub Actions 2020-11-28 17:47:26 +01:00
e7740316e3 Split TestFailureException into its own header 2020-11-28 11:18:54 +01:00
72525a3053 Rename catch_common.hpp -> catch_source_line_info.hpp
Previous splitting of catch_common.hpp left it containing only one
actual thing, which is the `SourceLineInfo` type. Given that, there
is no reason to keep the old name.

Also found out that it was included in some places for no reason
(primarily Matchers).
2020-11-28 11:18:51 +01:00
1982c0d5ee Split out the StreamEndStop helper into its own header 2020-11-28 11:18:46 +01:00
0442229dc9 Start using GitHub actions for Linux builds
For now we switched only some of the builds, those that do not
need manually installed compiler packages to function.
2020-11-28 11:13:16 +01:00
130bf835b5 Simple Doxygen file 2020-11-26 18:43:31 +01:00
c3e8ae642f Remove pointless virtual in WildcardPattern 2020-11-26 18:35:20 +01:00
3bd5fd6bc5 Mention GENERATE decaying inputs in release notes docs 2020-11-26 18:35:02 +01:00
f36e059453 Remove superfluous <memory> include from catch_reporter_cumulative_base.hpp 2020-11-26 18:06:21 +01:00
677adf8ade Replace shared_ptrs with unique_ptrs in CumulativeReporterBase nodes
Closes #2089
2020-11-26 14:45:27 +01:00
bfe5553416 CumulativeReporterBase doesn't use shared_ptrs for sectionStack 2020-11-26 10:21:12 +01:00
c673db7a4e Add explanatory comment to CumulativeReporterBase 2020-11-26 09:45:28 +01:00
b10a19545b CumulativeReporterBase stores TestRunNodes directly 2020-11-26 09:43:23 +01:00
e5ccb79bf8 Remove pointless virtual on StreamingReporterBase::Node destructor 2020-11-25 21:57:30 +01:00
3610eb81b1 Explicitly inline trivial function in CumulativeReporterBase impl 2020-11-17 21:02:11 +01:00
bd1e76cc3a Untypedef some types for clarity when refactoring 2020-11-17 21:01:35 +01:00
166c520598 Remove another unused member variable from CumulativeReporterBase 2020-11-14 16:17:48 +01:00
a29deeb129 Remove unused member in StreamingReporterBase
Part of #2089
2020-11-14 16:17:39 +01:00
1cef51b69b Default StreamingReporterBase::Node destructor 2020-11-14 16:17:35 +01:00
79c1bf9301 Use plain pointer to point to deepest section in CumulativeReporterBase
Part of #2089
2020-11-14 16:17:27 +01:00
4f14922aa3 Don't use shared_ptr to store test run info in CumulativeReporterBase
Part of #2089
2020-11-14 16:16:28 +01:00
0fa133a0c5 Run checking scripts directly 2020-11-10 15:25:52 +01:00
447b39cae0 Better names for misc-checks workflow steps 2020-11-10 15:25:24 +01:00
851a0e907e Make scripts executable 2020-11-10 15:22:15 +01:00
93312b369e Merge pull request #2099 from innerout/check-license
Adds license check in CI
2020-11-10 11:10:03 +01:00
d913837a5d Adds license check in CI 2020-11-10 00:51:24 +02:00
a9941d4231 Merge pull request #2098 from innerout/check-duplicates
Adds duplication check for source files in CI
2020-11-07 20:59:37 +01:00
39e13bf530 Adds duplication check for source files in CI 2020-11-07 21:35:41 +02:00
fefa001bb6 Fixup approvals 2020-11-07 20:16:53 +01:00
135103bacf Arg::parse accepts plain Args as argument 2020-11-07 18:00:27 +01:00
2baa472bcc Add GitHub action to check that include guards use standard pattern
Based on work by @innerout, but finished and merged by me, because
the GitHub action refused to run for him.

Closes #2075
Closes #2091
2020-11-06 21:36:31 +01:00
f97436a1f7 Normalize include guard in catch_config_ucnaught_exceptions.hpp 2020-11-06 18:01:27 +01:00
dd5652933a Silence -Wuseless-cast warning with GCC10
With GCC 10, the `static_cast<bool>` triggers the -Wuseless-cast warning. This commit changes the cast into `static_cast<const bool&>`: it achieves the same thing but doesn't trigger the warning thanks to the "gratuitous" type conversion to `const bool&`. As per references rules, `const bool&` should bind to anything, be it `const` or not, an rvalue or an lvalue, so I doubt that this change is breaking anything.
2020-11-02 16:01:17 +01:00
3a15433d52 Add support for Components 2020-11-02 15:37:35 +01:00
67a9561fb5 Fix slashes for Windows builds 2020-11-02 15:37:35 +01:00
2f31f9037d Inject conan_basic_setup to properly set MD/MT flags 2020-11-02 15:37:35 +01:00
33bcdc6bf5 Picked release notes for v2.13.3 2020-11-02 14:42:24 +01:00
74b397e6b8 Consider CMP0110 add_test() policy
CMake 3.19 introduces new add_test() behavior guarded with the policy
CMP0110.

See: https://cmake.org/cmake/help/latest/policy/CMP0110.html

Update the helper script ParseAndAddCatchTests to consider the policy and
handle the test case name accordingly.
2020-11-02 14:42:22 +01:00
730ec39a74 Fix indentation in ParseAndAddCatchTests
Consistently use 4 spaces instead of tabs
2020-11-02 14:42:19 +01:00
71328bae90 Fix CMake regex to add tests
Fix regex that requires two string arguments in the form of
TEST_CASE("a", "b") resulting in not finding TEST_CASE("a") entries.

See https://regex101.com/r/JygOND/1

Fixes: https://github.com/catchorg/Catch2/issues/2055
2020-11-02 14:42:17 +01:00
ed9ef85a34 Add AppMesh to Open Source projects using Catch 2020-11-02 14:42:13 +01:00
e4474021ff Merge pull request #2085 from AugustasV/patch-1
[DOCS] update from Hudson to Jenkins
2020-11-01 15:12:58 +01:00
79a5cd795c Merge pull request #2086 from AMS21/patch-1
Enable IPO for CMake versions that support it
2020-11-01 15:11:17 +01:00
b8ae2878b4 Enable IPO for CMake versions that support it 2020-11-01 09:36:42 +00:00
dc3c22f9ec update from Hudson to Jenkins
Hudson is discontinued in 2016. Changed to Jenkins
2020-10-31 21:48:28 +02:00
735f46ed6d Link to the migration docs from main readme and docs/readme 2020-10-31 09:08:58 +01:00
39aabede62 Add a short documentation for migrating from v2 to v3 2020-10-31 08:14:50 +01:00
d7ced69db2 Update README.md to point to v2 branch 2020-10-30 15:31:41 +01:00
f797ae7a8f Bump OpenCppCoverage to 0.9.9.0
Also modified the AppVeyor script to account for changes in what
happens to PATH with the new installer.

Closes #2059
2020-10-29 13:39:35 +01:00
40b9df567f Add missing cmake option to fix buildAndTest.sh
Note that the documentation was updated automatically, by running:
 ./tools/scripts/updateDocumentSnippets.py
2020-10-28 11:38:06 +01:00
c6352c3e1f Embed the script buildAndTest.sh inside the docs 2020-10-28 11:38:06 +01:00
4035beb988 Add a script - buildAndTest.sh
This contains the currently-documented steps for a developer build.
Also, make git ignore its output directory.
2020-10-28 11:38:06 +01:00
8c3970465d Add infrastructure for embedding code samples in docs. 2020-10-28 11:38:06 +01:00
f57689f888 Fix minor typo in docs. 2020-10-28 11:38:06 +01:00
967b82231c Sweep out two leftover references to master
This is basically cherry-picking the usable parts from #2067
2020-10-23 23:02:44 +02:00
7b9bf633be Tighten the test for matching filenames-as-tags 2020-10-23 23:02:09 +02:00
4c8454b5ec Fix potential infinite loops in generators combined with section filter
The problem was that under specific circumstances, namely that none
of their children progressed, `GeneratorTracker` will not progress.
This was changed recently, to allow for code like this, where a
`SECTION` follows a `GENERATE` at the same level:

```cpp
SECTION("A") {}
auto a = GENERATE(1, 2);
SECTION("B") {}
```

However, this interacted badly with `SECTION` filters (`-c foo`),
as they could deactivate all `SECTION`s below a generator, and thus
stop it from progressing forever. This commit makes GeneratorTracker
check whether there are any filters active, and if they are, it checks
whether its section-children can ever run.

Fixes #2025
2020-10-23 21:21:15 +02:00
8878f90323 Fix matching of non-lowercase filename tags
The bug was caused by forgetting to lower-case the filename tag
for matching against test spec.

Fixes #2064
2020-10-22 16:20:41 +02:00
0c7f96ba63 Update documentation not to reference master branch 2020-10-21 19:59:59 +02:00
923bcc5d6f Special case --list-tests --verbosity quiet
The new output (mostly) follows the old `--list-test-names-only`
format, with the exception of no longer supporting line output
for `--verbosity high`.

Fixes #2051
2020-10-20 15:09:48 +02:00
b6a3e2e26b Merge pull request #2058 from catchorg/devel-generators-cache
Fix types in generators: decay types, properly constrain forwarding
2020-10-20 12:46:31 +02:00
6ffac61719 Fix types in generators: decay types, properly constrain forwarding
Fixes #2040
Closes #2012
2020-10-20 10:57:37 +02:00
4b2bc8757c Update catch_compiler_capabilities.hpp 2020-10-10 17:56:43 +02:00
faffc29253 Enable "Try online" badge again 2020-10-10 15:13:24 +02:00
4ea18d6d17 Merge pull request #2044 from lkeegan/fix_cmake_catch_add_tests_warning
fix warning in extras/CatchAddTests.cmake
2020-10-08 17:15:05 +02:00
c44d9cc718 fix warning in extras/CatchAddTests.cmake
- don't warn on zero return code of --list-reporters
  - previously return code was the number of reporters (#1410, #1146)
  - as of 2c06ee9 return code is zero on success
2020-10-08 16:44:20 +02:00
b9853b4b35 Bump version to v3.0.0 preview 3 2020-10-08 15:26:30 +02:00
853565bfb8 Pick release notes for v2.13.2 2020-10-08 12:51:07 +02:00
3f9e779542 Improve detection of std::uncaught_exceptions support
The problem was that Catch2 did not reliably include `<exception>`
before it checked for the feature test macro for
`std::uncaught_exceptions`. To avoid overhead of including
`<exception>` everywhere, the configuration check was split out
into a separate header.

Closes #2021
2020-10-08 12:39:50 +02:00
863cc6a155 Add WORKING_DIRECTORY to CatchAddTests.cmake commands 2020-10-08 11:41:35 +02:00
b601b7faca Workaround AppleClang bug by renaming TestHasher constructor argument
As far as I understand the standard, if there is a function called
`rng` in the global namespace, and a function argument called `rng`,
then the argument should shadow the function. This then means that
uses of `rng` inside the function should refer to the argument.

This is not the case for AppleClang 12.0.0. Luckily the workaround
is simple enough; just rename the argument. Given that the function
is 3 lines and uncomplicated, the change of the name doesn't really
affect readability.

Still, WTF AppleClang?

Closes #2030
2020-10-08 11:41:05 +02:00
b841650253 Add missing syntax highlighting tag 2020-10-08 11:37:31 +02:00
1d01464730 Support template test cases in ParseAndAddCatchTests
* Change regex to allow parentheses inside the test macro for a type list
* Append a wildcard to the CTestName if the test case is a template
* Also change the regular expression so parentheses are allowed in names
  (fixes #1848)
2020-10-08 11:37:11 +02:00
c522e88afa Add REPORTER and OUTPUT_* args 2020-10-08 11:35:42 +02:00
b1df96e7e4 Docu/Show how to use CMake FetchContent (#2028) 2020-10-08 11:35:18 +02:00
a4dfcf9042 Disable __builtin_constant_p when compiling with nvcc 2020-10-08 11:34:57 +02:00
9e172c707e fixes bug in example - undeclared identifier
j was not declared in `SECTION("two")`
2020-10-08 11:34:25 +02:00
b0214ff862 Make experimental capture work on Windows with read-write temp file behavior 2020-10-08 11:28:50 +02:00
2454cfffb7 add fuzzer for columns 2020-10-07 17:38:27 +02:00
0098a76fef first take basing fuzz on v3 2020-10-07 17:38:27 +02:00
340ff00058 Rewrite contributing docs for v3 2020-10-06 12:36:20 +02:00
60dfec559f Provide 1 .hpp + 1 .cpp distribution of Catch2
This commits also adds a script that does the amalgamation of headers
and .cpp files into the distributable version, removes the old
`generateSingleHeader` script, and also adds a very simple compilation
test for the amalgamated distribution.
2020-09-09 13:02:50 +02:00
8b89a60bf6 Reenable VS2017+examples+extra-tests build on AppVeyor 2020-09-08 22:20:12 +02:00
99d70c0c9d Remove superfluous whitespace before #include 2020-09-07 20:01:04 +02:00
d1625f30b1 Pick release notes from v2.13.1 2020-09-07 14:34:36 +02:00
8f44e09a72 Remove superfluous values
The `0`s were needed for the expansion of parameter pack for the sake of
expander. Now when we have `index++` it is no more needed.
2020-09-07 14:30:58 +02:00
31d4831245 Support sentinel-based ranges in default stringify (#2004) 2020-09-07 14:23:47 +02:00
08fb5cbab2 Fix typo in generators docs 2020-09-07 13:02:38 +02:00
5ad1a4fe61 Issue 1992: Better c++17 std::byte detection
This fixed buld errors for ubuntu-16 + clang and similar situations
where C++17 is supported in the compiler but not the default
C++ standard library.
2020-09-07 13:01:51 +02:00
2c1c02f7e7 Update ParseAndAddCatchTests.cmake 2020-09-07 13:00:55 +02:00
8851e779cf console colour: fix unintended colouring of user's stderr on POSIX
At some places, the colour reset code is printed after a newline.
Since the default output buffering to console is line-based, the reset
code is not actually written out. If messages from user code are printed
to stderr (different stream, same console), they are printed before
the colour reset code, and thus they are coloured.

Explicitly flushing the stream after writing the colour escape code solves
this.
2020-09-07 11:53:45 +02:00
2d4f8ac8e6 Disable problematic test on Windows 2020-09-06 20:41:58 +02:00
9155a9ff20 Fix -Wparentheses issue on g++9 2020-09-06 16:02:01 +02:00
cc18bd719d Sweep out Wshadow
Most of the changes are completely pointless renaming of constructor
arguments so that they do not use the same name as the type members,
but 🤷

Closes #2015
2020-09-06 13:11:42 +02:00
90aeffb97d Add standardized copyright notice + SPDX identifier to source files
This should also be done for test files, but that has lower priority.
2020-08-30 15:43:45 +02:00
c2453c2bf8 Remove the rest of old-style copyright headers 2020-08-30 14:45:05 +02:00
a822cb9717 Standardize include guard patterns to FILE_NAME_EXTENSION_INCLUDED
This commit also strips the old copyright comment header in touched
files, as those will also be replaced with a more standardized and
machine-friendly version.
2020-08-30 14:09:27 +02:00
c26693df23 Fix doccomment on shouldShowDuration reporter helper 2020-08-29 19:55:17 +02:00
33ad1ee2ac Split EventListener base from streaming_base.hpp
The base was also renamed from `TestEventListenerBase` to
`EventListenerBase`, and modified to derive directly from the
reporter interface, rather than deriving from `StreamingReporterBase`.
2020-08-29 19:09:54 +02:00
f9fdc96cbf Push LazyStat down into catch_reporter_streaming_base.hpp 2020-08-24 11:27:21 +02:00
360b82620e Refactor serializeFilters implementation 2020-08-24 10:19:26 +02:00
2a8e317cfb Split various reporter helpers out from streaming_base.hpp
Due to also adding a new TU, there is no improvement to the
compilation times of the static library, but it improves the
compilation times of consumer's reporter TUs.
2020-08-24 10:19:24 +02:00
6a08d401aa Split out ReporterFactory out of catch_interfaces_reporter.hpp 2020-08-23 22:35:01 +02:00
9677df6d8b Split IReporterRegistry into its own header file
Doing this removes `<map>` from the include set of the base reporter
interface, and thus from bunch more TUs. This provides about 1.5%
improvements in the debug build of the static library, and 1% in
release build.
2020-08-23 21:01:04 +02:00
ed7eaf2df3 Split catch_reporter_bases.hpp into two separate headers
Each of the two reporter bases now has its own header file, and
cpp file. Even though this adds another TU to the compilation,
the total CPU time taken by compilation is reduced by about 1%
for debug build and ~0.5% for optimized build of the main library.
(The improvement would be roughly doubles without splitting the TUs,
but the maintainability hit is not worth it.)

The code size of the static library build has also somewhat decreased.

Follow up: Introduce combined TU for reporters, and further split
apart the catch_reporter_streaming_base.hpp header into its
constituent parts, as it still contains a whole bunch of other stuff.
2020-08-23 07:30:26 +02:00
24559493bf Anchor some Clara vtables into the cpp file
Anchoring the vtables does 2 things
1) Fixes some instances of `-Wweak-vtables`
2) Decreases code size and linker pressure

However, there are still some unanchored ones, and thus we have
to keep suppressing `-Wweak-vtables` warning for Clang.
2020-08-20 23:16:11 +02:00
7500ad1ffd Turn Clara::Detail::ResultBase::Type into its own enum class 2020-08-20 22:59:06 +02:00
1a97af45f1 Cleanup some stuff found by MSVC's /analyze
* Added some missing `noexcept`s on custom destructors.
* Fixed `std::move` being called on a const-reference.
* Initialized `ScopedMessage::m_moved` in class definition, instead
of doing so in constructors explicitly.
* Turned some `enum`s into `enum class`es.
* Initialized `StreamingReporterBase::currentTestCaseInfo` in class
definition.
* Some cleanups in SelfTest code.
2020-08-20 20:42:21 +02:00
2e480b6e56 Use StringRef for CasedString::caseSensitivitySuffix return type 2020-08-18 21:02:34 +02:00
f16be402f7 Make XmlEncoding tests slightly more efficient 2020-08-18 21:02:31 +02:00
e418e75c74 Turn CaseSensitive into proper enum class 2020-08-18 21:02:30 +02:00
6a46b344c0 Split out CaseSensitive enum out of catch_common.hpp
Only very few places actually use it, so there is no need to have
it be central to everywhere.
2020-08-18 21:02:25 +02:00
e7eb749815 Split the NonCopyable helper out of catch_common.hpp 2020-08-18 21:02:23 +02:00
c1bb699d45 Remove superfluous catch_reporter_registrars.hpp includes from reporters
The includes were an artifact of the old design where the built-in
reporter TUs used autoregistration to add the reporters to the
factory. Because the current design is to add them explicitly in
the central reporter factory TU, the includes are useless.
2020-08-14 09:33:53 +02:00
bf1e902ca1 Remove <algorithm> include from catch_commandline.cpp 2020-08-14 09:33:42 +02:00
49b55d53e4 Prune includes in catch_run_context.cpp 2020-08-14 09:33:28 +02:00
07fb96d42c Don't include catch_random_number_generator.hpp in catch_session.cpp 2020-08-14 09:33:16 +02:00
34d9a588bb Cleanup includes in catch_list.cpp 2020-08-14 09:32:59 +02:00
05d7014e75 ApproxMacher uses Approx's overloads for double directly
This saves a tiny little bit of compilation times when the
`ApproxMatcher` is used and `epsilon`, `margin`, or `scale` are
used to customize its behaviour.
2020-08-11 19:07:37 +02:00
aa28a917cb Turn UseColour into enum class 2020-08-11 17:47:54 +02:00
b0531404e4 Turn RunTests into TestRunOrder enum class 2020-08-11 17:47:00 +02:00
60cc4c293d Turn ShowDurations into an enum class 2020-08-11 15:03:37 +02:00
6dc8345261 IStreamingReporter::list* only uses IConfig instead of full Config
As the full `Config` is not needed, the TUs implementing the `list*`
functions can require the less heavy header `catch_interfaces_config.hpp`
instead of the much heavier `catch_config.hpp`.

This commit also fixes up some other TUs that include `Config`,
while using just `IConfig`, to cleanup the includes further.
2020-08-11 09:39:00 +02:00
24b83edf8a Clara cleanups
* Clara is now split between a header and a cpp file.
* Removed the deprecated `+` and `+=` operators for composing
  a parser.
* Renamed `clara` and `detail` namespaces to be inline with the
rest of Catch2 (they are now `Clara` and `Detail` respectively).
* Taken most of user-exposed types out of the `Detail` namespace
completely (instead of using `using` directives to bring them into
the outer namespace).
2020-08-10 20:30:40 +02:00
b824d06844 Fixup missing transitive includes
Found after updating to newest MSVC...
2020-08-10 20:28:14 +02:00
e7aa432850 Split TextFlow out from Clara
Now that it has its own header, various reporter TUs that want to
format text do not have to also include Clara. Together with
outlining implementations from a header into a separate TU, this
has noticeably improved the compilation times of the testing impl.

As part of this split, I also implemented some improvements to the
TextFlow code in comparison to the upstream code. These are:

* Replaced the `Spacer` type with a free function that constructs
special `Column` that does the same thing.
* Generic performance improvements, such as eliminating needless
allocations, reserving space in needed allocations, and using smarter
algorithms in some places.
* Because `Column` only ever stored 1 string in its vector, it now
holds the string directly instead.
2020-08-03 23:40:14 +02:00
81aa2d5582 Replace ostringstream with ReusableStringStream in ConsoleReporter 2020-08-03 22:59:43 +02:00
9d591f19ff Allow explicitly setting ReusableStringStream's serialized data 2020-08-03 22:59:09 +02:00
a4ac07d104 Split out default CATCH_CONFIG_CONSOLE_WIDTH into its own header
This means that code that uses it no longer has to include all of
catch_config.hpp, which seems to provide significant reduction in
size of unoptimized compilation of the final static library.
2020-08-03 22:57:43 +02:00
8b0845b1a2 Split catch_config.hpp out of reporter bases 2020-07-30 08:46:07 +02:00
c5037743e6 Avoid copying SectionInfo while construction Section 2020-07-29 21:51:10 +02:00
9d6ac62aff Remove unused member from Section 2020-07-29 21:51:08 +02:00
ed0ea30149 Apply some IWYU suggestions
This is not nearly all of them, because IWYU does not support the
way Catch2 manages includes -- it expects that non-system includes
are done using `#include "foo/bar/baz.hpp"`, while Catch2 uses
`<foo/bar/baz.hpp>`. This causes trouble, because IWYU suggests
removing every single internal header, and then adding them again,
but using `""` in the include directive... the resulting suggestions
cannot be used without a lot of manual work, as they are largely
bogus.

For bonus points, IWYU also _loves_ to suggest kinda-random stdlib
headers for `size_t` and similar. Still, the resulting inclusion
graph is somewhat better than it was before.
2020-07-29 21:51:05 +02:00
35098a62d8 Sanitize includes in random_number_generator.cpp
I have no idea how those got there, but now its dependency graph
is much saner.
2020-07-29 14:58:14 +02:00
ba57c17310 Remove some extraneous includes from run_context.hpp 2020-07-29 14:54:55 +02:00
4e0af77e29 Push includes for TestCaseInfo down into reporter TUs 2020-07-29 14:43:05 +02:00
eaf7113fd3 Cleanup stdlib includes in reporter_bases.hpp 2020-07-29 12:50:51 +02:00
d090074da7 Devirtualize handling of ReporterPreferences
The new scheme is that there is one protected member instance of
`ReporterPreferences` in the `IStreamingReporter` base class,
and derived classes can modify it to express their own preferences.

Retrieving the preferences is now a non-virtual operation, which
makes it much cheaper to read them frequently. Previously, we
avoided doing so by caching the preferences in another variable,
but we still read them at least once per test case run.
2020-07-28 09:24:57 +02:00
d218d6f9e2 Avoid recalculating string-literal size on root tracker construction
This is a tiiiiiiny performance optimization, but it's free.
2020-07-27 20:08:44 +02:00
ef92178058 Devirtualize nested tracker handling 2020-07-27 19:53:54 +02:00
125d4b4164 Devirtualize test case registration 2020-07-27 08:49:49 +02:00
c9b4867441 Move some impls of StreamingReporterBase members to .cpp file 2020-07-27 08:46:35 +02:00
258cac63f8 Move impls of CumulativeReporterBase member functions to .cpp file
`catch_reporter_bases.hpp` turned out fairly expensive for parsing
when building the main library, and the significant amount of code
in headers likely doesn't help. Since the reason it is in the header
is legacy from CRTP reporter bases, moving as much of the
implementations to the .cpp file is free compilation perf.
2020-07-27 07:32:21 +02:00
5f6990d746 Only start Section's timer if the duration will be used
This is a small potential runtime optimization on systems with
virtual syscalls, and a significant runtime optimization on systems
without.
2020-07-26 21:33:49 +02:00
5ca68829e1 Refactor how shortcircuiting of old style matchers is tested 2020-07-26 21:31:29 +02:00
ac54ba7e12 Add test for shortcircuiting behaviour of generic matcher combinators 2020-07-26 21:24:05 +02:00
95c0c88d84 Fix CMake add test helper for CMake 3.18.0
With CMake 3.18.0 the `add_test(NAME)` handling changed. The escaped
double quotes confuse the new call. Work around this upstream change.

fixes: https://github.com/catchorg/Catch2/issues/1984
2020-07-26 15:15:50 +02:00
6efeecc179 Cherry-pick doc updates for v2.13.0 2020-07-26 15:14:38 +02:00
6b3c563c38 Remove pointless CompactReporter::getPreferences override 2020-07-26 14:07:19 +02:00
a004423c7f Improve documentation for --min-duration 2020-07-26 14:06:52 +02:00
4b344f11ea Document GENERATE's new usage between SECTIONs 2020-07-26 14:06:34 +02:00
87d0197cbd Update catch_reporter_tap.hpp
TAP format requires all results to be reported.
Removed extraneous preferences function (handled by parent)
Incorporated fix from 3d9e7db2e0
Simplified total printing
2020-07-26 14:00:10 +02:00
4565b826cf Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-26 11:35:06 +02:00
250d9b9c72 Fix how testRandomOrder.py builds tag arguments 2020-07-26 10:51:55 +02:00
90d6fd849e Increase tolerances in --min-duration tests
The underpowered and oversubscribed CI servers are hell.
2020-07-26 10:48:03 +02:00
13917c44b4 --min-duration is overriden by -d no 2020-07-26 10:48:01 +02:00
e6d947f6d4 Refactor tests for duration reporting threshold 2020-07-26 10:47:58 +02:00
80b0d6975c Add --min-duration option
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.

When there are many tests, this produces output spam which makes it hard
to find the test failure output.  Nevertheless, it is helpful to be
informed of tests which are unusually slow.

Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed.  This allows slow
tests to be visible without mentioning every test.
2020-07-26 10:47:53 +02:00
36131f7ffa Escaping literal "*" ("times") to fix markdown 2020-07-24 22:36:58 +02:00
f52018205d Cherry pick release notes for v2.12.4 2020-07-24 22:36:13 +02:00
b32d2fa016 Update FUNDING file (-Patreon, +PayPal) 2020-07-24 22:35:17 +02:00
a25c1a24af Fix for macOS on ARM 2020-07-24 22:34:22 +02:00
e28018c659 Cherry pick release notes for v2.12.3 2020-07-24 22:33:59 +02:00
2a25a267ea Cherry pick release notes for v2.12.2 2020-07-24 22:33:26 +02:00
7f58840163 Add OverallResultsCases element to XML reporter 2020-07-24 22:30:28 +02:00
3b0f8c7ff0 Replace a TODO comment in examples 2020-07-22 21:49:58 +02:00
2840ce1e70 Add an example of using GENERATE(table())
There are some examples on issue #850 of using this feature, but they
are not easily found from the documentation. Adding them here as an
example makes them more findable and ensures they keep working if the
API changes.
2020-07-22 21:49:54 +02:00
ed9be5a00b Add notes on compiling the examples.
This took me some time to figure out so document for others.
2020-07-22 21:43:53 +02:00
273111d1a6 Clang-format configuration added.
Some notes on the configuration options chosen:

* We want `AllowShortEnumsOnASingleLine` set to `false`, but that
option is clang-format-11 and up, which is not out yet.
* `IndentPPDirectives` is currently inconsistent, but `AfterHash`
is the preferred style in new code.
* `NamespaceIndentation` is a mess, but `All` is closer to the effect
we want than `Inner`.
* `SpacesInParentheses` set to `true` is not ideal due to it also
introducing extra spaces in preprocessor expressions, but using it
is much closer to the current style than not.

All in all, using this setting globally would reformat pretty much
every line of code in the codebase, but it is as close as possible
to the bespoke style currently used. Still, it should only be used
on the diffs.

Closes #1182
2020-07-22 21:43:30 +02:00
a862924601 Make scripts/updateDocumentToC.py executable.
On systems where the file system has excute permissions, this script was
not marked as executable in a clean git checkout and so could be run
without first changing the permissions. Fixed by setting the relevant
git flag.
2020-07-22 21:43:16 +02:00
0e77adee05 Add explicit test for shortcircuiting behaviour of combined matchers 2020-07-22 21:36:14 +02:00
b74996a29c Devirtualize NameAndLocation query on trackers 2020-07-22 21:23:12 +02:00
de53773e46 Fix copy paste error in 7-arg TEMPLATE_TEST_CASE_SIG implementation
Closes #1954
2020-07-22 21:18:52 +02:00
314bb7e632 Clarify documentation about nested generators
Closes #1947
2020-07-22 21:18:49 +02:00
9221a6ff65 Hide std::exception_ptr and friends if exceptions disabled
Some compilers, e.g. the Green Hills C++ compiler, react badly to the
appearance of std::exception_ptr, std::current_exception,
std::rethrow_exception and std::uncaught_exception(s). To allow usage of
Catch2 with these compilers when exceptions are disabled, hide the usage
of std::exception_ptr etc. when compiling with
CATCH_CONFIG_DISABLE_EXCEPTIONS.
2020-07-22 21:18:47 +02:00
657ebf5db2 Replace stray tabs with spaces 2020-07-22 20:46:58 +02:00
480f3f418b Improved generator tracking
* Successive executions of the same `GENERATE` macro (e.g. because
of a for loop) no longer lead to multiple nested generators.
* The same line can now contain multiple `GENERATE` macros without
issues.

Fixes #1913
2020-07-22 20:44:22 +02:00
3ceaad7d66 fixing UB 2020-07-22 17:17:33 +02:00
5c502320e8 Remove obsolete comment in UnorderedEquals vector matcher 2020-07-12 21:28:50 +02:00
f3fe2dcb11 Add status attribute to JUnit's section reporting
This brings our output inline with GTest's. We do not handle skipped
tests properly, but that should be currently less important than
having the attribute exist with proper value for non-skipped tests.

Thanks @joda-01.

Closes #1899
2020-07-12 21:22:15 +02:00
8b5f6e26d3 Fix compilation failure when using libstdc++10 (#1929)
The issue is caused by deleted `std::__detail::begin` declared in `bits/iterator_concepts.h`. This would be found by ADL, and because it is deleted, compilation would fail. This change makes it so that we SFINAE on `begin(std::declval<T>())` and `end(std::declval<T>())` being well-formed.
2020-07-12 21:01:49 +02:00
c24f7e5b34 Fix invalid isspace call detected by PREfast
D:\vcpkg\toolsrc\include\catch2\catch.hpp(11285): warning C6330: 'char' passed as _Param_(1) when 'unsigned char' is required in call to 'isspace'.
D:\vcpkg\toolsrc\include\catch2\catch.hpp(11288): warning C6330: 'char' passed as _Param_(1) when 'unsigned char' is required in call to 'isspace'.

ISO/IEC 9899:2011:
"7.4 Character handling <ctype.h>"/1
[...] In all cases the argument is an int, the value of which shall be
representable as an unsigned char or shall equal the value of the macro
EOF. If the argument has any other value, the behavior is undefined.

This means if isspace was passed a character like ñ it could corrupt
memory without the static_cast to treat it as a positive value after
integral promotion (and C libraries commonly use the int index supplied
as a key into a table which result in out of bounds access if the
resulting int is negative).
2020-07-12 21:00:25 +02:00
7dae3efad2 Silence clang-tidy's hicpp-vararg (alias of coreguidelines vararg)
Ideally, clang-tidy would be smart that if one alias of a warning
is suppressed, then the other one is suppressed as well, but as of
right now, it isn't. This means that for now we have to suppress
both aliases of this warning. Opened upstream issue to fix this:
https://bugs.llvm.org/show_bug.cgi?id=45859

Obviously, ideally clang-tidy would also not warn that we are calling
a vararg function when it is an unevaluated magic builtin, but that
also is not happening right now and I opened an issue for it:
https://bugs.llvm.org/show_bug.cgi?id=45860

Closes #1921
2020-07-12 20:59:29 +02:00
a71721801e Remove extra semi-colons 2020-07-10 13:40:42 +02:00
2cd5e70012 Silence a -Wsign-conversio warning in Clara under GCC 2020-07-09 12:24:25 +02:00
392e44ec21 Minor refactoring of CompactReporter 2020-05-31 22:34:37 +02:00
317145514f Add op+(StringRef, StringRef) -> std::string 2020-05-31 22:30:41 +02:00
f2b9508081 Remove last vestiges of shared ptr use with IConfig 2020-05-31 15:25:14 +02:00
1d1ccf8f3c Replace all uses of std::unique_ptr with Catch::Detail::unique_ptr
Doing some benchmarking with ClangBuildAnalyzer suggests that
compiling Catch2's `SelfTest` spends 10% of the time instantiating
`std::unique_ptr` for some interface types required for registering
and running tests.

The lesser compilation overhead of `Catch::Detail::unique_ptr` should
significantly reduce that time.

The compiled implementation was also changed to use the custom impl,
to avoid having to convert between using `std::unique_ptr` and
`Catch::Detail::unique_ptr`. This will likely also improve the compile
times of the implementation, but that is less important than improving
compilation times of the user's TUs with tests.
2020-05-31 15:20:24 +02:00
41bbaa6d57 Implement a simplified variant of std::unique_ptr<T>
This simplified variant supports only a subset of the functionality
in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports
single element pointer (no array support) with default deleter.

By removing the support for custom deleters, we also avoid requiring
significant machinery to support EBO, speeding up instantiations of
`unique_ptr<T>` significantly. Catch2 also currently does not need
to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-31 15:08:47 +02:00
66ab942903 Refactor StreamEndStop slightly 2020-05-30 15:44:46 +02:00
d05a8e2e24 Add Bazel support for the v3 branch 2020-05-26 14:49:49 +02:00
1356788ea8 Avoid using std::result_of when std::invoke_result is available 2020-05-21 22:58:33 +02:00
21d284df34 Session now holds Config in unique_ptr instead of shared_ptr 2020-05-20 20:25:51 +02:00
668454b36b Do not share ownership of Config in RunContext 2020-05-20 20:25:49 +02:00
458241cc90 Do not use shared_ptr<Config> when listing things 2020-05-20 20:25:48 +02:00
fa160cf3f2 Keep reporter pointer around in TestGroup 2020-05-20 20:25:45 +02:00
a17b9f754a Remove unused function 2020-05-20 20:25:43 +02:00
c2852c9944 Remove some uses of std::shared_ptr 2020-05-20 20:25:40 +02:00
4394d3ae65 Translate exceptions by const reference instead of plain ref 2020-05-20 08:15:27 +02:00
4b2f1da02a Split CATCH_TRANSLATE_EXCEPTION into its own header
As far as I know, only a few users actually use it, but these changes
allow us to avoid including a surprising amount of code in the main
compilation path.
2020-05-20 08:15:11 +02:00
0c6fda6e7d Cleanup benchmark headers a tiny bit 2020-05-20 08:00:24 +02:00
bad8b7c866 Fix make_shared used instead of make_unique 2020-05-20 07:15:21 +02:00
964303706a Fixup catch_discover_tests
Fixed its use of removed `--list-test-names-only` switch, and also
updated the return code check to reflect changes in v3.

Closes #1933
2020-05-20 07:14:06 +02:00
54882dbb11 Don't include <tuple> everywhere in benchmarks 2020-05-20 07:09:15 +02:00
b4a61cfd29 Remove superfluous overload of operator== 2020-05-18 21:32:08 +02:00
d86834e5b5 Push down #include <ostream> to .cpp files 2020-05-18 21:31:41 +02:00
39e093021c Remove some superfluous includes 2020-05-18 20:55:21 +02:00
e867ce7769 Fix missing backticks on release notes
Closes #1932
2020-05-15 10:54:20 +02:00
f7fbbac601 Update release notes for v3 2020-05-14 14:57:22 +02:00
ddde2f5e33 Remove obsolete configuration macros from the documentation 2020-05-14 14:52:33 +02:00
d5e87eabbb Add provided generic matchers to the documentation 2020-05-14 14:48:48 +02:00
29d4b3768c Add Wsuggest-override to warning flags 2020-05-13 16:09:34 +02:00
ae0ba81423 Use more warnings for develoment builds 2020-05-12 23:56:51 +02:00
03ef6b9f9a Explicitly default smfs when relevant to avoid Wdeprecated-copy-dtor 2020-05-12 23:56:34 +02:00
579dcd1a76 Ignore Wdeprecated for Clara 2020-05-12 23:56:24 +02:00
eb267b424b Tap reporter changes 2020-05-12 23:56:23 +02:00
2528247351 Delete Capturer's smf
As part of `-Wdeprecated-copy-dtor` sweep, I noticed that Capturer's
copies are defaulted. Given that the class would likely break horribly
in the event of actual copy happening, they are now deleted.
2020-05-12 23:56:08 +02:00
f56832d3ea Add Catch2::Catch2 alias for Catch2 CMake target 2020-05-11 20:05:10 +02:00
601ca1c670 Refactor some uses of virtual -> override 2020-05-11 16:34:55 +02:00
a39154e115 Do not explicitly default smfs if all of them are defaulted 2020-05-11 00:00:55 +02:00
7c622a79d4 Unvirtual bunch off destructors of simple data classes 2020-05-10 23:57:05 +02:00
04cbbb8a4b Replace getLineOfChars with non-templated, runtime version 2020-05-10 23:52:38 +02:00
f64487bf70 Move LazyExpr's and MessageInfo's implementation to the combined TU
Some of the implementations were inlined instead.
2020-05-10 20:21:04 +02:00
27f1756d8e Split out LazyExpr into its own header
This enables us to avoid `catch_reporter_bases.hpp` being indirectly
dependent on `catch_tostring.hpp`, cutting apart quite a bit indirect
inclusions.
2020-05-10 20:21:02 +02:00
824ffe6525 Inline and default LazyExpression's constructor 2020-05-10 20:20:59 +02:00
d5e08a4beb Rename catch_assertionhandler -> catch_assertion_handler 2020-05-10 20:20:56 +02:00
ed967fd7fc Split MessageInfo into its own header
This is first step towards splitting apart reporter implementation
and `catch_string.hpp`.
2020-05-10 20:20:52 +02:00
7030d7740d Remove some superfluous includes from reporters 2020-05-10 20:20:47 +02:00
7efbc83ae0 Introduce combined TUs for compiling small TUs
This should improve the compilation times by decreasing the number
of TUs compiled, without making overly big TUs that would cause
problems with heavy-tailed compilation times.

There is one "combined TU" for the top level part, and each subpart,
except for Reporters, which currently do not have any trivial TUs.
2020-05-10 15:37:29 +02:00
9e498278be Move StringRef header to internals 2020-05-10 10:09:01 +02:00
14533f5bb6 Make Colour's op<< hidden friend 2020-05-10 07:32:40 +02:00
895d0a0696 Small improvements for StringRef
* `operator[]` is constexpr
* `operator<<` and `operator+=` are hidden friends
2020-05-10 07:22:11 +02:00
094d840efe Cleanup SourceLineInfo implementation
Special member functions are now implicit, which should make them
both noexcept and constexpr. The `operator<<` has been made into
hidden friend as per best practices.
2020-05-10 06:57:02 +02:00
a595066ff9 Use internal linkage for float stringification helper 2020-05-10 06:54:23 +02:00
cb25c4a8a3 Reinline some StringMaker impls 2020-05-09 21:26:42 +02:00
b93cf932fb Use UDLs to construct StringRefs for decomposed operators directly 2020-05-09 21:07:55 +02:00
eef6c9b79b Microopt: stream single char instead of single char string 2020-05-09 21:02:12 +02:00
b5a287f09f Make rest of the generators final 2020-05-09 20:56:25 +02:00
e1a0cce82b Remove Clara from external folder
As Clara is no longer maintained as a separate project, the
implementation was moved to the internal subfolder of top-level
folder. This removes one folder and avoids potential user confusion.

Also simplified the convenience header checking script accordingly.
2020-05-09 19:40:57 +02:00
75b711a360 Fix ODR violation because of name clash in matchers 2020-05-09 18:00:51 +02:00
db32550898 Remove catch_default_main.hpp
There are two reasons for this:

1) It is highly unlikely that someone has use for this header,
which has no customization points and only provides simplest
possible main, and cannot link the static library which also
provides a default main implementation.
2) It being a header was causing extra complications with
the convenience headers, and our checking script. This would either
require special handling in the checking script, or would break user's
of the main convenience header.

All in all, it is simpler and better in the long term to remove it,
than to fix its problems.
2020-05-09 18:00:49 +02:00
e78b4f6be7 Remove file list checking from CMake
I do not think we need a safeguard against not including files in
CMake anymore, and as it is, it caused annoying false positive about
the default main implementation.
2020-05-06 21:23:13 +02:00
9766a7b200 Fix installation of extra utilities 2020-05-06 21:21:08 +02:00
7c816c7c0b Upload conan releases to catch2 remote (instead of Catch2)
At some point we moved over to catch2:catchorg (notice lowercase `c`)
instead of Catch2:catchorg, but we kept uploading the released
packages to the upper-cased repository... Time to fix this, and then
merge them again.
2020-05-06 20:53:42 +02:00
04c171f91f Update CATCH_CONFIG_WINDOWS_CRTDBG docs to reflect new distribution 2020-05-06 18:17:51 +02:00
fe405034b8 Add script checking convenience header correctness 2020-05-06 17:53:39 +02:00
2ccc48e108 Require Python3 for running tests 2020-05-06 14:56:33 +02:00
6020f8f27c Add convenience headers that include all headers in a subpart
The naming scheme is simple, to include all matchers, include header
`catch2/matchers/catch_matchers_all.hpp`. To include **everything**,
include `catch2/catch_all.hpp`.
2020-05-06 11:21:46 +02:00
26622f1620 Fix Conan recipe
Apparently cpp_info.libs is a list of _filenames_ and not _targets_.
Oh well.
2020-05-03 19:34:37 +02:00
c086746cc9 Use main-as-static-lib in ExtraTests 2020-05-03 19:21:58 +02:00
0c223bb751 Decrease chance of false positive in random generator testing 2020-05-03 19:01:21 +02:00
19ecad6f68 Rename CMake/pkg-config target Catch2Main to Catch2WithMain
This describes the reality better, as it also links in the rest
of Catch2.

The on-disk name of the static library remains just `Catch2Main`,
as that is what it is -- single main function -- and on-disk artifacts
cannot describe link dependencies.
2020-05-03 18:54:38 +02:00
33c58dad41 Remove duplicated test for #1027 2020-05-03 09:58:46 +02:00
68061bbed4 Remove mentions of single header version from README 2020-05-03 09:54:18 +02:00
e83c9fb674 Run unit tests in random order 2020-05-03 09:49:59 +02:00
b8221c8350 Remove the single_include folder
It is no longer used by v3, and contains obsolete versions of the
headers anyway. There are future plans for some sort of replacement,
but those are,
1) in the future
2) different than the single header model

so we can delete the folder completely.
2020-05-03 09:34:19 +02:00
31ff89709f Pick docs for v2.12.1 2020-05-03 07:54:07 +02:00
5b8cccaf6a Add support for bitwise xor to the decomposer 2020-05-03 07:52:47 +02:00
4aefbbcd02 Pick docs for v2.12.0 2020-05-03 07:49:05 +02:00
53434a2f32 Support bitand and bitor in REQUIRE/CHECK
This means that bit-flag-like types with conversion to bool can be
asserted on, like so `REQUIRE(var & Flags::AddNewline)`.
2020-05-03 07:45:04 +02:00
2a93a65bc2 Support custom allocators in vector Matchers (#1909)
-- Combined with f4fc2dab2c
   during cherry-picking.
2020-05-03 07:40:04 +02:00
dd35430a2b Add more tests for test spec parser
Originally the tests were from #1912, but as it turned out, the issue
was somewhere else. Still, the inputs provided were interesting, so
they are now part of our test suite.
2020-05-02 18:24:07 +02:00
bbbc7a0d7f Fix CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTION interaction with feature macro 2020-05-01 20:26:40 +02:00
89fab65382 Update documentation for --order 2020-05-01 14:46:14 +02:00
1bd7cac09f Use macro to determine whether std::uncaught_exceptions is available
Catch assumes std::uncaught_exceptions is available whenever C++17 is
available, but for macOS versions older than 10.12 this is not the case.

Instead of checking the C++ version, use a macro to check whether the
feature is available.
2020-05-01 14:45:56 +02:00
9b5fc9eaea Randomize test for subset invariant random ordering of tests
Also removed the iterative checking that seeds 1-100 do not create
the same output, because it used too much runtime.
2020-05-01 14:45:36 +02:00
630ba26278 Add test for consistent random ordering
-- Fixed up during cherrypicking
2020-05-01 14:30:01 +02:00
26b2c3e7e2 Change random test shuffling technique (refactored)
Previously a random test ordering was obtained by applying std::shuffle
to the tests in declaration order.  This has two problems:

- It depends on the declaration order, so the order in which the tests
  will be run will be platform-specific.
- When trying to debug accidental inter-test dependencies, it is helpful
  to be able to find a minimal subset of tests which exhibits the issue.
  However, any change to the set of tests being run will completely
  change the test ordering, making it difficult or impossible to reduce
  the set of tests being run in any reasonably efficient manner.

Therefore, change the randomization approach to resolve both these
issues.

Generate a random value based on the user-provided RNG seed.  Convert
every test case to an integer by hashing a combination of that value
with the test name.  Sort the test cases by this integer.

The test names and RNG are platform-independent, so this should be
consistent across platforms.  Also, removing one test does not change
the integer value associated with the remaining tests, so they remain in
the same order.

To hash, use the FNV-1a hash, except with the basis being our randomly
selected value rather than the fixed basis set in the algorithm.  Cannot
use std::hash, because it is important that the result be
platform-independent.
2020-05-01 12:32:38 +02:00
87a8b61d5a Fix bug in test spec parser handling of escaping in ORed patterns
It did not clear out all of its internal state when switching from
one pattern to another, so when it should've escaped `,`, it took
its position from its position in the original user-provided string,
rather than its position in the current pattern.

Fixes #1905
2020-05-01 09:56:34 +02:00
ca27b0dcc5 Remove usage of __builtin_constant_p under IBM XL 2020-05-01 09:31:59 +02:00
87c8055176 Suppress clang-tidy warning about vararg usage in assertion macros
CATCH_INTERNAL_IGNORE_BUT_WARN() introduced with b7b346c triggers
clang-tidy warning 'cppcoreguidelines-pro-type-vararg' for every usage
of assertion macros like CHECK() and REQUIRE(). Silence it via NOLINT
in the '#if defined(__clang__)' block only, as clang-tidy honors those.
2020-05-01 09:31:48 +02:00
46cc551b7a Add vcpkg installation instructions (#1898)
* Add vcpkg installation instructions

* Add index
2020-05-01 09:31:41 +02:00
f34aacfe5f Added toml++ to opensource-users.md 2020-05-01 09:31:12 +02:00
0d3e933d71 Fix alphabetical ordering of opensource-users.md 2020-05-01 09:31:03 +02:00
02a998598c C-header updates 2020-05-01 09:30:50 +02:00
8ea45bf50c Change PARSE_CATCH_TESTS_ADD_FIXTURE_IN_TEST_NAME conditional.
When no TEST_CASE_METHOD function, there is no fixture to get.
2020-05-01 09:30:31 +02:00
beb8c3a99d Fixed typo in "benchmark name" column width calculation. Closes #1885. 2020-05-01 09:30:17 +02:00
656b15d37b Pick doc changes for v2.11.3 2020-05-01 09:28:17 +02:00
5198fd3c9a Fix compilation error when lambdas are used in assertions
This is a partial revert of b7b346c3e5.
2020-05-01 09:24:26 +02:00
08f8a81b2c Picking v2.11.2 docs changes 2020-04-30 21:44:16 +02:00
0d8eeec557 Move all scripts to Python3 2020-04-29 21:28:52 +02:00
d3c0b36487 Update wandbox script to use https endpoint and Python3 2020-04-29 21:28:34 +02:00
95a2e54702 Different approach to proper colouring of filters 2020-04-29 21:28:25 +02:00
6badd7d9ed Switch back to uncolored output after printing filters
After printing the list of filters, switch back from yellow to black
before printing a newline to avoid the remaining output to be colored in
yellow.
2020-04-29 21:28:18 +02:00
60cfaa38fb Make warnings in assertions fire for GCC/Clang again
The old code caused warnings to fire under MSVC, and Clang <3.8.
I could not find a GCC version where it worked, but I assume that it
did at some point.

This new code causes all of MSVC, GCC, Clang, in current versions,
to emit signed/unsigned comparison warning in test like this:

```cpp
TEST_CASE() {
    int32_t i = -1;
    uint32_t j = 1;
    REQUIRE(i != j);
}
```

Where previously only MSVC would emit the warning.

Fixes #1880
2020-04-29 21:27:57 +02:00
38a0dfca6d Update tutorial.md
Fix: typo; remove trailing ","
2020-04-29 21:17:54 +02:00
b014d988fe Update slow-compiles.md
Add missing "."
2020-04-29 21:17:30 +02:00
7a0f8ff4b8 Add king as user 2020-04-29 16:55:03 +02:00
efbfaa1704 Don't cast nanoseconds to integers when writing output 2020-04-29 16:54:46 +02:00
c4e5b05cfc include <iterator> in catch_stats.hpp
needed for std::back_inserter on some platforms
2020-04-29 16:54:39 +02:00
0fdeb10c65 Cleanup how configuration-dependent test macros are defined 2020-04-29 15:36:40 +02:00
783ab5ef87 Move around message macros and implementation 2020-04-29 14:02:53 +02:00
8d50f04419 Move around matcher macro implementation 2020-04-26 21:33:55 +02:00
804e2df099 Move Benchmarking macros to catch_benchmark.hpp 2020-04-26 19:24:36 +02:00
0470794a68 Split TEMPLATE_TEST* macros into their own set of files
These files are not included by the default
`#include <catch2/catch_test_macros.hpp>` path, so that users do
not have to pay for them if they do not use them. Follow up is to
split out the small part of `catch_preprocessor.hpp` used by the
default test macros (AFAIK, it is just `INTERNAL_CATCH_REMOVE_PARENS`
macro), so that it is not included by the default path either.

Also fixes #1892 by providing the missing macros.
2020-04-26 16:25:43 +02:00
5150fa4476 Rename catch_capture.hpp to more descriptive catch_test_macro_impl.hpp 2020-04-26 13:32:17 +02:00
d776a93a39 Development build defaults to off 2020-04-25 18:18:45 +02:00
c078373f3f Rename internal macro for suppression of unused variable warnings 2020-04-25 18:13:17 +02:00
517839fb3f Small cleanup of fatal condition handler 2020-04-25 14:07:50 +02:00
b955355ec4 Avoid recompiling main for each example
Instead, link against `Catch2Main` CMake target as intended.
2020-04-24 21:13:07 +02:00
c5ec936a72 Inline TagAlias constructor
It is trivial enough that the function call is not worth it, and
lets us remove one TU from compilation.
2020-04-24 21:09:10 +02:00
8d44c2450c Add some explanatory comments to ErrnoGuard 2020-04-24 21:04:54 +02:00
7c97554565 Remove Obj-C(++) support 2020-04-24 20:42:52 +02:00
e1e6872c4c Standardize header names and file locations
This is both a really big and a really small commit. It is small in
that it only contains renaming, moving and modification of include
directives caused by this.

It is really big in the obvious way of touching something like 200
files.

The new rules for naming files is simple: headers use the `.hpp`
extension. The rules for physical file layout is still kinda in
progress, but the basics are also simple:
 * Significant parts of functionality get their own subfolder
   * Benchmarking is in `catch2/benchmark`
   * Matchers are in `catch2/matchers`
   * Generators are in `catch2/generators`
   * Reporters are in `catch2/reporters`
   * Baseline testing facilities are in `catch2/`
 * Various top level folders also contain `internal` subfolder,
   with files that users probably do not want to include directly,
   at least not until they have to write something like their own
   reporter.
    * The exact files in these subfolders is likely to change later
      on

Note that while some includes were cleaned up in this commit, it
is only the low hanging fruit and further cleanup using automatic
tooling will happen later.

Also note that various include guards, copyright notices and file
headers will also be standardized later, rather than in this commit.
2020-04-24 18:58:44 +02:00
3836aa9ceb Remove catch_user_interfaces
It used to be a file that would collect interfaces we always wanted
to provide to users, so that the single header stitching script
would place them in the common part of the single header version.

As v3 is moving to separate headers model, the file is no longer
useful.
2020-04-24 15:34:12 +02:00
3f2ada03d5 Remove unused catch_impl.hpp header
It became useless when we moved away from the single-header model,
because it was used an entry point for stitching CPP into the single
header file.
2020-04-24 14:25:33 +02:00
7892954c99 Remove global warning manipulation headers
They were unused except for one test file anyway.
2020-04-05 19:14:16 +02:00
54a7eb1aed Rename reporters to use .hpp suffix
I want to standardize _all_ of Catch2 headers on the `.hpp` suffix.
2020-04-05 19:10:57 +02:00
151dccbd31 Remove catch.hpp
This was an old "include all" header, that we no longer want to be
usable, to make the include differences in new versions explicit.

We will introduce new "include all" headers later, in the form of
`catch_all.hpp`, `catch_matchers_all.hpp` and so on...
2020-03-29 21:32:13 +02:00
4d63c36402 Update release notes with matcher changes 2020-03-29 14:57:29 +02:00
a25d83d8c4 Matcher type flattening overloads of && and || are now hidden friends 2020-03-29 14:03:29 +02:00
f7d7aa9eb2 Fix and extend tests for composing generic matchers 2020-03-29 14:03:12 +02:00
ca5af2e85b Cleanup vector matchers
Once again, added doxygen comments and removed the inner-most namespace.
2020-03-28 15:17:12 +01:00
904c47a634 Cleanup string matchers
Removed nested `StdString` namespace and added Doxygen comments.
Also renamed some matchers to avoid colisions now that there are
less separate namespaces for matchers to go to. Since this is a
breaking release anyway, it shouldn't matter, and the factory
functions that the users should use remain the same anyway.
2020-03-28 15:15:31 +01:00
afc8b28c07 Cleanup for floating point matchers
Removed the `Floating` nested namespace and added Doxygen comments
for the factory methods.
2020-03-28 12:48:19 +01:00
a6baa6dda6 Cleanup in exception matchers
No more nested namespace for the matcher, and there is now a doxygen
explanatory comment on the factory function.
2020-03-28 11:06:14 +01:00
5c9367d4f1 Small cleanup for PredicateMatcher
Removed the `generic` nested namespace, so PredicateMatcher now
lives in `Catch::Matchers` namespace, just like other matchers.
Also cleaned up and doxygenized comments on the `Predicate` factory
function for `PredicateMatcher`.
2020-03-27 14:56:33 +01:00
ab0ca2f566 Rename some matcher-related files
The two changes are
`catch_matchers_templates` -> `catch_matchers_templated` and
`catch_matchers_generic` -> `catch_matchers_predicate`. The former
is mostly cosmetic, but the second was previously significantly
misleading, and as the library is now to be consumed by including
specific headers, this needed to be fixed.
2020-03-27 10:24:08 +01:00
3a3efebd16 Add IsEmpty and SizeIs matchers for ranges/containers
`SizeIs` can accept both `size_t` and a matcher. In the first case,
it checks whether the size of the range is equal to specified size.
In the second case, it checks whether the provided matcher accepts
the size of the range.
2020-03-27 10:24:08 +01:00
f52a58e857 Make concrete matchers final
Outside of `MatcherBase` and `GenericMatcherBase`, matchers are not
designed to be overriden. This means that doing so can easily lead
to errors, and matchers are generally fairly simple functionality-wise.
so there is not much code reuse to be gained anyway.

Thus, Catch2-provided concrete matchers are now final.
2020-03-27 10:22:25 +01:00
007efc173a Add generic Contains matcher
It matches a range iff the range contains a specific element,
or an element in the range matches the provided matcher.
2020-03-27 10:22:25 +01:00
89e857349b Use _t form of traits 2020-03-27 10:22:25 +01:00
c2daf468bb Standardize matcher headers to use .hpp suffix 2020-03-27 10:22:25 +01:00
64d7f9b98a New and hopefully improved documentation for matchers 2020-03-27 10:22:25 +01:00
121f04ffcf Update Catch2-provided pkg-config
Also fixes an installation issue, where some installation steps
would not properly pick up `CMAKE_INSTALL_PREFIX` when set.
2020-03-18 10:51:35 +01:00
0e7e6b210a Remove obsolete surrogate TUs
They will be returned later, but better organized and covering
_all_ headers.
2020-03-17 23:34:55 +01:00
a15ffb735d Make ListeningReporter final
In general, for Catch2 v3 we are making virtual types `final`,
unless they were explicitly designed to be derived-from.
`ListeningReporter` is definitely not designed to be derived-from.
2020-03-09 21:15:30 +01:00
727b26ab35 Remove IStreamingReporter::isMulti query
This was previously used to avoid `dynamic_cast` inside our code,
when we were creating more than one reporter, or a reporter
together with listeners. However, since then the offending code
was refactored to be smarter instead, and this query member function
is no longer needed nor used.
2020-03-09 21:07:32 +01:00
9de6eae6bb ConsoleReporter no longer creates a string when writing time units
This is minor perf improvement only, but it is free and trivial.
2020-03-08 14:04:35 +01:00
d1ffaf55a1 Fix warnings in ExtraTests and Examples 2020-02-26 16:07:54 +01:00
33b47f7309 Fix warnings from stringifying functions
The only way to stream those is to use the `bool` overload of `op<<`.
However, to convert a function to bool, GCC creates AST equivalent
of `A? true : false`. Then, because `A` is a function, it warns that
it will never be `false`. 🤦

As a bonus, newer GCC versions issue _two_ different warnings about
this, but older GCC versions do not know both of them, so we also
have to suppress warning about unknown warning suppression.
2020-02-26 16:07:41 +01:00
8d1e7ca896 Suppress unused variable warning for template test cases
The variable initialization has test registration as a side-effect,
but as far as GCC is concerned, the variable itself is unused.
Because the macro substitution always happens at global scope, we
cannot use cast to `void` as is usually done.
2020-02-26 16:07:39 +01:00
e601a5dc4f Suppress empty-variadic macros warning under GCC 2020-02-26 16:07:36 +01:00
e9caeb7d0b Fix Wparentheses for GCC
Sadly most versions still cannot properly handle the suppression
via `_Pragma`, so it has to leak to the users when they use older
GCC versions to compile their code
2020-02-26 16:07:34 +01:00
6e270958a2 Add development build option to CMake and enable it on CI
Development build enables warnings and and `Werror` or equivalent.
2020-02-26 16:07:26 +01:00
50b2cfa5de Fix -Wmissing-braces in new matchers 2020-02-21 23:10:34 +01:00
34e7a5e0cf Bunch of warning fixes 2020-02-21 23:10:22 +01:00
04f18d996b Suppress global constructor warning in benchmarking 2020-02-21 23:10:12 +01:00
3bb9fcd916 Remove trailing semi in REGISTER_TEST_CASE 2020-02-21 23:10:01 +01:00
c3013a6251 Move matcher implementation to their own subfolder
In the future we can expect many more matchers, so let's give them
a place to live.

Also moved matcher-related internal files to `internal` subfolder.
Ideally we should sort out all of our source code, but that will
have to come later.
2020-02-20 17:42:11 +01:00
40e35d4318 Reorganize base headers for matchers 2020-02-20 13:39:04 +01:00
b83a12b12c Replace enable_if with enable_if_t 2020-02-20 13:03:38 +01:00
d33af93e17 Cleanup visibility in generic not matcher 2020-02-20 13:03:36 +01:00
25c5ae240c Disable copies on generic matcher combinators 2020-02-20 13:03:34 +01:00
260263b9bf Combined matchers are now final 2020-02-20 13:03:32 +01:00
cf6575576f Start fixing up Matchers: namespaces, composition ops
This commit also forbids composing lvalues of composed matchers, as
per previous deprecation notice. I do not expect this to be contentious
in practice, because there was a bug in that usage for years, and
nobody complained.
2020-02-20 13:03:30 +01:00
a1be19aa1b Remove the separately compiled test for Benchmarking Macros
Thanks to the changes to compilation model, the tests for benchmarking
macros have been made part of the normal test run. This means that
the only purpose these separately compiled tests served was to waste
CI time.
2020-02-18 13:48:34 +01:00
c745adb81c Temporarily disable the test for DebugBreakMacro
The change to static library means that the new implementation needs
to be somewhat different, and I do not want to fix it right now.
2020-02-18 13:43:37 +01:00
06c135706e Move X12-CustomDebugBreakMacro.cpp to the right place
This is a small fixup because cherry-picking the relevant commit
threw it into the wrong place...
2020-02-18 10:20:37 +01:00
ae1d21315c Specialize CATCH_TRAP() for iOS + thumb instruction set combo
Fixes #1862
2020-02-16 16:11:29 +01:00
6a2c025bfc Add command line option 'never' to --wait-for-keypress (#1866)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-16 16:11:11 +01:00
2441c2faab stdio.h replaced with cstdio. 2020-02-16 16:05:30 +01:00
442283ee11 Add link to list of commercial user projects 2020-02-16 16:05:18 +01:00
3f81dd753a Fix typo in link in docs 2020-02-16 16:05:08 +01:00
f8794634c2 Fix significant bug with storing composed matchers
Given that in the 2 or so years that matchers are thing nobody complained,
it seems that people do not actually write this sort of code, and the
possibility will be removed in v3. However, to avoid correctness bugs,
we will have to support this weird code in v2.
2020-02-16 16:02:31 +01:00
d6b2a3793b Check for Windows instead of WIN32 for wmain entry point
Closes #1849
2020-02-16 15:54:24 +01:00
548de655fd Cleanup nextafter workaround 2020-02-16 15:52:14 +01:00
89f18f15ca Add a test for custom debug break macros
See #1846
2020-02-16 15:47:00 +01:00
3c7e737a7b Allow configuring of benchmark warmup time 2020-02-16 15:44:23 +01:00
e880da93bd Make CATCH_BREAK_INTO_DEBUGGER be user-configurable 2020-02-16 15:27:21 +01:00
3e01d4b239 catch_compiler_capabilities.h: use proper math define
C++11 math requires _GLIBCXX_USE_C99_MATH_TR1 to be true with gcc/clang.

Also fixes an issue with uClibc-ng where __UCLIBC__ is defined in features.h but
that is not included here and is thus no-op.
2020-02-16 15:27:04 +01:00
06c32862b3 Some refactorings:
- Overrides added
 - usages of push_back() replaced with emplace_back()
 - Loop variable made const-refernce
 - NULL replaced with nullptr
 - Names used in the declaration and definition unified
 - size() replaced with empty
 - Identical cases merged
2020-02-16 15:25:23 +01:00
ab520f4e97 Removed unneeded 'using uchar = unsigned char' 2020-02-16 15:12:07 +01:00
32617f42d0 Cherry-pick release notes for v2.11.1 2020-02-16 15:11:30 +01:00
17c4b2d093 Feature: generic matchers (#1843)
This commit extends the Matchers feature with the ability to have type-independent (e.g. templated) matchers. This is done by adding a new base type that Matchers can extend, `MatcherGenericBase`, and overloads of operators `!`, `&&` and `||` that handle matchers extending `MatcherGenericBase` in a special manner.

These new matchers can also take their arguments as values and non-const references.

Closes #1307 
Closes #1553 
Closes #1554 

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-16 11:19:10 +01:00
db1a0465dc Outline GeneratorException from generators header 2020-02-14 16:15:47 +01:00
b2a6523d85 Fix Clang-3.8 compilation of tests
As far as I know, the compilation error is a compiler bug, but I
do not want to just drop the support for something that is trivial
to work around.
2020-02-13 16:57:37 +01:00
b009d190bf Avoid detecting Clang as having MSVC's traditional preprocessor
Fixes #1806
2020-02-13 16:28:30 +01:00
ac83087bc2 catch_debugger.h - implement break into debugger assembler instructions for iOS 2020-02-13 16:28:02 +01:00
123b449f8d catch_console_colour.cpp - adjust useColourOnPlatform for iOS 2020-02-13 16:27:56 +01:00
6ad743a62b catch_debugger.cpp - debugger detection is identical on Mac OS X and iOS 2020-02-13 16:27:46 +01:00
0f47fe16bd Provide const overload of ObjectStorage::stored_object()
Fixes #1820
2020-02-13 16:26:06 +01:00
82baef62e2 Fix forwarding in SingleValueGenerator and generator creation
Fixes #1809
2020-02-13 16:19:50 +01:00
0fbf4f3e15 Fix wrong namespacing of benchmarking constructor helpers 2020-02-13 16:15:50 +01:00
ad3f50bbc1 Suppress using-namespace lint in GENERATE* macros
Closes #1799
2020-02-13 15:31:09 +01:00
13e01d273a Cherry pick v2.11.0 release notes and docs 2020-02-13 15:30:46 +01:00
2788897051 Minor cleanup in the benchmarking tests 2020-02-13 15:12:57 +01:00
2945b80f61 Add more constexpr to StringRef 2020-02-13 15:01:03 +01:00
63b7d6f98e Improve erasure of test numbers in TAP approvals 2020-02-13 14:22:18 +01:00
c50ba09cde Split [.foo] into [.][foo] when parsing test specs
b77cec05c0 fixed this problem for tagging tests, so that a test
case tagged with `[.foo]` would be parsed as tagged with `[.][foo]`.
This does the same for the test spec parsing.

Fixes #1798
2020-02-13 13:35:10 +01:00
c165bd15c5 Forbid copying ReusableStringStream
Copying a `ReusableStringStream` would lead to "double free" of
the stream, and thus it could be used in multiple places at the
same time, breaking the output.
2020-02-12 19:21:15 +01:00
4f0de7bbad Fix error when period of steady_clock is not nano
On systems where std::chrono::steady_clock::period is not std::nano, benchmark tests fail to compile due to trying to convert analysis.samples from a vector of duration<double, clock::period> to a vector of std::chrono::duration<double, std::nano>.
2020-02-12 19:20:41 +01:00
21b24e8326 Remove catch_external_interfaces.h
Its intent was to show which headers are expected to be useable by
Catch2's users, and to enforce their inclusion in the single header
distribution at the right place.

Given the new library model, the second use case is not needed and
the first one is better served with documentation and physical file
layout.
2020-02-12 16:57:15 +01:00
0b2874b6b1 Move SonarQube reporter implementation to a cpp file 2020-02-07 11:26:33 +01:00
e6ea53ab49 Move Automake reporter to a cpp file 2020-02-07 11:26:30 +01:00
338572a4f7 Move TAP reporter implementation to a cpp file 2020-02-07 11:26:28 +01:00
70836d49ba Move include of catch_enforce out of the reporter bases 2020-02-07 11:26:24 +01:00
db148c42d7 Move TeamCity implementation to a cpp file 2020-02-07 11:26:21 +01:00
cd7d7a1c67 Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle
Now that Catch2 is a proper library, we can always build the full
library (comparatively minor slowdown) and the user can avoid
including benchmarking headers to avoid the compilation slowdown.
2020-02-06 11:36:46 +01:00
86e19b952d NoAssertions runs only 1 test case instead of all of them 2020-02-04 10:11:55 +01:00
bce5b364d3 Unconditionally provide <chrono> StringMakers 2020-02-03 20:53:36 +01:00
34bc56340d Normalize TAP approvals to avoid massive diffs for every change 2020-02-03 20:29:36 +01:00
c3a5e21648 Move Approx out of the Detail namespace 2020-02-03 15:14:59 +01:00
bd9520c0f9 GCC pragma fixed. 2020-02-03 09:27:34 +01:00
a3ffc20f57 Provide CTest a hint on test costs (and thus ordering)
When running tests in parallel, CTest runs the tests in decreasing
order of cost (time required), to get the largest speed up from
parallelism. However, the initial cost estimates for all tests are
0, and they are only updated after a test run. This works on a dev
machine, where the tests are ran over and over again, because
eventually the estimates become quite precise, but CI always does
a clean build with 0 estimates.

Because we have 2 slow tests, we want them to run first to avoid
losing parallelism. To do this, we provide them with a cost estimate
manually.
2020-02-03 09:07:23 +01:00
b86ab20154 OpenCppCoverage skips coverage of non-source directories
Previously, we would collect coverage data for all source files in
Catch2's directory, including tests and examples, and we would then
ask codecov.io to ignore those. With this change, OpenCppCoverage
only collects coverage data for source files in the `src/` directory.
This cuts the size of the coverage report in half, and also speeds
up the coverage collection.
2020-02-02 23:15:22 +01:00
1327946785 Add all in-repo reporters to approval tests 2020-02-02 15:04:19 +01:00
a49ab0a162 Do not count test files into coverage 2020-02-02 14:51:41 +01:00
3b297cf9b5 Outline Config destructor to avoid including superfluous header 2020-02-02 14:43:26 +01:00
66fe591477 Replace C++11 use of ::type with the _t suffix for std traits 2020-02-02 12:59:19 +01:00
ea6db67063 Use std::make_unique instead of our polyfill or naked new
The use we previously used the polyfill or naked new is that we
supported C++11, which did not yet have `std::make_unique`. However,
with the move to C++14 as the minimum, `std::make_unique` can be
expected to be always available.
2020-02-01 23:34:00 +01:00
a7b3e087a0 Fix coverage collection on AppVeyor 2020-01-31 14:44:40 +01:00
ddd0e7218d Remove the !hide special tag 2020-01-28 20:48:32 +01:00
49e000b505 [conan] Fix the Conan package for multiple static library use 2020-01-25 22:39:50 +01:00
2e1ce37faa Try uploading in-development versions of v3 as testing packages 2020-01-25 21:09:48 +01:00
d0257fc1ff Do not use shared_ptrs for filters and patterns 2020-01-25 20:39:17 +01:00
df2379218b Reduce the use of shared_ptrs for various Config objects
Ideally they would not be used at all, but the main config lifetime
is a weird mess right now and will require further refactoring.
2020-01-25 17:41:38 +01:00
7134ad9913 Replace shared_ptr with unique_ptr for reporters 2020-01-25 14:07:29 +01:00
827733fe81 Improve approval tests handling off tagged devel versions 2020-01-25 11:23:40 +01:00
2f4a7dda68 Have the in-development version report as v3.0.0
Because some of the tooling used by Catch2 does not properly support
version postfixes, such as `preview-1`, we will report the
in-development version is `v3.0.0`, and the first real release will
have to be `v3.0.1`.

Closes #1824
2020-01-25 09:15:46 +01:00
6c3a5ef625 Remove CATCH_CONFIG_DISABLE_MATCHERS
Now that the recommended distribution and usage method is proper
library, users can just avoid including the matcher headers to get
basically the same effect.
2020-01-25 09:07:36 +01:00
c770a9c8b5 Conan topic cleanup. 2020-01-24 15:10:43 +01:00
d63681f707 Remove testing of CATCH_CONFIG_FALLBACK_STRINGIFIER from SelfTest
This removes a potential ODR violation and the configuration option
is tested in its own separate binary anyway.
2020-01-21 15:14:24 +01:00
2b696c4388 Piecemeal includes in extra-tests 2020-01-21 15:03:07 +01:00
17281c09c3 Convert examples to piecemeal includes 2020-01-21 14:46:07 +01:00
26f78f96aa Start using piecemeal includes in test files 2020-01-21 10:03:54 +01:00
c381b49c60 Split the test macros into their own file 2020-01-19 16:01:50 +01:00
acf975cab1 Separate version macros to its own header 2020-01-19 15:39:11 +01:00
ec7280379e Adjust release scripts for v3 2020-01-19 15:39:11 +01:00
21868deeab Conanfile updated to build the static libraries. 2020-01-19 14:59:33 +01:00
4005d87460 Use proper linkage between Catch2Main and Catch2 2019-12-18 20:14:34 +01:00
0dc30e51c0 Update codecov ignorelist 2019-12-18 18:04:31 +01:00
0c62a50392 Add the catch_main source file 2019-12-18 18:02:57 +01:00
68cf4ca883 Add back coverage CMake option 2019-12-18 17:31:10 +01:00
9c07e2a416 Add test for tag aliases 2019-12-18 17:30:41 +01:00
a4c31ecd16 Small CMakeLists cleanup 2019-12-18 17:30:08 +01:00
1cc05122d7 Add Catch2Main CMake target that contains test main 2019-12-18 17:25:04 +01:00
add7068f21 Fixup installation 2019-12-16 22:33:56 +01:00
ebeeaaeec6 Reenable appveyor for CI 2019-12-16 00:15:30 +01:00
69bd213c40 Reenable travis for CI 2019-12-16 00:15:10 +01:00
5fbf04cd59 Redo how the separate compilation tests are handled 2019-12-15 20:33:39 +01:00
8b42acc328 Reintegrate extra tests 2019-12-09 10:04:26 +01:00
29b441949c Port the last example 2019-12-08 20:58:52 +01:00
70ef2f7f12 Update building of examples for static lib 2019-12-08 15:55:04 +01:00
248f922465 Cleanup impl and main from the main header 2019-12-08 11:31:47 +01:00
91ee07e08c Moved scripts/ to tools/scripts/ 2019-12-06 11:53:31 +01:00
6eb04667ad Move misc/ to tools/misc 2019-12-06 11:40:53 +01:00
604ededf77 Move artwork/ to data/artwork/ 2019-12-06 11:36:12 +01:00
918aa327fa Rename contrib/ to extras/ 2019-12-05 16:24:57 +01:00
0fea081ad1 Move tests from projects/ to tests/ 2019-12-05 16:00:20 +01:00
90e2549cec Rewrite includes in tests to use library's include paths 2019-12-02 12:23:10 +01:00
82cc5bc034 Rewrite includes inside the library to use <catch2/> form 2019-12-02 12:04:55 +01:00
fef4f217b1 Unit tests build and run (todo: projects, includes, extras) 2019-11-30 20:45:34 +01:00
e341b11467 WIP: CMake changes to build static lib + tests separately 2019-11-30 17:41:41 +01:00
443fa0fc88 Fixing paths in SelfTest 2019-11-30 17:41:17 +01:00
4385951a55 Move files into catch2/ subdir 2019-11-30 17:30:32 +01:00
7c6de33977 Fixed up include paths 2019-11-30 17:13:22 +01:00
e1a43b5b0f Move files from internal/ up a level 2019-11-30 12:23:37 +01:00
8e56b8b0ec Rename include/ to src/ 2019-11-30 12:17:16 +01:00
6923a168a1 Remove unused header 2019-11-28 13:23:48 +01:00
c9067b2253 Refactor SectionInfo constructors 2019-11-16 21:53:32 +01:00
d36c15c3ca Store tags in one big pre-allocated string and only work with refs
This should decrease the number of allocations before main is entered
significantly, but complicates the code somewhat in return.

Assuming I used `massif` right, doing just `SelfTest --list-tests`
went from 929 allocations at "Remove gcc-4.9 from the travis builds"
(2 commits up), to 614 allocations with this commit.
2019-11-14 10:53:12 +01:00
302e2c0b06 Do not copy around TestCaseInfo
Now a `TEST_CASE` macro should create a single TestCaseInfo and then
it should never be copied around. This, together with latter changes,
should significantly decrease the number of allocations made before
`main` is even entered.
2019-11-14 10:52:34 +01:00
019b0a0fe0 Use Python3.5 (well, PIP) on AppVeyor to solve deprecation warnings 2019-11-13 20:59:19 +01:00
9ff2b81164 Remove gcc-4.9 from the travis builds
According to CMake, it does not support templated variables. Strictly
speaking, we added that to the target so that the project defaults
to C++14, but I am planning to use them inside Catch2 anyway.
2019-11-07 15:58:37 +01:00
d6f7f1fbed Set CMakeLists to default to C++14 for our internal projects 2019-11-07 13:02:43 +01:00
6ddd84a67a Re-inline NameAndTags constructor 2019-11-07 12:45:16 +01:00
09b66ccfde Remove unused method on TestCase 2019-11-05 10:45:59 +01:00
d1cb727e85 Remove workaround for GCC 4.8, as it is no longer supported
Closes #1442
2019-11-05 10:45:57 +01:00
e6b9b854b5 Change how and when anonymous test cases are named 2019-11-04 13:45:39 +01:00
478c324534 Redo AppVeyor build matrix
Added VS 2019 builds, C++17/C++20 builds and performed some general
cleanup.
2019-11-04 13:12:59 +01:00
eabb8a6af7 Cleanup tests for C++20 2019-11-04 13:08:10 +01:00
ae10e4ef72 Sections no longer save their description
Users can still write a description for their sections, but it will
no longer be saved as part of the `SectionInfo` struct. This ability
has also been added to the documentation.

Closes #1319
2019-11-03 23:13:25 +01:00
2c6ace04a7 Redo build matrix on .travis for C++14 and up
* Use Xenial as the base distribution
* Remove C++11 builds
* Add a lot more C++14 builds
* Add some C++17 builds
* Include newer versions of Clang and GCC
2019-10-31 16:19:56 +01:00
7c48ea6016 Do not introduce VTable when using the NonCopyable helper 2019-10-30 18:51:14 +01:00
ca4c6218d4 Remove SourceLineInfo::empty() 2019-10-30 18:01:38 +01:00
ac4958395c Rejigger OSX images on Travis 2019-10-29 23:32:24 +01:00
316a5c0572 Remove type erasure in predicate matcher
Now the type of the predicate is part of the type of the
PredicateMatcher.
2019-10-29 23:28:08 +01:00
c781301cd4 Stop support for gcc 4.8, XCode 7.3 and VS 2015 2019-10-29 14:07:18 +01:00
c46deee024 Untemplatize reporter bases 2019-10-29 14:07:18 +01:00
4f47d1c6c1 Hidden tests now require positive filter match to be selected
This also required some refactoring of how the pattern matching
works. This means that the concepts of include and exclude patterns
are no longer unified, with exclusion patterns working as just
negation of an inclusion patterns (which led to including hidden
tags by default, as they did not match the exclusion), but rather
both include and exclude patterns are handled separately.

The new logic is that given a filter and a test case, the test
case must match _all_ include patterns and _no_ exclude patterns
to be included by the filter. Furthermore, if the test case is
hidden, then the filter must have at least one include pattern
for the test case to be used.

Closes #1184
2019-10-29 14:07:18 +01:00
2bcf1b3db6 Remove support for test case descriptions
Closes #1189
2019-10-29 14:07:18 +01:00
be44cfa63b Specialize XML reporter's --list output 2019-10-29 14:07:18 +01:00
85b129c741 Pipe --list* through reporters
This allows us to provide machine-readable listings through the
XMLReporter and it will also allow users to define their own listing
format that does whatever their own tools need.
2019-10-29 14:07:18 +01:00
0f39438aae Stop checking verbosities up-front 2019-10-29 14:07:18 +01:00
c582e351ce Remove --list-test-names-only
People should use verbosity modifiers with `--list-tests` instead.

Closes #1190
2019-10-29 14:07:18 +01:00
d2cddfc9c5 Make --list-* exit code be 0
Previously it returned the sum of listed things because ???. This
was completely useless and in many ways actively counterproductive
because of the success/failure conventions around exit codes.

Closes #1410
2019-10-29 14:07:18 +01:00
557b336125 Remove superfluous semicolon in the INFO macro
Closes #1456
2019-10-29 14:07:18 +01:00
141761745a Remove ANON_TEST_CASE
It can be fully replaced with `TEST_CASE` without any arguments,
so there is no need to keep it around.

Closes #1220
2019-10-29 14:07:18 +01:00
3136c4fb6a Refactored XMLWriter to provide finer-grained control over formatting 2019-10-29 13:59:18 +01:00
74e0e737a6 Remove useless test 2019-10-29 11:44:24 +01:00
0685216175 Mark type erasure in PredicateFunction as deprecated 2019-10-28 15:15:13 +01:00
fc320f6b8f Extract FunctionReturnType to catch_meta.hpp 2019-10-28 15:15:13 +01:00
5290d4bedc Merge pull request #1791 from catchorg/dev-reorganize-warning-suppression
Rework how warning suppression in macros is done
2019-10-28 13:13:23 +01:00
7ada02e21e Avoid technically UB type punning when determining endianness 2019-10-27 22:07:10 +01:00
849f2848bd Rework how warning suppression in macros is done
Previously, each warning suppression was self-contained, with its
own pair of `SUPPRESS_X_WARNING` and `UNSUPPRESS_X_WARNING` macros.
This had the obvious advantage of being self-containing, but it
also meant that if we needed to suppress more than one warning
in a single place, then we would manipulate the compiler's warning
state multiple times, even though logically we would only need one
layer.

The new way of suppressing warnings in macros is to push compiler's
warning state with `CATCH_INTERNAL_START_WARNINGS_SUPPRESSION` macro,
then disable whatever macros we need with the
`CATCH_INTERNAL_SUPPRESS_X_WARNINGS` macro, and then return to the
previous state using `CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION`.
2019-10-27 21:07:21 +01:00
2fbd66c51c Improve JUnit failure report
The JUnit report is improved in that:
* The message shows the testing condition, not the result
* The actual message has similar output than the console one
2019-10-27 18:02:14 +01:00
51b29ced1a Add SonarQube Generic Test Data reporter
It outputs reports in the `Generic Execution Test Data` format, see
https://docs.sonarqube.org/latest/analysis/generic-test/, specifically
https://docs.sonarqube.org/latest/analysis/generic-test/#header-2

Close #1738 (this is a cherry-pick and fixup of that PR)
2019-10-27 10:02:41 +01:00
9a558171d8 Merge pull request #1776 from mvd4/bugfix/enable-floating-point-ranges
Enable floating point ranges
2019-10-27 09:17:15 +01:00
c5c688820c Document that matcher combinators (&&, ||, !) do not take ownership
Closes #1781
2019-10-26 21:07:38 +02:00
6a08225863 Add documentation for floating point range generator 2019-10-26 20:52:40 +02:00
4327baba40 Enable range generator for floating point types 2019-10-26 20:52:09 +02:00
50cc14c94c Rework StringRef interface and internals
Now it no longer tries to be this weird hybrid between an owning
and non-owning reference, and is only ever non-owning. This is also
reflected in its interface, for example `StringRef::isNullTerminated`
is now public, and `StringRef::c_str()` has the precondition that it
is true.

Overview of the changes:
* The `StringRef::m_data` member has been completely removed, as it
had no more uses.
* `StringRef::isSubstring()` has been made public and renamed to
`StringRef::isNullTerminated()`, so that the name reflects what the
method actually does.
* `StringRef::currentData()` has been renamed to `StringRef::data()`,
to be in line with common C++ containers and container-alikes.
* `StringRef::c_str()` will no longer silently make copies. It instead
has a precondition that `isNullTerminated()` is true.
* If the user needs a null-terminated string, they should use the
`std::string` conversion operator and call `c_str()` on the resulting
`std::string`.
* Some small optimizations in various places.
* Basic functionality is now `constexpr`.
2019-10-25 13:57:52 +02:00
87b745da66 v2.10.2 2019-10-24 18:41:25 +02:00
7d0b205564 Prevent warning suppression from leaking when registering a listener 2019-10-22 00:10:01 +02:00
8fb1219013 docs: command-line: Add example to specify tests by file name. 2019-10-21 23:51:33 +02:00
23c80bcc92 Provide workaround for platforms where INFINITY is double
Fixes #1782
2019-10-21 18:33:26 +02:00
a2c8dce85c v2.10.1 2019-10-20 21:03:22 +02:00
1e379de9d7 Fix "ldd" -> "lld" typo in docs 2019-10-20 20:57:55 +02:00
4eea438b73 Update updateWandbox script to account for the new Wandbox API response 2019-10-20 20:56:20 +02:00
407ee0af2f Add a section on slow linking under MinGW to "Known Limitations" docs
Unless someone steps up to fix the long link times with a set of
unobtrusive changes, the recommended solution will remain "use a better
linker".

Related to #1205, #1247, and #1637

Closes #1247
Closes #1637
2019-10-20 19:52:39 +02:00
060a41ec7b Suppress false positive from clang-analyzer
Fixes issue #1230
2019-10-20 17:27:09 +02:00
90825a4f7a Add more tests for reading test specs from file
Related to #1770
2019-10-20 15:14:50 +02:00
9e8ae7d470 Use scientific notation for the WithinULP matcher
This should now properly handle small numbers which would previously
output something like `[0.00000000000000019, 0.00000000000000019]`,
which does not allow user to read the numbers properly.

Closes #1760
2019-10-20 12:30:21 +02:00
84856844e1 Fixes #1766: Catch terminates when parsing invalid test name 2019-10-19 21:14:06 +02:00
01ef7076f5 Allow in-tree builds where Catch2 is just a subproject
Closes #1773
Closes #1774
2019-10-18 18:49:44 +02:00
ae14a47360 TemplateTests: suppress -Wunused-template warning in template test cases
this warning was introduced by rework to support NTTPs
since we have implementation macro for NTTPs and normal template test cases
warning is going to be suppressed

Fixes #1762
2019-10-18 18:38:39 +02:00
f2b23db6d1 TemplateTests: fix compilation with ICC
ICC in some cases fails on trailing return type with decltype

Closes #1748
2019-10-18 12:35:09 +02:00
1aa98c76ac add a note how to run selftests using multiconfig generators 2019-10-17 20:23:51 +02:00
3195c242c2 Remove JSON library from users 2019-10-17 20:20:02 +02:00
31906d83ec Add parenthesis to prevent macro expansions of min/max
Closes #1772
2019-10-17 16:40:37 +02:00
91fa55303b Add test for including unguarded windows.h
If you do this, you are wrong, but apparently people expect libraries
to work around intrusive lower cased macros. Oh well.
2019-10-17 11:15:42 +02:00
7c9f92bc1c v2.10.0 2019-10-13 23:44:18 +02:00
a92a7d0229 Rewrite documentation for floating point matchers 2019-10-13 21:31:48 +02:00
e4d61e4cd8 Fix baselines 2019-10-13 21:26:51 +02:00
9ba48e2c9b Remove superfluous includes in exception matchers 2019-10-13 20:49:36 +02:00
2cc0c71856 Add a matcher that checks exception's message
Only works for exceptions that publicly derive from `std::exception`
and the matching is done exactly, including case and whitespace.

Closes #1649
Closes #1728

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch master
# Your branch is up-to-date with 'origin/master'.
#
# Changes to be committed:
#	modified:   ../docs/matchers.md
#	modified:   ../include/internal/catch_capture_matchers.h
#	modified:   ../projects/CMakeLists.txt
#	modified:   ../projects/SelfTest/Baselines/compact.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/console.std.approved.txt
#	modified:   ../projects/SelfTest/Baselines/console.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/junit.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/xml.sw.approved.txt
#	modified:   ../projects/SelfTest/UsageTests/Matchers.tests.cpp
#
# Untracked files:
#	./
#	../clang-full/
#	../clang-test/
#	../clang10-build/
#	../coverage-build/
#	../gcc-build/
#	../gcc-full/
#	../include/internal/catch_matchers_exception.cpp
#	../include/internal/catch_matchers_exception.hpp
#	../misc-build/
#	../msvc-sln/
#	../notes.txt
#	../test-install/
#
2019-10-13 20:37:07 +02:00
28663fb959 Use the right overload of std::nextafter in tests 2019-10-13 13:36:22 +02:00
d2d418a9cb Add a Relative Comparison matcher for floating point numbers
It checks Knuth's _close enough with tolerance_ relationship, that
is `|lhs - rhs| <= epsilon * max(|lhs|, |rhs|)`, rather then the
_very close with tolerance_ relationship that can be written down as
`|lhs - rhs| <= epsilon * min(|lhs|, |rhs|)`.

This is because it is the more common model around the internet, and
as such is likely to be less surprising to the users. In the future
we might want to provide the other model as well.

Closes #1746
2019-10-13 11:56:50 +02:00
c8db4e77c4 Add a from_range(Container) overload to the generator helpers 2019-10-09 14:51:36 +02:00
1c5749669e Merge pull request #1769 from amitherman95/amitherman-1767
Patch:Failure to pass special chars through input file
2019-10-09 14:16:50 +02:00
3109add95c Add tests for the -f/--input-file command line argument 2019-10-09 12:54:13 +02:00
adb4789136 Patch:Failure to apply filters through external file 2019-10-09 11:43:06 +02:00
75200e199e Merge pull request #1768 from RT222/master
Simplified benchmark output with --benchmark-no-analysis
2019-10-08 21:07:35 +02:00
a5a22cdadb Have the random generators use the global rng instance
This means that if you nest multiple random generators inside one
test case, they will not return the same sequence of numbers.

Idea taken from #1736 by Amit Herman.

Closes #1736
Closes #1734
2019-10-07 21:53:07 +02:00
535da5c513 Introduce Catch's own RNG based on the PCG family of RNGs
In the future, we will also want to introduce our own
`uniform_int_distribution` and `uniform_real_distribution` to get
repeatable test runs across different platforms.
2019-10-07 19:56:23 +02:00
2331249a8d Simplified benchmark output with --benchmark-no-analysis
Removed unused informations from benchmarks output when the --benchmark-no-analysis flag is used, making the results easier to read.
2019-10-07 15:40:00 +02:00
319cb9e1da Add a generator that takes an iterator pair 2019-10-06 13:55:10 +02:00
b8b765d55e Merge pull request #1765 from kevingranade/patch-1
Add Cataclysm:Dark Days Ahead to project list
2019-10-06 12:41:46 +02:00
a0ebd63806 Add Cataclysm:Dark Days Ahead to project list 2019-10-05 20:18:31 -07:00
4bd2c3ad6a Fix compilation error with CATCH_CONFIG_GLOBAL_NEXTAFTER
Wrong nesting of namespaces resulted in the `Catch` namespace
being ambigous between `::Catch` and `::{anon}::Catch` namespaces.
This should fix it.

Closes #1761
2019-10-05 21:03:46 +02:00
c38a5caa2e Allow full range of target ULP values for the ULPMatcher
Previously it was limited to roughly 2 billion ULPs, rather than
the roughly 2^64 possible ones.
2019-10-05 13:38:22 +02:00
ebc5609484 Add test for INF == Approx(1) 2019-10-04 14:19:39 +02:00
fcda35f645 update name of Value Generators in doc to match class names 2019-10-04 13:31:37 +02:00
02ee130bd0 Special case Approx(inf) to better follow user expectations 2019-10-04 13:28:43 +02:00
815f99541d Do not explicitly install conan
Instead, let it be installed as a dependency of `conan-package-tools`
to avoid trouble with the fact that pip is really bad at version
resolution, and that up-to-date version of the `conan` package is not
supported by up-to-date version of the `conan-package-tools` package.
2019-10-03 00:00:17 +02:00
da0062f7c1 Merge pull request #1752 from amitherman95/amitherman-1737
Escape characters bug patch(v2.9.2)
2019-09-22 10:48:08 +02:00
de42f8a93e Fix escaping of quotes in coverage helper 2019-09-21 23:19:45 +02:00
af84f1350e Add documentation for the table generator helper 2019-09-21 01:00:53 +02:00
fc2066bf18 Add tests for escaping special characters in CLI 2019-09-21 00:50:17 +02:00
2bcff9dd35 Escape characters bug patch(v2.9.2) 2019-09-21 00:46:58 +02:00
3beccfb429 Remove the no longer used is_unique type trait
It was used in checking that types in TEMPLATE_TEST_CASE and friends
were unique, but this was removed for v2.8.0 (#1628). Since there
are no further uses of this trait, the simplest thing to do is to
just remove it.

Fixes #1757
2019-09-20 23:28:19 +02:00
af8b2538a6 Ignore leading/trailing whitespace in test/section specs
The leading/trailing whitespace is problematic because of e.g.
`WHEN` macro having preceeding whitespace for alignment, and it is
generally messy.

Credits to Phil who did lot of the original work.

Closes #1708
2019-09-09 14:28:11 +02:00
a156440b19 Reinline some StringRef methods with trivial bodies 2019-09-08 21:08:37 +02:00
dab0296b64 Add test for past-the-end substr on StringRef 2019-09-08 21:07:21 +02:00
9f4c4777a5 Remove (mostly) unused overloads of StringRef operator + 2019-09-08 21:01:33 +02:00
293012a002 Inline SourceLineInfo::empty definition 2019-09-08 18:20:49 +02:00
e2b3443fe7 Do not blindly access empty StringRefs 2019-09-08 17:44:56 +02:00
7b865daccc Make StringRef's operator std::string explicit
This way it is explicit when there is a `StringRef` -> `std::string`
conversion and makes it easier to look for allocations that could
be avoided.

Doing this has already removed one allocation per registered test
case, as there was a completely pointless `StringRef` -> `std::string`
conversion when parsing tags of a test case.
2019-09-08 15:58:10 +02:00
14362533bb Check for empty expression properly
The old code was a left-over from the times when the
`capturedExpression` member was a `const char*`, which could always
be indexed. With the change to use `StringRef`, blindly indexing 0th
element is invalid, as it is not indexable part of a StringRef.
2019-09-08 14:35:01 +02:00
a5bb3e3d91 Small updates to include directives (#1726)
Fixed some inconsistencies in include directives.
2019-09-08 14:25:23 +02:00
923db16322 Use StringRefs through the enum registration 2019-09-08 14:14:46 +02:00
fbbaadb704 Use StringRef literal for unknown enum values instead of string lit 2019-09-08 14:14:14 +02:00
dd1f0f1c72 Add test for --filename-as-tags 2019-09-07 20:43:14 +02:00
d27d580d0b Add test for --libidentify command line flag 2019-09-07 20:36:12 +02:00
6da00c1b64 Split out the rest of string manipulation tests 2019-09-07 20:22:36 +02:00
fe967b1f41 Remove the ill-considered StringRef::numberOfCharacters
It never counted characters, only codepoints. If you think these
are interchangeable, you should not touch non-ascii text.
2019-09-07 20:13:22 +02:00
f2c2711bdc Add trim for StringRef 2019-09-07 11:31:00 +02:00
b77ab74b72 Start adding tests for string manip functions 2019-09-06 18:50:57 +02:00
4038ee6bc6 Stop compilation if registering an enum could truncate its value
We probably want to use a bigger type (e.g. `int64_t`) later on,
but the is a mess right now anyway.
2019-09-06 18:36:59 +02:00
789f3591ef Do not write singular characters into stream as C-strings 2019-09-06 18:30:16 +02:00
6e8d769775 Unify handling of __has_include checks 2019-09-06 13:38:58 +02:00
1189a73be2 Do not assume that version placeholders contain ")" 2019-09-06 13:25:39 +02:00
071bacad5e Properly linkthrough additions to the configuration docs 2019-09-06 13:24:51 +02:00
addf799040 Workaround missing std::nextafter in uClibc
Luckily, the rest of C++11 features seem to be supported...

Closes #1739
2019-09-06 13:08:44 +02:00
155274f0df Add disabling the use of Android's logging at compile time
This is done via the new compile time toggle,
`CATCH_CONFIG_ANDROID_LOGWRITE`.

Closes #1743
2019-09-06 12:44:06 +02:00
18d597cf10 Allow to use non-copyable and non-movable types in TEMPLATE_LIST_TEST_CASE
The parameter given to `convert` may not be copyable therefore it has to be
captured by const reference. For example an `std::tuple` that contains a
non-copyable type is itself non-copyable.

The NonDefaultConstructible test-case was reduced by one example type
because it did not add any value.
2019-09-06 12:15:09 +02:00
6629c11ef8 Optimize in-repo PNGs 2019-09-06 12:12:47 +02:00
c6bf56b3d5 Fix Travis builds with clang 5/6
There is some weird broken dependency in their version of Trusty
(14.04 LTE), so we are using Xenial (16.04 LTE) instead.
2019-09-06 12:07:29 +02:00
623e348d9e Merge pull request #1744 from ux3d/add-ux3d-to-commercial-users
add UX3D to commercial users
2019-09-06 11:37:55 +02:00
46f767e602 add UX3D to commercial users 2019-09-06 11:27:33 +02:00
ce42deb72f Add Android's log library to Catch2's CMake interface 2019-08-10 20:51:15 +02:00
46a70071a7 Use __android_log_write instead of __android_log_print
`print` version of the logging functions supports `printf`-like
formatting, which we do not use and given our current debug print
internals, will never use. This should be slightly more efficient
and expresses the intent better.
2019-08-10 14:51:38 +02:00
378cc1a670 Rename template parameter names to avoid name conflict on Solaris.
Closes #1722
2019-08-10 10:55:17 +02:00
e2d863b090 Actually run the random generator tests 2019-08-09 11:21:05 +02:00
ebe6a07c23 Remove the new tests from approvals
They lead to stringification of file (which is ok) and file line
(not ok) to the approvals, which makes them exceedingly brittle
and not worth approval testing. Instead we just run them as part
of the base test run.
2019-08-09 11:13:28 +02:00
edcfd7fc62 Add missed item to v2.9.2 release notes 2019-08-09 10:57:59 +02:00
738818ae1d Add a test for the internals 2019-08-09 10:50:53 +02:00
2c869e17e4 v2.9.2 2019-08-08 14:05:24 +02:00
0ab11aa9b4 (Hopefully) Fix Obj-C++ compilation of matchers
Thanks to bdb for the patch, related to #1661
2019-08-07 20:35:27 +02:00
7a6af7ba76 Merge pull request #1717 from ax3l/fix-storageClassFirst
static constexpr: storage class first
2019-08-07 09:23:47 +02:00
fa096b26d1 Merge pull request #1715 from sfranzen/fix-compact-reporter
Fix infinite loop in compact reporter printer
2019-08-07 09:12:38 +02:00
820b1f12bf static constexpr: storage class first
Seen with PGI compiler for `static constexpr`:
```
warning: storage class is not first
```
2019-08-06 19:17:16 -05:00
6070745cab Improve reporting of unmatched filters (#1684)
This PR ultimately does 3 things
* Separately tracks matched tests per each filter part (that is, a set of filters separated by an OR (`,`)), which allows Catch2 to report each of the alternative filters that don't match any tests.
* Fixes `-w NoTests` to return non-zero in the process
* Adds tests for `-w NoTests`.
2019-08-06 20:51:19 +02:00
3d9e7db2e0 Fix infinite loop in compact reporter printer
Also simplify some variables with auto deduction.
2019-08-06 20:19:42 +02:00
cf55cfd76f Tiny speedup when listing tags
Noticed that the code was originally concatenating strings just to
then append the result to another string. Now it does not create
temporaries and also preallocates the string buffer.
2019-08-05 19:12:29 +02:00
3701c2e2e6 Small cleanup of self tests 2019-08-04 13:50:25 +02:00
7dc7d77af2 Add a test for Catch version output 2019-08-03 20:16:46 +02:00
06bc20cf37 Improve handling of newlines in release scripts
Under WSL, Python in text mode will translate `\n` into `\r\n`, even
though other tools and utilities use `\n` (because WSL is basically
Linux). This leads to the update scripts leaving the files with
Windows newlines even though git and similar expect them to have
Linux newlines.

By instead handling files in binary mode, we can keep the original
newlines. This commits switches parts of the update process to
binary mode, but not all because some of the will require a lot of
work to fix.
2019-08-03 19:59:35 +02:00
7a4beed6a6 Make release script update version placeholders in docs 2019-08-03 10:44:57 +02:00
67b4ada6b0 Add 'Introduced in Catch 2.3.0.' text 2019-08-02 19:18:47 +02:00
119569a67e Add 'Introduced in Catch 2.4.0.' text 2019-08-02 19:18:47 +02:00
ab713894cc Add docs for AND_GIVEN macro - see #1360 2019-08-02 19:18:47 +02:00
69fc94d6f8 Add 'Introduced in Catch 2.4.1.' text 2019-08-02 19:18:47 +02:00
49cd7c96b4 Add 'Introduced in Catch 2.4.2.' text 2019-08-02 19:18:47 +02:00
e998d152cc Add 'Introduced in Catch 2.5.0.' text 2019-08-02 19:18:47 +02:00
42a5903188 Add 'Introduced in Catch 2.6.0.' text 2019-08-02 19:18:47 +02:00
c071f07e1a Add 'Introduced in Catch 2.7.0.' note 2019-08-02 19:18:47 +02:00
53776a90cf Add 'Introduced in Catch 2.7.1.' note 2019-08-02 19:18:47 +02:00
4511dc0c16 Add 'Introduced in Catch 2.8.0.' text 2019-08-02 19:18:47 +02:00
e7c3bdb351 Add 'Introduced in Catch 2.8.0.' text 2019-08-02 19:18:47 +02:00
9aab958667 Add section headers, so I can later say when features were introduced 2019-08-02 19:18:47 +02:00
8cd58f75ec Add missing struct Template_Fixture_2 so that the example compiles.
Code copied from projects/SelfTest/UsageTests/Class.tests.cpp
2019-08-02 19:18:47 +02:00
d5a69cd400 Add 'Introduced in Catch 2.9.0.' text 2019-08-02 19:18:47 +02:00
1d13d88833 Helper script to create "Introduced in ..." text - see #1667 2019-08-02 19:18:47 +02:00
de0674c116 Add a template for creating new documentation page
Related to #1695
2019-08-02 13:17:22 +02:00
3d7282c2bd Add a section on documentation to contributing.md
Related to #1695
2019-08-02 10:58:51 +02:00
e5c0e3322d Add Android implementation of writeToDebugConsole that prints to Logcat (#1710) 2019-07-31 23:12:34 +02:00
dc8c8e957f Use an up to date conan package version again
Last time it was fixed to a specific version because the `conan`
and the `conan-package-tools` package that `pip install` would
gather were not compatible, let's hope it won't happen again.
2019-07-31 15:59:16 +02:00
ba9193370b Add autoconfiguration for RTX
Closes #1693
2019-07-30 11:28:28 +02:00
7b70b11c23 Add PopHead to Open Source projects using Catch 2019-07-30 09:37:05 +02:00
ab80277a86 Improve the REQUIRE_THROWS_MATCHES documentation a bit 2019-07-29 20:39:02 +02:00
7e7ab0e28b Fix xml reporter crashing when a benchmark failed. 2019-07-29 15:30:33 +02:00
425957dc63 Update vector matcher documentation
As it turns out, there were 2 undocumented vector matchers. I also
improved the documentation of them all.

Closes #1699
2019-07-27 18:41:42 +02:00
d017f6d18f Document CATCH_CONFIG_CPP17_BYTE macro
Leftover from #1686
2019-07-26 21:43:46 +02:00
91244d30a7 Document CATCH_CONFIG_CPP17_OPTIONAL macro 2019-07-26 21:42:54 +02:00
62b3f6c3c2 Merge pull request #1698 from sfranzen/fix-#1634
Fix CTest command issue with square brackets
2019-07-22 13:15:35 +02:00
e7c26f09d1 Fix non-default-constructible type lists used in TEMPLATE_LIST_TEST_CASE (#1697)
* Fix non-default-constructible type lists used in TEMPLATE_LIST_TEST_CASE

std::tuple is not default constructible when the first type is not
default-constuctible. Therefore it can not be instantiated.
to circumvent this, we have to use std::declval in the unevaluate decltype
context.
2019-07-22 12:56:33 +02:00
a22b7df46c Merge pull request #1700 from claremacrae/patch-3
Fix typo in 2.7.1 release notes
2019-07-22 12:13:29 +02:00
032068b889 Fix typo in 2.7.1 release notes
GENERATE_VAR -> GENERATE_REF
2019-07-21 22:04:50 +01:00
2aed6233cf Fix CTest command issue with square brackets 2019-07-21 16:29:46 +02:00
fb74bb133c Fix coverage report merging
For some time now (I'd guess almost a year 🤷), the coverage
merging on Windows has been failing, because the reports have been
generated in a different folder than expected. Our merge script did
not report failure because it was not checking the returned error
code from OpenCppCoverage, and for some reason, the `codecov` tool
happily returned 0 even though it did not find the file it was
supposed to upload...

The former is also fixed by this commit.
2019-07-20 21:12:17 +02:00
0b42ada60d Coverage helper now passes-on test return value
This allows #1684 to proceed forward.
2019-07-20 21:05:31 +02:00
c424ca47f9 Revert "Do not resolve symbolic path when sanitizing filepath during approvals"
This reverts commit 52f3abadbb.

Opens #1691
2019-07-19 18:16:21 +02:00
52f3abadbb Do not resolve symbolic path when sanitizing filepath during approvals
Fixes #1691
2019-07-18 16:33:57 +02:00
53281b471f Prevent compilation of generators that use bool in dangerous manner
Closes #1692
2019-07-18 11:54:18 +02:00
03ffc1014c Add a notice that benchmarking supports needs opt-in 2019-07-14 15:49:17 +02:00
87739ad3fe Add std::byte stringification support (#1686)
* Add `std::byte` support

1. Add byte detection in Catch config
2. Add a `std::byte`specialisation for Catch2::Detail::StringMaker
2019-07-13 14:47:56 +02:00
0c27554af5 Update Contents in docs #1667 2019-07-06 17:25:55 +02:00
11488e63b6 Use _WIN32 macro for detecting MSVC Windows
Previously we used `WIN32`, which sometimes does exist, but according to
https://docs.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=vs-2019
`_WIN32` is the one that is actually provided by the compiler.

Closes #1676.
2019-07-06 16:55:10 +02:00
820271bf24 Remove the weird backronym from readme 2019-07-02 20:43:37 +02:00
56d4510138 Default the SelfTest project to C++11
Unlike the previous approach, this will still obey
`-DCMAKE_CXX_STANDARD=X` properly.
2019-07-02 20:42:14 +02:00
c0d3a2e08f Merge pull request #1672 from sommd/master
Fix ChunkGenerator with chunk-size 0
2019-07-01 18:07:37 +02:00
2c3018a9d5 Merge pull request #1673 from sfranzen/fix-sections
Fix TrackerBase::close children completion check
2019-06-30 09:17:01 +02:00
9a6551b22b Add integration test for the children completion check fix 2019-06-30 00:00:41 +02:00
800f1b1d3d Fix TrackerBase::close children completion check
Now verifies that all children have completed, instead of only the last.
2019-06-29 18:02:28 +02:00
9cf5897a11 Fix ChunkGenerator with chunk-size 0
Fixes #1671
2019-06-29 23:48:30 +10:00
6f32c67ea7 Adding to documentation for --order and --rng-seed 2019-06-21 15:17:55 +02:00
7eea3ab245 Actually fix the CMake platform-independence problem 2019-06-17 19:41:17 +02:00
80af9ca687 Rollback ctest labels in CatchAddTests.cmake
Parsing --list-tests is broken, as Catch automatically line wraps the
line when it gets too long, stripping any whitespace in the process.
This means that it's impossible to reproduce the exact name of the
test if the test's name is long enough to line-wrap.

Furthermore, overwriting the LABELS property with the discovered labels
breaks users who manually added custom ctest labels.

Rolling back to using --list-test-names-only for now, as it does not
wrap lines even on very long test names.

We may be able parse the output of --list-tags to produce the ctest labels.
However, the straightforward way of doing this is to use CMake's
get_property(TEST ...) and set_property(TEST ... APPEND ...), which don't
work if the test name has spaces or other special characters. We would
need to mangle the test name to a valid CMake identifier to do it that way.
2019-06-17 19:17:32 +02:00
33286fdc37 Fix bug with making arch-independent CMake config file
Closes #1660
2019-06-17 19:08:16 +02:00
2f631bb808 v2.9.1 2019-06-17 12:02:14 +02:00
25cc09dcec Fix benchmarking in stand-alone files 2019-06-17 11:58:40 +02:00
f9dce28e7d v2.9.0 2019-06-16 10:42:59 +02:00
b87caafd91 Remove explicit setting of CXX_STANDARD for SelfTest target
- The current setup tries to detect USE_CPP14/USE_CPP17 and sets the
    CXX_STANDARD property for the SelfTest target. This is not ideal, since
    CMAKE_CXX_STANDARD can be provided by the toolchain file or as command line
    option and should be used by the library internally correctly.  Hence, the
    whole set of the relevant lines from `projects/CMakeLists.txt` have been
    removed.

  - The above can also cause subtle issues where the user is expecting the tests
    to compile with C++17 after setting CMAKE_CXX_STANDARD and then getting
    results of compilation with C++11 as USE_CPP17 has not been set.

  - The current build matrix used the above code to run the tests. So, even
    though the it should not required anymore to build Catch2, it was still
    required to send correct options to build matrix. In that respect,
    .travis.yml has been modified to send correct options to the build command
    in the new setup.
2019-06-15 20:33:28 +02:00
geh
bbbd5c4e08 Added null-ptr check in Colour::use() 2019-06-15 18:06:59 +02:00
f41051f22a test approvals for TEMPLATE_LIST_TEST_CASE 2019-06-15 15:40:39 +02:00
e90d5a86e4 docs for TEMPLATE_LIST_TEST_CASE 2019-06-15 15:40:39 +02:00
dbc1295354 tests for TEMPLATE_LIST_TEST_CASE 2019-06-15 15:40:39 +02:00
f2cfc2b852 TEMPLATE_LIST_TEST_CASE implementation
Closes #1627
2019-06-15 15:40:39 +02:00
c365ac392b Standardize and document the parallel processing toggle 2019-06-15 12:50:36 +02:00
e640c3837a Move the async-using parts of benchmarking into a .cpp file
This keeps it out of the main include path when benchmarking is
enabled, somewhat reducing the compilation-time penalty.

Also moved some other functions into the .cpp file, especially
helpers that could be given internal linkage, and concretized some
iterator-templated code that only ever used
`std::vector<double>::iterator`.
2019-06-15 11:43:25 +02:00
b468d7cbff Link benchmarking docs from the doc main page 2019-06-15 09:23:00 +02:00
7142d5a8c9 Show bound when stringifying the WithinULP matcher
Closes #1581
2019-06-15 07:12:27 +02:00
1967feac49 Introduce stubs for throwing specific exception types
This allows us to move <stdexcept> out of the common path, and replace
it with just <exception>. The difference between these two headers is
~13k lines after preprocessing on libstdc++ (16k vs 3k) and ~17k lines
for MS's STL(33k vs 16k).

Note that this is only beneficial if no other stdlib header we use
includes <stdexcept>. AFAIK this is true for the newest MS's STL,
but I have no idea of the applicability for libstdc++ and libc++.
2019-06-14 19:13:50 +02:00
f0b7b0ca11 Add a FUNDING file 2019-06-07 19:15:35 +02:00
4b1252547c Merge pull request #1616 from fodinabor/integrate_nonius
Integrate nonius to provide more advanced benchmarking
2019-06-07 07:01:47 +02:00
10067a47da Add an extra test for benchmarking macros
Also updated baselines
2019-06-06 21:29:25 +02:00
e340ab8db6 Various improvements to the benchmarking support
* Units from <ratio> are no longer redeclared in our own namespace
* The default clock is `steady_clock`, not `high_resolution_clock`,
because, as HH says "high_resolution_clock is useless. If you want
measure the passing of time, use steady_clock. If you want user
friendly time, use system_clock".
* Benchmarking support is opt-in, not opt-out, to avoid the large
(~10%) compile time penalty.
* Benchmarking-related options in CLI are always present, to decrease
the amount of code that is only compiled conditionally and making
the whole shebang more maintainble.
2019-06-06 21:28:56 +02:00
ce2560ca95 Integrate Nonius benchmark into Catch2
Changes done to Nonius:
* Moved things into "Catch::Benchmark" namespace
* Benchmarks were integrated with `TEST_CASE`/`SECTION`/`GENERATE` macros
* Removed Nonius's parameters for benchmarks, Generators should be used instead
* Added relevant methods to the reporter interface (default-implemented, to avoid
breaking existing 3rd party reporters)
* Async processing is guarded with `_REENTRANT` macro for GCC/Clang, used by default
on MSVC
* Added a macro `CATCH_CONFIG_DISABLE_BENCHMARKING` that removes all traces of
benchmarking from Catch
2019-06-06 19:33:37 +02:00
00347f1e79 v2.8.0 2019-05-26 22:47:00 +02:00
a5a2d08fbb Remove commented out code 2019-05-26 21:19:55 +02:00
97602b248b Remove obsolete file with wrong advice 2019-05-25 21:20:37 +02:00
e28e162795 Merge pull request #1638 from janisozaur/patch-1
Prevent building in source dir
2019-05-25 21:15:58 +02:00
90378f4a59 Have conan test-build the package in a separate subdirectory 2019-05-25 19:58:49 +02:00
84f8e806b8 Nttp support tests approvals 2019-05-23 21:03:47 +02:00
732e4b06db docs for signature based parametrised test cases 2019-05-23 21:03:47 +02:00
0c43f98fa2 Nttp support
Closes #1531
2019-05-23 21:03:47 +02:00
bd703dd74b Merge pull request #1611 from cincodenada/unscoped-disabled-prefixed
Add UNSCOPED_INFO to prefix/disabled sections
2019-05-23 14:19:22 +02:00
99602787cd Fix error message when unmatched quotes are encountered by CAPTURE
Thanks to @ledvinap for noticing
2019-05-23 13:54:51 +02:00
bfb4ee1597 Prevent building in source dir
Fixes #1636
2019-05-21 06:38:06 +02:00
31537c43d9 Rebased approvals 2019-05-21 00:05:39 +01:00
96355da34e StringRef no longer repoints m_start to m_data after c_str() on a substring.
This fixes an issue where a self-assignment of a StringRef copy would point into internally (and now dangling) data.
(now self-assignment check is no longer needed)
2019-05-21 00:04:44 +01:00
71fce429af Fix clang warning on self-assignment 2019-05-21 00:04:44 +01:00
d13e094598 Remove unique type check
Closes #1628
2019-05-20 21:13:05 +02:00
d30f1dda02 Rebased approvals 2019-05-18 20:53:28 +01:00
3bce8ba14b Fix StringRef self-assignment after substring
Thanks to Alex Tkachenko for spotting it.
2019-05-18 17:54:39 +01:00
e680c4b9fb fix SEGFAULT during construction of Session
coloured output tried to write startup exceptions
to stream which was not initialized
2019-05-15 20:14:25 +02:00
f1e14a1168 Add missing word 2019-05-09 22:40:59 +02:00
92ad9ee355 Merge pull request #1614 from mlimber/master
Allow custom precision in error reports for floating-point numbers
2019-05-03 17:46:03 +02:00
e2862a8d71 Add documentation for custom precision in float stringification 2019-05-03 15:40:21 +02:00
1161011dd0 Refactor custom precision in floating point stringification
Also fixup tests.
2019-05-03 15:38:06 +02:00
53a83e855e Add support for custom precision in floating point stringification
Closes #1612
2019-05-02 21:34:47 +02:00
9c741fe960 Allow quotes in CAPTURE arguments (#1608)
* Allow quotes in CAPTURE arguments

Fix CAPTURE to handle string and character literals properly
2019-05-01 19:12:44 +02:00
979bbf03bb Removed another redundant ; (from docs) 2019-04-27 18:52:38 +01:00
33ce3f3953 added cpp to another code block in docs 2019-04-27 18:51:26 +01:00
87a9424c9d Removed redundant ; 2019-04-27 18:50:05 +01:00
00cb0035c9 Enclosed generated StringMaker for enums in Catch namespace, rather than qualified 2019-04-26 18:24:52 +01:00
6267b06089 Added #include for <memory> (for unique_ptr) 2019-04-26 17:06:44 +01:00
9837c35df1 Rebased approvals for stringified enum tests 2019-04-26 11:30:19 +01:00
46066ede17 Merge branch enum stringification work 2019-04-26 11:26:45 +01:00
6981783178 Added some missing #includes 2019-04-25 14:19:00 +01:00
08c8df1e3b include StringRef, rather than fwd decl, for splitString 2019-04-25 10:32:55 +01:00
daeb5a87e6 Removed global qualification of specialisation 2019-04-25 10:23:58 +01:00
f2ee4f17ad Moved enum tests that depend on internals to IntrospectiveTests.
- also factored out makeEnumInfo, so tests don't need to touch registry
- and added usage test that involves namespace
2019-04-25 10:13:11 +01:00
182fc3e46e fix example's mistake in slow-compiles.md
the example lack the generation of factorial.o file which lead to an undefined reference error.
2019-04-25 09:48:22 +02:00
6b5b72651d Add UNSCOPED_INFO to prefix/disabled sections 2019-04-24 12:56:39 -07:00
f45bb00351 Add failing test for CATCH_UNSCOPED_CAPTURE
Actually fails to compile, which is good
2019-04-24 12:55:11 -07:00
7c37501b07 v2.7.2 2019-04-22 23:15:59 +02:00
4a1ca1ab55 TrackerContext: Remove misleading instance static method.
TrackerContext is not used as singleton, so just remove this misleading
method
2019-04-22 16:37:22 +02:00
e02d9e788f Document CATCH_REGISTER_ENUM 2019-04-21 20:32:20 +03:00
541f1ed1b3 Only provide CATCH_REGISTER_ENUM
No longer have version without the CATCH_ prefix
2019-04-21 20:26:46 +03:00
346723c9b6 Renamed STRINGIFY_ENUM to REGISTER_ENUM 2019-04-21 20:15:26 +03:00
5a74fcc9c9 Removed IEnumInfo (just use EnumInfo directly) 2019-04-21 20:09:58 +03:00
9d5d719868 Changed splitString to splitStringRef
Now takes and returns StringRefs
2019-04-21 20:03:44 +03:00
91b617c462 Added a vector approximate matcher 2019-04-20 10:02:17 +02:00
45e552528d Remove home-rolled algorithm replacements
Previously we had them to avoid including <algorithm> in the vector
matchers, but
* we included it anyway, even though we did not use it
* we use <algorithm> anyways in the generators
2019-04-19 17:54:21 +02:00
3978e9653b Add more documentation on contributing 2019-04-18 16:18:30 +02:00
d6fce7bf34 Fix warnings generated with -Wshadow 2019-04-18 15:30:38 +02:00
c3c82f539c Merge pull request #1336 from ax3l/topic-overrideVirtualRedundant
Refactor: override implies virtual
2019-04-18 14:10:24 +02:00
c7653811a6 Add basic CPack support
@ThijsWithaar is responsible for giving me the idea, but his PR
had couple of things that meant it was simpler to rewrite it than
to fix and merge it.

Supersedes and closes #1599
2019-04-18 13:47:33 +02:00
79417b9afc Revert "draft of nttp support"
This reverts commit 0c39409da7.
2019-04-18 11:35:33 +02:00
11cdd72db9 Merge remote-tracking branch 'refs/remotes/origin/nttp_support' 2019-04-18 11:26:33 +02:00
0c39409da7 draft of nttp support
added _SIG macros
2019-04-17 21:41:19 +02:00
edfac75347 Add test for omitting the 'Filters' heading when there were none 2019-04-16 23:49:22 +02:00
ac94bd0520 ParseAndAddCatchTests: Ignore cmake object libraries 2019-04-14 19:10:15 +02:00
d4eec016a9 CatchAddTests now adds tags as labels for ctest
- `ctest --print-labels` now will show list of available labels
- `ctest -L <regex>` will allow to run tests with given labels(tags)
2019-04-14 18:59:53 +02:00
36fb856163 Fix JUnit reporter output so that it conforms to JUnit schema
The `properties` can only go under the `testsuite` tag, not under
the `testsuites` tag.

Fixes #1598
2019-04-11 13:04:54 +02:00
4e32e0a563 Fix vendored dep because upstream is non-responsive
Eventually this needs to be fixed in the textflow project by Phil,
but he has not done so in the half a year this bug has been known
to be there, so...

Closes #1470
Closes #1455
2019-04-10 20:17:25 +02:00
1e2270b370 ParseAndAddCatchTests: Do not change CMAKE_MINIMUM_REQUIRED_VERSION when included 2019-04-10 10:50:13 +02:00
5096e39297 ParseAndAddCatchTests: Add scope to functions in order to avoid name collisions 2019-04-10 10:50:13 +02:00
15ccced6da ParseAndAddCatchTests: Document AdditionalCatchParameters variable 2019-04-10 10:50:13 +02:00
682617b5b7 ParseAndAddCatchTests: Fix tests not found
When using an optional launcher the target name is not interpreted as a
target, therefore it is not replaced automatically with its path.
2019-04-10 10:50:13 +02:00
15150c7b46 ParseAndAddCatchTests: Set the ParseAndAddCatchTests_TESTS property
After the script, the ParseAndAddCatchTests_TESTS property for the
target, and for each source file in the target is set, and contains the
list of the tests extracted from that target, or from that file.

This is useful, for example to add further labels or properties to the
tests.
2019-04-10 10:50:13 +02:00
5ce355a38c For macOS builds, disable isDebuggerActive() for non-AppleClang targets. Fixes #1588 2019-04-10 10:19:54 +02:00
edde6f4736 Fix typos identified by codespell.
Self test baselines also modified accordingly, due to
one typo found in a string in test code.
2019-04-10 09:42:11 +02:00
6bc5d172ee Merge pull request #1596 from 50ty/patch-1
IndexOfHideLabel is never used
2019-04-10 09:31:00 +02:00
3079b514d4 IndexOfHideLabel is never used 2019-04-10 08:00:08 +02:00
e99f1efd28 Avoid adding a default test spec when none was provided 2019-04-09 11:50:59 +02:00
b9dd1936e5 v2.7.1 2019-04-08 13:38:00 +02:00
293d617c49 Merge pull request #1592 from robinlinden/fix-cmake-integration-typo
Use correct filename in CMake integration docs
2019-04-08 13:20:41 +02:00
7be35af167 Use correct filename in CMake integration docs 2019-04-07 15:41:34 +02:00
02f13cf95a Made onto dev build and regenerated single header 2019-04-04 16:02:58 +01:00
43428c6093 First commit of STRINGIFY_ENUM 2019-04-04 15:55:46 +01:00
08147a23f9 Fix ObjC matchers
Closes #1571
2019-04-03 20:32:11 +02:00
8af8704089 support for printing test filters (PR #1585) 2019-04-03 20:24:52 +02:00
3816e99d0c Add GENERATE_COPY and GENERATE_VAR capturing generator macros 2019-03-31 14:11:10 +02:00
b77cec05c0 Fix test tag parsing to split [.foo] into [.][foo] 2019-03-29 10:48:56 +01:00
54089c4c8c Deducing return type of map generator helper (#1576)
* Deduce map return type implicitly

Giving the first template argument to map generator function to deduce
return type is now optional even if the return type is different from
the type generated by mapped generator.
2019-03-24 15:44:22 +01:00
296d447452 Merge pull request #1575 from alabuzhev/warning_ntstatus
Suppress warning caused by ntstatus.h inclusion
2019-03-20 15:22:27 +01:00
0531965349 Suppress warning caused by ntstatus.h inclusion 2019-03-19 23:57:56 +00:00
a1cdff4f18 raspigcd - opensource-users
Added raspigcd that uses Catch2 for tests
2019-03-19 20:35:59 +01:00
4611125801 generators.md: fix extraneous back tick in doc 2019-03-14 09:14:02 +01:00
e509012e64 Update blog URL and fic typos in tutorial.md 2019-03-09 18:02:29 +01:00
448825db03 Readability improvements for assertions.md 2019-03-09 18:02:29 +01:00
0fff8e7791 Add ApprovalTests.cpp to list of open source users 2019-03-09 18:02:29 +01:00
68a3c129ac Merge pull request #1562 from claremacrae/patch-2
Update url of blog post
2019-03-09 17:35:08 +01:00
1ce5ec9b74 Merge pull request #1565 from inobelar/patch-1
Added override keyword near IntRange::describe() method
2019-03-09 17:33:42 +01:00
37a4e32319 Added override keyword near IntRange::describe() method 2019-03-09 15:05:27 +02:00
0424c9a62c Update url of blog post 2019-03-08 08:56:00 +00:00
d633072794 v2.7.0 2019-03-07 21:44:57 +01:00
51ed08be22 Take changes to Clara from master 2019-03-07 20:48:36 +01:00
1701325caa Document unscoped info 2019-03-06 22:04:35 +01:00
7aee973a4a Add tests for unscoped info
Update approval tests as new tests are added for messaging.
2019-03-06 22:04:35 +01:00
99575b45db Add unscoped info functionality
This adds UNSCOPED_INFO macro, creating a log message that is stored
until the end of next assertion or the end of test case, whichever comes
first. These messages are not scoped locally, unlike messages created by
INFO macro.
2019-03-06 22:04:35 +01:00
1a03918455 Force flush streams in tests for #1514
This avoids the problem where writes to stderr/stdout stop being
line-buffered when stderr/stdout is redirected to a file, which led
to different order of outputs between Linux and Windows in our tests.
2019-03-06 21:53:26 +01:00
bd667f4d69 Add comma-escaping in test specs to ParseAndAddCatchTests.cmake
Fixes #1527
2019-03-03 13:11:11 +01:00
28db5ed4c9 Add tests for #1514
If the regression comes back, it will only be caught by approvals,
but that's better than nothing.
2019-03-02 21:22:47 +01:00
7d2451f119 Fix output redirection in failing tests (#1525)
Fixes #1514
2019-03-02 21:18:48 +01:00
5bf6e47381 Add another example for generators 2019-02-26 12:52:21 +01:00
29b3b7ae6b Namespace our type traits in catch_meta
Previously they could conflict with user-defined type traits that
were in the global namespace.

Fixes #1548
2019-02-23 21:06:20 +01:00
ef5fd8d42f Add another test for FilterGenerator 2019-02-23 20:37:35 +01:00
693647c43f Add ChunkGenerator
This generator collects values from the underlying generator until it
has a specified amount of them, and then returns them in one "chunk".
In case the underlying generator does not have enough elements for
a specific chunk, the left-over elements are discarded.

Closes #1538
2019-02-23 20:28:58 +01:00
288387fa10 Cleanup: move things around 2019-02-23 17:16:28 +01:00
165de9b072 Add RangeGenerator 2019-02-23 16:58:34 +01:00
bf4771a7ed Script helper getBuildExecutable now works properly on Windows 2019-02-23 14:44:40 +01:00
7012a31a39 Fix VS2015 warning issue 2019-02-20 10:14:46 +01:00
269303d9d9 Introduce random number (Integral and Float) generators 2019-02-20 00:09:15 +01:00
e8bfd882e8 Fix -Wdefaulted-function-deleted warning
Clang8 warns

catch_interfaces_reporter.h:84:25: error: explicitly defaulted move assignment operator is implicitly deleted [-Werror,-Wdefaulted-function-deleted]
|         AssertionStats& operator = ( AssertionStats && )     = default;
|                         ^

Signed-off-by: Khem Raj <raj.khem@gmail.com>
2019-02-18 20:42:42 +01:00
2bd0722470 fix "code will never be executed" under clang 2019-02-18 19:01:43 +01:00
45ebf17ec7 Do not open the output file twice
7f229b4f caused the output file to get opened twice, while
some types of files (e.g. named pipes) can be only opened once.

After this change Session::applyCommandLine opens the output file
only when there is an error to print.
2019-02-18 18:59:54 +01:00
093b72416d Remove non-ASCII characters from the distributable parts
Closes #1542
2019-02-18 10:33:41 +01:00
c99a346490 Use correct type name instead of index for TEMPLATE_PRODUCT_TEST_CASE (#1544)
Previously, for a TEMPLATE_PRODUCT_TEST_CASE("Test" ..., T, (P1, P2)),
the generated test case names were

  Test - 0
  Test - 1

With this commit, the correct typename is used:

  Test - T<P1>
  Test - T<P2>

-----------

MSVC needs another indirection to evaluate INTERNAL_CATCH_STRINGIZE
and also inserts a space before theINTERNAL_CATCH_STRINGIZE_WITHOUT_PARENS
parameter, which we can get rid of by pointer arithmetic.
2019-02-17 21:52:22 +01:00
359a54b6bd Merge pull request #1539 from mattclarke/patch-1
Docs: missing hyphens and full-stops added
2019-02-17 21:24:42 +01:00
711d750ca7 use "std" namespace for "sprintf"
Some compilers don't export C compatibility functions in `::`. Using
`std::sprintf` should work for those (and all other compilers).

Reference: https://en.cppreference.com/w/cpp/header#C_compatibility_headers
2019-02-17 21:02:21 +01:00
95f7712808 Missing hyphens and full-stops added
The use of full-stops is inconsistent, so I have added them where there are none.
The other option was to remove them instead.
2019-02-13 10:13:14 +01:00
dbbab8727c v2.6.1 2019-02-12 19:58:12 +01:00
5d4061af12 Update release process docs to include PGP-signing 2019-02-12 19:33:08 +01:00
9ccea82d7f Catch: resolve inconsistency in CATCH_REQUIRE_THROWS
The REQUIRE_THROWS and CATCH_REQUIRE_THROWS macros have
a subtle inconsistency in their implementation which can
cause a warning if [-Wunused-value] is used.

This commit changes CATCH_REQUIRE_THROWS so it has the
same implementation as REQUIRE_THROWS

It looks like REQUIRE_THROWS was change in commit
fae0fa4ec but not CATCH_REQUIRE_THROWS.

Similar changes for CATCH_CHECK_THROWS
2019-02-09 17:42:53 +01:00
dd3d27de57 Fix TAP protocol reporter
According to TAP protocol version 13, the comments after the test name
only may contain Directives - # TODO or # SKIP. We should put
the comment (aka suite name) on a separate line before the test.

See http://testanything.org/tap-version-13-specification.html#directives
2019-02-08 10:48:32 +01:00
7f229b4ff1 fix SEGFAULTs
issue was that mutable context had no config thus trying to get stream
caused SEGFAULT

Closes #1533, #1534
2019-02-08 10:41:23 +01:00
c03b23c84b Added Makimo to list of commercial users
Closes #1528
2019-02-04 20:04:48 +01:00
17686ba571 junit reporter reports random seed (--rng-seed)
Fixes #1520
2019-02-01 10:34:20 +01:00
5347ff9e5f Refactor: override implies virtual
If not used with `final`, override implies `virtual`.
Detected via CodeFactor score.

Another reference on SO:
  https://stackoverflow.com/questions/43466863/isnt-virtual-keyword-redundant-when-override-or-final-specifiers-are-used
2018-07-12 14:27:06 +02:00
689 changed files with 169778 additions and 59705 deletions

4
.bazelrc Normal file
View File

@ -0,0 +1,4 @@
build:gcc9 --cxxopt=-std=c++2a
build:clang13 --cxxopt=-std=c++17
build:vs2019 --cxxopt=/std:c++17
build:vs2022 --cxxopt=/std:c++17

25
.clang-format Normal file
View File

@ -0,0 +1,25 @@
---
AccessModifierOffset: '-4'
AlignEscapedNewlines: Left
AllowAllConstructorInitializersOnNextLine: 'true'
BinPackArguments: 'false'
BinPackParameters: 'false'
BreakConstructorInitializers: AfterColon
ConstructorInitializerAllOnOneLineOrOnePerLine: 'true'
DerivePointerAlignment: 'false'
FixNamespaceComments: 'true'
IncludeBlocks: Regroup
IndentCaseLabels: 'false'
IndentPPDirectives: AfterHash
IndentWidth: '4'
Language: Cpp
NamespaceIndentation: All
PointerAlignment: Left
SpaceBeforeCtorInitializerColon: 'false'
SpaceInEmptyParentheses: 'false'
SpacesInParentheses: 'true'
Standard: Cpp11
TabWidth: '4'
UseTab: Never
...

View File

@ -26,12 +26,12 @@ class BuilderSettings(object):
""" Set Catch2 repository to be used on upload.
The upload server address could be customized by env var
CONAN_UPLOAD. If not defined, the method will check the branch name.
Only master or CONAN_STABLE_BRANCH_PATTERN will be accepted.
The master branch will be pushed to testing channel, because it does
Only devel or CONAN_STABLE_BRANCH_PATTERN will be accepted.
The devel branch will be pushed to testing channel, because it does
not match the stable pattern. Otherwise it will upload to stable
channel.
"""
return os.getenv("CONAN_UPLOAD", "https://api.bintray.com/conan/catchorg/Catch2")
return os.getenv("CONAN_UPLOAD", "https://api.bintray.com/conan/catchorg/catch2")
@property
def upload_only_when_stable(self):
@ -47,9 +47,9 @@ class BuilderSettings(object):
@property
def reference(self):
""" Read project version from branch create Conan referece
""" Read project version from branch create Conan reference
"""
return os.getenv("CONAN_REFERENCE", "Catch2/{}".format(self._version))
return os.getenv("CONAN_REFERENCE", "catch2/{}".format(self._version))
@property
def channel(self):
@ -85,7 +85,7 @@ if __name__ == "__main__":
reference=settings.reference,
channel=settings.channel,
upload=settings.upload,
upload_only_when_stable=settings.upload_only_when_stable,
upload_only_when_stable=False,
stable_branch_pattern=settings.stable_branch_pattern,
login_username=settings.login_username,
username=settings.username,

View File

@ -1,11 +1,12 @@
cmake_minimum_required(VERSION 3.2.0)
project(test_package CXX)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup(TARGETS)
include("${CMAKE_BINARY_DIR}/conanbuildinfo.cmake")
conan_basic_setup()
find_package(Catch2 REQUIRED CONFIG)
add_executable(${PROJECT_NAME} test_package.cpp)
target_link_libraries(${PROJECT_NAME} CONAN_PKG::Catch2)
set_target_properties(${PROJECT_NAME} PROPERTIES CXX_STANDARD 11)
target_link_libraries(${PROJECT_NAME} Catch2::Catch2WithMain)
set_target_properties(${PROJECT_NAME} PROPERTIES CXX_STANDARD 14)

View File

@ -6,7 +6,7 @@ import os
class TestPackageConan(ConanFile):
settings = "os", "compiler", "build_type", "arch"
generators = "cmake"
generators = "cmake_find_package_multi", "cmake"
def build(self):
cmake = CMake(self)
@ -14,6 +14,7 @@ class TestPackageConan(ConanFile):
cmake.build()
def test(self):
assert os.path.isfile(os.path.join(self.deps_cpp_info["Catch2"].rootpath, "licenses", "LICENSE.txt"))
assert os.path.isfile(os.path.join(
self.deps_cpp_info["catch2"].rootpath, "licenses", "LICENSE.txt"))
bin_path = os.path.join("bin", "test_package")
self.run("%s -s" % bin_path, run_environment=True)

View File

@ -1,6 +1,4 @@
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
int Factorial( int number ) {
return number <= 1 ? 1 : Factorial( number - 1 ) * number;

2
.github/FUNDING.yml vendored Normal file
View File

@ -0,0 +1,2 @@
github: "horenmar"
custom: "https://www.paypal.me/horenmar"

View File

@ -12,7 +12,7 @@ at docs/contributing.md. It will tell you how to properly test your changes.
<!--
Describe the what and the why of your pull request. Remember that these two
are usually a bit different. As an example, if you have made various changes
to decrease the number of new strings allocated, thats what. The why probably
to decrease the number of new strings allocated, that's what. The why probably
was that you have a large set of tests and found that this speeds them up.
-->

103
.github/workflows/linux-other-builds.yml vendored Normal file
View File

@ -0,0 +1,103 @@
# The builds in this file are more complex (e.g. they need custom CMake
# configuration) and thus are unsuitable to the simple build matrix
# approach used in simple-builds
name: Linux builds (complex)
on: [push, pull_request]
jobs:
build:
name: ${{matrix.build_description}}, ${{matrix.cxx}}, C++${{matrix.std}} ${{matrix.build_type}}
runs-on: ubuntu-20.04
strategy:
matrix:
# We add builds one by one in this case, because there are no
# dimensions that are shared across the builds
include:
# Single surrogate header build
- cxx: clang++-10
build_description: Surrogates build
build_type: Debug
std: 14
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_SURROGATES=ON
# Extras and examples with gcc-7
- cxx: g++-7
build_description: Extras + Examples
build_type: Debug
std: 14
other_pkgs: g++-7
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
- cxx: g++-7
build_description: Extras + Examples
build_type: Release
std: 14
other_pkgs: g++-7
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
# Extras and examples with Clang-10
- cxx: clang++-10
build_description: Extras + Examples
build_type: Debug
std: 17
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
- cxx: clang++-10
build_description: Extras + Examples
build_type: Release
std: 17
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
# Configure tests with Clang-10
- cxx: clang++-10
build_description: CMake configuration tests
build_type: Debug
std: 14
other_pkgs: clang-10
cmake_configurations: -DCATCH_ENABLE_CONFIGURE_TESTS=ON
# Valgrind test Clang-10
- cxx: clang++-10
build_description: Valgrind tests
build_type: Debug
std: 14
other_pkgs: clang-10 valgrind
cmake_configurations: -DMEMORYCHECK_COMMAND=`which valgrind` -DMEMORYCHECK_COMMAND_OPTIONS="-q --track-origins=yes --leak-check=full --num-callers=50 --show-leak-kinds=definite --error-exitcode=1"
other_ctest_args: -T memcheck -LE uses-python
steps:
- uses: actions/checkout@v2
- name: Prepare environment
run: sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} \
-DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCMAKE_CXX_EXTENSIONS=OFF \
-DCATCH_DEVELOPMENT_BUILD=ON \
${{matrix.cmake_configurations}} \
-G Ninja
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: ninja
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2 ${{matrix.other_ctest_args}}

View File

@ -0,0 +1,97 @@
name: Linux builds (basic)
on: [push, pull_request]
jobs:
build:
name: ${{matrix.cxx}}, C++${{matrix.std}}, ${{matrix.build_type}}
runs-on: ubuntu-20.04
strategy:
matrix:
cxx:
# - g++-6
- g++-7
- g++-8
- g++-9
- g++-10
- clang++-6.0
- clang++-7
- clang++-8
- clang++-9
- clang++-10
build_type: [Debug, Release]
std: [14]
include:
# cannot be installed on ubuntu-20.04 be default?
# - cxx: g++-6
# other_pkgs: g++-6
- cxx: g++-7
other_pkgs: g++-7
- cxx: g++-8
other_pkgs: g++-8
- cxx: g++-9
other_pkgs: g++-9
- cxx: g++-10
other_pkgs: g++-10
- cxx: clang++-6.0
other_pkgs: clang-6.0
- cxx: clang++-7
other_pkgs: clang-7
- cxx: clang++-8
other_pkgs: clang-8
- cxx: clang++-9
other_pkgs: clang-9
- cxx: clang++-10
other_pkgs: clang-10
# Clang 6 + C++17
# does not work with the default libstdc++ version thanks
# to a disagreement on variant implementation.
# - cxx: clang++-6.0
# build_type: Debug
# std: 17
# other_pkgs: clang-6.0
# - cxx: clang++-6.0
# build_type: Release
# std: 17
# other_pkgs: clang-6.0
# Clang 10 + C++17
- cxx: clang++-10
build_type: Debug
std: 17
other_pkgs: clang-10
- cxx: clang++-10
build_type: Release
std: 17
other_pkgs: clang-10
steps:
- uses: actions/checkout@v2
- name: Prepare environment
run: sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} \
-DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCMAKE_CXX_EXTENSIONS=OFF \
-DCATCH_DEVELOPMENT_BUILD=ON \
-G Ninja
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: ninja
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2

45
.github/workflows/mac-builds.yml vendored Normal file
View File

@ -0,0 +1,45 @@
name: Mac builds
on: [push, pull_request]
jobs:
build:
runs-on: macos-10.15
strategy:
matrix:
cxx:
- g++-9
- clang++
build_type: [Debug, Release]
std: [14, 17]
include:
- build_type: Debug
examples: ON
extra_tests: ON
steps:
- uses: actions/checkout@v2
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} -DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCATCH_DEVELOPMENT_BUILD=ON -DCATCH_BUILD_EXAMPLES=${{matrix.examples}} \
-DCATCH_BUILD_EXTRA_TESTS=${{matrix.examples}}
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: make -j 2
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2

View File

@ -0,0 +1,36 @@
name: Check header guards
on: [push, pull_request]
jobs:
build:
# Set the type of machine to run on
runs-on: ubuntu-20.04
steps:
- name: Checkout source code
uses: actions/checkout@v2
- name: Setup Dependencies
uses: actions/setup-python@v2
with:
python-version: '3.7'
- name: Install checkguard
run: pip install guardonce
- name: Check that include guards are properly named
run: |
wrong_files=$(checkguard -r src/catch2/ -p "name | append _INCLUDED | upper")
if [[ $wrong_files ]]; then
echo "Files with wrong header guard:"
echo $wrong_files
exit 1
fi
- name: Check that there are no duplicated filenames
run: |
./tools/scripts/checkDuplicateFilenames.py
- name: Check that all source files have the correct license header
run: |
./tools/scripts/checkLicense.py

16
.gitignore vendored
View File

@ -11,11 +11,6 @@ Release
xcuserdata
CatchSelfTest.xcscheme
Breakpoints.xcbkptlist
projects/VS2010/TestCatch/_UpgradeReport_Files/
projects/VS2010/TestCatch/TestCatch/TestCatch.vcxproj.filters
projects/VisualStudio/TestCatch/UpgradeLog.XML
projects/CMake/.idea
projects/CMake/cmake-build-debug
UpgradeLog.XML
Resources/DWARF
projects/Generated
@ -25,6 +20,17 @@ DerivedData
Build
.idea
.vs
.vscode
cmake-build-*
benchmark-dir
.conan/test_package/build
bazel-*
build-fuzzers
debug-build
.vscode
msvc-sln*
# Currently we use Doxygen for dep graphs and the full docs are only slowly
# being filled in, so we definitely do not want git to deal with the docs.
docs/doxygen
*.cache
compile_commands.json

View File

@ -1,324 +0,0 @@
language: cpp
branches:
except:
- /dev-appveyor.*/
common_sources: &all_sources
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty
- llvm-toolchain-trusty-3.9
- llvm-toolchain-trusty-4.0
- llvm-toolchain-trusty-5.0
- llvm-toolchain-trusty-6.0
matrix:
include:
# 1/ Linux Clang Builds
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.5']
env: COMPILER='clang++-3.5'
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.6']
env: COMPILER='clang++-3.6'
# Clang 3.7 is intentionally skipped as we cannot get it easily on
# TravisCI container
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['lcov', 'clang-3.8']
env: COMPILER='clang++-3.8'
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.9']
env: COMPILER='clang++-3.9'
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-4.0']
env: COMPILER='clang++-4.0'
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-5.0']
env: COMPILER='clang++-5.0'
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0']
env: COMPILER='clang++-6.0'
# 2/ Linux GCC Builds
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-4.8']
env: COMPILER='g++-4.8'
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-4.9']
env: COMPILER='g++-4.9'
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-5']
env: COMPILER='g++-5'
- os: linux
compiler: gcc
addons: &gcc6
apt:
sources: *all_sources
packages: ['g++-6']
env: COMPILER='g++-6'
- os: linux
compiler: gcc
addons: &gcc7
apt:
sources: *all_sources
packages: ['g++-7']
env: COMPILER='g++-7'
- os: linux
compiler: gcc
addons: &gcc8
apt:
sources: *all_sources
packages: ['g++-8']
env: COMPILER='g++-8'
# 3b/ Linux C++14 Clang builds
# Note that we need newer libstdc++ for C++14 support
- os: linux
compiler: clang
addons:
apt:
packages: ['clang-3.8', 'libstdc++-6-dev']
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty
env: COMPILER='clang++-3.8' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.9', 'libstdc++-6-dev']
env: COMPILER='clang++-3.9' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-4.0', 'libstdc++-6-dev']
env: COMPILER='clang++-4.0' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-5.0', 'libstdc++-6-dev']
env: COMPILER='clang++-5.0' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0', 'libstdc++-6-dev']
env: COMPILER='clang++-6.0' CPP14=1
# 4a/ Linux C++14 GCC builds
- os: linux
compiler: gcc
addons: *gcc6
env: COMPILER='g++-6' CPP14=1
- os: linux
compiler: gcc
addons: *gcc7
env: COMPILER='g++-7' CPP14=1
- os: linux
compiler: gcc
addons: *gcc8
env: COMPILER='g++-8' CPP14=1
# 5/ OSX Clang Builds
- os: osx
osx_image: xcode7.3
compiler: clang
env: COMPILER='clang++'
- os: osx
osx_image: xcode8
compiler: clang
env: COMPILER='clang++'
- os: osx
osx_image: xcode9
compiler: clang
env: COMPILER='clang++'
- os: osx
osx_image: xcode9.1
compiler: clang
env: COMPILER='clang++'
- os: osx
osx_image: xcode9.1
compiler: clang
env: COMPILER='clang++' CPP14=1
# 6/ Special builds -- examples, coverage, valgrind, etc.
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['lcov', 'g++-7']
env: COMPILER='g++-7' CPP14=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
- os: linux
compiler: clang
addons:
apt:
packages: ['clang-3.8', 'lcov']
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty
env: COMPILER='clang++-3.8' EXAMPLES=1 COVERAGE=1 EXTRAS=1
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['valgrind', 'lcov', 'g++-7']
env: COMPILER='g++-7' CPP14=1 VALGRIND=1
- os: osx
osx_image: xcode9.1
compiler: clang
env: COMPILER='clang++' CPP14=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
# 7/ C++17 builds
- os: linux
compiler: gcc
addons: *gcc7
env: COMPILER='g++-7' CPP17=1
- os: linux
compiler: gcc
addons: *gcc7
env: COMPILER='g++-7' EXAMPLES=1 COVERAGE=1 EXTRAS=1 CPP17=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0', 'libstdc++-8-dev']
env: COMPILER='clang++-6.0' CPP17=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0', 'libstdc++-8-dev']
env: COMPILER='clang++-6.0' CPP17=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
# 8/ Conan
- language: python
python:
- "3.7"
dist: xenial
install:
- pip install conan==1.10.2 conan-package-tools
env:
- CONAN_GCC_VERSIONS=8
- CONAN_DOCKER_IMAGE=conanio/gcc8
script:
- python .conan/build.py
install:
- DEPS_DIR="${TRAVIS_BUILD_DIR}/deps"
- mkdir -p ${DEPS_DIR} && cd ${DEPS_DIR}
- |
if [[ "${TRAVIS_OS_NAME}" == "linux" ]]; then
CMAKE_URL="http://cmake.org/files/v3.8/cmake-3.8.2-Linux-x86_64.tar.gz"
mkdir cmake && travis_retry wget --no-check-certificate --quiet -O - ${CMAKE_URL} | tar --strip-components=1 -xz -C cmake
export PATH=${DEPS_DIR}/cmake/bin:${PATH}
elif [[ "${TRAVIS_OS_NAME}" == "osx" ]]; then
which cmake || brew install cmake;
fi
before_script:
- export CXX=${COMPILER}
- cd ${TRAVIS_BUILD_DIR}
# Regenerate single header file, so it is tested in the examples...
- python scripts/generateSingleHeader.py
# Use Debug builds for running Valgrind and building examples
- cmake -H. -BBuild-Debug -DCMAKE_BUILD_TYPE=Debug -Wdev -DUSE_CPP14=${CPP14} -DUSE_CPP17=${CPP17} -DCATCH_USE_VALGRIND=${VALGRIND} -DCATCH_BUILD_EXAMPLES=${EXAMPLES} -DCATCH_ENABLE_COVERAGE=${COVERAGE} -DCATCH_BUILD_EXTRA_TESTS=${EXTRAS}
# Don't bother with release build for coverage build
- cmake -H. -BBuild-Release -DCMAKE_BUILD_TYPE=Release -Wdev -DUSE_CPP14=${CPP14} -DUSE_CPP17=${CPP17}
script:
- cd Build-Debug
- make -j 2
- CTEST_OUTPUT_ON_FAILURE=1 ctest -j 2
# Coverage collection does not work for OS X atm
- |
if [[ "${TRAVIS_OS_NAME}" == "linux" ]] && [[ "${COVERAGE}" == "1" ]]; then
make gcov
make lcov
bash <(curl -s https://codecov.io/bash) -X gcov || echo "Codecov did not collect coverage reports"
fi
- # Go to release build
- cd ../Build-Release
- make -j 2
- CTEST_OUTPUT_ON_FAILURE=1 ctest -j 2

91
BUILD.bazel Normal file
View File

@ -0,0 +1,91 @@
# Load the cc_library rule.
load("@rules_cc//cc:defs.bzl", "cc_library")
load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
expand_template(
name = "catch_user_config",
out = "catch2/catch_user_config.hpp",
substitutions = {
"#cmakedefine CATCH_CONFIG_ANDROID_LOGWRITE": "",
"#cmakedefine CATCH_CONFIG_BAZEL_SUPPORT": "#define CATCH_CONFIG_BAZEL_SUPPORT",
"#cmakedefine CATCH_CONFIG_NO_COLOUR_WIN32": "",
"#cmakedefine CATCH_CONFIG_COLOUR_WIN32": "",
"#cmakedefine CATCH_CONFIG_COUNTER": "",
"#cmakedefine CATCH_CONFIG_CPP11_TO_STRING": "",
"#cmakedefine CATCH_CONFIG_CPP17_BYTE": "",
"#cmakedefine CATCH_CONFIG_CPP17_OPTIONAL": "",
"#cmakedefine CATCH_CONFIG_CPP17_STRING_VIEW": "",
"#cmakedefine CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_CPP17_VARIANT": "",
"#cmakedefine CATCH_CONFIG_DISABLE_EXCEPTIONS_CUSTOM_HANDLER": "",
"#cmakedefine CATCH_CONFIG_DISABLE_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_DISABLE_STRINGIFICATION": "",
"#cmakedefine CATCH_CONFIG_DISABLE": "",
"#cmakedefine CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS": "",
"#cmakedefine CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_PAIR_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_TUPLE_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_EXPERIMENTAL_REDIRECT": "",
"#cmakedefine CATCH_CONFIG_FALLBACK_STRINGIFIER @CATCH_CONFIG_FALLBACK_STRINGIFIER@": "",
"#cmakedefine CATCH_CONFIG_FAST_COMPILE": "",
"#cmakedefine CATCH_CONFIG_GLOBAL_NEXTAFTER": "",
"#cmakedefine CATCH_CONFIG_NO_COUNTER": "",
"#cmakedefine CATCH_CONFIG_NO_CPP11_TO_STRING": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_BYTE": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_OPTIONAL": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_STRING_VIEW": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_VARIANT": "",
"#cmakedefine CATCH_CONFIG_NO_GLOBAL_NEXTAFTER": "",
"#cmakedefine CATCH_CONFIG_NO_POSIX_SIGNALS": "",
"#cmakedefine CATCH_CONFIG_NO_USE_ASYNC": "",
"#cmakedefine CATCH_CONFIG_NO_WCHAR": "",
"#cmakedefine CATCH_CONFIG_NO_WINDOWS_SEH": "",
"#cmakedefine CATCH_CONFIG_NOSTDOUT": "",
"#cmakedefine CATCH_CONFIG_POSIX_SIGNALS": "",
"#cmakedefine CATCH_CONFIG_PREFIX_ALL": "",
"#cmakedefine CATCH_CONFIG_USE_ASYNC": "",
"#cmakedefine CATCH_CONFIG_WCHAR": "",
"#cmakedefine CATCH_CONFIG_WINDOWS_CRTDBG": "",
"#cmakedefine CATCH_CONFIG_WINDOWS_SEH": "",
"#cmakedefine CATCH_CONFIG_NO_ANDROID_LOGWRITE": "",
"@CATCH_CONFIG_DEFAULT_REPORTER@": "console",
"@CATCH_CONFIG_CONSOLE_WIDTH@": "80",
},
template = "src/catch2/catch_user_config.hpp.in",
)
# Generated header library, modifies the include prefix to account for
# generation path so that we can include <catch2/catch_user_config.hpp>
# correctly.
cc_library(
name = "catch2_generated",
hdrs = ["catch2/catch_user_config.hpp"],
include_prefix = ".", # to manipulate -I of dependenices
visibility = ["//visibility:public"],
)
# Static library, without main.
cc_library(
name = "catch2",
srcs = glob(
["src/catch2/**/*.cpp"],
exclude = ["src/catch2/internal/catch_main.cpp"],
),
hdrs = glob(["src/catch2/**/*.hpp"]),
includes = ["src/"],
linkstatic = True,
visibility = ["//visibility:public"],
deps = [":catch2_generated"],
)
# Static library, with main.
cc_library(
name = "catch2_main",
srcs = ["src/catch2/internal/catch_main.cpp"],
includes = ["src/"],
linkstatic = True,
visibility = ["//visibility:public"],
deps = [":catch2"],
)

View File

@ -0,0 +1,77 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
##
# This file contains options that are materialized into the Catch2
# compiled library. All of them default to OFF, as even the positive
# forms correspond to the user _forcing_ them to ON, while being OFF
# means that Catch2 can use its own autodetection.
#
# For detailed docs look into docs/configuration.md
macro(AddOverridableConfigOption OptionBaseName)
option(CATCH_CONFIG_${OptionBaseName} "Read docs/configuration.md for details" OFF)
option(CATCH_CONFIG_NO_${OptionBaseName} "Read docs/configuration.md for details" OFF)
endmacro()
macro(AddConfigOption OptionBaseName)
option(CATCH_CONFIG_${OptionBaseName} "Read docs/configuration.md for details" OFF)
endmacro()
set(_OverridableOptions
"ANDROID_LOGWRITE"
"BAZEL_SUPPORT"
"COLOUR_WIN32"
"COUNTER"
"CPP11_TO_STRING"
"CPP17_BYTE"
"CPP17_OPTIONAL"
"CPP17_STRING_VIEW"
"CPP17_UNCAUGHT_EXCEPTIONS"
"CPP17_VARIANT"
"GLOBAL_NEXTAFTER"
"POSIX_SIGNALS"
"USE_ASYNC"
"WCHAR"
"WINDOWS_SEH"
)
foreach(OptionName ${_OverridableOptions})
AddOverridableConfigOption(${OptionName})
endforeach()
set(_OtherConfigOptions
"DISABLE_EXCEPTIONS"
"DISABLE_EXCEPTIONS_CUSTOM_HANDLER"
"DISABLE"
"DISABLE_STRINGIFICATION"
"ENABLE_ALL_STRINGMAKERS"
"ENABLE_OPTIONAL_STRINGMAKER"
"ENABLE_PAIR_STRINGMAKER"
"ENABLE_TUPLE_STRINGMAKER"
"ENABLE_VARIANT_STRINGMAKER"
"EXPERIMENTAL_REDIRECT"
"FAST_COMPILE"
"NOSTDOUT"
"PREFIX_ALL"
"WINDOWS_CRTDBG"
)
foreach(OptionName ${_OtherConfigOptions})
AddConfigOption(${OptionName})
endforeach()
set(CATCH_CONFIG_DEFAULT_REPORTER "console" CACHE STRING "Read docs/configuration.md for details. The name of the reporter should be without quotes.")
set(CATCH_CONFIG_CONSOLE_WIDTH "80" CACHE STRING "Read docs/configuration.md for details. Must form a valid integer literal.")
# There is no good way to both turn this into a CMake cache variable,
# and keep reasonable default semantics inside the project. Thus we do
# not define it and users have to provide it as an outside variable.
#set(CATCH_CONFIG_FALLBACK_STRINGIFIER "" CACHE STRING "Read docs/configuration.md for details.")

View File

@ -0,0 +1,120 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
include(CheckCXXCompilerFlag)
function(add_cxx_flag_if_supported_to_targets flagname targets)
check_cxx_compiler_flag("${flagname}" HAVE_FLAG_${flagname})
if (HAVE_FLAG_${flagname})
foreach(target ${targets})
target_compile_options(${target} PUBLIC ${flagname})
endforeach()
endif()
endfunction()
# Assumes that it is only called for development builds, where warnings
# and Werror is desired, so it also enables Werror.
function(add_warnings_to_targets targets)
LIST(LENGTH targets TARGETS_LEN)
# For now we just assume 2 possibilities: msvc and msvc-like compilers,
# and other.
if (MSVC)
foreach(target ${targets})
# Force MSVC to consider everything as encoded in utf-8
target_compile_options( ${target} PRIVATE /utf-8 )
# Enable Werror equivalent
if (CATCH_ENABLE_WERROR)
target_compile_options( ${target} PRIVATE /WX )
endif()
# MSVC is currently handled specially
if ( CMAKE_CXX_COMPILER_ID MATCHES "MSVC" )
STRING(REGEX REPLACE "/W[0-9]" "/W4" CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS}) # override default warning level
target_compile_options( ${target} PRIVATE /w44265 /w44061 /w44062 /w45038 )
endif()
endforeach()
endif()
if (NOT MSVC)
set(CHECKED_WARNING_FLAGS
"-Wabsolute-value"
"-Wall"
"-Wc++20-compat"
"-Wcall-to-pure-virtual-from-ctor-dtor"
"-Wcast-align"
"-Wcatch-value"
"-Wdangling"
"-Wdeprecated"
"-Wdeprecated-register"
"-Wexceptions"
"-Wexit-time-destructors"
"-Wextra"
"-Wextra-semi"
"-Wfloat-equal"
"-Wglobal-constructors"
"-Winit-self"
"-Wmisleading-indentation"
"-Wmismatched-new-delete"
"-Wmismatched-return-types"
"-Wmismatched-tags"
"-Wmissing-braces"
"-Wmissing-declarations"
"-Wmissing-noreturn"
"-Wmissing-prototypes"
"-Wmissing-variable-declarations"
"-Wnull-dereference"
"-Wold-style-cast"
"-Woverloaded-virtual"
"-Wparentheses"
"-Wpedantic"
"-Wreorder"
"-Wreturn-std-move"
"-Wshadow"
"-Wstrict-aliasing"
"-Wsuggest-destructor-override"
"-Wsuggest-override"
"-Wundef"
"-Wuninitialized"
"-Wunneeded-internal-declaration"
"-Wunreachable-code"
"-Wunused"
"-Wunused-function"
"-Wunused-parameter"
"-Wvla"
"-Wweak-vtables"
# This is a useful warning, but our tests sometimes rely on
# functions being present, but not picked (e.g. various checks
# for stringification implementation ordering).
# Ergo, we should use it every now and then, but we cannot
# enable it by default.
# "-Wunused-member-function"
)
foreach(warning ${CHECKED_WARNING_FLAGS})
add_cxx_flag_if_supported_to_targets(${warning} "${targets}")
endforeach()
if (CATCH_ENABLE_WERROR)
foreach(target ${targets})
# Enable Werror equivalent
target_compile_options( ${target} PRIVATE -Werror )
endforeach()
endif()
endif()
endfunction()
# Adds flags required for reproducible build to the target
# Currently only supports GCC and Clang
function(add_build_reproducibility_settings target)
# Make the build reproducible on versions of g++ and clang that supports -ffile-prefix-map
if(("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND NOT ${CMAKE_CXX_COMPILER_VERSION} VERSION_LESS 8) OR
("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang" AND NOT ${CMAKE_CXX_COMPILER_VERSION} VERSION_LESS 10))
target_compile_options(${target} PRIVATE "-ffile-prefix-map=${CATCH_DIR}=.")
endif()
endfunction()

View File

@ -19,7 +19,7 @@ set(CMAKE_REQUIRED_QUIET ${codecov_FIND_QUIETLY})
get_property(ENABLED_LANGUAGES GLOBAL PROPERTY ENABLED_LANGUAGES)
foreach (LANG ${ENABLED_LANGUAGES})
# Gcov evaluation is dependend on the used compiler. Check gcov support for
# Gcov evaluation is dependent on the used compiler. Check gcov support for
# each compiler that is used. If gcov binary was already found for this
# compiler, do not try to find it again.
if (NOT GCOV_${CMAKE_${LANG}_COMPILER_ID}_BIN)

View File

@ -74,7 +74,7 @@ set(CMAKE_REQUIRED_QUIET ${codecov_FIND_QUIETLY})
get_property(ENABLED_LANGUAGES GLOBAL PROPERTY ENABLED_LANGUAGES)
foreach (LANG ${ENABLED_LANGUAGES})
# Coverage flags are not dependend on language, but the used compiler. So
# Coverage flags are not dependent on language, but the used compiler. So
# instead of searching flags foreach language, search flags foreach compiler
# used.
set(COMPILER ${CMAKE_${LANG}_COMPILER_ID})

View File

@ -1,26 +0,0 @@
#checks that the given hard-coded list contains all headers + sources in the given folder
function(CheckFileList LIST_VAR FOLDER)
set(MESSAGE " should be added to the variable ${LIST_VAR}")
set(MESSAGE "${MESSAGE} in ${CMAKE_CURRENT_LIST_FILE}\n")
file(GLOB GLOBBED_LIST "${FOLDER}/*.cpp"
"${FOLDER}/*.hpp"
"${FOLDER}/*.h")
list(REMOVE_ITEM GLOBBED_LIST ${${LIST_VAR}})
foreach(EXTRA_ITEM ${GLOBBED_LIST})
string(REPLACE "${CATCH_DIR}/" "" RELATIVE_FILE_NAME "${EXTRA_ITEM}")
message(AUTHOR_WARNING "The file \"${RELATIVE_FILE_NAME}\"${MESSAGE}")
endforeach()
endfunction()
function(CheckFileListRec LIST_VAR FOLDER)
set(MESSAGE " should be added to the variable ${LIST_VAR}")
set(MESSAGE "${MESSAGE} in ${CMAKE_CURRENT_LIST_FILE}\n")
file(GLOB_RECURSE GLOBBED_LIST "${FOLDER}/*.cpp"
"${FOLDER}/*.hpp"
"${FOLDER}/*.h")
list(REMOVE_ITEM GLOBBED_LIST ${${LIST_VAR}})
foreach(EXTRA_ITEM ${GLOBBED_LIST})
string(REPLACE "${CATCH_DIR}/" "" RELATIVE_FILE_NAME "${EXTRA_ITEM}")
message(AUTHOR_WARNING "The file \"${RELATIVE_FILE_NAME}\"${MESSAGE}")
endforeach()
endfunction()

View File

@ -0,0 +1,10 @@
includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
libdir=@CMAKE_INSTALL_FULL_LIBDIR@
pkg_version=@Catch2_VERSION@
Name: Catch2-With-Main
Description: A modern, C++-native test framework for C++14 and above (links in default main)
Version: ${pkg_version}
Requires: catch2 = ${pkg_version}
Cflags: -I${includedir}
Libs: -L${libdir} -lCatch2Main

View File

@ -1,7 +1,11 @@
prefix=@CMAKE_INSTALL_PREFIX@
exec_prefix=${prefix}
includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
libdir=@CMAKE_INSTALL_FULL_LIBDIR@
Name: Catch2
Description: A modern, C++-native, header-only, test framework for C++11
Description: A modern, C++-native, test framework for C++14 and above
URL: https://github.com/catchorg/Catch2
Version: @Catch2_VERSION@
Cflags: -I${includedir}
Libs: -L${libdir} -lCatch2

View File

@ -4,45 +4,95 @@ cmake_minimum_required(VERSION 3.5)
# disable testsuite in that case
if(NOT DEFINED PROJECT_NAME)
set(NOT_SUBPROJECT ON)
else()
set(NOT_SUBPROJECT OFF)
endif()
project(Catch2 LANGUAGES CXX VERSION 2.6.0)
option(CATCH_INSTALL_DOCS "Install documentation alongside library" ON)
option(CATCH_INSTALL_EXTRAS "Install extras (CMake scripts, debugger helpers) alongside library" ON)
option(CATCH_DEVELOPMENT_BUILD "Build tests, enable warnings, enable Werror, etc" OFF)
# Provide path for scripts
include(CMakeDependentOption)
cmake_dependent_option(CATCH_BUILD_TESTING "Build the SelfTest project" ON "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_EXAMPLES "Build code examples" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_EXTRA_TESTS "Build extra tests" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_FUZZERS "Build fuzzers" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_COVERAGE "Generate coverage for codecov.io" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_WERROR "Enables Werror during build" ON "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_SURROGATES "Enable generating and building surrogate TUs for the main headers" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_CONFIGURE_TESTS "Enable CMake configuration tests. WARNING: VERY EXPENSIVE" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
# Catch2's build breaks if done in-tree. You probably should not build
# things in tree anyway, but we can allow projects that include Catch2
# as a subproject to build in-tree as long as it is not in our tree.
if (CMAKE_BINARY_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
message(FATAL_ERROR "Building in-source is not supported! Create a build dir and remove ${CMAKE_SOURCE_DIR}/CMakeCache.txt")
endif()
if(CMAKE_VERSION VERSION_GREATER 3.8)
# Enable IPO for CMake versions that support it
cmake_policy(SET CMP0069 NEW)
endif()
project(Catch2
VERSION 3.1.0 # CML version placeholder, don't delete
LANGUAGES CXX
# HOMEPAGE_URL is not supported until CMake version 3.12, which
# we do not target yet.
# HOMEPAGE_URL "https://github.com/catchorg/Catch2"
DESCRIPTION "A modern, C++-native, unit test framework."
)
# Provide path for scripts. We first add path to the scripts we don't use,
# but projects including us might, and set the path up to parent scope.
# Then we also add path that we use to configure the project, but is of
# no use to top level projects.
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/extras")
if (NOT NOT_SUBPROJECT)
set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH}" PARENT_SCOPE)
endif()
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/CMake")
include(GNUInstallDirs)
include(CMakePackageConfigHelpers)
include(CTest)
include(CatchConfigOptions)
if(CATCH_DEVELOPMENT_BUILD)
include(CTest)
endif()
option(CATCH_USE_VALGRIND "Perform SelfTests with Valgrind" OFF)
option(CATCH_BUILD_TESTING "Build SelfTest project" ON)
option(CATCH_BUILD_EXAMPLES "Build documentation examples" OFF)
option(CATCH_BUILD_EXTRA_TESTS "Build extra tests" OFF)
option(CATCH_ENABLE_COVERAGE "Generate coverage for codecov.io" OFF)
option(CATCH_ENABLE_WERROR "Enable all warnings as errors" ON)
option(CATCH_INSTALL_DOCS "Install documentation alongside library" ON)
option(CATCH_INSTALL_HELPERS "Install contrib alongside library" ON)
# This variable is used in some subdirectories, so we need it here, rather
# than later in the install block
set(CATCH_CMAKE_CONFIG_DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/Catch2")
set_property(GLOBAL PROPERTY USE_FOLDERS ON)
# define some folders
set(CATCH_DIR ${CMAKE_CURRENT_SOURCE_DIR})
set(SELF_TEST_DIR ${CATCH_DIR}/projects/SelfTest)
set(BENCHMARK_DIR ${CATCH_DIR}/projects/Benchmark)
set(HEADER_DIR ${CATCH_DIR}/include)
if(USE_WMAIN)
# We have some Windows builds that test `wmain` entry point,
# and we need this change to be present in all binaries that
# are built during these tests, so this is required here, before
# the subdirectories are added.
if(CATCH_TEST_USE_WMAIN)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /ENTRY:wmainCRTStartup")
endif()
# Basic paths
set(CATCH_DIR ${CMAKE_CURRENT_SOURCE_DIR})
set(SOURCES_DIR ${CATCH_DIR}/src/catch2)
set(SELF_TEST_DIR ${CATCH_DIR}/tests/SelfTest)
set(BENCHMARK_DIR ${CATCH_DIR}/tests/Benchmark)
set(EXAMPLES_DIR ${CATCH_DIR}/examples)
# We need to bring-in the variables defined there to this scope
add_subdirectory(src)
# Build tests only if requested
if (BUILD_TESTING AND CATCH_BUILD_TESTING AND NOT_SUBPROJECT)
find_package(PythonInterp)
find_package(PythonInterp 3 REQUIRED)
if (NOT PYTHONINTERP_FOUND)
message(FATAL_ERROR "Python not found, but required for tests")
endif()
add_subdirectory(projects)
add_subdirectory(tests)
endif()
if(CATCH_BUILD_EXAMPLES)
@ -50,53 +100,21 @@ if(CATCH_BUILD_EXAMPLES)
endif()
if(CATCH_BUILD_EXTRA_TESTS)
add_subdirectory(projects/ExtraTests)
add_subdirectory(tests/ExtraTests)
endif()
# add catch as a 'linkable' target
add_library(Catch2 INTERFACE)
if(CATCH_BUILD_FUZZERS)
add_subdirectory(fuzzing)
endif()
# depend on some obvious c++11 features so the dependency is transitively added dependents
target_compile_features(Catch2
INTERFACE
cxx_alignas
cxx_alignof
cxx_attributes
cxx_auto_type
cxx_constexpr
cxx_defaulted_functions
cxx_deleted_functions
cxx_final
cxx_lambdas
cxx_noexcept
cxx_override
cxx_range_for
cxx_rvalue_references
cxx_static_assert
cxx_strong_enums
cxx_trailing_return_types
cxx_unicode_literals
cxx_user_literals
cxx_variadic_macros
)
target_include_directories(Catch2
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}/single_include>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
# provide a namespaced alias for clients to 'link' against if catch is included as a sub-project
add_library(Catch2::Catch2 ALIAS Catch2)
if (CATCH_DEVELOPMENT_BUILD)
add_warnings_to_targets("${CATCH_WARNING_TARGETS}")
endif()
# Only perform the installation steps when Catch is not being used as
# a subproject via `add_subdirectory`, or the destinations will break,
# see https://github.com/catchorg/Catch2/issues/1373
if (NOT_SUBPROJECT)
set(CATCH_CMAKE_CONFIG_DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/Catch2")
configure_package_config_file(
${CMAKE_CURRENT_LIST_DIR}/CMake/Catch2Config.cmake.in
${CMAKE_CURRENT_BINARY_DIR}/Catch2Config.cmake
@ -104,50 +122,11 @@ if (NOT_SUBPROJECT)
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# create and install an export set for catch target as Catch2::Catch
install(
TARGETS
Catch2
EXPORT
Catch2Targets
DESTINATION
${CMAKE_INSTALL_LIBDIR}
)
install(
EXPORT
Catch2Targets
NAMESPACE
Catch2::
DESTINATION
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# By default, FooConfigVersion is tied to architecture that it was
# generated on. Because Catch2 is header-only, it is arch-independent
# and thus Catch2ConfigVersion should not be tied to the architecture
# it was generated on.
#
# CMake does not provide a direct customization point for this in
# `write_basic_package_version_file`, but it can be accomplished
# indirectly by temporarily undefining `CMAKE_SIZEOF_VOID_P`.
set(CATCH2_CMAKE_SIZEOF_VOID_P ${CMAKE_SIZEOF_VOID_P})
unset(CMAKE_SIZEOF_VOID_P)
write_basic_package_version_file(
"${CMAKE_CURRENT_BINARY_DIR}/Catch2ConfigVersion.cmake"
COMPATIBILITY
SameMajorVersion
)
set(CMAKE_SIZEOF_VOID_P ${CATCH2_CMAKE_SIZEOF_VOID_P})
install(
DIRECTORY
"single_include/"
DESTINATION
"${CMAKE_INSTALL_INCLUDEDIR}"
)
install(
FILES
@ -164,28 +143,29 @@ if (NOT_SUBPROJECT)
docs/
DESTINATION
"${CMAKE_INSTALL_DOCDIR}"
PATTERN "doxygen" EXCLUDE
)
endif()
if(CATCH_INSTALL_HELPERS)
# Install CMake scripts
install(
FILES
"contrib/ParseAndAddCatchTests.cmake"
"contrib/Catch.cmake"
"contrib/CatchAddTests.cmake"
DESTINATION
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# Install debugger helpers
install(
FILES
"contrib/gdbinit"
"contrib/lldbinit"
DESTINATION
${CMAKE_INSTALL_DATAROOTDIR}/Catch2
)
if(CATCH_INSTALL_EXTRAS)
# Install CMake scripts
install(
FILES
"extras/ParseAndAddCatchTests.cmake"
"extras/Catch.cmake"
"extras/CatchAddTests.cmake"
DESTINATION
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# Install debugger helpers
install(
FILES
"extras/gdbinit"
"extras/lldbinit"
DESTINATION
${CMAKE_INSTALL_DATAROOTDIR}/Catch2
)
endif()
## Provide some pkg-config integration
@ -198,11 +178,28 @@ if (NOT_SUBPROJECT)
${CMAKE_CURRENT_BINARY_DIR}/catch2.pc
@ONLY
)
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/CMake/catch2-with-main.pc.in
${CMAKE_CURRENT_BINARY_DIR}/catch2-with-main.pc
@ONLY
)
install(
FILES
"${CMAKE_CURRENT_BINARY_DIR}/catch2.pc"
"${CMAKE_CURRENT_BINARY_DIR}/catch2-with-main.pc"
DESTINATION
${PKGCONFIG_INSTALL_DIR}
)
# CPack/CMake started taking the package version from project version 3.12
# So we need to set the version manually for older CMake versions
if(${CMAKE_VERSION} VERSION_LESS "3.12.0")
set(CPACK_PACKAGE_VERSION ${PROJECT_VERSION})
endif()
set(CPACK_PACKAGE_CONTACT "https://github.com/catchorg/Catch2/")
include( CPack )
endif(NOT_SUBPROJECT)

25
CMakePresets.json Normal file
View File

@ -0,0 +1,25 @@
{
"version": 3,
"configurePresets": [
{
"name": "basic-tests",
"displayName": "Basic development build",
"description": "Enables development build with basic tests that are cheap to build and run",
"cacheVariables": {
"CATCH_DEVELOPMENT_BUILD": "ON"
}
},
{
"name": "all-tests",
"inherits": "basic-tests",
"displayName": "Full development build",
"description": "Enables development build with examples and ALL tests",
"cacheVariables": {
"CATCH_BUILD_EXAMPLES": "ON",
"CATCH_BUILD_EXTRA_TESTS": "ON",
"CATCH_BUILD_SURROGATES": "ON",
"CATCH_ENABLE_CONFIGURE_TESTS": "ON"
}
}
]
}

2484
Doxyfile Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,29 +1,41 @@
<a id="top"></a>
![catch logo](artwork/catch2-logo-small.png)
![Catch2 logo](data/artwork/catch2-logo-small.png)
[![Github Releases](https://img.shields.io/github/release/catchorg/catch2.svg)](https://github.com/catchorg/catch2/releases)
[![Build Status](https://travis-ci.org/catchorg/Catch2.svg?branch=master)](https://travis-ci.org/catchorg/Catch2)
[![Build status](https://ci.appveyor.com/api/projects/status/github/catchorg/Catch2?svg=true)](https://ci.appveyor.com/project/catchorg/catch2)
[![codecov](https://codecov.io/gh/catchorg/Catch2/branch/master/graph/badge.svg)](https://codecov.io/gh/catchorg/Catch2)
[![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://wandbox.org/permlink/85qcYBOdUKzlnpl5)
[![Linux build status](https://github.com/catchorg/Catch2/actions/workflows/linux-simple-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/linux-simple-builds.yml)
[![Linux build status](https://github.com/catchorg/Catch2/actions/workflows/linux-other-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/linux-other-builds.yml)
[![MacOS build status](https://github.com/catchorg/Catch2/actions/workflows/mac-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/mac-builds.yml)
[![Build Status](https://ci.appveyor.com/api/projects/status/github/catchorg/Catch2?svg=true&branch=devel)](https://ci.appveyor.com/project/catchorg/catch2)
[![Code Coverage](https://codecov.io/gh/catchorg/Catch2/branch/devel/graph/badge.svg)](https://codecov.io/gh/catchorg/Catch2)
[![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://godbolt.org/z/EdoY15q9G)
[![Join the chat in Discord: https://discord.gg/4CWS9zD](https://img.shields.io/badge/Discord-Chat!-brightgreen.svg)](https://discord.gg/4CWS9zD)
<a href="https://github.com/catchorg/Catch2/releases/download/v2.6.0/catch.hpp">The latest version of the single header can be downloaded directly using this link</a>
## What's the Catch2?
## Catch2 is released!
Catch2 is mainly a unit testing framework for C++, but it also
provides basic micro-benchmarking features, and simple BDD macros.
If you've been using an earlier version of Catch, please see the
Breaking Changes section of [the release notes](https://github.com/catchorg/Catch2/releases/tag/v2.0.1)
before moving to Catch2. You might also like to read [this blog post](http://www.levelofindirection.com/journal/2017/11/3/catch2-released.html) for more details.
Catch2's main advantage is that using it is both simple and natural.
Tests autoregister themselves and do not have to be named with valid
identifiers, assertions look like normal C++ code, and sections provide
a nice way to share set-up and tear-down code in tests.
## What's the Catch?
Catch2 stands for C++ Automated Test Cases in a Header and is a
multi-paradigm test framework for C++. which also supports Objective-C
(and maybe C).
It is primarily distributed as a single header file, although certain
extensions may require additional headers.
## Catch2 v3 is being developed!
You are on the `devel` branch, where the next major version, v3, of
Catch2 is being developed. As it is a significant rework, you will
find that parts of this documentation are likely still stuck on v2.
For stable (and documentation-matching) version of Catch2, [go to the
`v2.x` branch](https://github.com/catchorg/Catch2/tree/v2.x).
For migrating from the v2 releases to v3, you should look at [our
documentation](docs/migrate-v2-to-v3.md#top). It provides a simple
guidelines on getting started, and collects most common migration
problems.
## How to use it
This documentation comprises these three parts:
@ -32,7 +44,9 @@ This documentation comprises these three parts:
* [Tutorial](docs/tutorial.md#top) - getting started
* [Reference section](docs/Readme.md#top) - all the details
## More
* Issues and bugs can be raised on the [Issue tracker on GitHub](https://github.com/catchorg/Catch2/issues)
* For discussion or questions please use [the dedicated Google Groups forum](https://groups.google.com/forum/?fromgroups#!forum/catch-forum) or our [Discord](https://discord.gg/4CWS9zD)
* See [who else is using Catch2](docs/opensource-users.md#top)
* For discussion or questions please use [our Discord](https://discord.gg/4CWS9zD)
* See who else is using Catch2 in [Open Source Software](docs/opensource-users.md#top)
or [commercially](docs/commercial-users.md#top).

19
SECURITY.md Normal file
View File

@ -0,0 +1,19 @@
# Security Policy
## Supported Versions
* Versions 1.x (branch Catch1.x) are no longer supported.
* Versions 2.x (branch v2.x) are currently supported.
* `devel` branch serves for stable-ish development and is supported,
but branches `devel-*` are considered short lived and are not supported separately.
## Reporting a Vulnerability
Due to its nature as a _unit_ test framework, Catch2 shouldn't interact
with untrusted inputs and there shouldn't be many security vulnerabilities
in it.
However, if you find one you send email to martin <dot> horenovsky <at>
gmail <dot> com. If you want to encrypt the email, my pgp key is
`E29C 46F3 B8A7 5028 6079 3B7D ECC9 C20E 314B 2360`.

14
WORKSPACE Normal file
View File

@ -0,0 +1,14 @@
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "bazel_skylib",
strip_prefix = "bazel-skylib-2a87d4a62af886fb320883aba102255aba87275e",
urls = [
"https://github.com/bazelbuild/bazel-skylib/archive/2a87d4a62af886fb320883aba102255aba87275e.tar.gz",
],
sha256 = "d847b08d6702d2779e9eb399b54ff8920fa7521dc45e3e53572d1d8907767de7",
)
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()

View File

@ -1,88 +1,37 @@
# version string format -- This will be overwritten later anyway
version: "{build}"
version: "{build}-{branch}"
# If we ever get a backlog larger than clone_depth, builds will fail
# spuriously. I do not think we will ever get 20 deep commits deep though.
clone_depth: 20
# We want to build everything, except for branches that are explicitly
# for messing around with travis.
branches:
except:
- /dev-travis.+/
os:
- Visual Studio 2017
- Visual Studio 2015
environment:
matrix:
- additional_flags: "/permissive- /std:c++latest"
wmain: 0
- additional_flags: ""
wmain: 0
- additional_flags: "/D_UNICODE /DUNICODE"
wmain: 1
coverage: 0
# Have a coverage dimension
- additional_flags: ""
wmain: 0
coverage: 1
# Have an examples dimension
- additional_flags: ""
wmain: 0
examples: 1
matrix:
exclude:
- os: Visual Studio 2015
additional_flags: "/permissive- /std:c++latest"
- os: Visual Studio 2015
additional_flags: "/D_UNICODE /DUNICODE"
# Exclude unwanted coverage configurations
- coverage: 1
platform: Win32
- coverage: 1
os: Visual Studio 2015
- coverage: 1
configuration: Release
# Exclude unwanted examples configurations
- examples: 1
platform: Win32
- examples: 1
os: Visual Studio 2015
- examples: 1
configuration: Release
# We need a more up to date pip because Python 2.7 is EOL soon
init:
- set PATH=C:\Python35;C:\Python35\Scripts;%PATH%
install:
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { python -m pip --disable-pip-version-check install codecov }
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { .\misc\installOpenCppCoverage.ps1 }
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { pip --disable-pip-version-check install codecov }
# This removes our changes to PATH. Keep this step last!
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { .\tools\misc\installOpenCppCoverage.ps1 }
# Win32 and x64 are CMake-compatible solution platform names.
# This allows us to pass %PLATFORM% to CMake -A.
platform:
- Win32
- x64
# build Configurations, i.e. Debug, Release, etc.
configuration:
- Debug
- Release
#Cmake will autodetect the compiler, but we set the arch
before_build:
# We need to modify PATH again, because it was reset since the "init" step
- set PATH=C:\Python35;C:\Python35\Scripts;%PATH%
- set CXXFLAGS=%additional_flags%
# If we are building examples/extra-tests, we need to regenerate the amalgamated files
- cmd: if "%examples%"=="1" ( python .\tools\scripts\generateAmalgamatedFiles.py )
# Indirection because appveyor doesn't handle multiline batch scripts properly
# https://stackoverflow.com/questions/37627248/how-to-split-a-command-over-multiple-lines-in-appveyor-yml/37647169#37647169
# https://help.appveyor.com/discussions/questions/3888-multi-line-cmd-or-powershell-warning-ignore
- cmd: .\misc\appveyorBuildConfigurationScript.bat
- cmd: .\tools\misc\appveyorBuildConfigurationScript.bat
# build with MSBuild
@ -93,4 +42,88 @@ build:
test_script:
- set CTEST_OUTPUT_ON_FAILURE=1
- cmd: .\misc\appveyorTestRunScript.bat
- cmd: .\tools\misc\appveyorTestRunScript.bat
# Sadly we cannot use the standard "dimensions" based approach towards
# specifying the different builds, as there is no way to add one-offs
# builds afterwards. This means that we will painfully specify each
# build explicitly.
environment:
matrix:
- FLAVOR: VS 2019 x64 Debug Surrogates Configure Tests
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
surrogates: 1
configure_tests: 1
platform: x64
configuration: Debug
- FLAVOR: VS 2019 x64 Release
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
platform: x64
configuration: Release
- FLAVOR: VS 2019 x64 Debug Coverage Examples
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
examples: 1
coverage: 1
platform: x64
configuration: Debug
- FLAVOR: VS 2019 x64 Debug WMain
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
wmain: 1
additional_flags: "/D_UNICODE /DUNICODE"
platform: x64
configuration: Debug
- FLAVOR: VS 2019 Win32 Debug
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
platform: Win32
configuration: Debug
- FLAVOR: VS 2019 x64 Debug Latest Strict
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
additional_flags: "/permissive- /std:c++latest"
platform: x64
configuration: Debug
- FLAVOR: VS 2017 x64 Debug
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
platform: x64
configuration: Debug
- FLAVOR: VS 2017 x64 Release
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
platform: x64
configuration: Release
- FLAVOR: VS 2017 x64 Release Coverage
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
coverage: 1
platform: x64
configuration: Debug
- FLAVOR: VS 2017 Win32 Debug
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
platform: Win32
configuration: Debug
- FLAVOR: VS 2017 Win32 Debug Examples
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
examples: 1
platform: Win32
configuration: Debug
- FLAVOR: VS 2017 Win32 Debug WMain
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
wmain: 1
additional_flags: "/D_UNICODE /DUNICODE"
platform: Win32
configuration: Debug
- FLAVOR: VS 2017 x64 Debug Latest Strict
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
additional_flags: "/permissive- /std:c++latest"
platform: x64
configuration: Debug

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 28 KiB

View File

@ -10,15 +10,13 @@ coverage:
default:
target: 80%
ignore:
- "projects/SelfTest"
- "**/catch_reporter_tap.hpp"
- "**/catch_reporter_automake.hpp"
- "**/catch_reporter_teamcity.hpp"
- "**/external/clara.hpp"
- "tests"
codecov:
branch: master
branch: devel
max_report_age: off
comment:
layout: "diff"

View File

@ -1,27 +1,60 @@
#!/usr/bin/env python
from conans import ConanFile, CMake
from conans import ConanFile, CMake, tools
class CatchConan(ConanFile):
name = "Catch2"
description = "A modern, C++-native, header-only, framework for unit-tests, TDD and BDD"
topics = ("conan", "catch2", "header-only", "unit-test", "tdd", "bdd")
name = "catch2"
description = "A modern, C++-native, framework for unit-tests, TDD and BDD"
topics = ("conan", "catch2", "unit-test", "tdd", "bdd")
url = "https://github.com/catchorg/Catch2"
homepage = url
license = "BSL-1.0"
exports = "LICENSE.txt"
exports_sources = ("single_include/*", "CMakeLists.txt", "CMake/*", "contrib/*")
exports_sources = ("src/*", "CMakeLists.txt", "CMake/*", "extras/*")
settings = "os", "compiler", "build_type", "arch"
generators = "cmake"
def package(self):
def _configure_cmake(self):
cmake = CMake(self)
cmake.definitions["BUILD_TESTING"] = "OFF"
cmake.definitions["CATCH_INSTALL_DOCS"] = "OFF"
cmake.definitions["CATCH_INSTALL_HELPERS"] = "ON"
cmake.configure()
cmake.definitions["CATCH_INSTALL_EXTRAS"] = "ON"
cmake.configure(build_folder="build")
return cmake
def build(self):
# We need this workaround until the toolchains feature
# to inject stuff like MD/MT
line_to_replace = 'list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/CMake")'
tools.replace_in_file("CMakeLists.txt", line_to_replace,
'''{}
include("{}/conanbuildinfo.cmake")
conan_basic_setup()'''.format(line_to_replace, self.install_folder.replace("\\", "/")))
cmake = self._configure_cmake()
cmake.build()
def package(self):
self.copy(pattern="LICENSE.txt", dst="licenses")
cmake = self._configure_cmake()
cmake.install()
self.copy(pattern="LICENSE.txt", dst="licenses")
def package_info(self):
lib_suffix = "d" if self.settings.build_type == "Debug" else ""
def package_id(self):
self.info.header_only()
self.cpp_info.names["cmake_find_package"] = "Catch2"
self.cpp_info.names["cmake_find_package_multi"] = "Catch2"
# Catch2
self.cpp_info.components["catch2base"].names["cmake_find_package"] = "Catch2"
self.cpp_info.components["catch2base"].names["cmake_find_package_multi"] = "Catch2"
self.cpp_info.components["catch2base"].names["pkg_config"] = "Catch2"
self.cpp_info.components["catch2base"].libs = ["Catch2" + lib_suffix]
self.cpp_info.components["catch2base"].builddirs.append("lib/cmake/Catch2")
# Catch2WithMain
self.cpp_info.components["catch2main"].names["cmake_find_package"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].names["cmake_find_package_multi"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].names["pkg_config"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].libs = ["Catch2Main" + lib_suffix]
self.cpp_info.components["catch2main"].requires = ["catch2base"]

View File

@ -1,78 +0,0 @@
# Distributed under the OSI-approved BSD 3-Clause License. See accompanying
# file Copyright.txt or https://cmake.org/licensing for details.
set(prefix "${TEST_PREFIX}")
set(suffix "${TEST_SUFFIX}")
set(spec ${TEST_SPEC})
set(extra_args ${TEST_EXTRA_ARGS})
set(properties ${TEST_PROPERTIES})
set(script)
set(suite)
set(tests)
function(add_command NAME)
set(_args "")
foreach(_arg ${ARGN})
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]") # form a bracket_argument
else()
set(_args "${_args} ${_arg}")
endif()
endforeach()
set(script "${script}${NAME}(${_args})\n" PARENT_SCOPE)
endfunction()
# Run test executable to get list of available tests
if(NOT EXISTS "${TEST_EXECUTABLE}")
message(FATAL_ERROR
"Specified test executable '${TEST_EXECUTABLE}' does not exist"
)
endif()
execute_process(
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-test-names-only
OUTPUT_VARIABLE output
RESULT_VARIABLE result
)
# Catch --list-test-names-only reports the number of tests, so 0 is... surprising
if(${result} EQUAL 0)
message(WARNING
"Test executable '${TEST_EXECUTABLE}' contains no tests!\n"
)
elseif(${result} LESS 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${result}\n"
" Output: ${output}\n"
)
endif()
string(REPLACE "\n" ";" output "${output}")
# Parse output
foreach(line ${output})
set(test ${line})
# use escape commas to handle properly test cases with commans inside the name
string(REPLACE "," "\\," test_name ${test})
# ...and add to script
add_command(add_test
"${prefix}${test}${suffix}"
${TEST_EXECUTOR}
"${TEST_EXECUTABLE}"
"${test_name}"
${extra_args}
)
add_command(set_tests_properties
"${prefix}${test}${suffix}"
PROPERTIES
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
${properties}
)
list(APPEND tests "${prefix}${test}${suffix}")
endforeach()
# Create a list of all discovered tests, which users may use to e.g. set
# properties on the tests
add_command(set ${TEST_LIST} ${tests})
# Write CTest script
file(WRITE "${CTEST_FILE}" "${script}")

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

View File

@ -14,6 +14,7 @@ Writing tests:
* [Event Listeners](event-listeners.md#top)
* [Data Generators](generators.md#top)
* [Other macros](other-macros.md#top)
* [Micro benchmarking](benchmarks.md#top)
Fine tuning:
* [Supplying your own main()](own-main.md#top)
@ -24,16 +25,17 @@ Running:
* [Command line](command-line.md#top)
Odds and ends:
* [Frequently Asked Questions (FAQ)](faq.md#top)
* [Best practices and other tips](usage-tips.md#top)
* [CMake integration](cmake-integration.md#top)
* [CI and other miscellaneous pieces](ci-and-misc.md#top)
FAQ:
* [Why are my tests slow to compile?](slow-compiles.md#top)
* [Known limitations](limitations.md#top)
Other:
* [Why Catch?](why-catch.md#top)
* [Open Source Projects using Catch](opensource-users.md#top)
* [Why Catch2?](why-catch.md#top)
* [Migrating from v2 to v3](migrate-v2-to-v3.md#top)
* [Open Source Projects using Catch2](opensource-users.md#top)
* [Commercial Projects using Catch2](commercial-users.md#top)
* [Contributing](contributing.md#top)
* [Release Notes](release-notes.md#top)
* [Deprecations and incoming changes](deprecations.md#top)

View File

@ -10,7 +10,7 @@
Most test frameworks have a large collection of assertion macros to capture all possible conditional forms (```_EQUALS```, ```_NOTEQUALS```, ```_GREATER_THAN``` etc).
Catch is different. Because it decomposes natural C-style conditional expressions most of these forms are reduced to one or two that you will use all the time. That said there are a rich set of auxiliary macros as well. We'll describe all of these here.
Catch is different. Because it decomposes natural C-style conditional expressions most of these forms are reduced to one or two that you will use all the time. That said there is a rich set of auxiliary macros as well. We'll describe all of these here.
Most of these macros come in two forms:
@ -61,7 +61,7 @@ Catch provides a way to perform tolerant comparisons of floating point values th
REQUIRE( performComputation() == Approx( 2.1 ) );
```
Catch also provides a UDL for `Approx`; `_a`. It resides in
Catch also provides a user-defined literal for `Approx`; `_a`. It resides in
the `Catch::literals` namespace and can be used like so:
```cpp
using namespace Catch::literals;
@ -136,7 +136,7 @@ REQUIRE_THROWS_WITH( dismantleHal(), "My mind is going" );
* **REQUIRE_THROWS_MATCHES(** _expression_, _exception type_, _matcher for given exception type_ **)** and
* **CHECK_THROWS_MATCHES(** _expression_, _exception type_, _matcher for given exception type_ **)**
Expects that exception of _exception type_ is thrown and it matches provided matcher (see next section for Matchers).
Expects that exception of _exception type_ is thrown and it matches provided matcher (see the [documentation for Matchers](matchers.md#top)).
_Please note that the `THROW` family of assertions expects to be passed a single expression, not a statement or series of statements. If you want to check a more complicated sequence of operations, you can use a C++11 lambda function._

251
docs/benchmarks.md Normal file
View File

@ -0,0 +1,251 @@
<a id="top"></a>
# Authoring benchmarks
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
Writing benchmarks is not easy. Catch simplifies certain aspects but you'll
always need to take care about various aspects. Understanding a few things about
the way Catch runs your code will be very helpful when writing your benchmarks.
First off, let's go over some terminology that will be used throughout this
guide.
- *User code*: user code is the code that the user provides to be measured.
- *Run*: one run is one execution of the user code. Sometimes also referred
to as an _iteration_.
- *Sample*: one sample is one data point obtained by measuring the time it takes
to perform a certain number of runs. One sample can consist of more than one
run if the clock available does not have enough resolution to accurately
measure a single run. All samples for a given benchmark execution are obtained
with the same number of runs.
## Execution procedure
Now I can explain how a benchmark is executed in Catch. There are three main
steps, though the first does not need to be repeated for every benchmark.
1. *Environmental probe*: before any benchmarks can be executed, the clock's
resolution is estimated. A few other environmental artifacts are also estimated
at this point, like the cost of calling the clock function, but they almost
never have any impact in the results.
2. *Estimation*: the user code is executed a few times to obtain an estimate of
the amount of runs that should be in each sample. This also has the potential
effect of bringing relevant code and data into the caches before the actual
measurement starts.
3. *Measurement*: all the samples are collected sequentially by performing the
number of runs estimated in the previous step for each sample.
This already gives us one important rule for writing benchmarks for Catch: the
benchmarks must be repeatable. The user code will be executed several times, and
the number of times it will be executed during the estimation step cannot be
known beforehand since it depends on the time it takes to execute the code.
User code that cannot be executed repeatedly will lead to bogus results or
crashes.
## Benchmark specification
Benchmarks can be specified anywhere inside a Catch test case.
There is a simple and a slightly more advanced version of the `BENCHMARK` macro.
Let's have a look how a naive Fibonacci implementation could be benchmarked:
```c++
std::uint64_t Fibonacci(std::uint64_t number) {
return number < 2 ? 1 : Fibonacci(number - 1) + Fibonacci(number - 2);
}
```
Now the most straight forward way to benchmark this function, is just adding a `BENCHMARK` macro to our test case:
```c++
TEST_CASE("Fibonacci") {
CHECK(Fibonacci(0) == 1);
// some more asserts..
CHECK(Fibonacci(5) == 8);
// some more asserts..
// now let's benchmark:
BENCHMARK("Fibonacci 20") {
return Fibonacci(20);
};
BENCHMARK("Fibonacci 25") {
return Fibonacci(25);
};
BENCHMARK("Fibonacci 30") {
return Fibonacci(30);
};
BENCHMARK("Fibonacci 35") {
return Fibonacci(35);
};
}
```
There's a few things to note:
- As `BENCHMARK` expands to a lambda expression it is necessary to add a semicolon after
the closing brace (as opposed to the first experimental version).
- The `return` is a handy way to avoid the compiler optimizing away the benchmark code.
Running this already runs the benchmarks and outputs something similar to:
```
-------------------------------------------------------------------------------
Fibonacci
-------------------------------------------------------------------------------
C:\path\to\Catch2\Benchmark.tests.cpp(10)
...............................................................................
benchmark name samples iterations estimated
mean low mean high mean
std dev low std dev high std dev
-------------------------------------------------------------------------------
Fibonacci 20 100 416439 83.2878 ms
2 ns 2 ns 2 ns
0 ns 0 ns 0 ns
Fibonacci 25 100 400776 80.1552 ms
3 ns 3 ns 3 ns
0 ns 0 ns 0 ns
Fibonacci 30 100 396873 79.3746 ms
17 ns 17 ns 17 ns
0 ns 0 ns 0 ns
Fibonacci 35 100 145169 87.1014 ms
468 ns 464 ns 473 ns
21 ns 15 ns 34 ns
```
### Advanced benchmarking
The simplest use case shown above, takes no arguments and just runs the user code that needs to be measured.
However, if using the `BENCHMARK_ADVANCED` macro and adding a `Catch::Benchmark::Chronometer` argument after
the macro, some advanced features are available. The contents of the simple benchmarks are invoked once per run,
while the blocks of the advanced benchmarks are invoked exactly twice:
once during the estimation phase, and another time during the execution phase.
```c++
BENCHMARK("simple"){ return long_computation(); };
BENCHMARK_ADVANCED("advanced")(Catch::Benchmark::Chronometer meter) {
set_up();
meter.measure([] { return long_computation(); });
};
```
These advanced benchmarks no longer consist entirely of user code to be measured.
In these cases, the code to be measured is provided via the
`Catch::Benchmark::Chronometer::measure` member function. This allows you to set up any
kind of state that might be required for the benchmark but is not to be included
in the measurements, like making a vector of random integers to feed to a
sorting algorithm.
A single call to `Catch::Benchmark::Chronometer::measure` performs the actual measurements
by invoking the callable object passed in as many times as necessary. Anything
that needs to be done outside the measurement can be done outside the call to
`measure`.
The callable object passed in to `measure` can optionally accept an `int`
parameter.
```c++
meter.measure([](int i) { return long_computation(i); });
```
If it accepts an `int` parameter, the sequence number of each run will be passed
in, starting with 0. This is useful if you want to measure some mutating code,
for example. The number of runs can be known beforehand by calling
`Catch::Benchmark::Chronometer::runs`; with this one can set up a different instance to be
mutated by each run.
```c++
std::vector<std::string> v(meter.runs());
std::fill(v.begin(), v.end(), test_string());
meter.measure([&v](int i) { in_place_escape(v[i]); });
```
Note that it is not possible to simply use the same instance for different runs
and resetting it between each run since that would pollute the measurements with
the resetting code.
It is also possible to just provide an argument name to the simple `BENCHMARK` macro to get
the same semantics as providing a callable to `meter.measure` with `int` argument:
```c++
BENCHMARK("indexed", i){ return long_computation(i); };
```
### Constructors and destructors
All of these tools give you a lot mileage, but there are two things that still
need special handling: constructors and destructors. The problem is that if you
use automatic objects they get destroyed by the end of the scope, so you end up
measuring the time for construction and destruction together. And if you use
dynamic allocation instead, you end up including the time to allocate memory in
the measurements.
To solve this conundrum, Catch provides class templates that let you manually
construct and destroy objects without dynamic allocation and in a way that lets
you measure construction and destruction separately.
```c++
BENCHMARK_ADVANCED("construct")(Catch::Benchmark::Chronometer meter) {
std::vector<Catch::Benchmark::storage_for<std::string>> storage(meter.runs());
meter.measure([&](int i) { storage[i].construct("thing"); });
};
BENCHMARK_ADVANCED("destroy")(Catch::Benchmark::Chronometer meter) {
std::vector<Catch::Benchmark::destructable_object<std::string>> storage(meter.runs());
for(auto&& o : storage)
o.construct("thing");
meter.measure([&](int i) { storage[i].destruct(); });
};
```
`Catch::Benchmark::storage_for<T>` objects are just pieces of raw storage suitable for `T`
objects. You can use the `Catch::Benchmark::storage_for::construct` member function to call a constructor and
create an object in that storage. So if you want to measure the time it takes
for a certain constructor to run, you can just measure the time it takes to run
this function.
When the lifetime of a `Catch::Benchmark::storage_for<T>` object ends, if an actual object was
constructed there it will be automatically destroyed, so nothing leaks.
If you want to measure a destructor, though, we need to use
`Catch::Benchmark::destructable_object<T>`. These objects are similar to
`Catch::Benchmark::storage_for<T>` in that construction of the `T` object is manual, but
it does not destroy anything automatically. Instead, you are required to call
the `Catch::Benchmark::destructable_object::destruct` member function, which is what you
can use to measure the destruction time.
### The optimizer
Sometimes the optimizer will optimize away the very code that you want to
measure. There are several ways to use results that will prevent the optimiser
from removing them. You can use the `volatile` keyword, or you can output the
value to standard output or to a file, both of which force the program to
actually generate the value somehow.
Catch adds a third option. The values returned by any function provided as user
code are guaranteed to be evaluated and not optimised out. This means that if
your user code consists of computing a certain value, you don't need to bother
with using `volatile` or forcing output. Just `return` it from the function.
That helps with keeping the code in a natural fashion.
Here's an example:
```c++
// may measure nothing at all by skipping the long calculation since its
// result is not used
BENCHMARK("no return"){ long_calculation(); };
// the result of long_calculation() is guaranteed to be computed somehow
BENCHMARK("with return"){ return long_calculation(); };
```
However, there's no other form of control over the optimizer whatsoever. It is
up to you to write a benchmark that actually measures what you want and doesn't
just measure the time to do a whole bunch of nothing.
To sum up, there are two simple rules: whatever you would do in handwritten code
to control optimization still works in Catch; and Catch makes return values
from user code into observable effects that can't be optimized away.
<i>Adapted from nonius' documentation.</i>

View File

@ -12,7 +12,7 @@ Build Systems may refer to low-level tools, like CMake, or larger systems that r
## Continuous Integration systems
Probably the most important aspect to using Catch with a build server is the use of different reporters. Catch comes bundled with three reporters that should cover the majority of build servers out there - although adding more for better integration with some is always a possibility (currently we also offer TeamCity, TAP and Automake reporters).
Probably the most important aspect to using Catch with a build server is the use of different reporters. Catch comes bundled with three reporters that should cover the majority of build servers out there - although adding more for better integration with some is always a possibility (currently we also offer TeamCity, TAP, Automake and SonarQube reporters).
Two of these reporters are built in (XML and JUnit) and the third (TeamCity) is included as a separate header. It's possible that the other two may be split out in the future too - as that would make the core of Catch smaller for those that don't need them.
@ -65,6 +65,10 @@ The Automake Reporter writes out the [meta tags](https://www.gnu.org/software/au
Because of the incremental nature of Catch's test suites and ability to run specific tests, our implementation of TAP reporter writes out the number of tests in a suite last.
### SonarQube Reporter
```-r sonarqube```
[SonarQube Generic Test Data](https://docs.sonarqube.org/latest/analysis/generic-test/) XML format for tests metrics.
## Low-level tools
### Precompiled headers (PCHs)
@ -91,7 +95,7 @@ can use `pkg-config` to get its include path: `pkg-config --cflags catch2`.
### gdb and lldb scripts
Catch2's `contrib` folder also contains two simple debugger scripts,
Catch2's `extras` folder also contains two simple debugger scripts,
`gdbinit` for `gdb` and `lldbinit` for `lldb`. If loaded into their
respective debugger, these will tell it to step over Catch2's internals
when stepping through code.

View File

@ -2,10 +2,12 @@
# CMake integration
**Contents**<br>
[CMake target](#cmake-target)<br>
[CMake targets](#cmake-targets)<br>
[Automatic test registration](#automatic-test-registration)<br>
[CMake project options](#cmake-project-options)<br>
[`CATCH_CONFIG_*` customization options in CMake](#catch_config_-customization-options-in-cmake)<br>
[Installing Catch2 from git repository](#installing-catch2-from-git-repository)<br>
[Installing Catch2 from vcpkg](#installing-catch2-from-vcpkg)<br>
Because we use CMake to build Catch2, we also provide a couple of
integration points for our users.
@ -14,41 +16,67 @@ integration points for our users.
2) Catch2's repository contains CMake scripts for automatic registration
of `TEST_CASE`s in CTest
## CMake target
## CMake targets
Catch2's CMake build exports an interface target `Catch2::Catch2`. Linking
against it will add the proper include path and all necessary capabilities
to the resulting binary.
Catch2's CMake build exports two targets, `Catch2::Catch2`, and
`Catch2::Catch2WithMain`. If you do not need custom `main` function,
you should be using the latter (and only the latter). Linking against
it will add the proper include paths and link your target together with
2 static libraries that implement Catch2 and its main respectively.
If you need custom `main`, you should link only against `Catch2::Catch2`.
This means that if Catch2 has been installed on the system, it should be
enough to do:
This means that if Catch2 has been installed on the system, it should
be enough to do
```cmake
find_package(Catch2 REQUIRED)
target_link_libraries(tests Catch2::Catch2)
find_package(Catch2 3 REQUIRED)
# These tests can use the Catch2-provided main
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)
# These tests need their own main
add_executable(custom-main-tests test.cpp test-main.cpp)
target_link_libraries(custom-main-tests PRIVATE Catch2::Catch2)
```
These targets are also provided when Catch2 is used as a subdirectory.
Assuming Catch2 has been cloned to `lib/Catch2`, you only need to replace
the `find_package` call with `add_subdirectory(lib/Catch2)` and the snippet
above still works.
This target is also provided when Catch2 is used as a subdirectory.
Assuming that Catch2 has been cloned to `lib/Catch2`:
Another possibility is to use [FetchContent](https://cmake.org/cmake/help/latest/module/FetchContent.html):
```cmake
add_subdirectory(lib/Catch2)
target_link_libraries(tests Catch2::Catch2)
Include(FetchContent)
FetchContent_Declare(
Catch2
GIT_REPOSITORY https://github.com/catchorg/Catch2.git
GIT_TAG v3.0.1 # or a later release
)
FetchContent_MakeAvailable(Catch2)
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)
```
## Automatic test registration
Catch2's repository also contains two CMake scripts that help users
Catch2's repository also contains three CMake scripts that help users
with automatically registering their `TEST_CASE`s with CTest. They
can be found in the `contrib` folder, and are
can be found in the `extras` folder, and are
1) `Catch.cmake` (and its dependency `CatchAddTests.cmake`)
2) `ParseAndAddCatchTests.cmake`
2) `ParseAndAddCatchTests.cmake` (deprecated)
3) `CatchShardTests.cmake` (and its dependency `CatchShardTestsImpl.cmake`)
If Catch2 has been installed in system, both of these can be used after
doing `find_package(Catch2 REQUIRED)`. Otherwise you need to add them
to your CMake module path.
### `Catch.cmake` and `AddCatchTests.cmake`
<a id="catch_discover_tests"></a>
### `Catch.cmake` and `CatchAddTests.cmake`
`Catch.cmake` provides function `catch_discover_tests` to get tests from
a target. This function works by running the resulting executable with
@ -63,13 +91,25 @@ project(baz LANGUAGES CXX VERSION 0.0.1)
find_package(Catch2 REQUIRED)
add_executable(foo test.cpp)
target_link_libraries(foo Catch2::Catch2)
target_link_libraries(foo PRIVATE Catch2::Catch2)
include(CTest)
include(Catch)
catch_discover_tests(foo)
```
When using `FetchContent`, `include(Catch)` will fail unless
`CMAKE_MODULE_PATH` is explicitly updated to include the extras
directory.
```cmake
# ... FetchContent ...
#
list(APPEND CMAKE_MODULE_PATH ${catch2_SOURCE_DIR}/extras)
include(CTest)
include(Catch)
catch_discover_tests()
```
#### Customization
`catch_discover_tests` can be given several extra argumets:
@ -82,6 +122,10 @@ catch_discover_tests(target
[TEST_SUFFIX suffix]
[PROPERTIES name1 value1...]
[TEST_LIST var]
[REPORTER reporter]
[OUTPUT_DIR dir]
[OUTPUT_PREFIX prefix]
[OUTPUT_SUFFIX suffix]
)
```
@ -128,13 +172,46 @@ default `<target>_TESTS`. This can be useful when the same test
executable is being used in multiple calls to `catch_discover_tests()`.
Note that this variable is only available in CTest.
* `REPORTER reporter`
Use the specified reporter when running the test case. The reporter will
be passed to the test runner as `--reporter reporter`.
* `OUTPUT_DIR dir`
If specified, the parameter is passed along as
`--out dir/<test_name>` to test executable. The actual file name is the
same as the test name. This should be used instead of
`EXTRA_ARGS --out foo` to avoid race conditions writing the result output
when using parallel test execution.
* `OUTPUT_PREFIX prefix`
May be used in conjunction with `OUTPUT_DIR`.
If specified, `prefix` is added to each output file name, like so
`--out dir/prefix<test_name>`.
* `OUTPUT_SUFFIX suffix`
May be used in conjunction with `OUTPUT_DIR`.
If specified, `suffix` is added to each output file name, like so
`--out dir/<test_name>suffix`. This can be used to add a file extension to
the output file name e.g. ".xml".
### `ParseAndAddCatchTests.cmake`
⚠ This script is [deprecated](https://github.com/catchorg/Catch2/pull/2120)
in Catch2 2.13.4 and superseded by the above approach using `catch_discover_tests`.
See [#2092](https://github.com/catchorg/Catch2/issues/2092) for details.
`ParseAndAddCatchTests` works by parsing all implementation files
associated with the provided target, and registering them via CTest's
`add_test`. This approach has some limitations, such as the fact that
commented-out tests will be registered anyway.
commented-out tests will be registered anyway. More serious, only a
subset of the assertion macros currently available in Catch can be
detected by this script and tests with any macros that cannot be
parsed are *silently ignored*.
#### Usage
@ -146,7 +223,7 @@ project(baz LANGUAGES CXX VERSION 0.0.1)
find_package(Catch2 REQUIRED)
add_executable(foo test.cpp)
target_link_libraries(foo Catch2::Catch2)
target_link_libraries(foo PRIVATE Catch2::Catch2)
include(CTest)
include(ParseAndAddCatchTests)
@ -160,7 +237,7 @@ ParseAndAddCatchTests(foo)
* `PARSE_CATCH_TESTS_VERBOSE` -- When `ON`, the script prints debug
messages. Defaults to `OFF`.
* `PARSE_CATCH_TESTS_NO_HIDDEN_TESTS` -- When `ON`, hidden tests (tests
tagged with any of `[!hide]`, `[.]` or `[.foo]`) will not be registered.
tagged with either of `[.]` or `[.foo]`) will not be registered.
Defaults to `OFF`.
* `PARSE_CATCH_TESTS_ADD_FIXTURE_IN_TEST_NAME` -- When `ON`, adds fixture
class name to the test name in CTest. Defaults to `ON`.
@ -182,10 +259,68 @@ unset(OptionalCatchTestLauncher)
ParseAndAddCatchTests(bar)
```
### `CatchShardTests.cmake`
> `CatchShardTests.cmake` was introduced in Catch2 3.1.0.
`CatchShardTests.cmake` provides a function
`catch_add_sharded_tests(TEST_BINARY)` that splits tests from `TEST_BINARY`
into multiple shards. The tests in each shard and their order is randomized,
and the seed changes every invocation of CTest.
Currently there are 3 customization points for this script:
* SHARD_COUNT - number of shards to split target's tests into
* REPORTER - reporter spec to use for tests
* TEST_SPEC - test spec used for filtering tests
Example usage:
```
include(CatchShardTests)
catch_add_sharded_tests(foo-tests
SHARD_COUNT 4
REPORTER "xml::out=-"
TEST_SPEC "A"
)
catch_add_sharded_tests(tests
SHARD_COUNT 8
REPORTER "xml::out=-"
TEST_SPEC "B"
)
```
This registers total of 12 CTest tests (4 + 8 shards) to run shards
from `foo-tests` test binary, filtered by a test spec.
_Note that this script is currently a proof-of-concept for reseeding
shards per CTest run, and thus does not support (nor does it currently
aim to support) all customization points from
[`catch_discover_tests`](#catch_discover_tests)._
## CMake project options
Catch2's CMake project also provides some options for other projects
that consume it. These are
that consume it. These are:
* `BUILD_TESTING` -- When `ON` and the project is not used as a subproject,
Catch2's test binary will be built. Defaults to `ON`.
* `CATCH_INSTALL_DOCS` -- When `ON`, Catch2's documentation will be
included in the installation. Defaults to `ON`.
* `CATCH_INSTALL_EXTRAS` -- When `ON`, Catch2's extras folder (the CMake
scripts mentioned above, debugger helpers) will be included in the
installation. Defaults to `ON`.
* `CATCH_DEVELOPMENT_BUILD` -- When `ON`, configures the build for development
of Catch2. This means enabling test projects, warnings and so on.
Defaults to `OFF`.
Enabling `CATCH_DEVELOPMENT_BUILD` also enables further configuration
customization options:
* `CATCH_BUILD_TESTING` -- When `ON`, Catch2's SelfTest project will be
built. Defaults to `ON`. Note that Catch2 also obeys `BUILD_TESTING` CMake
@ -193,12 +328,40 @@ variable, so _both_ of them need to be `ON` for the SelfTest to be built,
and either of them can be set to `OFF` to disable building SelfTest.
* `CATCH_BUILD_EXAMPLES` -- When `ON`, Catch2's usage examples will be
built. Defaults to `OFF`.
* `CATCH_INSTALL_DOCS` -- When `ON`, Catch2's documentation will be
included in the installation. Defaults to `ON`.
* `CATCH_INSTALL_HELPERS` -- When `ON`, Catch2's contrib folder will be
included in the installation. Defaults to `ON`.
* `BUILD_TESTING` -- When `ON` and the project is not used as a subproject,
Catch2's test binary will be built. Defaults to `ON`.
* `CATCH_BUILD_EXTRA_TESTS` -- When `ON`, Catch2's extra tests will be
built. Defaults to `OFF`.
* `CATCH_BUILD_FUZZERS` -- When `ON`, Catch2 fuzzing entry points will
be built. Defaults to `OFF`.
* `CATCH_ENABLE_WERROR` -- When `ON`, adds `-Werror` or equivalent flag
to the compilation. Defaults to `ON`.
* `CATCH_BUILD_SURROGATES` -- When `ON`, each header in Catch2 will be
compiled separately to ensure that they are self-sufficient.
Defaults to `OFF`.
## `CATCH_CONFIG_*` customization options in CMake
> CMake support for `CATCH_CONFIG_*` options was introduced in Catch2 3.0.1
Due to the new separate compilation model, all the options from the
[Compile-time configuration docs](configuration.md#top) can also be set
through Catch2's CMake. To set them, define the option you want as `ON`,
e.g. `-DCATCH_CONFIG_NOSTDOUT=ON`.
Note that setting the option to `OFF` doesn't disable it. To force disable
an option, you need to set the `_NO_` form of it to `ON`, e.g.
`-DCATCH_CONFIG_NO_COLOUR_WIN32=ON`.
To summarize the configuration option behaviour with an example:
| `-DCATCH_CONFIG_COLOUR_WIN32` | `-DCATCH_CONFIG_NO_COLOUR_WIN32` | Result |
|-------------------------------|----------------------------------|-------------|
| `ON` | `ON` | error |
| `ON` | `OFF` | force-on |
| `OFF` | `ON` | force-off |
| `OFF` | `OFF` | auto-detect |
## Installing Catch2 from git repository
@ -220,6 +383,19 @@ when configuring the build, and then modify your calls to
[find_package](https://cmake.org/cmake/help/latest/command/find_package.html)
accordingly.
## Installing Catch2 from vcpkg
Alternatively, you can build and install Catch2 using [vcpkg](https://github.com/microsoft/vcpkg/) dependency manager:
```
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install catch2
```
The catch2 port in vcpkg is kept up to date by microsoft team members and community contributors.
If the version is out of date, please [create an issue or pull request](https://github.com/Microsoft/vcpkg) on the vcpkg repository.
---

View File

@ -15,24 +15,29 @@
[Warnings](#warnings)<br>
[Reporting timings](#reporting-timings)<br>
[Load test names to run from a file](#load-test-names-to-run-from-a-file)<br>
[Just test names](#just-test-names)<br>
[Specify the order test cases are run](#specify-the-order-test-cases-are-run)<br>
[Specify a seed for the Random Number Generator](#specify-a-seed-for-the-random-number-generator)<br>
[Identify framework and version according to the libIdentify standard](#identify-framework-and-version-according-to-the-libidentify-standard)<br>
[Wait for key before continuing](#wait-for-key-before-continuing)<br>
[Specify multiples of clock resolution to run benchmarks for](#specify-multiples-of-clock-resolution-to-run-benchmarks-for)<br>
[Skip all benchmarks](#skip-all-benchmarks)<br>
[Specify the number of benchmark samples to collect](#specify-the-number-of-benchmark-samples-to-collect)<br>
[Specify the number of resamples for bootstrapping](#specify-the-number-of-resamples-for-bootstrapping)<br>
[Specify the confidence-interval for bootstrapping](#specify-the-confidence-interval-for-bootstrapping)<br>
[Disable statistical analysis of collected benchmark samples](#disable-statistical-analysis-of-collected-benchmark-samples)<br>
[Specify the amount of time in milliseconds spent on warming up each test](#specify-the-amount-of-time-in-milliseconds-spent-on-warming-up-each-test)<br>
[Usage](#usage)<br>
[Specify the section to run](#specify-the-section-to-run)<br>
[Filenames as tags](#filenames-as-tags)<br>
[Override output colouring](#override-output-colouring)<br>
[Test Sharding](#test-sharding)<br>
[Allow running the binary without tests](#allow-running-the-binary-without-tests)<br>
[Output verbosity](#output-verbosity)<br>
Catch works quite nicely without any command line options at all - but for those times when you want greater control the following options are available.
Click one of the followings links to take you straight to that option - or scroll on to browse the available options.
Click one of the following links to take you straight to that option - or scroll on to browse the available options.
<a href="#specifying-which-tests-to-run"> ` <test-spec> ...`</a><br />
<a href="#usage"> ` -h, -?, --help`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -l, --list-tests`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -t, --list-tags`</a><br />
<a href="#showing-results-for-successful-tests"> ` -s, --success`</a><br />
<a href="#breaking-into-the-debugger"> ` -b, --break`</a><br />
<a href="#eliding-assertions-expected-to-throw"> ` -e, --nothrow`</a><br />
@ -51,14 +56,25 @@ Click one of the followings links to take you straight to that option - or scrol
</br>
<a href="#list-test-names-only"> ` --list-test-names-only`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-tests`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-tags`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-reporters`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-listeners`</a><br />
<a href="#order"> ` --order`</a><br />
<a href="#rng-seed"> ` --rng-seed`</a><br />
<a href="#libidentify"> ` --libidentify`</a><br />
<a href="#wait-for-keypress"> ` --wait-for-keypress`</a><br />
<a href="#benchmark-resolution-multiple"> ` --benchmark-resolution-multiple`</a><br />
<a href="#use-colour"> ` --use-colour`</a><br />
<a href="#skip-benchmarks"> ` --skip-benchmarks`</a><br />
<a href="#benchmark-samples"> ` --benchmark-samples`</a><br />
<a href="#benchmark-resamples"> ` --benchmark-resamples`</a><br />
<a href="#benchmark-confidence-interval"> ` --benchmark-confidence-interval`</a><br />
<a href="#benchmark-no-analysis"> ` --benchmark-no-analysis`</a><br />
<a href="#benchmark-warmup-time"> ` --benchmark-warmup-time`</a><br />
<a href="#colour-mode"> ` --colour-mode`</a><br />
<a href="#test-sharding"> ` --shard-count`</a><br />
<a href="#test-sharding"> ` --shard-index`</a><br />
<a href=#no-tests-override> ` --allow-running-no-tests`</a><br />
<a href=#output-verbosity> ` --verbosity`</a><br />
</br>
@ -85,7 +101,8 @@ Inclusions and exclusions are evaluated in left-to-right order.
Test case examples:
<pre>thisTestOnly Matches the test case called, 'thisTestOnly'
```
thisTestOnly Matches the test case called, 'thisTestOnly'
"this test only" Matches the test case called, 'this test only'
these* Matches all cases starting with 'these'
exclude:notThis Matches all tests except, 'notThis'
@ -93,7 +110,9 @@ exclude:notThis Matches all tests except, 'notThis'
~*private* Matches all tests except those that contain 'private'
a* ~ab* abc Matches all tests that start with 'a', except those that
start with 'ab', except 'abc', which is included
</pre>
~[tag1] Matches all tests except those tagged with '[tag1]'
-# [#somefile] Matches all tests from the file 'somefile.cpp'
```
Names within square brackets are interpreted as tags.
A series of tags form an AND expression whereas a comma-separated sequence forms an OR expression. e.g.:
@ -107,18 +126,48 @@ Test names containing special characters, such as `,` or `[` can specify them on
<a id="choosing-a-reporter-to-use"></a>
## Choosing a reporter to use
<pre>-r, --reporter &lt;reporter></pre>
<pre>-r, --reporter &lt;reporter[::key=value]*&gt;</pre>
A reporter is an object that formats and structures the output of running tests, and potentially summarises the results. By default a console reporter is used that writes, IDE friendly, textual output. Catch comes bundled with some alternative reporters, but more can be added in client code.<br />
The bundled reporters are:
Reporters are how the output from Catch2 (results of assertions, tests,
benchmarks and so on) is formatted and written out. The default reporter
is called the "Console" reporter and is intended to provide relatively
verbose and human-friendly output.
<pre>-r console
-r compact
-r xml
-r junit
</pre>
Reporters are also individually configurable. To pass configuration options
to the reporter, you append `::key=value` to the reporter specification
as many times as you want, e.g. `--reporter xml::out=someFile.xml`.
The keys must either be prefixed by "X", in which case they are not parsed
by Catch2 and are only passed down to the reporter, or one of options
hardcoded into Catch2. Currently there are only 2,
["out"](#sending-output-to-a-file), and ["colour-mode"](#colour-mode).
_Note that the reporter might still check the X-prefixed options for
validity, and throw an error if they are wrong._
> Support for passing arguments to reporters through the `-r`, `--reporter` flag was introduced in Catch2 3.0.1
There are multiple built-in reporters, you can see what they do by using the
[`--list-reporter`](command-line.md#listing-available-tests-tags-or-reporters)
flag. If you need a reporter providing custom format outside of the already
provided ones, look at the ["write your own reporter" part of the reporter
documentation](reporters.md#writing-your-own-reporter).
This option may be passed multiple times to use multiple (different)
reporters at the same time. See the [reporter documentation](reporters.md#multiple-reporters)
for details on what the resulting behaviour is. Also note that at most one
reporter can be provided without the output-file part of reporter spec.
This reporter will use the "default" output destination, based on
the [`-o`, `--out`](#sending-output-to-a-file) option.
> Support for using multiple different reporters at the same time was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 3.0.1
_Note: There is currently no way to escape `::` in the reporter spec,
and thus the reporter names, or configuration keys and values, cannot
contain `::`. As `::` in paths is relatively obscure (unlike ':'), we do
not consider this an issue._
The JUnit reporter is an xml format that follows the structure of the JUnit XML Report ANT task, as consumed by a number of third-party tools, including Continuous Integration servers such as Hudson. If not otherwise needed, the standard XML reporter is preferred as this is a streaming reporter, whereas the Junit reporter needs to hold all its results until the end so it can write the overall results into attributes of the root node.
<a id="breaking-into-the-debugger"></a>
## Breaking into the debugger
@ -148,24 +197,62 @@ Sometimes this results in a flood of failure messages and you'd rather just see
<a id="listing-available-tests-tags-or-reporters"></a>
## Listing available tests, tags or reporters
<pre>-l, --list-tests
-t, --list-tags
```
--list-tests
--list-tags
--list-reporters
</pre>
--list-listeners
```
```-l``` or ```--list-tests``` will list all registered tests, along with any tags.
If one or more test-specs have been supplied too then only the matching tests will be listed.
> The `--list*` options became customizable through reporters in Catch2 3.0.1
```-t``` or ```--list-tags``` lists all available tags, along with the number of test cases they match. Again, supplying test specs limits the tags that match.
> The `--list-listeners` option was added in Catch2 3.0.1
```--list-reporters``` lists the available reporters.
`--list-tests` lists all registered tests matching specified test spec.
Usually this listing also includes tags, and potentially also other
information, like source location, based on verbosity and reporter's design.
`--list-tags` lists all tags from registered tests matching specified test
spec. Usually this also includes number of tests cases they match and
similar information.
`--list-reporters` lists all available reporters and their descriptions.
`--list-listeners` lists all registered listeners and their descriptions.
The [`--verbosity` argument](#output-verbosity) modifies the level of detail provided by the default `--list*` options
as follows:
| Option | `normal` (default) | `quiet` | `high` |
|--------------------|---------------------------------|---------------------|-----------------------------------------|
| `--list-tests` | Test names and tags | Test names only | Same as `normal`, plus source code line |
| `--list-tags` | Tags and counts | Same as `normal` | Same as `normal` |
| `--list-reporters` | Reporter names and descriptions | Reporter names only | Same as `normal` |
| `--list-listeners` | Listener names and descriptions | Same as `normal` | Same as `normal` |
<a id="sending-output-to-a-file"></a>
## Sending output to a file
<pre>-o, --out &lt;filename>
<pre>-o, --out &lt;filename&gt;
</pre>
Use this option to send all output to a file. By default output is sent to stdout (note that uses of stdout and stderr *from within test cases* are redirected and included in the report - so even stderr will effectively end up on stdout).
Use this option to send all output to a file, instead of stdout. You can
use `-` as the filename to explicitly send the output to stdout (this is
useful e.g. when using multiple reporters).
> Support for `-` as the filename was introduced in Catch2 3.0.1
Filenames starting with "%" (percent symbol) are reserved by Catch2 for
meta purposes, e.g. using `%debug` as the filename opens stream that
writes to platform specific debugging/logging mechanism.
Catch2 currently recognizes 3 meta streams:
* `%debug` - writes to platform specific debugging/logging output
* `%stdout` - writes to stdout
* `%stderr` - writes to stderr
> Support for `%stdout` and `%stderr` was introduced in Catch2 3.0.1
<a id="naming-a-test-run"></a>
## Naming a test run
@ -196,16 +283,24 @@ This option transforms tabs and newline characters into ```\t``` and ```\n``` re
## Warnings
<pre>-w, --warn &lt;warning name></pre>
Enables reporting of suspicious test states. There are currently two
available warnings
You can think of Catch2's warnings as the equivalent of `-Werror` (`/WX`)
flag for C++ compilers. It turns some suspicious occurences, like a section
without assertions, into errors. Because these might be intended, warnings
are not enabled by default, but user can opt in.
You can enable multiple warnings at the same time.
There are currently two warnings implemented:
```
NoAssertions // Fail test case / leaf section if no assertions
// (e.g. `REQUIRE`) is encountered.
NoTests // Return non-zero exit code when no test cases were run
// Also calls reporter's noMatchingTestCases method
NoAssertions // Fail test case / leaf section if no assertions
// (e.g. `REQUIRE`) is encountered.
UnmatchedTestSpec // Fail test run if any of the CLI test specs did
// not match any tests.
```
> `UnmatchedTestSpec` was introduced in Catch2 3.0.1.
<a id="reporting-timings"></a>
## Reporting timings
@ -213,19 +308,27 @@ available warnings
When set to ```yes``` Catch will report the duration of each test case, in milliseconds. Note that it does this regardless of whether a test case passes or fails. Note, also, the certain reporters (e.g. Junit) always report test case durations regardless of this option being set or not.
<pre>-D, --min-duration &lt;value></pre>
> `--min-duration` was [introduced](https://github.com/catchorg/Catch2/pull/1910) in Catch2 2.13.0
When set, Catch will report the duration of each test case that took more
than &lt;value> seconds, in milliseconds. This option is overriden by both
`-d yes` and `-d no`, so that either all durations are reported, or none
are.
<a id="input-file"></a>
## Load test names to run from a file
<pre>-f, --input-file &lt;filename></pre>
Provide the name of a file that contains a list of test case names - one per line. Blank lines are skipped and anything after the comment character, ```#```, is ignored.
Provide the name of a file that contains a list of test case names,
one per line. Blank lines are skipped.
A useful way to generate an initial instance of this file is to use the <a href="#list-test-names-only">list-test-names-only</a> option. This can then be manually curated to specify a specific subset of tests - or in a specific order.
<a id="list-test-names-only"></a>
## Just test names
<pre>--list-test-names-only</pre>
This option lists all available tests in a non-indented form, one on each line. This makes it ideal for saving to a file and feeding back into the <a href="#input-file">```-f``` or ```--input-file```</a> option.
A useful way to generate an initial instance of this file is to combine
the [`--list-tests`](#listing-available-tests-tags-or-reporters) flag with
the [`--verbosity quiet`](#output-verbosity) option. You can also
use test specs to filter this list down to what you want first.
<a id="order"></a>
@ -234,25 +337,48 @@ This option lists all available tests in a non-indented form, one on each line.
Test cases are ordered one of three ways:
### decl
Declaration order. The order the tests were originally declared in. Note that ordering between files is not guaranteed and is implementation dependent.
Declaration order (this is the default order if no --order argument is provided).
Tests in the same TU are sorted using their declaration orders, different
TUs are in an implementation (linking) dependent order.
### lex
Lexicographically sorted. Tests are sorted, alpha-numerically, by name.
Lexicographic order. Tests are sorted by their name, their tags are ignored.
### rand
Randomly sorted. Test names are sorted using ```std::random_shuffle()```. By default the random number generator is seeded with 0 - and so the order is repeatable. To control the random seed see <a href="#rng-seed">rng-seed</a>.
Randomly ordered. The order is dependent on Catch2's random seed (see
[`--rng-seed`](#rng-seed)), and is subset invariant. What this means
is that as long as the random seed is fixed, running only some tests
(e.g. via tag) does not change their relative order.
> The subset stability was introduced in Catch2 v2.12.0
Since the random order was made subset stable, we promise that given
the same random seed, the order of test cases will be the same across
different platforms, as long as the tests were compiled against identical
version of Catch2. We reserve the right to change the relative order
of tests cases between Catch2 versions, but it is unlikely to happen often.
<a id="rng-seed"></a>
## Specify a seed for the Random Number Generator
<pre>--rng-seed &lt;'time'|number&gt;</pre>
<pre>--rng-seed &lt;'time'|'random-device'|number&gt;</pre>
Sets a seed for the random number generator using ```std::srand()```.
If a number is provided this is used directly as the seed so the random pattern is repeatable.
Alternatively if the keyword ```time``` is provided then the result of calling ```std::time(0)``` is used and so the pattern becomes unpredictable.
Sets the seed for random number generators used by Catch2. These are used
e.g. to shuffle tests when user asks for tests to be in random order.
Using `time` as the argument asks Catch2 generate the seed through call
to `std::time(nullptr)`. This provides very weak randomness and multiple
runs of the binary can generate the same seed if they are started close
to each other.
Using `random-device` asks for `std::random_device` to be used instead.
If your implementation provides working `std::random_device`, it should
be preferred to using `time`. Catch2 uses `std::random_device` by default.
In either case the actual value for the seed is printed as part of Catch's output so if an issue is discovered that is sensitive to test ordering the ordering can be reproduced - even if it was originally seeded from ```std::time(0)```.
<a id="libidentify"></a>
## Identify framework and version according to the libIdentify standard
@ -262,18 +388,71 @@ See [The LibIdentify repo for more information and examples](https://github.com/
<a id="wait-for-keypress"></a>
## Wait for key before continuing
<pre>--wait-for-keypress &lt;start|exit|both&gt;</pre>
<pre>--wait-for-keypress &lt;never|start|exit|both&gt;</pre>
Will cause the executable to print a message and wait until the return/ enter key is pressed before continuing -
either before running any tests, after running all tests - or both, depending on the argument.
<a id="benchmark-resolution-multiple"></a>
## Specify multiples of clock resolution to run benchmarks for
<pre>--benchmark-resolution-multiple &lt;multiplier&gt;</pre>
<a id="skip-benchmarks"></a>
## Skip all benchmarks
<pre>--skip-benchmarks</pre>
When running benchmarks the clock resolution is estimated. Benchmarks are then run for exponentially increasing
numbers of iterations until some multiple of the estimated resolution is exceed. By default that multiple is 100, but
it can be overridden here.
> [Introduced](https://github.com/catchorg/Catch2/issues/2408) in Catch2 3.0.1.
This flag tells Catch2 to skip running all benchmarks. Benchmarks in this
case mean code blocks in `BENCHMARK` and `BENCHMARK_ADVANCED` macros, not
test cases with the `[!benchmark]` tag.
<a id="benchmark-samples"></a>
## Specify the number of benchmark samples to collect
<pre>--benchmark-samples &lt;# of samples&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
When running benchmarks a number of "samples" is collected. This is the base data for later statistical analysis.
Per sample a clock resolution dependent number of iterations of the user code is run, which is independent of the number of samples. Defaults to 100.
<a id="benchmark-resamples"></a>
## Specify the number of resamples for bootstrapping
<pre>--benchmark-resamples &lt;# of resamples&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
After the measurements are performed, statistical [bootstrapping] is performed
on the samples. The number of resamples for that bootstrapping is configurable
but defaults to 100000. Due to the bootstrapping it is possible to give
estimates for the mean and standard deviation. The estimates come with a lower
bound and an upper bound, and the confidence interval (which is configurable but
defaults to 95%).
[bootstrapping]: http://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29
<a id="benchmark-confidence-interval"></a>
## Specify the confidence-interval for bootstrapping
<pre>--benchmark-confidence-interval &lt;confidence-interval&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
The confidence-interval is used for statistical bootstrapping on the samples to
calculate the upper and lower bounds of mean and standard deviation.
Must be between 0 and 1 and defaults to 0.95.
<a id="benchmark-no-analysis"></a>
## Disable statistical analysis of collected benchmark samples
<pre>--benchmark-no-analysis</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
When this flag is specified no bootstrapping or any other statistical analysis is performed.
Instead the user code is only measured and the plain mean from the samples is reported.
<a id="benchmark-warmup-time"></a>
## Specify the amount of time in milliseconds spent on warming up each test
<pre>--benchmark-warmup-time</pre>
> [Introduced](https://github.com/catchorg/Catch2/pull/1844) in Catch2 2.11.2.
Configure the amount of time spent warming up each test.
<a id="usage"></a>
## Usage
@ -320,7 +499,7 @@ There are some limitations of this feature to be aware of:
- Code outside of sections being skipped will still be executed - e.g. any set-up code in the TEST_CASE before the
start of the first section.</br>
- At time of writing, wildcards are not supported in section names.
- If you specify a section without narrowing to a test case first then all test cases will be executed
- If you specify a section without narrowing to a test case first then all test cases will be executed
(but only matching sections within them).
@ -328,21 +507,77 @@ start of the first section.</br>
## Filenames as tags
<pre>-#, --filenames-as-tags</pre>
When this option is used then every test is given an additional tag which is formed of the unqualified
When this option is used then every test is given an additional tag which is formed of the unqualified
filename it is found in, with any extension stripped, prefixed with the `#` character.
So, for example, tests within the file `~\Dev\MyProject\Ferrets.cpp` would be tagged `[#Ferrets]`.
<a id="use-colour"></a>
<a id="colour-mode"></a>
## Override output colouring
<pre>--use-colour &lt;yes|no|auto&gt;</pre>
<pre>--colour-mode &lt;ansi|win32|none|default&gt;</pre>
Catch colours output for terminals, but omits colouring when it detects that
output is being sent to a pipe. This is done to avoid interfering with automated
processing of output.
> The `--colour-mode` option replaced the old `--colour` option in Catch2 3.0.1
Catch2 support two different ways of colouring terminal output, and by
default it attempts to make a good guess on which implementation to use
(and whether to even use it, e.g. Catch2 tries to avoid writing colour
codes when writing the results into a file).
`--colour-mode` allows the user to explicitly select what happens.
* `--colour-mode ansi` tells Catch2 to always use ANSI colour codes, even
when writing to a file
* `--colour-mode win32` tells Catch2 to use colour implementation based
on Win32 terminal API
* `--colour-mode none` tells Catch2 to disable colours completely
* `--colour-mode default` lets Catch2 decide
`--colour-mode default` is the default setting.
<a id="test-sharding"></a>
## Test Sharding
<pre>--shard-count <#number of shards>, --shard-index <#shard index to run></pre>
> [Introduced](https://github.com/catchorg/Catch2/pull/2257) in Catch2 3.0.1.
When `--shard-count <#number of shards>` is used, the tests to execute
will be split evenly in to the given number of sets, identified by indices
starting at 0. The tests in the set given by
`--shard-index <#shard index to run>` will be executed. The default shard
count is `1`, and the default index to run is `0`.
_It is an error to specify a shard index greater than the number of shards._
Sharding is useful when you want to split test execution across multiple
processes, as is done with the [Bazel test sharding](https://docs.bazel.build/versions/main/test-encyclopedia.html#test-sharding).
<a id="no-tests-override"></a>
## Allow running the binary without tests
<pre>--allow-running-no-tests</pre>
> Introduced in Catch2 3.0.1.
By default, Catch2 test binaries return non-0 exit code if no tests were
run, e.g. if the binary was compiled with no tests, or the provided test
spec matched no tests. This flag overrides that, so a test run with no
tests still returns 0.
## Output verbosity
```
-v, --verbosity <quiet|normal|high>
```
Changing verbosity might change how much details Catch2's reporters output.
However, you should consider changing the verbosity level as a _suggestion_.
Not all reporters support all verbosity levels, e.g. because the reporter's
format cannot meaningfully change. In that case, the verbosity level is
ignored.
Verbosity defaults to _normal_.
`--use-colour yes` forces coloured output, `--use-colour no` disables coloured
output. The default behaviour is `--use-colour auto`.
---

View File

@ -1,18 +1,23 @@
<a id="top"></a>
# Commercial users of Catch
# Commercial users of Catch2
As well as [Open Source](opensource-users.md#top) users Catch is widely used within proprietary code bases too.
Many organisations like to keep this information internal, and that's fine,
but if you're more open it would be great if we could list the names of as
many organisations as possible that use Catch somewhere in their codebase.
Enterprise environments often tend to be far more conservative in their tool adoption -
and being aware that other companies are using Catch can ease the path in.
Catch2 is also widely used in proprietary code bases. This page contains
some of them that are willing to share this information.
So if you are aware of Catch usage in your organisation, and are fairly confident there is no issue with sharing this
fact then please let us know - either directly, via a PR or
[issue](https://github.com/philsquared/Catch/issues), or on the [forums](https://groups.google.com/forum/?fromgroups#!forum/catch-forum).
If you want to add your organisation, please check that there is no issue
with you sharing this fact.
- Bloomberg
- [Bloomlife](https://bloomlife.com)
- NASA
- [Inscopix Inc.](https://www.inscopix.com/)
- Locksley.CZ
- [Makimo](https://makimo.pl/)
- NASA
- [Nexus Software Systems](https://nexwebsites.com)
- [UX3D](https://ux3d.io)
- [King](https://king.com)
---
[Home](Readme.md#top)

View File

@ -2,39 +2,26 @@
# Compile-time configuration
**Contents**<br>
[main()/ implementation](#main-implementation)<br>
[Reporter / Listener interfaces](#reporter--listener-interfaces)<br>
[Prefixing Catch macros](#prefixing-catch-macros)<br>
[Terminal colour](#terminal-colour)<br>
[Console width](#console-width)<br>
[stdout](#stdout)<br>
[Fallback stringifier](#fallback-stringifier)<br>
[Default reporter](#default-reporter)<br>
[Bazel support](#bazel-support)<br>
[C++11 toggles](#c11-toggles)<br>
[C++17 toggles](#c17-toggles)<br>
[Other toggles](#other-toggles)<br>
[Windows header clutter](#windows-header-clutter)<br>
[Enabling stringification](#enabling-stringification)<br>
[Disabling exceptions](#disabling-exceptions)<br>
[Overriding Catch's debug break (`-b`)](#overriding-catchs-debug-break--b)<br>
Catch is designed to "just work" as much as possible. For most people the only configuration needed is telling Catch which source file should host all the implementation code (```CATCH_CONFIG_MAIN```).
Catch2 is designed to "just work" as much as possible, and most of the
configuration options below are changed automatically during compilation,
according to the detected environment. However, this detection can also
be overriden by users, using macros documented below, and/or CMake options
with the same name.
Nonetheless there are still some occasions where finer control is needed. For these occasions Catch exposes a set of macros for configuring how it is built.
## main()/ implementation
CATCH_CONFIG_MAIN // Designates this as implementation file and defines main()
CATCH_CONFIG_RUNNER // Designates this as implementation file
Although Catch is header only it still, internally, maintains a distinction between interface headers and headers that contain implementation. Only one source file in your test project should compile the implementation headers and this is controlled through the use of one of these macros - one of these identifiers should be defined before including Catch in *exactly one implementation file in your project*.
## Reporter / Listener interfaces
CATCH_CONFIG_EXTERNAL_INTERFACES // Brings in necessary headers for Reporter/Listener implementation
Brings in various parts of Catch that are required for user defined Reporters and Listeners. This means that new Reporters and Listeners can be defined in this file as well as in the main file.
Implied by both `CATCH_CONFIG_MAIN` and `CATCH_CONFIG_RUNNER`.
## Prefixing Catch macros
@ -45,19 +32,18 @@ To keep test code clean and uncluttered Catch uses short macro names (e.g. ```TE
## Terminal colour
CATCH_CONFIG_COLOUR_NONE // completely disables all text colouring
CATCH_CONFIG_COLOUR_WINDOWS // forces the Win32 console API to be used
CATCH_CONFIG_COLOUR_ANSI // forces ANSI colour codes to be used
CATCH_CONFIG_COLOUR_WIN32 // Force enables compiling colouring impl based on Win32 console API
CATCH_CONFIG_NO_COLOUR_WIN32 // Force disables ...
Yes, I am English, so I will continue to spell "colour" with a 'u'.
Yes, Catch2 uses the british spelling of colour.
When sending output to the terminal, if it detects that it can, Catch will use colourised text. On Windows the Win32 API, ```SetConsoleTextAttribute```, is used. On POSIX systems ANSI colour escape codes are inserted into the stream.
Catch2 attempts to autodetect whether the Win32 console colouring API,
`SetConsoleTextAttribute`, is available, and if it is available it compiles
in a console colouring implementation that uses it.
For finer control you can define one of the above identifiers (these are mutually exclusive - but that is not checked so may behave unexpectedly if you mix them):
This option can be used to override Catch2's autodetection and force the
compilation either ON or OFF.
Note that when ANSI colour codes are used "unistd.h" must be includable - along with a definition of ```isatty()```
Typically you should place the ```#define``` before #including "catch.hpp" in your main source file - but if you prefer you can define it for your whole project by whatever your IDE or build system provides for you to do so.
## Console width
@ -71,7 +57,7 @@ By default a console width of 80 is assumed but this can be controlled by defini
CATCH_CONFIG_NOSTDOUT
To support platforms that do not provide `std::cout`, `std::cerr` and
`std::clog`, Catch does not usem the directly, but rather calls
`std::clog`, Catch does not use them directly, but rather calls
`Catch::cout`, `Catch::cerr` and `Catch::clog`. You can replace their
implementation by defining `CATCH_CONFIG_NOSTDOUT` and implementing
them yourself, their signatures are:
@ -89,7 +75,7 @@ them yourself, their signatures are:
By default, when Catch's stringification machinery has to stringify
a type that does not specialize `StringMaker`, does not overload `operator<<`,
is not an enumeration and is not a range, it uses `"{?}"`. This can be
overriden by defining `CATCH_CONFIG_FALLBACK_STRINGIFIER` to name of a
overridden by defining `CATCH_CONFIG_FALLBACK_STRINGIFIER` to name of a
function that should perform the stringification instead.
All types that do not provide `StringMaker` specialization or `operator<<`
@ -111,6 +97,14 @@ This means that defining `CATCH_CONFIG_DEFAULT_REPORTER` to `"console"`
is equivalent with the out-of-the-box experience.
## Bazel support
When `CATCH_CONFIG_BAZEL_SUPPORT` is defined or when `BAZEL_TEST=1` (which is set by the Bazel inside of a test environment),
Catch2 will register a `JUnit` reporter writing to a path pointed by `XML_OUTPUT_FILE` provided by Bazel.
> `CATCH_CONFIG_BAZEL_SUPPORT` was [introduced](https://github.com/catchorg/Catch2/pull/2399) in Catch2 3.0.1.
> `CATCH_CONFIG_BAZEL_SUPPORT` was [deprecated](https://github.com/catchorg/Catch2/pull/2459) in Catch2 3.1.0.
## C++11 toggles
CATCH_CONFIG_CPP11_TO_STRING // Use `std::to_string`
@ -126,9 +120,13 @@ Catch's selection, by defining either `CATCH_CONFIG_CPP11_TO_STRING` or
## C++17 toggles
CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTIONS // Use std::uncaught_exceptions instead of std::uncaught_exception
CATCH_CONFIG_CPP17_STRING_VIEW // Provide StringMaker specialization for std::string_view
CATCH_CONFIG_CPP17_VARIANT // Override C++17 detection for CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER
CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTIONS // Override std::uncaught_exceptions (instead of std::uncaught_exception) support detection
CATCH_CONFIG_CPP17_STRING_VIEW // Override std::string_view support detection (Catch provides a StringMaker specialization by default)
CATCH_CONFIG_CPP17_VARIANT // Override std::variant support detection (checked by CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER)
CATCH_CONFIG_CPP17_OPTIONAL // Override std::optional support detection (checked by CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER)
CATCH_CONFIG_CPP17_BYTE // Override std::byte support detection (Catch provides a StringMaker specialization by default)
> `CATCH_CONFIG_CPP17_STRING_VIEW` was [introduced](https://github.com/catchorg/Catch2/issues/1376) in Catch2 2.4.1.
Catch contains basic compiler/standard detection and attempts to use
some C++17 features whenever appropriate. This automatic detection
@ -142,19 +140,26 @@ by using `_NO_` in the macro, e.g. `CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS`.
CATCH_CONFIG_COUNTER // Use __COUNTER__ to generate unique names for test cases
CATCH_CONFIG_WINDOWS_SEH // Enable SEH handling on Windows
CATCH_CONFIG_FAST_COMPILE // Sacrifices some (rather minor) features for compilation speed
CATCH_CONFIG_DISABLE_MATCHERS // Do not compile Matchers in this compilation unit
CATCH_CONFIG_POSIX_SIGNALS // Enable handling POSIX signals
CATCH_CONFIG_WINDOWS_CRTDBG // Enable leak checking using Windows's CRT Debug Heap
CATCH_CONFIG_DISABLE_STRINGIFICATION // Disable stringifying the original expression
CATCH_CONFIG_DISABLE // Disables assertions and test case registration
CATCH_CONFIG_WCHAR // Enables use of wchart_t
CATCH_CONFIG_EXPERIMENTAL_REDIRECT // Enables the new (experimental) way of capturing stdout/stderr
CATCH_CONFIG_USE_ASYNC // Force parallel statistical processing of samples during benchmarking
CATCH_CONFIG_ANDROID_LOGWRITE // Use android's logging system for debug output
CATCH_CONFIG_GLOBAL_NEXTAFTER // Use nextafter{,f,l} instead of std::nextafter
> [`CATCH_CONFIG_ANDROID_LOGWRITE`](https://github.com/catchorg/Catch2/issues/1743) and [`CATCH_CONFIG_GLOBAL_NEXTAFTER`](https://github.com/catchorg/Catch2/pull/1739) were introduced in Catch2 2.10.0
Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support.
`CATCH_CONFIG_POSIX_SIGNALS` is on by default, except when Catch is compiled under `Cygwin`, where it is disabled by default (but can be force-enabled by defining `CATCH_CONFIG_POSIX_SIGNALS`).
`CATCH_CONFIG_WINDOWS_CRTDBG` is off by default. If enabled, Windows's CRT is used to check for memory leaks, and displays them after the tests finish running.
`CATCH_CONFIG_WINDOWS_CRTDBG` is off by default. If enabled, Windows's
CRT is used to check for memory leaks, and displays them after the tests
finish running. This option only works when linking against the default
main, and must be defined for the whole library build.
`CATCH_CONFIG_WCHAR` is on by default, but can be disabled. Currently
it is only used in support for DJGPP cross-compiler.
@ -173,11 +178,6 @@ should not lead to false negatives.
`CATCH_CONFIG_FAST_COMPILE` has to be either defined, or not defined,
in all translation units that are linked into single test binary.
### `CATCH_CONFIG_DISABLE_MATCHERS`
When `CATCH_CONFIG_DISABLE_MATCHERS` is defined, all mentions of Catch's Matchers are ifdef-ed away from the translation unit. Doing so will speed up compilation of that TU.
_Note: If you define `CATCH_CONFIG_DISABLE_MATCHERS` in the same file as Catch's main is implemented, your test executable will fail to link if you use Matchers anywhere._
### `CATCH_CONFIG_DISABLE_STRINGIFICATION`
This toggle enables a workaround for VS 2017 bug. For details see [known limitations](limitations.md#visual-studio-2017----raw-string-literal-in-assert-fails-to-compile).
@ -188,13 +188,6 @@ This feature is considered experimental and might change at any point.
_Inspired by Doctest's `DOCTEST_CONFIG_DISABLE`_
## Windows header clutter
On Windows Catch includes `windows.h`. To minimize global namespace clutter in the implementation file, it defines `NOMINMAX` and `WIN32_LEAN_AND_MEAN` before including it. You can control this behaviour via two macros:
CATCH_CONFIG_NO_NOMINMAX // Stops Catch from using NOMINMAX macro
CATCH_CONFIG_NO_WIN32_LEAN_AND_MEAN // Stops Catch from using WIN32_LEAN_AND_MEAN macro
## Enabling stringification
@ -202,14 +195,18 @@ By default, Catch does not stringify some types from the standard library. This
CATCH_CONFIG_ENABLE_PAIR_STRINGMAKER // Provide StringMaker specialization for std::pair
CATCH_CONFIG_ENABLE_TUPLE_STRINGMAKER // Provide StringMaker specialization for std::tuple
CATCH_CONFIG_ENABLE_CHRONO_STRINGMAKER // Provide StringMaker specialization for std::chrono::duration, std::chrono::timepoint
CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER // Provide StringMaker specialization for std::variant, std::monostate (on C++17)
CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER // Provide StringMaker specialization for std::optional (on C++17)
CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS // Defines all of the above
> `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1380) in Catch2 2.4.1.
> `CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1510) in Catch2 2.6.0.
## Disabling exceptions
> Introduced in Catch2 2.4.0.
By default, Catch2 uses exceptions to signal errors and to abort tests
when an assertion from the `REQUIRE` family of assertions fails. We also
provide an experimental support for disabling exceptions. Catch2 should
@ -242,6 +239,18 @@ namespace Catch {
}
```
## Overriding Catch's debug break (`-b`)
> [Introduced](https://github.com/catchorg/Catch2/pull/1846) in Catch2 2.11.2.
You can override Catch2's break-into-debugger code by defining the
`CATCH_BREAK_INTO_DEBUGGER()` macro. This can be used if e.g. Catch2 does
not know your platform, or your platform is misdetected.
The macro will be used as is, that is, `CATCH_BREAK_INTO_DEBUGGER();`
must compile and must break into debugger.
---
[Home](Readme.md#top)

View File

@ -1,82 +1,317 @@
<a id="top"></a>
# Contributing to Catch
# Contributing to Catch2
So you want to contribute something to Catch? That's great! Whether it's a bug fix, a new feature, support for
additional compilers - or just a fix to the documentation - all contributions are very welcome and very much appreciated.
Of course so are bug reports and other comments and questions.
**Contents**<br>
[Using Git(Hub)](#using-github)<br>
[Testing your changes](#testing-your-changes)<br>
[Writing documentation](#writing-documentation)<br>
[Writing code](#writing-code)<br>
[CoC](#coc)<br>
If you are contributing to the code base there are a few simple guidelines to keep in mind. This also includes notes to
help you find your way around. As this is liable to drift out of date please raise an issue or, better still, a pull
request for this file, if you notice that.
So you want to contribute something to Catch2? That's great! Whether it's
a bug fix, a new feature, support for additional compilers - or just
a fix to the documentation - all contributions are very welcome and very
much appreciated. Of course so are bug reports, other comments, and
questions, but generally it is a better idea to ask questions in our
[Discord](https://discord.gg/4CWS9zD), than in the issue tracker.
## Branches
Ongoing development is currently on _master_. At some point an integration branch will be set-up and PRs should target
that - but for now it's all against master. You may see feature branches come and go from time to time, too.
This page covers some guidelines and helpful tips for contributing
to the codebase itself.
## Directory structure
## Using Git(Hub)
_Users_ of Catch primarily use the single header version. _Maintainers_ should work with the full source (which is still,
primarily, in headers). This can be found in the `include` folder. There are a set of test files, currently under
`projects/SelfTest`. The test app can be built via CMake from the `CMakeLists.txt` file in the root, or you can generate
project files for Visual Studio, XCode, and others (instructions in the `projects` folder). If you have access to CLion,
it can work with the CMake file directly.
Ongoing development happens in the `devel` branch for Catch2 v3, and in
`v2.x` for maintenance updates to the v2 versions.
As well as the runtime test files you'll also see a `SurrogateCpps` directory under `projects/SelfTest`.
This contains a set of .cpp files that each `#include` a single header.
While these files are not essential to compilation they help to keep the implementation headers self-contained.
At time of writing this set is not complete but has reasonable coverage.
If you add additional headers please try to remember to add a surrogate cpp for it.
Commits should be small and atomic. A commit is atomic when, after it is
applied, the codebase, tests and all, still works as expected. Small
commits are also preferred, as they make later operations with git history,
whether it is bisecting, reverting, or something else, easier.
The other directories are `scripts` which contains a set of python scripts to help in testing Catch as well as
generating the single include, and `docs`, which contains the documentation as a set of markdown files.
_When submitting a pull request please do not include changes to the
amalgamated distribution files. This means do not include them in your
git commits!_
When addressing review comments in a MR, please do not rebase/squash the
commits immediately. Doing so makes it harder to review the new changes,
slowing down the process of merging a MR. Instead, when addressing review
comments, you should append new commits to the branch and only squash
them into other commits when the MR is ready to be merged. We recommend
creating new commits with `git commit --fixup` (or `--squash`) and then
later squashing them with `git rebase --autosquash` to make things easier.
__When submitting a pull request please do not include changes to the single include, or to the version number file
as these are managed by the scripts!__
## Testing your changes
Obviously all changes to Catch's code should be tested. If you added new
functionality, you should add tests covering and showcasing it. Even if you have
only made changes to Catch internals (i.e. you implemented some performance
improvements), you should still test your changes.
_Note: Running Catch2's tests requires Python3_
This means 2 things
* Compiling Catch's SelfTest project:
Catch2 has multiple layers of tests that are then run as part of our CI.
The most obvious one are the unit tests compiled into the `SelfTest`
binary. These are then used in "Approval tests", which run (almost) all
tests from `SelfTest` through a specific reporter and then compare the
generated output with a known good output ("Baseline"). By default, new
tests should be placed here.
However, not all tests can be written as plain unit tests. For example,
checking that Catch2 orders tests randomly when asked to, and that this
random ordering is subset-invariant, is better done as an integration
test using an external check script. Catch2 integration tests are written
using CTest, either as a direct command invocation + pass/fail regex,
or by delegating the check to a Python script.
Catch2 is slowly gaining more and more types of tests, currently Catch2
project also has buildable examples, "ExtraTests", and CMake config tests.
Examples present a small and self-contained snippets of code that
use Catch2's facilities for specific purpose. Currently they are assumed
passing if they compile.
ExtraTests then are expensive tests, that we do not want to run all the
time. This can be either because they take a long time to run, or because
they take a long time to compile, e.g. because they test compile time
configuration and require separate compilation.
Finally, CMake config tests test that you set Catch2's compile-time
configuration options through CMake, using CMake options of the same name.
None of these tests are enabled by default. To enable them, add
`-DCATCH_BUILD_EXAMPLES=ON`, `-DCATCH_BUILD_EXTRA_TESTS=ON`, and
`-DCATCH_ENABLE_CONFIGURE_TESTS=ON` when configuration the CMake build.
Bringing this all together, the steps below should configure, build,
and run all tests in the `Debug` compilation.
<!-- snippet: catch2-build-and-test -->
<a id='snippet-catch2-build-and-test'></a>
```sh
# 1. Regenerate the amalgamated distribution
./tools/scripts/generateAmalgamatedFiles.py
# 2. Configure the full test build
cmake -Bdebug-build -H. -DCMAKE_BUILD_TYPE=Debug -DCATCH_DEVELOPMENT_BUILD=ON -DCATCH_BUILD_EXAMPLES=ON -DCATCH_BUILD_EXTRA_TESTS=ON
# 3. Run the actual build
cmake --build debug-build
# 4. Run the tests using CTest
cd debug-build
ctest -j 4 --output-on-failure -C Debug
```
$ cd Catch2
$ cmake -Bdebug-build -H. -DCMAKE_BUILD_TYPE=Debug
$ cmake --build debug-build
<sup><a href='/tools/scripts/buildAndTest.sh#L6-L19' title='File snippet `catch2-build-and-test` was extracted from'>snippet source</a> | <a href='#snippet-catch2-build-and-test' title='Navigate to start of snippet `catch2-build-and-test`'>anchor</a></sup>
<!-- endSnippet -->
For convenience, the above commands are in the script `tools/scripts/buildAndTest.sh`, and can be run like this:
```bash
cd Catch2
./tools/scripts/buildAndTest.sh
```
because code that does not compile is evidently incorrect. Obviously,
you are not expected to have access to all the compilers and platforms
supported by Catch2, but you should at least smoke test your changes
on your platform. Our CI pipeline will check your PR against most of
the supported platforms, but it takes an hour to finish -- compiling
locally takes just a few minutes.
A Windows version of the script is available at `tools\scripts\buildAndTest.cmd`.
If you added new tests, you will likely see `ApprovalTests` failure.
After you check that the output difference is expected, you should
run `tools/scripts/approve.py` to confirm them, and include these changes
in your commit.
* Running the tests via CTest:
```
$ cd debug-build
$ ctest -j 2 --output-on-failure
```
If you added new tests, approval tests are very likely to fail. If they
do not, it means that your changes weren't run as part of them. This
_might_ be intentional, but usually is not.
## Writing documentation
The approval tests compare current output of the SelfTest binary in various
configurations against known good outputs. The reason it fails is,
_usually_, that you've added new tests but have not yet approved the changes
they introduce. This is done with the `scripts/approve.py` script, but
before you do so, you need to check that the introduced changes are indeed
intentional.
If you have added new feature to Catch2, it needs documentation, so that
other people can use it as well. This section collects some technical
information that you will need for updating Catch2's documentation, and
possibly some generic advise as well.
### Technicalities
First, the technicalities:
* If you have introduced a new document, there is a simple template you
should use. It provides you with the top anchor mentioned to link to
(more below), and also with a backlink to the top of the documentation:
```markdown
<a id="top"></a>
# Cool feature
> [Introduced](https://github.com/catchorg/Catch2/pull/123456) in Catch2 X.Y.Z
Text that explains how to use the cool feature.
*this document is still in-progress...*
---
[Home](Readme.md#top)
```
* Crosslinks to different pages should target the `top` anchor, like this
`[link to contributing](contributing.md#top)`.
* We introduced version tags to the documentation, which show users in
which version a specific feature was introduced. This means that newly
written documentation should be tagged with a placeholder, that will
be replaced with the actual version upon release. There are 2 styles
of placeholders used through the documentation, you should pick one that
fits your text better (if in doubt, take a look at the existing version
tags for other features).
* `> [Introduced](link-to-issue-or-PR) in Catch2 X.Y.Z` - this
placeholder is usually used after a section heading
* `> X (Y and Z) was [introduced](link-to-issue-or-PR) in Catch2 X.Y.Z`
- this placeholder is used when you need to tag a subpart of something,
e.g. a list
* For pages with more than 4 subheadings, we provide a table of contents
(ToC) at the top of the page. Because GitHub markdown does not support
automatic generation of ToC, it has to be handled semi-manually. Thus,
if you've added a new subheading to some page, you should add it to the
ToC. This can be done either manually, or by running the
`updateDocumentToC.py` script in the `scripts/` folder.
### Contents
Now, for some content tips:
* Usage examples are good. However, having large code snippets inline
can make the documentation less readable, and so the inline snippets
should be kept reasonably short. To provide more complex compilable
examples, consider adding new .cpp file to `examples/`.
* Don't be afraid to introduce new pages. The current documentation
tends towards long pages, but a lot of that is caused by legacy, and
we know that some of the pages are overly big and unfocused.
* When adding information to an existing page, please try to keep your
formatting, style and changes consistent with the rest of the page.
* Any documentation has multiple different audiences, that desire
different information from the text. The 3 basic user-types to try and
cover are:
* A beginner to Catch2, who requires closer guidance for the usage of Catch2.
* Advanced user of Catch2, who want to customize their usage.
* Experts, looking for full reference of Catch2's capabilities.
## Writing code
If want to contribute code, this section contains some simple rules
and tips on things like code formatting, code constructions to avoid,
and so on.
### C++ standard version
Catch2 currently targets C++14 as the minimum supported C++ version.
Features from higher language versions should be used only sparingly,
when the benefits from using them outweight the maintenance overhead.
Example of good use of polyfilling features is our use of `conjunction`,
where if available we use `std::conjunction` and otherwise provide our
own implementation. The reason it is good is that the surface area for
maintenance is quite small, and `std::conjunction` can directly use
compiler built-ins, thus providing significant compilation benefits.
Example of bad use of polyfilling features would be to keep around two
sets of metaprogramming in the stringification implementation, once
using C++14 compliant TMP and once using C++17's `if constexpr`. While
the C++17 would provide significant compilation speedups, the maintenance
cost would be too high.
### Formatting
To make code formatting simpler for the contributors, Catch2 provides
its own config for `clang-format`. However, because it is currently
impossible to replicate existing Catch2's formatting in clang-format,
using it to reformat a whole file would cause massive diffs. To keep
the size of your diffs reasonable, you should only use clang-format
on the newly changed code.
### Code constructs to watch out for
This section is a (sadly incomplete) listing of various constructs that
are problematic and are not always caught by our CI infrastructure.
#### Naked exceptions and exceptions-related function
If you are throwing an exception, it should be done via `CATCH_ERROR`
or `CATCH_RUNTIME_ERROR` in `internal/catch_enforce.hpp`. These macros will handle
the differences between compilation with or without exceptions for you.
However, some platforms (IAR) also have problems with exceptions-related
functions, such as `std::current_exceptions`. We do not have IAR in our
CI, but luckily there should not be too many reasons to use these.
However, if you do, they should be kept behind a
`CATCH_CONFIG_DISABLE_EXCEPTIONS` macro.
#### Avoid `std::move` and `std::forward`
`std::move` and `std::forward` provide nice semantic name for a specific
`static_cast`. However, being function templates they have surprisingly
high cost during compilation, and can also have a negative performance
impact for low-optimization builds.
You should be using `CATCH_MOVE` and `CATCH_FORWARD` macros from
`internal/catch_move_and_forward.hpp` instead. They expand into the proper
`static_cast`, and avoid the overhead of `std::move` and `std::forward`.
#### Unqualified usage of functions from C's stdlib
If you are using a function from C's stdlib, please include the header
as `<cfoo>` and call the function qualified. The common knowledge that
there is no difference is wrong, QNX and VxWorks won't compile if you
include the header as `<cfoo>` and call the function unqualified.
#### User-Defined Literals (UDL) for Catch2' types
Due to messy standardese and ... not great ... implementation of
`-Wreserved-identifier` in Clang, avoid declaring UDLs as
```cpp
Approx operator "" _a(long double);
```
and instead declare them as
```cpp
Approx operator ""_a(long double);
```
Notice that the second version does not have a space between the `""` and
the literal suffix.
### New source file template
If you are adding new source file, there is a template you should use.
Specifically, every source file should start with the licence header:
```cpp
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
```
The include guards for header files should follow the pattern `{FILENAME}_INCLUDED`.
This means that for file `catch_matchers_foo.hpp`, the include guard should
be `CATCH_MATCHERS_FOO_HPP_INCLUDED`, for `catch_generators_bar.hpp`, the include
guard should be `CATCH_GENERATORS_BAR_HPP_INCLUDED`, and so on.
## CoC
This project has a [CoC](../CODE_OF_CONDUCT.md). Please adhere to it
while contributing to Catch2.
-----------
_This documentation will always be in-progress as new information comes
up, but we are trying to keep it as up to date as possible._
---

View File

@ -9,84 +9,22 @@ either of these is a breaking change, and thus will not happen until
at least the next major release.
## Deprecations
### `ParseAndAddCatchTests.cmake`
### `--list-*` return values
The return codes of the `--list-*` family of command line arguments
will no longer be equal to the number of tests/tags/etc found, instead
it will be 0 for success and non-zero for failure.
The CMake/CTest integration using `ParseAndAddCatchTests.cmake` is deprecated,
as it can be replaced by `Catch.cmake` that provides the function
`catch_discover_tests` to get tests directly from a CMake target via the
command line interface instead of parsing C++ code with regular expressions.
### `--list-test-names-only`
### `CATCH_CONFIG_BAZEL_SUPPORT`
`--list-test-names-only` command line argument will be removed.
### `ANON_TEST_CASE`
`ANON_TEST_CASE` is scheduled for removal, as it can be fully replaced
by a `TEST_CASE` with no arguments.
### Secondary description amongst tags
Currently, the tags part of `TEST_CASE` (and others) macro can also
contain text that is not part of tags. This text is then separated into
a "description" of the test case, but the description is then never used
apart from writing it out for `--list-tests -v high`.
Because it isn't actually used nor documented, and brings complications
to Catch2's internals, description support will be removed.
## Planned changes
### Reporter verbosities
The current implementation of verbosities, where the reporter is checked
up-front whether it supports the requested verbosity, is fundamentally
misguided and will be changed. The new implementation will no longer check
whether the specified reporter supports the requested verbosity, instead
it will be up to the reporters to deal with verbosities as they see fit
(with an expectation that unsupported verbosities will be, at most,
warnings, but not errors).
### Output format of `--list-*` command line parameters
The various list operations will be piped through reporters. This means
that e.g. XML reporter will write the output as machine-parseable XML,
while the Console reporter will keep the current, human-oriented output.
### `CHECKED_IF` and `CHECKED_ELSE`
To make the `CHECKED_IF` and `CHECKED_ELSE` macros more useful, they will
be marked as "OK to fail" (`Catch::ResultDisposition::SuppressFail` flag
will be added), which means that their failure will not fail the test,
making the `else` actually useful.
### Change semantics of `[.]` and tag exclusion
Currently, given these 2 tests
```cpp
TEST_CASE("A", "[.][foo]") {}
TEST_CASE("B", "[.][bar]") {}
```
specifying `[foo]` as the testspec will run test "A" and specifying
`~[foo]` will run test "B", even though it is hidden. Also, specifying
`~[baz]` will run both tests. This behaviour is often surprising and will
be changed so that hidden tests are included in a run only if they
positively match a testspec.
### Console Colour API
The API for Catch2's console colour will be changed to take an extra
argument, the stream to which the colour code should be applied.
Catch2 supports writing the Bazel JUnit XML output file when it is aware
that is within a bazel testing environment. Originally there was no way
to accurately probe the environment for this information so the flag
`CATCH_CONFIG_BAZEL_SUPPORT` was added. This now deprecated. Bazel has now had a change
where it will export `BAZEL_TEST=1` for purposes like the above. Catch2
will now instead inspect the environment instead of relying on build configuration.
---

View File

@ -1,74 +1,43 @@
<a id="top"></a>
# Event Listeners
A `Listener` is a class you can register with Catch that will then be passed events,
such as a test case starting or ending, as they happen during a test run.
`Listeners` are actually types of `Reporters`, with a few small differences:
1. Once registered in code they are automatically used - you don't need to specify them on the command line
2. They are called in addition to (just before) any reporters, and you can register multiple listeners.
3. They derive from `Catch::TestEventListenerBase`, which has default stubs for all the events,
so you are not forced to implement events you're not interested in.
4. You register a listener with `CATCH_REGISTER_LISTENER`
An event listener is a bit like a reporter, in that it responds to various
reporter events in Catch2, but it is not expected to write any output.
Instead, an event listener performs actions within the test process, such
as performing global initialization (e.g. of a C library), or cleaning out
in-memory logs if they are not needed (the test case passed).
Unlike reporters, each registered event listener is always active. Event
listeners are always notified before reporter(s).
## Implementing a Listener
Simply derive a class from `Catch::TestEventListenerBase` and implement the methods you are interested in, either in
the main source file (i.e. the one that defines `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`), or in a
file that defines `CATCH_CONFIG_EXTERNAL_INTERFACES`.
To write your own event listener, you should derive from `Catch::TestEventListenerBase`,
as it provides empty stubs for all reporter events, allowing you to
only override events you care for. Afterwards you have to register it
with Catch2 using `CATCH_REGISTER_LISTENER` macro, so that Catch2 knows
about it and instantiates it before running tests.
Then register it using `CATCH_REGISTER_LISTENER`.
Example event listener:
```cpp
#include <catch2/reporters/catch_reporter_event_listener.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
For example ([complete source code](../examples/210-Evt-EventListeners.cpp)):
class testRunListener : public Catch::EventListenerBase {
public:
using Catch::EventListenerBase::EventListenerBase;
```c++
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
struct MyListener : Catch::TestEventListenerBase {
using TestEventListenerBase::TestEventListenerBase; // inherit constructor
virtual void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
// Perform some setup before a test case is run
void testRunStarting(Catch::TestRunInfo const&) override {
lib_foo_init();
}
virtual void testCaseEnded( Catch::TestCaseStats const& testCaseStats ) override {
// Tear-down after a test case is run
}
};
CATCH_REGISTER_LISTENER( MyListener )
CATCH_REGISTER_LISTENER(testRunListener)
```
_Note that you should not use any assertion macros within a Listener!_
## Events that can be hooked
[You can find the list of events that the listeners can react to on its
own page](reporter-events.md#top).
The following are the methods that can be overridden in the Listener:
```c++
// The whole test run, starting and ending
virtual void testRunStarting( TestRunInfo const& testRunInfo );
virtual void testRunEnded( TestRunStats const& testRunStats );
// Test cases starting and ending
virtual void testCaseStarting( TestCaseInfo const& testInfo );
virtual void testCaseEnded( TestCaseStats const& testCaseStats );
// Sections starting and ending
virtual void sectionStarting( SectionInfo const& sectionInfo );
virtual void sectionEnded( SectionStats const& sectionStats );
// Assertions before/ after
virtual void assertionStarting( AssertionInfo const& assertionInfo );
virtual bool assertionEnded( AssertionStats const& assertionStats );
// A test is being skipped (because it is "hidden")
virtual void skipTest( TestCaseInfo const& testInfo );
```
More information about the events (e.g. name of the test case) is contained in the structs passed as arguments -
just look in the source code to see what fields are available.
---

62
docs/faq.md Normal file
View File

@ -0,0 +1,62 @@
<a id="top"></a>
# Frequently Asked Questions (FAQ)
**Contents**<br>
[How do I run global setup/teardown only if tests will be run?](#how-do-i-run-global-setupteardown-only-if-tests-will-be-run)<br>
[How do I clean up global state between running different tests?](#how-do-i-clean-up-global-state-between-running-different-tests)<br>
[Why cannot I derive from the built-in reporters?](#why-cannot-i-derive-from-the-built-in-reporters)<br>
[What is Catch2's ABI stability policy?](#what-is-catch2s-abi-stability-policy)<br>
[What is Catch2's API stability policy?](#what-is-catch2s-api-stability-policy)<br>
## How do I run global setup/teardown only if tests will be run?
Write a custom [event listener](event-listeners.md#top) and place the
global setup/teardown code into the `testRun*` events.
## How do I clean up global state between running different tests?
Write a custom [event listener](event-listeners.md#top) and place the
cleanup code into either `testCase*` or `testCasePartial*` events,
depending on how often the cleanup needs to happen.
## Why cannot I derive from the built-in reporters?
They are not made to be overriden, in that we do not attempt to maintain
a consistent internal state if a member function is overriden, and by
forbidding users from using them as a base class, we can refactor them
as needed later.
## What is Catch2's ABI stability policy?
Catch2 provides no ABI stability guarantees whatsoever. Catch2 provides
rich C++ interface, and trying to freeze its ABI would take a lot of
pointless work.
Catch2 is not designed to be distributed as dynamic library, and you
should really be able to compile everything with the same compiler binary.
## What is Catch2's API stability policy?
Catch2 follows [semver](https://semver.org/) to the best of our ability.
This means that we will not knowingly make backwards-incompatible changes
without incrementing the major version number.
## Does Catch2 support running tests in parallel?
Not natively, no. We see running tests in parallel as the job of an
external test runner, that can also run them in separate processes,
support test execution timeouts and so on.
However, Catch2 provides some tools that make the job of external test
runners easier. [See the relevant section in our page on best
practices](usage-tips.md#parallel-tests).
---
[Home](Readme.md#top)

View File

@ -1,6 +1,8 @@
<a id="top"></a>
# Data Generators
> Introduced in Catch2 2.6.0.
Data generators (also known as _data driven/parametrized test cases_)
let you reuse the same set of assertions across different input values.
In Catch2, this means that they respect the ordering and nesting
@ -10,23 +12,88 @@ are run once per each value in a generator.
This is best explained with an example:
```cpp
TEST_CASE("Generators") {
auto i = GENERATE(1, 2, 3);
SECTION("one") {
auto j = GENERATE( -3, -2, -1 );
REQUIRE(j < i);
}
auto i = GENERATE(1, 3, 5);
REQUIRE(is_odd(i));
}
```
The assertion in this test case will be run 9 times, because there
are 3 possible values for `i` (1, 2, and 3) and there are 3 possible
values for `j` (-3, -2, and -1).
The "Generators" `TEST_CASE` will be entered 3 times, and the value of
`i` will be 1, 3, and 5 in turn. `GENERATE`s can also be used multiple
times at the same scope, in which case the result will be a cartesian
product of all elements in the generators. This means that in the snippet
below, the test case will be run 6 (2\*3) times.
```cpp
TEST_CASE("Generators") {
auto i = GENERATE(1, 2);
auto j = GENERATE(3, 4, 5);
}
```
There are 2 parts to generators in Catch2, the `GENERATE` macro together
with the already provided generators, and the `IGenerator<T>` interface
that allows users to implement their own generators.
## Combining `GENERATE` and `SECTION`.
`GENERATE` can be seen as an implicit `SECTION`, that goes from the place
`GENERATE` is used, to the end of the scope. This can be used for various
effects. The simplest usage is shown below, where the `SECTION` "one"
runs 4 (2\*2) times, and `SECTION` "two" is run 6 times (2\*3).
```cpp
TEST_CASE("Generators") {
auto i = GENERATE(1, 2);
SECTION("one") {
auto j = GENERATE(-3, -2);
REQUIRE(j < i);
}
SECTION("two") {
auto k = GENERATE(4, 5, 6);
REQUIRE(i != k);
}
}
```
The specific order of the `SECTION`s will be "one", "one", "two", "two",
"two", "one"...
The fact that `GENERATE` introduces a virtual `SECTION` can also be used
to make a generator replay only some `SECTION`s, without having to
explicitly add a `SECTION`. As an example, the code below reports 3
assertions, because the "first" section is run once, but the "second"
section is run twice.
```cpp
TEST_CASE("GENERATE between SECTIONs") {
SECTION("first") { REQUIRE(true); }
auto _ = GENERATE(1, 2);
SECTION("second") { REQUIRE(true); }
}
```
This can lead to surprisingly complex test flows. As an example, the test
below will report 14 assertions:
```cpp
TEST_CASE("Complex mix of sections and generates") {
auto i = GENERATE(1, 2);
SECTION("A") {
SUCCEED("A");
}
auto j = GENERATE(3, 4);
SECTION("B") {
SUCCEED("B");
}
auto k = GENERATE(5, 6);
SUCCEED();
}
```
> The ability to place `GENERATE` between two `SECTION`s was [introduced](https://github.com/catchorg/Catch2/issues/1938) in Catch2 2.13.0.
## Provided generators
Catch2's provided generator functionality consists of three parts,
@ -34,26 +101,49 @@ Catch2's provided generator functionality consists of three parts,
* `GENERATE` macro, that serves to integrate generator expression with
a test case,
* 2 fundamental generators
* `ValueGenerator<T>` -- contains only single element
* `ValuesGenerator<T>` -- contains multiple elements
* 4 generic generators that modify other generators
* `SingleValueGenerator<T>` -- contains only single element
* `FixedValuesGenerator<T>` -- contains multiple elements
* 5 generic generators that modify other generators
* `FilterGenerator<T, Predicate>` -- filters out elements from a generator
for which the predicate returns "false"
* `TakeGenerator<T>` -- takes first `n` elements from a generator
* `RepeatGenerator<T>` -- repeats output from a generator `n` times
* `MapGenerator<T, U, Func>` -- returns the result of applying `Func`
on elements from a different generator
* `ChunkGenerator<T>` -- returns chunks (inside `std::vector`) of n elements from a generator
* 4 specific purpose generators
* `RandomIntegerGenerator<Integral>` -- generates random Integrals from range
* `RandomFloatGenerator<Float>` -- generates random Floats from range
* `RangeGenerator<T>(first, last)` -- generates all values inside a `[first, last)` arithmetic range
* `IteratorGenerator<T>` -- copies and returns values from an iterator range
> `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch2 2.7.0.
> `IteratorGenerator<T>` was introduced in Catch2 2.10.0.
The generators also have associated helper functions that infer their
type, making their usage much nicer. These are
* `value(T&&)` for `ValueGenerator<T>`
* `values(std::initializer_list<T>)` for `ValuesGenerator<T>`
* `value(T&&)` for `SingleValueGenerator<T>`
* `values(std::initializer_list<T>)` for `FixedValuesGenerator<T>`
* `table<Ts...>(std::initializer_list<std::tuple<Ts...>>)` for `FixedValuesGenerator<std::tuple<Ts...>>`
* `filter(predicate, GeneratorWrapper<T>&&)` for `FilterGenerator<T, Predicate>`
* `take(count, GeneratorWrapper<T>&&)` for `TakeGenerator<T>`
* `repeat(repeats, GeneratorWrapper<T>&&)` for `RepeatGenerator<T>`
* `map(func, GeneratorWrapper<T>&&)` for `MapGenerator<T, T, Func>` (map `T` to `T`)
* `map(func, GeneratorWrapper<T>&&)` for `MapGenerator<T, U, Func>` (map `U` to `T`, deduced from `Func`)
* `map<T>(func, GeneratorWrapper<U>&&)` for `MapGenerator<T, U, Func>` (map `U` to `T`)
* `chunk(chunk-size, GeneratorWrapper<T>&&)` for `ChunkGenerator<T>`
* `random(IntegerOrFloat a, IntegerOrFloat b)` for `RandomIntegerGenerator` or `RandomFloatGenerator`
* `range(Arithemtic start, Arithmetic end)` for `RangeGenerator<Arithmetic>` with a step size of `1`
* `range(Arithmetic start, Arithmetic end, Arithmetic step)` for `RangeGenerator<Arithmetic>` with a custom step size
* `from_range(InputIterator from, InputIterator to)` for `IteratorGenerator<T>`
* `from_range(Container const&)` for `IteratorGenerator<T>`
> `chunk()`, `random()` and both `range()` functions were introduced in Catch2 2.7.0.
> `from_range` has been introduced in Catch2 2.10.0
> `range()` for floating point numbers has been introduced in Catch2 2.11.0
And can be used as shown in the example below to create a generator
that returns 100 odd random number:
@ -69,20 +159,24 @@ TEST_CASE("Generating random ints", "[example][generator]") {
}
```
_Note that `random` is currently not a part of the first-party generators_.
Apart from registering generators with Catch2, the `GENERATE` macro has
one more purpose, and that is to provide simple way of generating trivial
generators, as seen in the first example on this page, where we used it
as `auto i = GENERATE(1, 2, 3);`. This usage converted each of the three
literals into a single `ValueGenerator<int>` and then placed them all in
literals into a single `SingleValueGenerator<int>` and then placed them all in
a special generator that concatenates other generators. It can also be
used with other generators as arguments, such as `auto i = GENERATE(0, 2,
take(100, random(300, 3000)));`. This is useful e.g. if you know that
specific inputs are problematic and want to test them separately/first.
**For safety reasons, you cannot use variables inside the `GENERATE` macro.**
**For safety reasons, you cannot use variables inside the `GENERATE` macro.
This is done because the generator expression _will_ outlive the outside
scope and thus capturing references is dangerous. If you need to use
variables inside the generator expression, make sure you thought through
the lifetime implications and use `GENERATE_COPY` or `GENERATE_REF`.**
> `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch2 2.7.1.
You can also override the inferred type by using `as<type>` as the first
argument to the macro. This can be useful when dealing with string literals,
@ -90,7 +184,7 @@ if you want them to come out as `std::string`:
```cpp
TEST_CASE("type conversion", "[generators]") {
auto str = GENERATE(as<std::string>{}, "a", "bb", "ccc");`
auto str = GENERATE(as<std::string>{}, "a", "bb", "ccc");
REQUIRE(str.size() > 0);
}
```

View File

@ -45,6 +45,15 @@ the `REQUIRE` family of macros), Catch2 does not know that there are no
more sections in that test case and must run the test case again.
### MinGW/CygWin compilation (linking) is extremely slow
Compiling Catch2 with MinGW can be exceedingly slow, especially during
the linking step. As far as we can tell, this is caused by deficiencies
in its default linker. If you can tell MinGW to instead use lld, via
`-fuse-ld=lld`, the link time should drop down to reasonable length
again.
## Features
This section outlines some missing features, what is their status and their possible workarounds.
@ -85,39 +94,57 @@ to remove this limitation in the future.
### Process isolation in a test
Catch does not support running tests in isolated (forked) processes. While this might in the future, the fact that Windows does not support forking and only allows full-on process creation and the desire to keep code as similar as possible across platforms, mean that this is likely to take significant development time, that is not currently available.
### Running multiple tests in parallel
Catch's test execution is strictly serial. If you find yourself with a test suite that takes too long to run and you want to make it parallel, there are 2 feasible solutions
* You can split your tests into multiple binaries and then run these binaries in parallel.
* You can have Catch list contained test cases and then run the same test binary multiple times in parallel, passing each instance list of test cases it should run.
Both of these solutions have their problems, but should let you wring parallelism out of your test suite.
### Running multiple tests in parallel
Catch2 keeps test execution in one process strictly serial, and there
are no plans to change this. If you find yourself with a test suite
that takes too long to run and yo uwant to make it parallel, you have
to run multiple processes side by side.
There are 2 basic ways to do that,
* you can split your tests into multiple binaries, and run those binaries
in parallel
* you can run the same test binary multiple times, but run a different
subset of the tests in each process
There are multiple ways to achieve the latter, the easiest way is to use
[test sharding](command-line.md#test-sharding).
## 3rd party bugs
This section outlines known bugs in 3rd party components (this means compilers, standard libraries, standard runtimes).
### Visual Studio 2017 -- raw string literal in assert fails to compile
There is a known bug in Visual Studio 2017 (VC 15), that causes compilation error when preprocessor attempts to stringize a raw string literal (`#` preprocessor is applied to it). This snippet is sufficient to trigger the compilation error:
There is a known bug in Visual Studio 2017 (VC 15), that causes compilation
error when preprocessor attempts to stringize a raw string literal
(`#` preprocessor directive is applied to it). This snippet is sufficient
to trigger the compilation error:
```cpp
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
#include <catch2/catch_test_macros.hpp>
TEST_CASE("test") {
CHECK(std::string(R"("\)") == "\"\\");
}
```
Catch provides a workaround, it is possible to disable stringification of original expressions by defining `CATCH_CONFIG_DISABLE_STRINGIFICATION`:
Catch2 provides a workaround, by letting the user disable stringification
of the original expression by defining `CATCH_CONFIG_DISABLE_STRINGIFICATION`,
like so:
```cpp
#define CATCH_CONFIG_FAST_COMPILE
#define CATCH_CONFIG_DISABLE_STRINGIFICATION
#include "catch.hpp"
#include <catch2/catch_test_macros.hpp>
TEST_CASE("test") {
CHECK(std::string(R"("\)") == "\"\\");
}
```
_Do note that this changes the output somewhat_
_Do note that this changes the output:_
```
catchwork\test1.cpp(6):
PASSED:
@ -126,26 +153,11 @@ with expansion:
""\" == ""\"
```
### Visual Studio 2015 -- Alignment compilation error (C2718)
VS 2015 has a known bug, where `declval<T>` can cause compilation error
if `T` has alignment requirements that it cannot meet.
A workaround is to explicitly specialize `Catch::is_range` for given
type (this avoids code path that uses `declval<T>` in a SFINAE context).
### Visual Studio 2015 -- Wrong line number reported in debug mode
VS 2015 has a known bug where `__LINE__` macro can be improperly expanded under certain circumstances, while compiling multi-file project in Debug mode.
A workaround is to compile the binary in Release mode.
### Clang/G++ -- skipping leaf sections after an exception
Some versions of `libc++` and `libstdc++` (or their runtimes) have a bug with `std::uncaught_exception()` getting stuck returning `true` after rethrow, even if there are no active exceptions. One such case is this snippet, which skipped the sections "a" and "b", when compiled against `libcxxrt` from master
```cpp
#define CATCH_CONFIG_MAIN
#include <catch.hpp>
#include <catch2/catch_test_macros.hpp>
TEST_CASE("a") {
CHECK_THROWS(throw 3);
@ -161,11 +173,6 @@ TEST_CASE("b") {
If you are seeing a problem like this, i.e. a weird test paths that trigger only under Clang with `libc++`, or only under very specific version of `libstdc++`, it is very likely you are seeing this. The only known workaround is to use a fixed version of your standard library.
### Clang/G++ -- `Matches` string matcher always returns false
This is a bug in `libstdc++-4.8`, where all matching methods from `<regex>` return false. Since `Matches` uses `<regex>` internally, if the underlying implementation does not work, it doesn't work either.
Workaround: Use newer version of `libstdc++`.
### libstdc++, `_GLIBCXX_DEBUG` macro and random ordering of tests

View File

@ -3,19 +3,19 @@
## Already available
- Catch main: [Catch-provided main](../examples/000-CatchMain.cpp)
- Test Case: [Single-file](../examples/010-TestCase.cpp)
- Test Case: [Multiple-file 1](../examples/020-TestCase-1.cpp), [2](../examples/020-TestCase-2.cpp)
- Assertion: [REQUIRE, CHECK](../examples/030-Asn-Require-Check.cpp)
- Fixture: [Sections](../examples/100-Fix-Section.cpp)
- Fixture: [Class-based fixtures](../examples/110-Fix-ClassFixture.cpp)
- BDD: [SCENARIO, GIVEN, WHEN, THEN](../examples/120-Bdd-ScenarioGivenWhenThen.cpp)
- Report: [Catch-provided main](../examples/200-Rpt-CatchMain.cpp)
- Report: [TeamCity reporter](../examples/207-Rpt-TeamCityReporter.cpp)
- Listener: [Listeners](../examples/210-Evt-EventListeners.cpp)
- Configuration: [Provide your own output streams](../examples/231-Cfg-OutputStreams.cpp)
- Generators: [Create your own generator](../examples/300-Gen-OwnGenerator.cpp)
- Generators: [Use map to convert types in GENERATE expression](../examples/301-Gen-MapTypeConversion.cpp)
- Generators: [Run test with a table of input values](../examples/302-Gen-Table.cpp)
- Generators: [Use variables in generator expressions](../examples/310-Gen-VariablesInGenerators.cpp)
- Generators: [Use custom variable capture in generator expressions](../examples/311-Gen-CustomCapture.cpp)
## Planned

View File

@ -1,7 +1,7 @@
<a id="top"></a>
# Logging macros
Additional messages can be logged during a test case. Note that the messages are scoped and thus will not be reported if failure occurs in scope preceding the message declaration. An example:
Additional messages can be logged during a test case. Note that the messages logged with `INFO` are scoped and thus will not be reported if failure occurs in scope preceding the message declaration. An example:
```cpp
TEST_CASE("Foo") {
@ -28,10 +28,66 @@ The number is 1
```
When the last `CHECK` fails in the "Bar" test case, then only one message will be printed: `Test case start`.
## Logging without local scope
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch2 2.7.0.
`UNSCOPED_INFO` is similar to `INFO` with two key differences:
- Lifetime of an unscoped message is not tied to its own scope.
- An unscoped message can be reported by the first following assertion only, regardless of the result of that assertion.
In other words, lifetime of `UNSCOPED_INFO` is limited by the following assertion (or by the end of test case/section, whichever comes first) whereas lifetime of `INFO` is limited by its own scope.
These differences make this macro useful for reporting information from helper functions or inner scopes. An example:
```cpp
void print_some_info() {
UNSCOPED_INFO("Info from helper");
}
TEST_CASE("Baz") {
print_some_info();
for (int i = 0; i < 2; ++i) {
UNSCOPED_INFO("The number is " << i);
}
CHECK(false);
}
TEST_CASE("Qux") {
INFO("First info");
UNSCOPED_INFO("First unscoped info");
CHECK(false);
INFO("Second info");
UNSCOPED_INFO("Second unscoped info");
CHECK(false);
}
```
"Baz" test case prints:
```
Info from helper
The number is 0
The number is 1
```
With "Qux" test case, two messages will be printed when the first `CHECK` fails:
```
First info
First unscoped info
```
"First unscoped info" message will be cleared after the first `CHECK`, while "First info" message will persist until the end of the test case. Therefore, when the second `CHECK` fails, three messages will be printed:
```
First info
Second info
Second unscoped info
```
## Streaming macros
All these macros allow heterogenous sequences of values to be streaming using the insertion operator (```<<```) in the same way that std::ostream, std::cout, etc support it.
All these macros allow heterogeneous sequences of values to be streaming using the insertion operator (```<<```) in the same way that std::ostream, std::cout, etc support it.
E.g.:
```c++
@ -43,11 +99,17 @@ These macros come in three forms:
**INFO(** _message expression_ **)**
The message is logged to a buffer, but only reported with the next assertion that is logged. This allows you to log contextual information in case of failures which is not shown during a successful test run (for the console reporter, without -s). Messages are removed from the buffer at the end of their scope, so may be used, for example, in loops.
The message is logged to a buffer, but only reported with next assertions that are logged. This allows you to log contextual information in case of failures which is not shown during a successful test run (for the console reporter, without -s). Messages are removed from the buffer at the end of their scope, so may be used, for example, in loops.
_Note that in Catch2 2.x.x `INFO` can be used without a trailing semicolon as there is a trailing semicolon inside macro.
This semicolon will be removed with next major version. It is highly advised to use a trailing semicolon after `INFO` macro._
**UNSCOPED_INFO(** _message expression_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch2 2.7.0.
Similar to `INFO`, but messages are not limited to their own scope: They are removed from the buffer after each assertion, section or test case, whichever comes first.
**WARN(** _message expression_ **)**
The message is always reported but does not fail the test.

View File

@ -1,65 +1,213 @@
<a id="top"></a>
# Matchers
Matchers are an alternative way to do assertions which are easily extensible and composable.
This makes them well suited to use with more complex types (such as collections) or your own custom types.
Matchers were first popularised by the [Hamcrest](https://en.wikipedia.org/wiki/Hamcrest) family of frameworks.
**Contents**<br>
[Using Matchers](#using-matchers)<br>
[Built-in matchers](#built-in-matchers)<br>
[Writing custom matchers (old style)](#writing-custom-matchers-old-style)<br>
[Writing custom matchers (new style)](#writing-custom-matchers-new-style)<br>
## In use
Matchers, as popularized by the [Hamcrest](https://en.wikipedia.org/wiki/Hamcrest)
framework are an alternative way to write assertions, useful for tests
where you work with complex types or need to assert more complex
properties. Matchers are easily composable and users can write their
own and combine them with the Catch2-provided matchers seamlessly.
Matchers are introduced with the `REQUIRE_THAT` or `CHECK_THAT` macros, which take two arguments.
The first argument is the thing (object or value) under test. The second part is a match _expression_,
which consists of either a single matcher or one or more matchers combined using `&&`, `||` or `!` operators.
For example, to assert that a string ends with a certain substring:
```c++
using Catch::Matchers::EndsWith; // or Catch::EndsWith
std::string str = getStringFromSomewhere();
REQUIRE_THAT( str, EndsWith( "as a service" ) );
```
## Using Matchers
The matcher objects can take multiple arguments, allowing more fine tuning.
The built-in string matchers, for example, take a second argument specifying whether the comparison is
case sensitive or not:
Matchers are most commonly used in tandem with the `REQUIRE_THAT` or
`CHECK_THAT` macros. The `REQUIRE_THAT` macro takes two arguments,
the first one is the input (object/value) to test, the second argument
is the matcher itself.
```c++
REQUIRE_THAT( str, EndsWith( "as a service", Catch::CaseSensitive::No ) );
```
For example, to assert that a string ends with the "as a service"
substring, you can write the following assertion
And matchers can be combined:
```cpp
using Catch::Matchers::EndsWith;
```c++
REQUIRE_THAT( str,
EndsWith( "as a service" ) ||
(StartsWith( "Big data" ) && !Contains( "web scale" ) ) );
REQUIRE_THAT( getSomeString(), EndsWith("as a service") );
```
## Built in matchers
Catch currently provides some matchers, they are in the `Catch::Matchers` and `Catch` namespaces.
Individual matchers can also be combined using the C++ logical
operators, that is `&&`, `||`, and `!`, like so:
### String matchers
The string matchers are `StartsWith`, `EndsWith`, `Contains`, `Equals` and `Matches`. The first four match a literal (sub)string against a result, while `Matches` takes and matches an ECMAScript regex. Do note that `Matches` matches the string as a whole, meaning that "abc" will not match against "abcd", but "abc.*" will.
```cpp
using Catch::Matchers::EndsWith;
using Catch::Matchers::ContainsSubstring;
Each of the provided `std::string` matchers also takes an optional second argument, that decides case sensitivity (by-default, they are case sensitive).
REQUIRE_THAT( getSomeString(),
EndsWith("as a service") && ContainsSubstring("web scale"));
```
The example above asserts that the string returned from `getSomeString`
_both_ ends with the suffix "as a service" _and_ contains the string
"web scale" somewhere.
Both of the string matchers used in the examples above live in the
`catch_matchers_string.hpp` header, so to compile the code above also
requires `#include <catch2/matchers/catch_matchers_string.hpp>`.
**IMPORTANT**: The combining operators do not take ownership of the
matcher objects being combined. This means that if you store combined
matcher object, you have to ensure that the matchers being combined
outlive its last use. What this means is that the following code leads
to a use-after-free (UAF):
```cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_string.hpp>
TEST_CASE("Bugs, bugs, bugs", "[Bug]"){
std::string str = "Bugs as a service";
auto match_expression = Catch::Matchers::EndsWith( "as a service" ) ||
(Catch::Matchers::StartsWith( "Big data" ) && !Catch::Matchers::ContainsSubstring( "web scale" ) );
REQUIRE_THAT(str, match_expression);
}
```
## Built-in matchers
Every matcher provided by Catch2 is split into 2 parts, a factory
function that lives in the `Catch::Matchers` namespace, and the actual
matcher type that is in some deeper namespace and should not be used by
the user. In the examples above, we used `Catch::Matchers::Contains`.
This is the factory function for the
`Catch::Matchers::StdString::ContainsMatcher` type that does the actual
matching.
Out of the box, Catch2 provides the following matchers:
### `std::string` matchers
Catch2 provides 5 different matchers that work with `std::string`,
* `StartsWith(std::string str, CaseSensitive)`,
* `EndsWith(std::string str, CaseSensitive)`,
* `ContainsSubstring(std::string str, CaseSensitive)`,
* `Equals(std::string str, CaseSensitive)`, and
* `Matches(std::string str, CaseSensitive)`.
The first three should be fairly self-explanatory, they succeed if
the argument starts with `str`, ends with `str`, or contains `str`
somewhere inside it.
The `Equals` matcher matches a string if (and only if) the argument
string is equal to `str`.
Finally, the `Matches` matcher performs an ECMAScript regex match using
`str` against the argument string. It is important to know that
the match is performed against the string as a whole, meaning that
the regex `"abc"` will not match input string `"abcd"`. To match
`"abcd"`, you need to use e.g. `"abc.*"` as your regex.
The second argument sets whether the matching should be case-sensitive
or not. By default, it is case-sensitive.
> `std::string` matchers live in `catch2/matchers/catch_matchers_string.hpp`
### Vector matchers
The vector matchers are `Contains`, `VectorContains` and `Equals`. `VectorContains` looks for a single element in the matched vector, `Contains` looks for a set (vector) of elements inside the matched vector.
_Vector matchers have been deprecated in favour of the generic
range matchers with the same functionality._
Catch2 provides 5 built-in matchers that work on `std::vector`.
These are
* `Contains` which checks whether a specified vector is present in the result
* `VectorContains` which checks whether a specified element is present in the result
* `Equals` which checks whether the result is exactly equal (order matters) to a specific vector
* `UnorderedEquals` which checks whether the result is equal to a specific vector under a permutation
* `Approx` which checks whether the result is "approx-equal" (order matters, but comparison is done via `Approx`) to a specific vector
> Approx matcher was [introduced](https://github.com/catchorg/Catch2/issues/1499) in Catch2 2.7.2.
An example usage:
```cpp
std::vector<int> some_vec{ 1, 2, 3 };
REQUIRE_THAT(some_vec, Catch::Matchers::UnorderedEquals(std::vector<int>{ 3, 2, 1 }));
```
This assertions will pass, because the elements given to the matchers
are a permutation of the ones in `some_vec`.
> vector matchers live in `catch2/matchers/catch_matchers_vector.hpp`
### Floating point matchers
The floating point matchers are `WithinULP` and `WithinAbs`. `WithinAbs` accepts floating point numbers that are within a certain margin of target. `WithinULP` performs an [ULP](https://en.wikipedia.org/wiki/Unit_in_the_last_place)-based comparison of two floating point numbers and accepts them if they are less than certain number of ULPs apart.
Do note that ULP-based checks only make sense when both compared numbers are of the same type and `WithinULP` will use type of its argument as the target type. This means that `WithinULP(1.f, 1)` will expect to compare `float`s, but `WithinULP(1., 1)` will expect to compare `double`s.
Catch2 provides 3 matchers that target floating point numbers. These
are:
* `WithinAbs(double target, double margin)`,
* `WithinULP(FloatingPoint target, uint64_t maxUlpDiff)`, and
* `WithinRel(FloatingPoint target, FloatingPoint eps)`.
> `WithinRel` matcher was introduced in Catch2 2.10.0
### Generic matchers
Catch also aims to provide a set of generic matchers. Currently this set
contains only a matcher that takes arbitrary callable predicate and applies
it onto the provided object.
`WithinAbs` creates a matcher that accepts floating point numbers whose
difference with `target` is less than the `margin`.
`WithinULP` creates a matcher that accepts floating point numbers that
are no more than `maxUlpDiff`
[ULPs](https://en.wikipedia.org/wiki/Unit_in_the_last_place)
away from the `target` value. The short version of what this means
is that there is no more than `maxUlpDiff - 1` representeable floating
point numbers between the argument for matching and the `target` value.
**Important**: The WithinULP matcher requires the platform to use the
[IEEE-754](https://en.wikipedia.org/wiki/IEEE_754) representation for
floating point numbers.
`WithinRel` creates a matcher that accepts floating point numbers that
are _approximately equal_ with the `target` with tolerance of `eps.`
Specifically, it matches if
`|arg - target| <= eps * max(|arg|, |target|)` holds. If you do not
specify `eps`, `std::numeric_limits<FloatingPoint>::epsilon * 100`
is used as the default.
In practice, you will often want to combine multiple of these matchers,
together for an assertion, because all 3 options have edge cases where
they behave differently than you would expect. As an example, under
the `WithinRel` matcher, a `0.` only ever matches a `0.` (or `-0.`),
regardless of the relative tolerance specified. Thus, if you want to
handle numbers that are "close enough to 0 to be 0", you have to combine
it with the `WithinAbs` matcher.
For example, to check that our computation matches known good value
within 0.1%, or is close enough (no different to 5 decimal places)
to zero, we would write this assertion:
```cpp
REQUIRE_THAT( computation(input),
Catch::Matchers::WithinRel(expected, 0.001)
|| Catch::Matchers::WithinAbs(0, 0.000001) );
```
> floating point matchers live in `catch2/matchers/catch_matchers_floating.hpp`
### Miscellaneous matchers
Catch2 also provides some matchers and matcher utilities that do not
quite fit into other categories.
The first one of them is the `Predicate(Callable pred, std::string description)`
matcher. It creates a matcher object that calls `pred` for the provided
argument. The `description` argument allows users to set what the
resulting matcher should self-describe as if required.
Do note that you will need to explicitly specify the type of the
argument, like in this example:
Because of type inference limitations, the argument type of the predicate
has to be provided explicitly. Example:
```cpp
REQUIRE_THAT("Hello olleH",
Predicate<std::string>(
@ -68,69 +216,220 @@ REQUIRE_THAT("Hello olleH",
);
```
The second argument is an optional description of the predicate, and is
used only during reporting of the result.
> the predicate matcher lives in `catch2/matchers/catch_matchers_predicate.hpp`
## Custom matchers
It's easy to provide your own matchers to extend Catch or just to work with your own types.
The other miscellaneous matcher utility is exception matching.
You need to provide two things:
1. A matcher class, derived from `Catch::MatcherBase<T>` - where `T` is the type being tested.
The constructor takes and stores any arguments needed (e.g. something to compare against) and you must
override two methods: `match()` and `describe()`.
2. A simple builder function. This is what is actually called from the test code and allows overloading.
Here's an example for asserting that an integer falls within a given range
(note that it is all inline for the sake of keeping the example short):
#### Matching exceptions
Catch2 provides an utility macro for asserting that an expression
throws exception of specific type, and that the exception has desired
properties. The macro is `REQUIRE_THROWS_MATCHES(expr, ExceptionType, Matcher)`.
> `REQUIRE_THROWS_MATCHES` macro lives in `catch2/matchers/catch_matchers.hpp`
Catch2 currently provides only one matcher for exceptions,
`Message(std::string message)`. `Message` checks that the exception's
message, as returned from `what` is exactly equal to `message`.
Example use:
```cpp
REQUIRE_THROWS_MATCHES(throwsDerivedException(), DerivedException, Message("DerivedException::what"));
```
Note that `DerivedException` in the example above has to derive from
`std::exception` for the example to work.
> the exception message matcher lives in `catch2/matchers/catch_matchers_exception.hpp`
### Generic range Matchers
> Generic range matchers were introduced in Catch2 3.0.1
Catch2 also provides some matchers that use the new style matchers
definitions to handle generic range-like types. These are:
* `IsEmpty()`
* `SizeIs(size_t target_size)`
* `SizeIs(Matcher size_matcher)`
* `Contains(T&& target_element, Comparator = std::equal_to<>{})`
* `Contains(Matcher element_matcher)`
* `AllMatch(Matcher element_matcher)`
* `NoneMatch(Matcher element_matcher)`
* `AnyMatch(Matcher element_matcher)`
* `AllTrue()`
* `NoneTrue()`
* `AnyTrue()`
`IsEmpty` should be self-explanatory. It successfully matches objects
that are empty according to either `std::empty`, or ADL-found `empty`
free function.
`SizeIs` checks range's size. If constructed with `size_t` arg, the
matchers accepts ranges whose size is exactly equal to the arg. If
constructed from another matcher, then the resulting matcher accepts
ranges whose size is accepted by the provided matcher.
`Contains` accepts ranges that contain specific element. There are
again two variants, one that accepts the desired element directly,
in which case a range is accepted if any of its elements is equal to
the target element. The other variant is constructed from a matcher,
in which case a range is accepted if any of its elements is accepted
by the provided matcher.
`AllMatch`, `NoneMatch`, and `AnyMatch` match ranges for which either
all, none, or any of the contained elements matches the given matcher,
respectively.
`AllTrue`, `NoneTrue`, and `AnyTrue` match ranges for which either
all, none, or any of the contained elements are `true`, respectively.
It works for ranges of `bool`s and ranges of elements (explicitly)
convertible to `bool`.
## Writing custom matchers (old style)
The old style of writing matchers has been introduced back in Catch
Classic. To create an old-style matcher, you have to create your own
type that derives from `Catch::Matchers::MatcherBase<ArgT>`, where
`ArgT` is the type your matcher works for. Your type has to override
two methods, `bool match(ArgT const&) const`,
and `std::string describe() const`.
As the name suggests, `match` decides whether the provided argument
is matched (accepted) by the matcher. `describe` then provides a
human-oriented description of what the matcher does.
We also recommend that you create factory function, just like Catch2
does, but that is mostly useful for template argument deduction for
templated matchers (assuming you do not have CTAD available).
To combine these into an example, let's say that you want to write
a matcher that decides whether the provided argument is a number
within certain range. We will call it `IsBetweenMatcher<T>`:
```c++
// The matcher class
class IntRange : public Catch::MatcherBase<int> {
int m_begin, m_end;
public:
IntRange( int begin, int end ) : m_begin( begin ), m_end( end ) {}
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers.hpp>
// ...
// Performs the test for this matcher
virtual bool match( int const& i ) const override {
return i >= m_begin && i <= m_end;
template <typename T>
class IsBetweenMatcher : public Catch::Matchers::MatcherBase<T> {
T m_begin, m_end;
public:
IsBetweenMatcher(T begin, T end) : m_begin(begin), m_end(end) {}
bool match(T const& in) const override {
return in >= m_begin && in <= m_end;
}
// Produces a string describing what this matcher does. It should
// include any provided data (the begin/ end in this case) and
// be written as if it were stating a fact (in the output it will be
// preceded by the value under test).
virtual std::string describe() const {
std::string describe() const override {
std::ostringstream ss;
ss << "is between " << m_begin << " and " << m_end;
return ss.str();
}
};
// The builder function
inline IntRange IsBetween( int begin, int end ) {
return IntRange( begin, end );
template <typename T>
IsBetweenMatcher<T> IsBetween(T begin, T end) {
return { begin, end };
}
// ...
// Usage
TEST_CASE("Integers are within a range")
{
CHECK_THAT( 3, IsBetween( 1, 10 ) );
CHECK_THAT( 100, IsBetween( 1, 10 ) );
TEST_CASE("Numbers are within range") {
// infers `double` for the argument type of the matcher
CHECK_THAT(3., IsBetween(1., 10.));
// infers `int` for the argument type of the matcher
CHECK_THAT(100, IsBetween(1, 10));
}
```
Running this test gives the following in the console:
```
/**/TestFile.cpp:123: FAILED:
CHECK_THAT( 100, IsBetween( 1, 10 ) )
with expansion:
100 is between 1 and 10
Obviously, the code above can be improved somewhat, for example you
might want to `static_assert` over the fact that `T` is an arithmetic
type... or generalize the matcher to cover any type for which the user
can provide a comparison function object.
Note that while any matcher written using the old style can also be
written using the new style, combining old style matchers should
generally compile faster. Also note that you can combine old and new
style matchers arbitrarily.
> `MatcherBase` lives in `catch2/matchers/catch_matchers.hpp`
## Writing custom matchers (new style)
> New style matchers were introduced in Catch2 3.0.1
To create a new-style matcher, you have to create your own type that
derives from `Catch::Matchers::MatcherGenericBase`. Your type has to
also provide two methods, `bool match( ... ) const` and overriden
`std::string describe() const`.
Unlike with old-style matchers, there are no requirements on how
the `match` member function takes its argument. This means that the
argument can be taken by value or by mutating reference, but also that
the matcher's `match` member function can be templated.
This allows you to write more complex matcher, such as a matcher that
can compare one range-like (something that responds to `begin` and
`end`) object to another, like in the following example:
```cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_templated.hpp>
// ...
template<typename Range>
struct EqualsRangeMatcher : Catch::Matchers::MatcherGenericBase {
EqualsRangeMatcher(Range const& range):
range{ range }
{}
template<typename OtherRange>
bool match(OtherRange const& other) const {
using std::begin; using std::end;
return std::equal(begin(range), end(range), begin(other), end(other));
}
std::string describe() const override {
return "Equals: " + Catch::rangeToString(range);
}
private:
Range const& range;
};
template<typename Range>
auto EqualsRange(const Range& range) -> EqualsRangeMatcher<Range> {
return EqualsRangeMatcher<Range>{range};
}
TEST_CASE("Combining templated matchers", "[matchers][templated]") {
std::array<int, 3> container{{ 1,2,3 }};
std::array<int, 3> a{{ 1,2,3 }};
std::vector<int> b{ 0,1,2 };
std::list<int> c{ 4,5,6 };
REQUIRE_THAT(container, EqualsRange(a) || EqualsRange(b) || EqualsRange(c));
}
```
Do note that while you can rewrite any matcher from the old style to
a new style matcher, combining new style matchers is more expensive
in terms of compilation time. Also note that you can combine old style
and new style matchers arbitrarily.
> `MatcherGenericBase` lives in `catch2/matchers/catch_matchers_templated.hpp`
---
[Home](Readme.md#top)

85
docs/migrate-v2-to-v3.md Normal file
View File

@ -0,0 +1,85 @@
<a id="top"></a>
# Migrating from v2 to v3
v3 is the next major version of Catch2 and brings three significant changes:
* Catch2 is now split into multiple headers
* Catch2 is now compiled as a static library
* C++14 is the minimum required C++ version
There are many reasons why we decided to go from the old single-header
distribution model to a more standard library distribution model. The
big one is compile-time performance, but moving over to a split header
distribution model also improves the future maintainability and
extendability of the codebase. For example v3 adds a new kind of matchers
without impacting the compilation times of users that do not use matchers
in their tests. The new model is also more friendly towards package
managers, such as vcpkg and Conan.
The result of this move is a significant improvement in compilation
times, e.g. the inclusion overhead of Catch2 in the common case has been
reduced by roughly 80%. The improved ease of maintenance also led to
various runtime performance improvements and the introduction of new features.
For details, look at [the release notes of 3.0.1](release-notes.md#301).
_Note that we still provide one header + one TU distribution but do
not consider it the primarily supported option. You should also expect
that the compilation times will be worse if you use this option._
## How to migrate projects from v2 to v3
To migrate to v3, there are two basic approaches to do so.
1. Use `catch_amalgamated.hpp` and `catch_amalgamated.cpp`.
2. Build Catch2 as a proper (static) library, and move to piecewise headers
Doing 1 means downloading the [amalgamated header](/extras/catch_amalgamated.hpp)
and the [amalgamated sources](/extras/catch_amalgamated.cpp) from `extras`,
dropping them into your test project, and rewriting your includes from
`<catch2/catch.hpp>` to `"catch_amalgamated.hpp"` (or something similar,
based on how you set up your paths).
The disadvantage of using this approach are increased compilation times,
at least compared to the second approach, but it does let you avoid
dealing with consuming libraries in your build system of choice.
However, we recommend doing 2, and taking extra time to migrate to v3
properly. This lets you reap the benefits of significantly improved
compilation times in the v3 version. The basic steps to do so are:
1. Change your CMakeLists.txt to link against `Catch2WithMain` target if
you use Catch2's default main. (If you do not, keep linking against
the `Catch2` target.). If you use pkg-config, change `pkg-config catch2` to
`pkg-config catch2-with-main`.
2. Delete TU with `CATCH_CONFIG_RUNNER` or `CATCH_CONFIG_MAIN` defined,
as it is no longer needed.
3. Change `#include <catch2/catch.hpp>` to `#include <catch2/catch_all.hpp>`
4. Check that everything compiles. You might have to modify namespaces,
or perform some other changes (see the
[Things that can break during porting](#things-that-can-break-during-porting)
section for the most common things).
5. Start migrating your test TUs from including `<catch2/catch_all.hpp>`
to piecemeal includes. You will likely want to start by including
`<catch2/catch_test_macros.hpp>`, and then go from there. (see
[other notes](#other-notes) for further ideas)
## Other notes
* The main test include is now `<catch2/catch_test_macros.hpp>`
* Big "subparts" like Matchers, or Generators, have their own folder, and
also their own "big header", so if you just want to include all matchers,
you can include `<catch2/matchers/catch_matchers_all.hpp>`,
or `<catch2/generators/catch_generators_all.hpp>`
## Things that can break during porting
* The namespace on Matchers were cleaned up, they are no longer first declared
deep within an internal namespace and then brought up. All Matchers now live
in `Catch::Matchers`.
* The reporter interfaces changed in a breaking manner. If you wrote custom
reporter or listener, you might need to modify them a bit.
---
[Home](Readme.md#top)

View File

@ -1,54 +1,75 @@
<a id="top"></a>
# Open Source projects using Catch
# Open Source projects using Catch2
Catch is great for open source. With its [liberal license](../LICENSE.txt) and single-header, dependency-free, distribution
it's easy to just drop the header into your project and start writing tests - what's not to like?
Catch2 is great for open source. It is licensed under the [Boost Software
License (BSL)](../LICENSE.txt), has no further dependencies and supports
two file distribution.
As a result Catch is now being used in many Open Source projects, including some quite well known ones.
This page is an attempt to track those projects. Obviously it can never be complete.
This effort largely relies on the maintainers of the projects themselves updating this page and submitting a PR
(or, if you prefer contact one of the maintainers of Catch directly, use the
[forums](https://groups.google.com/forum/?fromgroups#!forum/catch-forum)), or raise an [issue](https://github.com/philsquared/Catch/issues) to let us know).
Of course users of those projects might want to update this page too. That's fine - as long you're confident the project maintainers won't mind.
If you're an Open Source project maintainer and see your project listed here but would rather it wasn't -
just let us know via any of the previously mentioned means - although I'm sure there won't be many who feel that way.
As a result, Catch2 is used for testing in many different Open Source
projects. This page lists at least some of them, even though it will
obviously never be complete (and does not have the ambition to be
complete). Note that the list below is intended to be in alphabetical
order, to avoid implications of relative importance of the projects.
_Please only add projects here if you are their maintainer, or have the
maintainer's explicit consent._
Listing a project here does not imply endorsement and the plan is to keep these ordered alphabetically to avoid an implication of relative importance.
## Libraries & Frameworks
### [Azmq](https://github.com/zeromq/azmq)
Boost Asio style bindings for ZeroMQ
### [accessorpp](https://github.com/wqking/accessorpp)
C++ library for implementing property and data binding.
### [ChakraCore](https://github.com/Microsoft/ChakraCore)
The core part of the Chakra JavaScript engine that powers Microsoft Edge
### [alpaka](https://github.com/alpaka-group/alpaka)
A header-only C++14 abstraction library for accelerator development.
### [ApprovalTests.cpp](https://github.com/approvals/ApprovalTests.cpp)
C++11 implementation of Approval Tests, for quick, convenient testing of legacy code.
### [args](https://github.com/Taywee/args)
A simple header-only C++ argument parser library.
### [Azmq](https://github.com/zeromq/azmq)
Boost Asio style bindings for ZeroMQ.
### [Cataclysm: Dark Days Ahead](https://github.com/CleverRaven/Cataclysm-DDA)
Post-apocalyptic survival RPG.
### [ChaiScript](https://github.com/ChaiScript/ChaiScript)
A, header-only, embedded scripting language designed from the ground up to directly target C++ and take advantage of modern C++ development techniques
A, header-only, embedded scripting language designed from the ground up to directly target C++ and take advantage of modern C++ development techniques.
### [ChakraCore](https://github.com/Microsoft/ChakraCore)
The core part of the Chakra JavaScript engine that powers Microsoft Edge.
### [Clara](https://github.com/philsquared/Clara)
A, single-header-only, type-safe, command line parser - which also prints formatted usage strings.
### [Couchbase-lite-core](https://github.com/couchbase/couchbase-lite-core)
The next-generation core storage and query engine for Couchbase Lite
The next-generation core storage and query engine for Couchbase Lite.
### [cppcodec](https://github.com/tplgy/cppcodec)
Header-only C++11 library to encode/decode base64, base64url, base32, base32hex and hex (a.k.a. base16) as specified in RFC 4648, plus Crockford's base32.
### [DtCraft](https://github.com/twhuang-uiuc/DtCraft)
A High-performance Cluster Computing Engine
A High-performance Cluster Computing Engine.
### [eventpp](https://github.com/wqking/eventpp)
C++ event library for callbacks, event dispatcher, and event queue. With eventpp you can easily implement signal and slot mechanism, publisher and subscriber pattern, or observer pattern.
### [forest](https://github.com/xorz57/forest)
Template Library of Tree Data Structures
Template Library of Tree Data Structures.
### [Fuxedo](https://github.com/fuxedo/fuxedo)
Open source Oracle Tuxedo-like XATMI middleware for C and C++.
### [HIP CPU Runtime](https://github.com/ROCm-Developer-Tools/HIP-CPU)
A header-only library that allows CPUs to execute unmodified HIP code. It is generic and does not assume a particular CPU vendor or architecture.
### [Inja](https://github.com/pantor/inja)
A header-only template engine for modern C++.
### [JSON for Modern C++](https://github.com/nlohmann/json)
A, single-header, JSON parsing library that takes advantage of what C++ has to offer.
### [LLAMA](https://github.com/alpaka-group/llama)
A C++17 template header-only library for the abstraction of memory access patterns.
### [libcluon](https://github.com/chrberger/libcluon)
A single-header-only library written in C++14 to glue distributed software components (UDP, TCP, shared memory) supporting natively Protobuf, LCM/ZCM, MsgPack, and JSON for dynamic message transformations in-between.
@ -62,49 +83,67 @@ A small C++ library wrapper for the native C ODBC API.
### [Nonius](https://github.com/libnonius/nonius)
A header-only framework for benchmarking small snippets of C++ code.
### [SOCI](https://github.com/SOCI/soci)
The C++ Database Access Library
### [OpenALpp](https://github.com/Laguna1989/OpenALpp)
A modern OOP C++14 audio library built on OpenAL for Windows, Linux and web (emscripten).
### [polymorphic_value](https://github.com/jbcoe/polymorphic_value)
A polymorphic value-type for C++
A polymorphic value-type for C++.
### [Ppconsul](https://github.com/oliora/ppconsul)
A C++ client library for Consul. Consul is a distributed tool for discovering and configuring services in your infrastructure
A C++ client library for Consul. Consul is a distributed tool for discovering and configuring services in your infrastructure.
### [Reactive-Extensions/ RxCpp](https://github.com/Reactive-Extensions/RxCpp)
A library of algorithms for values-distributed-in-time
A library of algorithms for values-distributed-in-time.
### [thor](https://github.com/xorz57/thor)
Wrapper Library for CUDA
### [SOCI](https://github.com/SOCI/soci)
The C++ Database Access Library.
### [TextFlowCpp](https://github.com/philsquared/textflowcpp)
A small, single-header-only, library for wrapping and composing columns of text
A small, single-header-only, library for wrapping and composing columns of text.
### [thor](https://github.com/xorz57/thor)
Wrapper Library for CUDA.
### [toml++](https://github.com/marzer/tomlplusplus)
A header-only TOML parser and serializer for modern C++.
### [Trompeloeil](https://github.com/rollbear/trompeloeil)
A thread safe header only mocking framework for C++14
### [args](https://github.com/Taywee/args)
A simple header-only C++ argument parser library.
A thread-safe header-only mocking framework for C++14.
## Applications & Tools
### [App Mesh](https://github.com/laoshanxi/app-mesh)
A high available cloud native micro-service application management platform implemented by modern C++.
### [ArangoDB](https://github.com/arangodb/arangodb)
ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values.
### [Cytopia](https://github.com/CytopiaTeam/Cytopia)
Cytopia is a free, open source retro pixel-art city building game with a big focus on mods. It utilizes a custom isometric rendering engine based on SDL2.
### [d-SEAMS](https://github.com/d-SEAMS/seams-core)
Open source molecular dynamics simulation structure analysis suite of tools in modern C++.
### [Giada - Your Hardcore Loop Machine](https://github.com/monocasual/giada)
Minimal, open-source and cross-platform audio tool for live music production.
### [MAME](https://github.com/mamedev/mame)
MAME originally stood for Multiple Arcade Machine Emulator
MAME originally stood for Multiple Arcade Machine Emulator.
### [Newsbeuter](https://github.com/akrennmair/newsbeuter)
Newsbeuter is an open-source RSS/Atom feed reader for text terminals.
### [PopHead](https://github.com/SPC-Some-Polish-Coders/PopHead)
A 2D, Zombie, RPG game which is being made on our own engine.
### [raspigcd](https://github.com/pantadeusz/raspigcd)
Low level CLI app and library for execution of GCODE on Raspberry Pi without any additional microcontrolers (just RPi + Stepsticks).
### [SpECTRE](https://github.com/sxs-collaboration/spectre)
SpECTRE is a code for multi-scale, multi-physics problems in astrophysics and gravitational physics.
### [Standardese](https://github.com/foonathan/standardese)
Standardese aims to be a nextgen Doxygen
Standardese aims to be a nextgen Doxygen.
---

View File

@ -15,6 +15,8 @@ stringification machinery to the _expr_ and records the result. As with
evaluates to `true`. `CHECKED_ELSE( expr )` work similarly, but the block
is entered only if the _expr_ evaluated to `false`.
> `CHECKED_X` macros were changed to not count as failure in Catch2 3.0.1.
Example:
```cpp
int a = ...;
@ -30,7 +32,7 @@ CHECKED_IF( a == b ) {
`CHECK_NOFAIL( expr )` is a variant of `CHECK` that does not fail the test
case if _expr_ evaluates to `false`. This can be useful for checking some
assumption, that might be violated without the test neccessarily failing.
assumption, that might be violated without the test necessarily failing.
Example output:
```
@ -57,7 +59,9 @@ TEST_CASE( "SUCCEED showcase" ) {
}
```
* `STATIC_REQUIRE`
* `STATIC_REQUIRE` and `STATIC_CHECK`
> `STATIC_REQUIRE` was [introduced](https://github.com/catchorg/Catch2/issues/1362) in Catch2 2.4.2.
`STATIC_REQUIRE( expr )` is a macro that can be used the same way as a
`static_assert`, but also registers the success with Catch2, so it is
@ -73,6 +77,20 @@ TEST_CASE("STATIC_REQUIRE showcase", "[traits]") {
}
```
> `STATIC_CHECK` was [introduced](https://github.com/catchorg/Catch2/pull/2318) in Catch2 3.0.1.
`STATIC_CHECK( expr )` is equivalent to `STATIC_REQUIRE( expr )`, with the
difference that when `CATCH_CONFIG_RUNTIME_STATIC_REQUIRE` is defined, it
becomes equivalent to `CHECK` instead of `REQUIRE`.
Example:
```cpp
TEST_CASE("STATIC_CHECK showcase", "[traits]") {
STATIC_CHECK( std::is_void<void>::value );
STATIC_CHECK_FALSE( std::is_void<int>::value );
}
```
## Test case related macros
* `METHOD_AS_TEST_CASE`
@ -115,23 +133,10 @@ is initiated. This means that it either needs to be done in a global
constructor, or before Catch2's session is created in user's own main._
* `ANON_TEST_CASE`
`ANON_TEST_CASE` is a `TEST_CASE` replacement that will autogenerate
unique name. The advantage of this is that you do not have to think
of a name for the test case,`the disadvantage is that the name doesn't
neccessarily remain stable across different links, and thus it might be
hard to run directly.
Example:
```cpp
ANON_TEST_CASE() {
SUCCEED("Hello from anonymous test case");
}
```
* `DYNAMIC_SECTION`
> Introduced in Catch2 2.3.0.
`DYNAMIC_SECTION` is a `SECTION` where the user can use `operator<<` to
create the final name for that section. This can be useful with e.g.
generators, or when creating a `SECTION` dynamically, within a loop.

View File

@ -2,63 +2,69 @@
# Supplying main() yourself
**Contents**<br>
[Let Catch take full control of args and config](#let-catch-take-full-control-of-args-and-config)<br>
[Amending the config](#amending-the-config)<br>
[Let Catch2 take full control of args and config](#let-catch2-take-full-control-of-args-and-config)<br>
[Amending the Catch2 config](#amending-the-catch2-config)<br>
[Adding your own command line options](#adding-your-own-command-line-options)<br>
[Version detection](#version-detection)<br>
The easiest way to use Catch is to let it supply ```main()``` for you and handle configuring itself from the command line.
The easiest way to use Catch2 is to use its own `main` function, and let
it handle the command line arguments. This is done by linking against
Catch2Main library, e.g. through the [CMake target](cmake-integration.md#cmake-targets),
or pkg-config files.
This is achieved by writing ```#define CATCH_CONFIG_MAIN``` before the ```#include "catch.hpp"``` in *exactly one* source file.
If you want to provide your own `main`, then you should link against
the static library (target) only, without the main part. You will then
have to write your own `main` and call into Catch2 test runner manually.
Sometimes, though, you need to write your own version of main(). You can do this by writing ```#define CATCH_CONFIG_RUNNER``` instead. Now you are free to write ```main()``` as normal and call into Catch yourself manually.
Below are some basic recipes on what you can do supplying your own main.
You now have a lot of flexibility - but here are three recipes to get your started:
## Let Catch take full control of args and config
## Let Catch2 take full control of args and config
If you just need to have code that executes before and/ or after Catch this is the simplest option.
This is useful if you just need to have code that executes before/after
Catch2 runs tests.
```c++
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
```cpp
#include <catch2/catch_session.hpp>
int main( int argc, char* argv[] ) {
// global setup...
// your setup ...
int result = Catch::Session().run( argc, argv );
// global clean-up...
// your clean-up...
return result;
}
```
## Amending the config
_Note that if you only want to run some set up before tests are run, it
might be simpler to use [event listeners](event-listeners.md#top) instead._
If you still want Catch to process the command line, but you want to programmatically tweak the config, you can do so in one of two ways:
## Amending the Catch2 config
If you want Catch2 to process command line arguments, but also want to
programmatically change the resulting configuration of Catch2 run,
you can do it in two ways:
```c++
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
int main( int argc, char* argv[] )
{
int main( int argc, char* argv[] ) {
Catch::Session session; // There must be exactly one instance
// writing to session.configData() here sets defaults
// this is the preferred way to set them
int returnCode = session.applyCommandLine( argc, argv );
if( returnCode != 0 ) // Indicates a command line error
return returnCode;
// writing to session.configData() or session.Config() here
// writing to session.configData() or session.Config() here
// overrides command line args
// only do this if you know you need to
int numFailed = session.run();
// numFailed is clamped to 255 as some unices only use the lower 8 bits.
// This clamping has already been applied, so just return it here
// You can also do any post run clean-up here
@ -66,38 +72,32 @@ int main( int argc, char* argv[] )
}
```
Take a look at the definitions of Config and ConfigData to see what you can do with them.
If you want full control of the configuration, don't call `applyCommandLine`.
To take full control of the config simply omit the call to ```applyCommandLine()```.
## Adding your own command line options
Catch embeds a powerful command line parser called [Clara](https://github.com/philsquared/Clara).
As of Catch2 (and Clara 1.0) Clara allows you to write _composable_ option and argument parsers,
so extending Catch's own command line options is now easy.
You can add new command line options to Catch2, by composing the premade
CLI parser (called Clara), and add your own options.
```c++
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
int main( int argc, char* argv[] )
{
```cpp
int main( int argc, char* argv[] ) {
Catch::Session session; // There must be exactly one instance
int height = 0; // Some user variable you want to be able to set
// Build a new parser on top of Catch's
using namespace Catch::clara;
auto cli
= session.cli() // Get Catch's composite command line parser
// Build a new parser on top of Catch2's
using namespace Catch::Clara;
auto cli
= session.cli() // Get Catch2's command line parser
| Opt( height, "height" ) // bind variable to a new option, with a hint string
["-g"]["--height"] // the option names it will respond to
("how high?"); // description string for the help output
// Now pass the new composite back to Catch so it uses that
session.cli( cli );
// Let Catch (using Clara) parse the command line
// Now pass the new composite back to Catch2 so it uses that
session.cli( cli );
// Let Catch2 (using Clara) parse the command line
int returnCode = session.applyCommandLine( argc, argv );
if( returnCode != 0 ) // Indicates a command line error
return returnCode;
@ -110,12 +110,13 @@ int main( int argc, char* argv[] )
}
```
See the [Clara documentation](https://github.com/philsquared/Clara/blob/master/README.md) for more details.
See the [Clara documentation](https://github.com/catchorg/Clara/blob/master/README.md)
for more details on how to use the Clara parser.
## Version detection
Catch provides a triplet of macros providing the header's version,
Catch2 provides a triplet of macros providing the header's version,
* `CATCH_VERSION_MAJOR`
* `CATCH_VERSION_MINOR`

View File

@ -2,6 +2,36 @@
# Release notes
**Contents**<br>
[3.1.0](#310)<br>
[3.0.1](#301)<br>
[2.13.7](#2137)<br>
[2.13.6](#2136)<br>
[2.13.5](#2135)<br>
[2.13.4](#2134)<br>
[2.13.3](#2133)<br>
[2.13.2](#2132)<br>
[2.13.1](#2131)<br>
[2.13.0](#2130)<br>
[2.12.4](#2124)<br>
[2.12.3](#2123)<br>
[2.12.2](#2122)<br>
[2.12.1](#2121)<br>
[2.12.0](#2120)<br>
[2.11.3](#2113)<br>
[2.11.2](#2112)<br>
[2.11.1](#2111)<br>
[2.11.0](#2110)<br>
[2.10.2](#2102)<br>
[2.10.1](#2101)<br>
[2.10.0](#2100)<br>
[2.9.2](#292)<br>
[2.9.1](#291)<br>
[2.9.0](#290)<br>
[2.8.0](#280)<br>
[2.7.2](#272)<br>
[2.7.1](#271)<br>
[2.7.0](#270)<br>
[2.6.1](#261)<br>
[2.6.0](#260)<br>
[2.5.0](#250)<br>
[2.4.2](#242)<br>
@ -19,6 +49,700 @@
[Older versions](#older-versions)<br>
[Even Older versions](#even-older-versions)<br>
## 3.1.0
### Improvements
* Improved suppression of `-Wparentheses` for older GCCs
* Turns out that even GCC 9 does not properly handle `_Pragma`s in the C++ frontend.
* Added type constraints onto `random` generator (#2433)
* These constraints copy what the standard says for the underlying `std::uniform_int_distribution`
* Suppressed -Wunused-variable from nvcc (#2306, #2427)
* Suppressed -Wunused-variable from MinGW (#2132)
* Added All/Any/NoneTrue range matchers (#2319)
* These check that all/any/none of boolean values in a range are true.
* The JUnit reporter now normalizes classnames from C++ namespaces to Java-like namespaces (#2468)
* This provides better support for other JUnit based tools.
* The Bazel support now understands `BAZEL_TEST` environment variable (#2459)
* The `CATCH_CONFIG_BAZEL_SUPPORT` configuration option is also still supported.
* Returned support for compiling Catch2 with GCC 5 (#2448)
* This required removing inherited constructors from Catch2's internals.
* I recommend updating to a newer GCC anyway.
* `catch_discover_tests` now has a new options for setting library load path(s) when running the Catch2 binary (#2467)
### Fixes
* Fixed crash when listing listeners without any registered listeners (#2442)
* Fixed nvcc compilation error in constructor benchmarking helper (#2477)
* Catch2's CMakeList supports pre-3.12 CMake again (#2428)
* The gain from requiring CMake 3.12 was very minor, but y'all should really update to newer CMake
### Miscellaneous
* Fixed SelfTest build on MinGW (#2447)
* The in-repo conan recipe exports the CMake helper (#2460)
* Added experimental CMake script to showcase using test case sharding together with CTest
* Compared to `catch_discover_tests`, it supports very limited number of options and customization
* Added documentation page on best practices when running Catch2 tests
* Catch2 can be built as a dynamic library (#2397, #2398)
* Note that Catch2 does not have visibility annotations, and you are responsible for ensuring correct visibility built into the resulting library.
## 3.0.1
**Catch2 now uses statically compiled library as its distribution model.
This also means that to get all of Catch2's functionality in a test file,
you have to include multiple headers.**
You probably want to look into the [migration docs](migrate-v2-to-v3.md#top),
which were written to help people coming from v2.x.x versions to the
v3 releases.
### FAQ
* Why is Catch2 moving to separate headers?
* The short answer is future extensibility and scalability. The long answer is complex and can be found on my blog, but at the most basic level, it is that providing single-header distribution is at odds with providing variety of useful features. When Catch2 was distributed in a single header, adding a new Matcher would cause overhead for everyone, but was useful only to a subset of users. This meant that the barrier to entry for new Matchers/Generators/etc is high in single header model, but much smaller in the new model.
* Will Catch2 again distribute single-header version in the future?
* No. But we do provide sqlite-style amalgamated distribution option. This means that you can download just 1 .cpp file and 1 header and place them next to your own sources. However, doing this has downsides similar to using the `catch_all.hpp` header.
* Why the big breaking change caused by replacing `catch.hpp` with `catch_all.hpp`?
* The convenience header `catch_all.hpp` exists for two reasons. One of them is to provide a way for quick migration from Catch2, the second one is to provide a simple way to test things with Catch2. Using it for migration has one drawback in that it is **big**. This means that including it _will_ cause significant compile time drag, and so using it to migrate should be a conscious decision by the user, not something they can just stumble into unknowingly.
### (Potentially) Breaking changes
* **Catch2 now uses statically compiled library as its distribution model**
* **Including `catch.hpp` no longer works**
* **Catch2 now uses C++14 as the minimum support language version**
* `ANON_TEST_CASE` has been removed, use `TEST_CASE` with no arguments instead (#1220)
* `--list*` commands no longer have non-zero return code (#1410)
* `--list-test-names-only` has been removed (#1190)
* You should use verbosity-modifiers for `--list-tests` instead
* `--list*` commands are now piped through the reporters
* The top-level reporter interface provides default implementation that works just as the old one
* XmlReporter outputs a machine-parseable XML
* `TEST_CASE` description support has been removed
* If the second argument has text outside tags, the text will be ignored.
* Hidden test cases are no longer included just because they don't match an exclusion tag
* Previously, a `TEST_CASE("A", "[.foo]")` would be included by asking for `~[bar]`.
* `PredicateMatcher` is no longer type erased.
* This means that the type of the provided predicate is part of the `PredicateMatcher`'s type
* `SectionInfo` no longer contains section description as a member (#1319)
* You can still write `SECTION("ShortName", "Long and wordy description")`, but the description is thrown away
* The description type now must be a `const char*` or be implicitly convertible to it
* The `[!hide]` tag has been removed.
* Use `[.]` or `[.foo]` instead.
* Lvalues of composed matchers cannot be composed further
* Uses of `REGISTER_TEST_CASE` macro need to be followed by a semicolon
* This does not change `TEST_CASE` and friends in any way
* `IStreamingReporter::IsMulti` member function was removed
* This is _very_ unlikely to actually affect anyone, as it was default-implemented in the interface, and only used internally
* Various classes not designed for user-extension have been made final
* `ListeningReporter` is now `final`
* Concrete Matchers (e.g. `UnorderedEquals` vector matcher) are now `final`
* All Generators are now `final`
* Matcher namespacing has been redone
* Matcher types are no longer in deeply nested namespaces
* Matcher factory functions are no longer brought into `Catch` namespace
* This means that all public-facing matcher-related functionality is now in `Catch::Matchers` namespace
* Defining `CATCH_CONFIG_MAIN` will no longer create main in that TU.
* Link with `libCatch2Main.a`, or the proper CMake/pkg-config target
* If you want to write custom main, include `catch2/catch_session.hpp`
* `CATCH_CONFIG_EXTERNAL_INTERFACES` has been removed.
* You should instead include the appropriate headers as needed.
* `CATCH_CONFIG_IMPL` has been removed.
* The implementation is now compiled into a static library.
* Event Listener interface has changed
* `TestEventListenerBase` was renamed to `EventListenerBase`
* `EventListenerBase` now directly derives from `IStreamingReporter`, instead of deriving from `StreamingReporterBase`
* `GENERATE` decays its arguments (#2012, #2040)
* This means that `str` in `auto str = GENERATE("aa", "bb", "cc");` is inferred to `char const*` rather than `const char[2]`.
* `--list-*` flags write their output to file specified by the `-o` flag
* Many changes to reporter interfaces
* With the exception of the XmlReporter, the outputs of first party reporters should remain the same
* New pair of events were added
* One obsolete event was removed
* The base class has been renamed
* The built-in reporter class hierarchy has been redone
* Catch2 generates a random seed if one hasn't been specified by the user
* The short flag for `--list-tests`, `-l`, has been removed.
* This is not a commonly used flag and does not need to use up valuable single-letter space.
* The short flag for `--list-tags`, `-t`, has been removed.
* This is not a commonly used flag and does not need to use up valuable single-letter space.
* The `--colour` option has been replaced with `--colour-mode` option
### Improvements
* Matchers have been extended with the ability to use different signatures of `match` (#1307, #1553, #1554, #1843)
* This includes having templated `match` member function
* See the [rewritten Matchers documentation](matchers.md#top) for details
* Catch2 currently provides _some_ generic matchers, but there should be more before final release of v3
* `IsEmpty`, `SizeIs` which check that the range has specific properties
* `Contains`, which checks whether a range contains a specific element
* `AllMatch`, `AnyMatch`, `NoneMatch` range matchers, which apply matchers over a range of elements
* Significant compilation time improvements
* including `catch_test_macros.hpp` is 80% cheaper than including `catch.hpp`
* Some runtime performance optimizations
* In all tested cases the v3 branch was faster, so the table below shows the speedup of v3 to v2 at the same task
<a id="v3-runtime-optimization-table"></a>
| task | debug build | release build |
|:------------------------------------------- | ------------:| -------------:|
| Run 1M `REQUIRE(true)` | 1.10 ± 0.01 | 1.02 ± 0.06 |
| Run 100 tests, 3^3 sections, 1 REQUIRE each | 1.27 ± 0.01 | 1.04 ± 0.01 |
| Run 3k tests, no names, no tags | 1.29 ± 0.01 | 1.05 ± 0.01 |
| Run 3k tests, names, tags | 1.49 ± 0.01 | 1.22 ± 0.01 |
| Run 1 out of 3k tests no names, no tags | 1.68 ± 0.02 | 1.19 ± 0.22 |
| Run 1 out of 3k tests, names, tags | 1.79 ± 0.02 | 2.06 ± 0.23 |
* POSIX platforms use `gmtime_r`, rather than `gmtime` when constructing a date string (#2008, #2165)
* `--list-*` flags write their output to file specified by the `-o` flag (#2061, #2163)
* `Approx::operator()` is now properly `const`
* Catch2's internal helper variables no longer use reserved identifiers (#578)
* `--rng-seed` now accepts string `"random-device"` to generate random seed using `std::random_device`
* Catch2 now supports test sharding (#2257)
* You can ask for the tests to be split into N groups and only run one of them.
* This greatly simplifies parallelization of tests in a binary through external runner.
* The embedded CLI parser now supports repeatedly callable lambdas
* A lambda-based option parser can opt into being repeatedly specifiable.
* Added `STATIC_CHECK` macro, similar to `STATIC_REQUIRE` (#2318)
* When deferred tu runtime, it behaves like `CHECK`, and not like `REQUIRE`.
* You can have multiple tests with the same name, as long as other parts of the test identity differ (#1915, #1999, #2175)
* Test identity includes test's name, test's tags and and test's class name if applicable.
* Added new warning, `UnmatchedTestSpec`, to error on test specs with no matching tests
* The `-w`, `--warn` warning flags can now be provided multiple times to enable multiple warnings
* The case-insensitive handling of tags is now more reliable and takes up less memory
* Test case and assertion counting can no longer reasonably overflow on 32 bit systems
* The count is now kept in `uint64_t` on all platforms, instead of using `size_t` type.
* The `-o`, `--out` output destination specifiers recognize `-` as stdout
* You have to provide it as `--out=-` to avoid CLI error about missing option
* The new reporter specification also recognizes `-` as stdout
* Multiple reporters can now run at the same time and write to different files (#1712, #2183)
* To support this, the `-r`, `--reporter` flag now also accepts optional output destination
* For full overview of the semantics of using multiple reporters, look into the reporter documentation
* To enable the new syntax, reporter names can no longer contain `::`.
* Console colour support has been rewritten and significantly improved
* The colour implementation based on ANSI colour codes is always available
* Colour implementations respect their associated stream
* previously e.g. Win32 impl would change console colour even if Catch2 was writing to a file
* The colour API is resilient against changing evaluation order of expressions
* The associated CLI flag and compile-time configuration options have changed
* For details see the docs for command-line and compile-time Catch2 configuration
* Added a support for Bazel integration with `XML_OUTPUT_FILE` env var (#2399)
* This has to be enabled during compilation.
* Added `--skip-benchmarks` flag to run tests without any `BENCHMARK`s (#2392, #2408)
* Added option to list all listeners in the binary via `--list-listeners`
### Fixes
* The `INFO` macro no longer contains superfluous semicolon (#1456)
* The `--list*` family of command line flags now return 0 on success (#1410, #1146)
* Various ways of failing a benchmark are now counted and reporter properly
* The ULP matcher now handles comparing numbers with different signs properly (#2152)
* Universal ADL-found operators should no longer break decomposition (#2121)
* Reporter selection is properly case-insensitive
* Previously it forced lower cased name, which would fail for reporters with upper case characters in name
* The cumulative reporter base stores benchmark results alongside assertion results
* Catch2's SE handling should no longer interferes with ASan on Windows (#2334)
* Fixed Windows console colour handling for tests that redirect stdout (#2345)
* Fixed issue with the `random` generators returning the same value over and over again
### Other changes
* `CATCH_CONFIG_DISABLE_MATCHERS` no longer exists.
* If you do not want to use Matchers in a TU, do not include their header.
* `CATCH_CONFIG_ENABLE_CHRONO_STRINGMAKER` no longer exists.
* `StringMaker` specializations for `<chrono>` are always provided
* Catch2's CMake now provides 2 targets, `Catch2` and `Catch2WithMain`.
* `Catch2` is the statically compiled implementation by itself
* `Catch2WithMain` also links in the default main
* Catch2's pkg-config integration also provides 2 packages
* `catch2` is the statically compiled implementation by itself
* `catch2-with-main` also links in the default main
* Passing invalid test specifications passed to Catch2 are now reported before tests are run, and are a hard error.
* Running 0 tests (e.g. due to empty binary, or test spec not matching anything) returns non-0 exit code
* Flag `--allow-running-no-tests` overrides this behaviour.
* `NoTests` warning has been removed because it is fully subsumed by this change.
* Catch2's compile-time configuration options (`CATCH_CONFIG_FOO`) can be set through CMake options of the same name
* They use the same semantics as C++ defines, including the `CATCH_CONFIG_NO_FOO` overrides,
* `-DCATCH_CONFIG_DEFAULT_REPORTER=compact` changes default reporter to "compact"
* `-DCATCH_CONFIG_NO_ANDROID_LOGWRITE=ON` forces android logwrite to off
* `-DCATCH_CONFIG_ANDROID_LOGWRITE=OFF` does nothing (the define will not exist)
## 2.13.7
### Fixes
* Added missing `<iterator>` include in benchmarking. (#2231)
* Fixed noexcept build with benchmarking enabled (#2235)
* Fixed build for compilers with C++17 support but without C++17 library support (#2195)
* JUnit only uses 3 decimal places when reporting durations (#2221)
* `!mayfail` tagged tests are now marked as `skipped` in JUnit reporter output (#2116)
## 2.13.6
### Fixes
* Disabling all signal handlers no longer breaks compilation (#2212, #2213)
### Miscellaneous
* `catch_discover_tests` should handle escaped semicolon (`;`) better (#2214, #2215)
## 2.13.5
### Improvements
* Detection of MAC and IPHONE platforms has been improved (#2140, #2157)
* Added workaround for bug in XLC 16.1.0.1 (#2155)
* Add detection for LCC when it is masquerading as GCC (#2199)
* Modified posix signal handling so it supports newer libcs (#2178)
* `MINSIGSTKSZ` was no longer usable in constexpr context.
### Fixes
* Fixed compilation of benchmarking when `min` and `max` macros are defined (#2159)
* Including `windows.h` without `NOMINMAX` remains a really bad idea, don't do it
### Miscellaneous
* The check whether Catch2 is being built as a subproject is now more reliable (#2202, #2204)
* The problem was that if the variable name used internally was defined the project including Catch2 as subproject, it would not be properly overwritten for Catch2's CMake.
## 2.13.4
### Improvements
* Improved the hashing algorithm used for shuffling test cases (#2070)
* `TEST_CASE`s that differ only in the last character should be properly shuffled
* Note that this means that v2.13.4 gives you a different order of test cases than 2.13.3, even given the same seed.
### Miscellaneous
* Deprecated `ParseAndAddCatchTests` CMake integration (#2092)
* It is impossible to implement it properly for all the different test case variants Catch2 provides, and there are better options provided.
* Use `catch_discover_tests` instead, which uses runtime information about available tests.
* Fixed bug in `catch_discover_tests` that would cause it to fail when used in specific project structures (#2119)
* Added Bazel build file
* Added an experimental static library target to CMake
## 2.13.3
### Fixes
* Fixed possible infinite loop when combining generators with section filter (`-c` option) (#2025)
### Miscellaneous
* Fixed `ParseAndAddCatchTests` not finding `TEST_CASE`s without tags (#2055, #2056)
* `ParseAndAddCatchTests` supports `CMP0110` policy for changing behaviour of `add_test` (#2057)
* This was the shortlived change in CMake 3.18.0 that temporarily broke `ParseAndAddCatchTests`
## 2.13.2
### Improvements
* Implemented workaround for AppleClang shadowing bug (#2030)
* Implemented workaround for NVCC ICE (#2005, #2027)
### Fixes
* Fixed detection of `std::uncaught_exceptions` support under non-msvc platforms (#2021)
* Fixed the experimental stdout/stderr capture under Windows (#2013)
### Miscellaneous
* `catch_discover_tests` has been improved significantly (#2023, #2039)
* You can now specify which reporter should be used
* You can now modify where the output will be written
* `WORKING_DIRECTORY` setting is respected
* `ParseAndAddCatchTests` now supports `TEMPLATE_TEST_CASE` macros (#2031)
* Various documentation fixes and improvements (#2022, #2028, #2034)
## 2.13.1
### Improvements
* `ParseAndAddCatchTests` handles CMake v3.18.0 correctly (#1984)
* Improved autodetection of `std::byte` (#1992)
* Simplified implementation of templated test cases (#2007)
* This should have a tiny positive effect on its compilation throughput
### Fixes
* Automatic stringification of ranges handles sentinel ranges properly (#2004)
## 2.13.0
### Improvements
* `GENERATE` can now follow a `SECTION` at the same level of nesting (#1938)
* The `SECTION`(s) before the `GENERATE` will not be run multiple times, the following ones will.
* Added `-D`/`--min-duration` command line flag (#1910)
* If a test takes longer to finish than the provided value, its name and duration will be printed.
* This flag is overriden by setting `-d`/`--duration`.
### Fixes
* `TAPReporter` no longer skips successful assertions (#1983)
## 2.12.4
### Improvements
* Added support for MacOS on ARM (#1971)
## 2.12.3
### Fixes
* `GENERATE` nested in a for loop no longer creates multiple generators (#1913)
* Fixed copy paste error breaking `TEMPLATE_TEST_CASE_SIG` for 6 or more arguments (#1954)
* Fixed potential UB when handling non-ASCII characters in CLI args (#1943)
### Improvements
* There can be multiple calls to `GENERATE` on a single line
* Improved `fno-except` support for platforms that do not provide shims for exception-related std functions (#1950)
* E.g. the Green Hills C++ compiler
* XmlReporter now also reports test-case-level statistics (#1958)
* This is done via a new element, `OverallResultsCases`
### Miscellaneous
* Added `.clang-format` file to the repo (#1182, #1920)
* Rewrote contributing docs
* They should explain the different levels of testing and so on much better
## 2.12.2
### Fixes
* Fixed compilation failure if `is_range` ADL found deleted function (#1929)
* Fixed potential UB in `CAPTURE` if the expression contained non-ASCII characters (#1925)
### Improvements
* `std::result_of` is not used if `std::invoke_result` is available (#1934)
* JUnit reporter writes out `status` attribute for tests (#1899)
* Suppresed clang-tidy's `hicpp-vararg` warning (#1921)
* Catch2 was already suppressing the `cppcoreguidelines-pro-type-vararg` alias of the warning
## 2.12.1
### Fixes
* Vector matchers now support initializer list literals better
### Improvements
* Added support for `^` (bitwise xor) to `CHECK` and `REQUIRE`
## 2.12.0
### Improvements
* Running tests in random order (`--order rand`) has been reworked significantly (#1908)
* Given same seed, all platforms now produce the same order
* Given same seed, the relative order of tests does not change if you select only a subset of them
* Vector matchers support custom allocators (#1909)
* `|` and `&` (bitwise or and bitwise and) are now supported in `CHECK` and `REQUIRE`
* The resulting type must be convertible to `bool`
### Fixes
* Fixed computation of benchmarking column widths in ConsoleReporter (#1885, #1886)
* Suppressed clang-tidy's `cppcoreguidelines-pro-type-vararg` in assertions (#1901)
* It was a false positive trigered by the new warning support workaround
* Fixed bug in test specification parser handling of OR'd patterns using escaping (#1905)
### Miscellaneous
* Worked around IBM XL's codegen bug (#1907)
* It would emit code for _destructors_ of temporaries in an unevaluated context
* Improved detection of stdlib's support for `std::uncaught_exceptions` (#1911)
## 2.11.3
### Fixes
* Fixed compilation error caused by lambdas in assertions under MSVC
## 2.11.2
### Improvements
* GCC and Clang now issue warnings for suspicious code in assertions (#1880)
* E.g. `REQUIRE( int != unsigned int )` will now issue mixed signedness comparison warning
* This has always worked on MSVC, but it now also works for GCC and current Clang versions
* Colorization of "Test filters" output should be more robust now
* `--wait-for-keypress` now also accepts `never` as an option (#1866)
* Reporters no longer round-off nanoseconds when reporting benchmarking results (#1876)
* Catch2's debug break now supports iOS while using Thumb instruction set (#1862)
* It is now possible to customize benchmark's warm-up time when running the test binary (#1844)
* `--benchmark-warmup-time {ms}`
* User can now specify how Catch2 should break into debugger (#1846)
### Fixes
* Fixes missing `<random>` include in benchmarking (#1831)
* Fixed missing `<iterator>` include in benchmarking (#1874)
* Hidden test cases are now also tagged with `[!hide]` as per documentation (#1847)
* Detection of whether libc provides `std::nextafter` has been improved (#1854)
* Detection of `wmain` no longer incorrectly looks for `WIN32` macro (#1849)
* Now it just detects Windows platform
* Composing already-composed matchers no longer modifies the partially-composed matcher expression
* This bug has been present for the last ~2 years and nobody reported it
## 2.11.1
### Improvements
* Breaking into debugger is supported on iOS (#1817)
* `google-build-using-namespace` clang-tidy warning is suppressed (#1799)
### Fixes
* Clang on Windows is no longer assumed to implement MSVC's traditional preprocessor (#1806)
* `ObjectStorage` now behaves properly in `const` contexts (#1820)
* `GENERATE_COPY(a, b)` now compiles properly (#1809, #1815)
* Some more cleanups in the benchmarking support
## 2.11.0
### Improvements
* JUnit reporter output now contains more details in case of failure (#1347, #1719)
* Added SonarQube Test Data reporter (#1738)
* It is in a separate header, just like the TAP, Automake, and TeamCity reporters
* `range` generator now allows floating point numbers (#1776)
* Reworked part of internals to increase throughput
### Fixes
* The single header version should contain full benchmarking support (#1800)
* `[.foo]` is now properly parsed as `[.][foo]` when used on the command line (#1798)
* Fixed compilation of benchmarking on platforms where `steady_clock::period` is not `std::nano` (#1794)
## 2.10.2
### Improvements
* Catch2 will now compile on platform where `INFINITY` is double (#1782)
### Fixes
* Warning suppressed during listener registration will no longer leak
## 2.10.1
### Improvements
* Catch2 now guards itself against `min` and `max` macros from `windows.h` (#1772)
* Templated tests will now compile with ICC (#1748)
* `WithinULP` matcher now uses scientific notation for stringification (#1760)
### Fixes
* Templated tests no longer trigger `-Wunused-templates` (#1762)
* Suppressed clang-analyzer false positive in context getter (#1230, #1735)
### Miscellaneous
* CMake no longer prohibits in-tree build when Catch2 is used as a subproject (#1773, #1774)
## 2.10.0
### Fixes
* `TEMPLATE_LIST_TEST_CASE` now properly handles non-copyable and non-movable types (#1729)
* Fixed compilation error on Solaris caused by a system header defining macro `TT` (#1722, #1723)
* `REGISTER_ENUM` will now fail at compilation time if the registered enum is too large
* Removed use of `std::is_same_v` in C++17 mode (#1757)
* Fixed parsing of escaped special characters when reading test specs from a file (#1767, #1769)
### Improvements
* Trailing and leading whitespace in test/section specs are now ignored.
* Writing to Android debug log now uses `__android_log_write` instead of `__android_log_print`
* Android logging support can now be turned on/off at compile time (#1743)
* The toggle is `CATCH_CONFIG_ANDROID_LOGWRITE`
* Added a generator that returns elements of a range
* Use via `from_range(from, to)` or `from_range(container)`
* Added support for CRTs that do not provide `std::nextafter` (#1739)
* They must still provide global `nextafter{f,l,}`
* Enabled via `CATCH_CONFIG_GLOBAL_NEXTAFTER`
* Special cased `Approx(inf)` not to match non-infinite values
* Very strictly speaking this might be a breaking change, but it should match user expectations better
* The output of benchmarking through the Console reporter when `--benchmark-no-analysis` is set is now much simpler (#1768)
* Added a matcher that can be used for checking an exceptions message (#1649, #1728)
* The matcher helper function is called `Message`
* The exception must publicly derive from `std::exception`
* The matching is done exactly, including case and whitespace
* Added a matcher that can be used for checking relative equality of floating point numbers (#1746)
* Unlike `Approx`, it considers both sides when determining the allowed margin
* Special cases `NaN` and `INFINITY` to match user expectations
* The matcher helper function is called `WithinRel`
* The ULP matcher now allows for any possible distance between the two numbers
* The random number generators now use Catch-global instance of RNG (#1734, #1736)
* This means that nested random number generators actually generate different numbers
### Miscellaneous
* In-repo PNGs have been optimized to lower overhead of using Catch2 via git clone
* Catch2 now uses its own implementation of the URBG concept
* In the future we also plan to use our own implementation of the distributions from `<random>` to provide cross-platform repeatability of random results
## 2.9.2
### Fixes
* `ChunkGenerator` can now be used with chunks of size 0 (#1671)
* Nested subsections are now run properly when specific section is run via the `-c` argument (#1670, #1673)
* Catch2 now consistently uses `_WIN32` to detect Windows platform (#1676)
* `TEMPLATE_LIST_TEST_CASE` now support non-default constructible type lists (#1697)
* Fixed a crash in the XMLReporter when a benchmark throws exception during warmup (#1706)
* Fixed a possible infinite loop in CompactReporter (#1715)
* Fixed `-w NoTests` returning 0 even when no tests were matched (#1449, #1683, #1684)
* Fixed matcher compilation under Obj-C++ (#1661)
### Improvements
* `RepeatGenerator` and `FixedValuesGenerator` now fail to compile when used with `bool` (#1692)
* Previously they would fail at runtime.
* Catch2 now supports Android's debug logging for its debug output (#1710)
* Catch2 now detects and configures itself for the RTX platform (#1693)
* You still need to pass `--benchmark-no-analysis` if you are using benchmarking under RTX
* Removed a "storage class is not first" warning when compiling Catch2 with PGI compiler (#1717)
### Miscellaneous
* Documentation now contains indication when a specific feature was introduced (#1695)
* These start with Catch2 v2.3.0, (a bit over a year ago).
* `docs/contributing.md` has been updated to provide contributors guidance on how to add these to newly written documentation
* Various other documentation improvements
* ToC fixes
* Documented `--order` and `--rng-seed` command line options
* Benchmarking documentation now clearly states that it requires opt-in
* Documented `CATCH_CONFIG_CPP17_OPTIONAL` and `CATCH_CONFIG_CPP17_BYTE` macros
* Properly documented built-in vector matchers
* Improved `*_THROWS_MATCHES` documentation a bit
* CMake config file is now arch-independent even if `CMAKE_SIZEOF_VOID_P` is in CMake cache (#1660)
* `CatchAddTests` now properly escapes `[` and `]` in test names (#1634, #1698)
* Reverted `CatchAddTests` adding tags as CTest labels (#1658)
* The script broke when test names were too long
* Overwriting `LABELS` caused trouble for users who set them manually
* CMake does not let users append to `LABELS` if the test name has spaces
## 2.9.1
### Fixes
* Fix benchmarking compilation failure in files without `CATCH_CONFIG_EXTERNAL_INTERFACES` (or implementation)
## 2.9.0
### Improvements
* The experimental benchmarking support has been replaced by integrating Nonius code (#1616)
* This provides a much more featurefull micro-benchmarking support.
* Due to the compilation cost, it is disabled by default. See the documentation for details.
* As far as backwards compatibility is concerned, this feature is still considered experimental in that we might change the interface based on user feedback.
* `WithinULP` matcher now shows the acceptable range (#1581)
* Template test cases now support type lists (#1627)
## 2.8.0
### Improvements
* Templated test cases no longer check whether the provided types are unique (#1628)
* This allows you to e.g. test over `uint32_t`, `uint64_t`, and `size_t` without compilation failing
* The precision of floating point stringification can be modified by user (#1612, #1614)
* We now provide `REGISTER_ENUM` convenience macro for generating `StringMaker` specializations for enums
* See the "String conversion" documentation for details
* Added new set of macros for template test cases that enables the use of NTTPs (#1531, #1609)
* See "Test cases and sections" documentation for details
### Fixes
* `UNSCOPED_INFO` macro now has a prefixed/disabled/prefixed+disabled versions (#1611)
* Reporting errors at startup should no longer cause a segfault under certain circumstances (#1626)
### Miscellaneous
* CMake will now prevent you from attempting in-tree build (#1636, #1638)
* Previously it would break with an obscure error message during the build step
## 2.7.2
### Improvements
* Added an approximate vector matcher (#1499)
### Fixes
* Filters will no longer be shown if there were none
* Fixed compilation error when using Homebrew GCC on OS X (#1588, #1589)
* Fixed the console reporter not showing messages that start with a newline (#1455, #1470)
* Modified JUnit reporter's output so that rng seed and filters are reported according to the JUnit schema (#1598)
* Fixed some obscure warnings and static analysis passes
### Miscellaneous
* Various improvements to `ParseAndAddCatchTests` (#1559, #1601)
* When a target is parsed, it receives `ParseAndAddCatchTests_TESTS` property which summarizes found tests
* Fixed problem with tests not being found if the `OptionalCatchTestLauncher` variables is used
* Including the script will no longer forcefully modify `CMAKE_MINIMUM_REQUIRED_VERSION`
* CMake object libraries are ignored when parsing to avoid needless warnings
* `CatchAddTests` now adds test's tags to their CTest labels (#1600)
* Added basic CPack support to our build
## 2.7.1
### Improvements
* Reporters now print out the filters applied to test cases (#1550, #1585)
* Added `GENERATE_COPY` and `GENERATE_REF` macros that can use variables inside the generator expression
* Because of the significant danger of lifetime issues, the default `GENERATE` macro still does not allow variables
* The `map` generator helper now deduces the mapped return type (#1576)
### Fixes
* Fixed ObjC++ compilation (#1571)
* Fixed test tag parsing so that `[.foo]` is now parsed as `[.][foo]`.
* Suppressed warning caused by the Windows headers defining SE codes in different manners (#1575)
## 2.7.0
### Improvements
* `TEMPLATE_PRODUCT_TEST_CASE` now uses the resulting type in the name, instead of the serial number (#1544)
* Catch2's single header is now strictly ASCII (#1542)
* Added generator for random integral/floating point types
* The types are inferred within the `random` helper
* Added back RangeGenerator (#1526)
* RangeGenerator returns elements within a certain range
* Added ChunkGenerator generic transform (#1538)
* A ChunkGenerator returns the elements from different generator in chunks of n elements
* Added `UNSCOPED_INFO` (#415, #983, #1522)
* This is a variant of `INFO` that lives until next assertion/end of the test case.
### Fixes
* All calls to C stdlib functions are now `std::` qualified (#1541)
* Code brought in from Clara was also updated.
* Running tests will no longer open the specified output file twice (#1545)
* This would cause trouble when the file was not a file, but rather a named pipe
* Fixes the CLion/Resharper integration with Catch
* Fixed `-Wunreachable-code` occurring with (old) ccache+cmake+clang combination (#1540)
* Fixed `-Wdefaulted-function-deleted` warning with Clang 8 (#1537)
* Catch2's type traits and helpers are now properly namespaced inside `Catch::` (#1548)
* Fixed std{out,err} redirection for failing test (#1514, #1525)
* Somehow, this bug has been present for well over a year before it was reported
### Contrib
* `ParseAndAddCatchTests` now properly escapes commas in the test name
## 2.6.1
### Improvements
* The JUnit reporter now also reports random seed (#1520, #1521)
### Fixes
* The TAP reporter now formats comments with test name properly (#1529)
* `CATCH_REQUIRE_THROWS`'s internals were unified with `REQUIRE_THROWS` (#1536)
* This fixes a potential `-Wunused-value` warning when used
* Fixed a potential segfault when using any of the `--list-*` options (#1533, #1534)
## 2.6.0
**With this release the data generator feature is now fully supported.**
@ -240,7 +964,7 @@ than `single_include/catch.hpp`.**
* CLR objects (`T^`) can now be stringified (#1216)
* This affects code compiled as C++/CLI
* Added `PredicateMatcher`, a matcher that takes an arbitrary predicate function (#1236)
* See [documentation for details](https://github.com/catchorg/Catch2/blob/master/docs/matchers.md)
* See [documentation for details](https://github.com/catchorg/Catch2/blob/devel/docs/matchers.md)
### Others
* Modified CMake-installed pkg-config to allow `#include <catch.hpp>`(#1239)
@ -268,7 +992,7 @@ than `single_include/catch.hpp`.**
* Added an option to warn (+ exit with error) when no tests were ran (#1158)
* Use as `-w NoTests`
* Added provisional support for Emscripten (#1114)
* [Added a way to override the fallback stringifier](https://github.com/catchorg/Catch2/blob/master/docs/configuration.md#fallback-stringifier) (#1024)
* [Added a way to override the fallback stringifier](https://github.com/catchorg/Catch2/blob/devel/docs/configuration.md#fallback-stringifier) (#1024)
* This allows project's own stringification machinery to be easily reused for Catch
* `Catch::Session::run()` now accepts `char const * const *`, allowing it to accept array of string literals (#1031, #1178)
* The embedded version of Clara was bumped to v1.1.3
@ -677,7 +1401,7 @@ Cygwin issue with `gettimeofday` - `#define` was not early enough
* Usage of `gettimeofday` inside Catch should no longer cause compilation errors.
* Improved `-Wparentheses` suppression for gcc (#674)
* When compiled with gcc 4.8 or newer, the suppression is localized to assertions only
* Otherwise it is supressed for the whole TU
* Otherwise it is suppressed for the whole TU
* Fixed test spec parser issue (with escapes in multiple names)
##### Other
@ -760,7 +1484,7 @@ Other:
##### Other:
* Types with overloaded `&&` operator are no longer evaluated twice when used in an assertion macro.
* The use of `__COUNTER__` is supressed when Catch is parsed by CLion
* The use of `__COUNTER__` is suppressed when Catch is parsed by CLion
* This change is not active when compiling a binary
* Approval tests can now be run on Windows
* CMake will now warn if a file is present in the `include` folder but not is not enumerated as part of the project

View File

@ -1,7 +1,7 @@
<a id="top"></a>
# How to release
When enough changes have accumulated, it is time to release new version of Catch. This document describes the process in doing so, that no steps are forgotten. Note that all referenced scripts can be found in the `scripts/` directory.
When enough changes have accumulated, it is time to release new version of Catch. This document describes the process in doing so, that no steps are forgotten. Note that all referenced scripts can be found in the `tools/scripts/` directory.
## Necessary steps
@ -32,7 +32,7 @@ Once a release is ready, release notes need to be written. They should summarize
### Commit and push update to GitHub
After version number is incremented, single-include header is regenerated and release notes are updated, changes should be commited and pushed to GitHub.
After version number is incremented, single-include header is regenerated and release notes are updated, changes should be committed and pushed to GitHub.
### Release on GitHub
@ -40,11 +40,27 @@ After version number is incremented, single-include header is regenerated and re
After pushing changes to GitHub, GitHub release *needs* to be created.
Tag version and release title should be same as the new version,
description should contain the release notes for the current release.
Single header version of `catch.hpp` *needs* to be attached as a binary,
as that is where the official download link links to. Preferably
it should use linux line endings. All non-bundled reporters (Automake,
TAP, TeamCity) should also be attached as binaries, as they might be
dependent on a specific version of the single-include header.
We also attach the two amalgamated files as "binaries".
Since 2.5.0, the release tag and the "binaries" (headers) should be PGP
signed.
Since 2.5.0, the release tag and the "binaries" (amalgamated files) should
be PGP signed.
#### Signing a tag
To create a signed tag, use `git tag -s <VERSION>`, where `<VERSION>`
is the version being released, e.g. `git tag -s v2.6.0`.
Use the version name as the short message and the release notes as
the body (long) message.
#### Signing the amalgamated files
This will create ASCII-armored signatures for the two amalgamated files
that are uploaded to the GitHub release:
```
gpg --armor --output extras/catch_amalgamated.hpp.asc --detach-sig extras/catch_amalgamated.hpp
gpg --armor --output extras/catch_amalgamated.cpp.asc --detach-sig extras/catch_amalgamated.cpp
```
_GPG does not support signing multiple files in single invocation._

175
docs/reporter-events.md Normal file
View File

@ -0,0 +1,175 @@
<a id="top"></a>
# Reporter events
**Contents**<br>
[Test running events](#test-running-events)<br>
[Benchmarking events](#benchmarking-events)<br>
[Listings events](#listings-events)<br>
[Miscellaneous events](#miscellaneous-events)<br>
Reporter events are one of the customization points for user code. They
are used by [reporters](reporters.md#top) to customize Catch2's output,
and by [event listeners](event-listeners.md#top) to perform in-process
actions under some conditions.
There are currently 21 reporter events in Catch2, split between 4 distinct
event groups:
* test running events (10 events)
* benchmarking (4 events)
* listings (3 events)
* miscellaneous (4 events)
## Test running events
Test running events are always paired so that for each `fooStarting` event,
there is a `fooEnded` event. This means that the 10 test running events
consist of 5 pairs of events:
* `testRunStarting` and `testRunEnded`,
* `testCaseStarting` and `testCaseEnded`,
* `testCasePartialStarting` and `testCasePartialEnded`,
* `sectionStarting` and `sectionEnded`,
* `assertionStarting` and `assertionEnded`
### `testRun` events
```cpp
void testRunStarting( TestRunInfo const& testRunInfo );
void testRunEnded( TestRunStats const& testRunStats );
```
The `testRun` events bookend the entire test run. `testRunStarting` is
emitted before the first test case is executed, and `testRunEnded` is
emitted after all the test cases have been executed.
### `testCase` events
```cpp
void testCaseStarting( TestCaseInfo const& testInfo );
void testCaseEnded( TestCaseStats const& testCaseStats );
```
The `testCase` events bookend one _full_ run of a specific test case.
Individual runs through a test case, e.g. due to `SECTION`s or `GENERATE`s,
are handled by a different event.
### `testCasePartial` events
> Introduced in Catch2 3.0.1
```cpp
void testCasePartialStarting( TestCaseInfo const& testInfo, uint64_t partNumber );
void testCasePartialEnded(TestCaseStats const& testCaseStats, uint64_t partNumber );
```
`testCasePartial` events bookend one _partial_ run of a specific test case.
This means that for any given test case, these events can be emitted
multiple times, e.g. due to multiple leaf sections.
In regards to nesting with `testCase` events, `testCasePartialStarting`
will never be emitted before the corresponding `testCaseStarting`, and
`testCasePartialEnded` will always be emitted before the corresponding
`testCaseEnded`.
### `section` events
```cpp
void sectionStarting( SectionInfo const& sectionInfo );
void sectionEnded( SectionStats const& sectionStats );
```
`section` events are emitted only for active `SECTION`s, that is, sections
that are entered. Sections that are skipped in this test case run-through
do not cause events to be emitted.
_Note that test cases always contain one implicit section. The event for
this section is emitted after the corresponding `testCasePartialStarting`
event._
### `assertion` events
```cpp
void assertionStarting( AssertionInfo const& assertionInfo );
void assertionEnded( AssertionStats const& assertionStats );
```
`assertionStarting` is called after the expression is captured, but before
the assertion expression is evaluated. This might seem like a minor
distinction, but what it means is that if you have assertion like
`REQUIRE( a + b == c + d )`, then what happens is that `a + b` and `c + d`
are evaluated before `assertionStarting` is emitted, while the `==` is
evaluated after the event.
## Benchmarking events
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
```cpp
void benchmarkPreparing( StringRef name ) override;
void benchmarkStarting( BenchmarkInfo const& benchmarkInfo ) override;
void benchmarkEnded( BenchmarkStats<> const& benchmarkStats ) override;
void benchmarkFailed( StringRef error ) override;
```
Due to the benchmark lifecycle being bit more complicated, the benchmarking
events have their own category, even though they could be seen as parallel
to the `assertion*` events. You should expect running a benchmark to
generate at least 2 of the events above.
To understand the explanation below, you should read the [benchmarking
documentation](benchmarks.md#top) first.
* `benchmarkPreparing` event is sent after the environmental probe
finishes, but before the user code is first estimated.
* `benchmarkStarting` event is sent after the user code is estimated,
but has not been benchmarked yet.
* `benchmarkEnded` event is sent after the user code has been benchmarked,
and contains the benchmarking results.
* `benchmarkFailed` event is sent if either the estimation or the
benchmarking itself fails.
## Listings events
> Introduced in Catch2 3.0.1.
Listings events are events that correspond to the test binary being
invoked with `--list-foo` flag.
There are currently 3 listing events, one for reporters, one for tests,
and one for tags. Note that they are not exclusive to each other.
```cpp
void listReporters( std::vector<ReporterDescription> const& descriptions );
void listTests( std::vector<TestCaseHandle> const& tests );
void listTags( std::vector<TagInfo> const& tagInfos );
```
## Miscellaneous events
```cpp
void reportInvalidTestSpec( StringRef unmatchedSpec );
void fatalErrorEncountered( StringRef error );
void noMatchingTestCases( StringRef unmatchedSpec );
```
These are one-off events that do not neatly fit into other categories.
`reportInvalidTestSpec` is sent for each [test specification command line
argument](command-line.md#specifying-which-tests-to-run) that wasn't
parsed into a valid spec.
`fatalErrorEncountered` is sent when Catch2's POSIX signal handling
or Windows SE handler is called into with a fatal signal/exception.
`noMatchingTestCases` is sent for each user provided test specification
that did not match any registered tests.
---
[Home](Readme.md#top)

View File

@ -1,45 +1,212 @@
<a id="top"></a>
# Reporters
Catch has a modular reporting system and comes bundled with a handful of useful reporters built in.
You can also write your own reporters.
Reporters are a customization point for most of Catch2's output, e.g.
formatting and writing out [assertions (whether passing or failing),
sections, test cases, benchmarks, and so on](reporter-events.md#top).
Catch2 comes with a bunch of reporters by default (currently 8), and
you can also write your own reporter. Because multiple reporters can
be active at the same time, your own reporters do not even have to handle
all reporter event, just the ones you are interested in, e.g. benchmarks.
## Using different reporters
The reporter to use can easily be controlled from the command line.
To specify a reporter use [`-r` or `--reporter`](command-line.md#choosing-a-reporter-to-use), followed by the name of the reporter, e.g.:
You can see which reporters are available by running the test binary
with `--list-reporters`. You can then pick one of them with the [`-r`,
`--reporter` option](command-line.md#choosing-a-reporter-to-use), followed
by the name of the desired reporter, like so:
```
-r xml
--reporter xml
```
If you don't specify a reporter then the console reporter is used by default.
There are four reporters built in to the single include:
You can also select multiple reporters to be used at the same time.
In that case you should read the [section on using multiple
reporters](#multiple-reporters) to avoid any surprises from doing so.
* `console` writes as lines of text, formatted to a typical terminal width, with colours if a capable terminal is detected.
* `compact` similar to `console` but optimised for minimal output - each entry on one line
* `junit` writes xml that corresponds to Ant's [junitreport](http://help.catchsoftware.com/display/ET/JUnit+Format) target. Useful for build systems that understand Junit.
Because of the way the junit format is structured the run must complete before anything is written.
* `xml` writes an xml format tailored to Catch. Unlike `junit` this is a streaming format so results are delivered progressively.
There are a few additional reporters, for specific build systems, in the Catch repository (in `include\reporters`) which you can `#include` in your project if you would like to make use of them.
Do this in one source file - the same one you have `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`.
<a id="multiple-reporters"></a>
## Using multiple reporters
* `teamcity` writes the native, streaming, format that [TeamCity](https://www.jetbrains.com/teamcity/) understands.
Use this when building as part of a TeamCity build to see results as they happen ([code example](../examples/207-Rpt-TeamCityReporter.cpp)).
* `tap` writes in the TAP ([Test Anything Protocol](https://en.wikipedia.org/wiki/Test_Anything_Protocol)) format.
* `automake` writes in a format that correspond to [automake .trs](https://www.gnu.org/software/automake/manual/html_node/Log-files-generation-and-test-results-recording.html) files
> Support for having multiple parallel reporters was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 3.0.1
Catch2 supports using multiple reporters at the same time while having
them write into different destinations. The two main uses of this are
* having both human-friendly and machine-parseable (e.g. in JUnit format)
output from one run of binary
* having "partial" reporters that are highly specialized, e.g. having one
reporter that writes out benchmark results as markdown tables and does
nothing else, while also having standard testing output separately
Specifying multiple reporter looks like this:
```
--reporter JUnit::out=result-junit.xml --reporter console::out=-::colour-mode=ansi
```
This tells Catch2 to use two reporters, `JUnit` reporter that writes
its machine-readable XML output to file `result-junit.xml`, and the
`console` reporter that writes its user-friendly output to stdout and
uses ANSI colour codes for colouring the output.
Using multiple reporters (or one reporter and one-or-more [event
listeners](event-listener.md#top)) can have surprisingly complex semantics
when using customization points provided to reporters by Catch2, namely
capturing stdout/stderr from test cases.
As long as at least one reporter (or listener) asks Catch2 to capture
stdout/stderr, captured stdout and stderr will be available to all
reporters and listeners.
Because this might be surprising to the users, if at least one active
_reporter_ is non-capturing, then Catch2 tries to roughly emulate
non-capturing behaviour by printing out the captured stdout/stderr
just before `testCasePartialEnded` event is sent out to the active
reporters and listeners. This means that stdout/stderr is no longer
printed out from tests as it is being written, but instead it is written
out in batch after each runthrough of a test case is finished.
You see what reporters are available from the command line by running with `--list-reporters`.
By default all these reports are written to stdout, but can be redirected to a file with [`-o` or `--out`](command-line.md#sending-output-to-a-file)
## Writing your own reporter
You can write your own custom reporter and register it with Catch.
At time of writing the interface is subject to some changes so is not, yet, documented here.
If you are determined you shouldn't have too much trouble working it out from the existing implementations -
but do keep in mind upcoming changes (these will be minor, simplifying, changes such as not needing to forward calls to the base class).
You can also write your own custom reporter and tell Catch2 to use it.
When writing your reporter, you have two options:
* Derive from `Catch::ReporterBase`. When doing this, you will have
to provide handling for all [reporter events](reporter-events.md#top).
* Derive from one of the provided [utility reporter bases in
Catch2](#utility-reporter-bases).
Generally we recommend doing the latter, as it is less work.
Apart from overriding handling of the individual reporter events, reporters
have access to some extra customization points, described below.
### Utility reporter bases
Catch2 currently provides two utility reporter bases:
* `Catch::StreamingReporterBase`
* `Catch::CumulativeReporterBase`
`StreamingReporterBase` is useful for reporters that can format and write
out the events as they come in. It provides (usually empty) implementation
for all reporter events, and if you let it handle the relevant events,
it also handles storing information about active test run and test case.
`CumulativeReporterBase` is a base for reporters that need to see the whole
test run, before they can start writing the output, such as the JUnit
and SonarQube reporters. This post-facto approach requires the assertions
to be stringified when it is finished, so that the assertion can be written
out later. Because the stringification can be expensive, and not all
cumulative reporters need the assertions, this base provides customization
point to change whether the assertions are saved or not, separate for
passing and failing assertions.
_Generally we recommend that if you override a member function from either
of the bases, you call into the base's implementation first. This is not
necessarily in all cases, but it is safer and easier._
Writing your own reporter then looks like this:
```cpp
#include <catch2/reporters/catch_reporter_streaming_base.hpp>
#include <catch2/catch_test_case_info.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
#include <iostream>
class PartialReporter : public Catch::StreamingReporterBase {
public:
using StreamingReporterBase::StreamingReporterBase;
static std::string getDescription() {
return "Reporter for testing TestCasePartialStarting/Ended events";
}
void testCasePartialStarting(Catch::TestCaseInfo const& testInfo,
uint64_t partNumber) override {
std::cout << "TestCaseStartingPartial: " << testInfo.name << '#' << partNumber << '\n';
}
void testCasePartialEnded(Catch::TestCaseStats const& testCaseStats,
uint64_t partNumber) override {
std::cout << "TestCasePartialEnded: " << testCaseStats.testInfo->name << '#' << partNumber << '\n';
}
};
CATCH_REGISTER_REPORTER("partial", PartialReporter)
```
This create a simple reporter that responds to `testCasePartial*` events,
and calls itself "partial" reporter, so it can be invoked with
`--reporter partial` command line flag.
### `ReporterPreferences`
Each reporter instance contains instance of `ReporterPreferences`, a type
that holds flags for the behaviour of Catch2 when this reporter run.
Currently there are two customization options:
* `shouldRedirectStdOut` - whether the reporter wants to handle
writes to stdout/stderr from user code, or not. This is useful for
reporters that output machine-parseable output, e.g. the JUnit
reporter, or the XML reporter.
* `shouldReportAllAssertions` - whether the reporter wants to handle
`assertionEnded` events for passing assertions as well as failing
assertions. Usually reporters do not report successful assertions
and don't need them for their output, but sometimes the desired output
format includes passing assertions even without the `-s` flag.
### Per-reporter configuration
> Per-reporter configuration was introduced in Catch2 3.0.1
Catch2 supports some configuration to happen per reporter. The configuration
options fall into one of two categories:
* Catch2-recognized options
* Reporter-specific options
The former is a small set of universal options that Catch2 handles for
the reporters, e.g. output file or console colour mode. The latter are
options that the reporters have to handle themselves, but the keys and
values can be arbitrary strings, as long as they don't contain `::`. This
allows writing reporters that can be significantly customized at runtime.
Reporter-specific options always have to be prefixed with "X" (large
letter X).
### Other expected functionality of a reporter
When writing a custom reporter, there are few more things that you should
keep in mind. These are not important for correctness, but they are
important for the reporter to work _nicely_.
* Catch2 provides a simple verbosity option for users. There are three
verbosity levels, "quiet", "normal", and "high", and if it makes sense
for reporter's output format, it should respond to these by changing
what, and how much, it writes out.
* Catch2 operates with an rng-seed. Knowing what seed a test run had
is important if you want to replicate it, so your reporter should
report the rng-seed, if at all possible given the target output format.
* Catch2 also operates with test filters, or test specs. If a filter
is present, you should also report the filter, if at all possible given
the target output format.
---

View File

@ -1,71 +0,0 @@
<a id="top"></a>
# Why do my tests take so long to compile?
**Contents**<br>
[Short answer](#short-answer)<br>
[Long answer](#long-answer)<br>
[Practical example](#practical-example)<br>
[Other possible solutions](#other-possible-solutions)<br>
Several people have reported that test code written with Catch takes much longer to compile than they would expect. Why is that?
Catch is implemented entirely in headers. There is a little overhead due to this - but not as much as you might think - and you can minimise it simply by organising your test code as follows:
## Short answer
Exactly one source file must ```#define``` either ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER``` before ```#include```-ing Catch. In this file *do not write any test cases*! In most cases that means this file will just contain two lines (the ```#define``` and the ```#include```).
## Long answer
Usually C++ code is split between a header file, containing declarations and prototypes, and an implementation file (.cpp) containing the definition, or implementation, code. Each implementation file, along with all the headers that it includes (and which those headers include, etc), is expanded into a single entity called a translation unit - which is then passed to the compiler and compiled down to an object file.
But functions and methods can also be written inline in header files. The downside to this is that these definitions will then be compiled in *every* translation unit that includes the header.
Because Catch is implemented *entirely* in headers you might think that the whole of Catch must be compiled into every translation unit that uses it! Actually it's not quite as bad as that. Catch mitigates this situation by effectively maintaining the traditional separation between the implementation code and declarations. Internally the implementation code is protected by ```#ifdef```s and is conditionally compiled into only one translation unit. This translation unit is that one that ```#define```s ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER```. Let's call this the main source file.
As a result the main source file *does* compile the whole of Catch every time! So it makes sense to dedicate this file to *only* ```#define```-ing the identifier and ```#include```-ing Catch (and implementing the runner code, if you're doing that). Keep all your test cases in other files. This way you won't pay the recompilation cost for the whole of Catch
## Practical example
Assume you have the `Factorial` function from the [tutorial](tutorial.md#top) in `factorial.cpp` (with forward declaration in `factorial.h`) and want to test it and keep the compile times down when adding new tests. Then you should have 2 files, `tests-main.cpp` and `tests-factorial.cpp`:
```cpp
// tests-main.cpp
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
```
```cpp
// tests-factorial.cpp
#include "catch.hpp"
#include "factorial.h"
TEST_CASE( "Factorials are computed", "[factorial]" ) {
REQUIRE( Factorial(1) == 1 );
REQUIRE( Factorial(2) == 2 );
REQUIRE( Factorial(3) == 6 );
REQUIRE( Factorial(10) == 3628800 );
}
```
After compiling `tests-main.cpp` once, it is enough to link it with separately compiled `tests-factorial.cpp`. This means that adding more tests to `tests-factorial.cpp`, will not result in recompiling Catch's main and the resulting compilation times will decrease substantially.
```
$ g++ tests-main.cpp -c
$ g++ tests-main.o tests-factorial.cpp -o tests && ./tests -r compact
Passed 1 test case with 4 assertions.
```
Now, the next time we change the file `tests-factorial.cpp` (say we add `REQUIRE( Factorial(0) == 1)`), it is enough to recompile the tests instead of recompiling main as well:
```
$ g++ tests-main.o tests-factorial.cpp -o tests && ./tests -r compact
tests-factorial.cpp:11: failed: Factorial(0) == 1 for: 0 == 1
Failed 1 test case, failed 1 assertion.
```
## Other possible solutions
You can also opt to sacrifice some features in order to speed-up Catch's compilation times. For details see the [documentation on Catch's compile-time configuration](configuration.md#other-toggles).
---
[Home](Readme.md#top)

View File

@ -6,6 +6,7 @@
[Tag aliases](#tag-aliases)<br>
[BDD-style test cases](#bdd-style-test-cases)<br>
[Type parametrised test cases](#type-parametrised-test-cases)<br>
[Signature based parametrised test cases](#signature-based-parametrised-test-cases)<br>
While Catch fully supports the traditional, xUnit, style of class-based fixtures containing test case methods this is not the preferred style.
@ -14,9 +15,17 @@ Instead Catch provides a powerful mechanism for nesting test case sections withi
Test cases and sections are very easy to use in practice:
* **TEST_CASE(** _test name_ \[, _tags_ \] **)**
* **SECTION(** _section name_ **)**
* **SECTION(** _section name_, \[, _section description_ \] **)**
_test name_ and _section name_ are free form, quoted, strings. The optional _tags_ argument is a quoted string containing one or more tags enclosed in square brackets. Tags are discussed below. Test names must be unique within the Catch executable.
_test name_ and _section name_ are free form, quoted, strings.
The optional _tags_ argument is a quoted string containing one or more
tags enclosed in square brackets, and are discussed below.
_section description_ can be used to provide long form description
of a section while keeping the _section name_ short for use with the
[`-c` command line parameter](command-line.md#specify-the-section-to-run).
**Test names must be unique within the Catch executable.**
For examples see the [Tutorial](tutorial.md#top)
@ -35,13 +44,21 @@ The tag expression, ```"[widget]"``` selects A, B & D. ```"[gadget]"``` selects
For more detail on command line selection see [the command line docs](command-line.md#specifying-which-tests-to-run)
Tag names are not case sensitive and can contain any ASCII characters. This means that tags `[tag with spaces]` and `[I said "good day"]` are both allowed tags and can be filtered on. Escapes are not supported however and `[\]]` is not a valid tag.
Tag names are not case sensitive and can contain any ASCII characters.
This means that tags `[tag with spaces]` and `[I said "good day"]`
are both allowed tags and can be filtered on. However, escapes are not
supported however and `[\]]` is not a valid tag.
The same tag can be specified multiple times for a single test case,
but only one of the instances of identical tags will be kept. Which one
is kept is functionally random.
### Special Tags
All tag names beginning with non-alphanumeric characters are reserved by Catch. Catch defines a number of "special" tags, which have meaning to the test runner itself. These special tags all begin with a symbol character. Following is a list of currently defined special tags and their meanings.
* `[!hide]` or `[.]` - causes test cases to be skipped from the default list (i.e. when no test cases have been explicitly selected through tag expressions or name wildcards). The hide tag is often combined with another, user, tag (for example `[.][integration]` - so all integration tests are excluded from the default run but can be run by passing `[integration]` on the command line). As a short-cut you can combine these by simply prefixing your user tag with a `.` - e.g. `[.integration]`. Because the hide tag has evolved to have several forms, all forms are added as tags if you use one of them.
* `[.]` - causes test cases to be skipped from the default list (i.e. when no test cases have been explicitly selected through tag expressions or name wildcards). The hide tag is often combined with another, user, tag (for example `[.][integration]` - so all integration tests are excluded from the default run but can be run by passing `[integration]` on the command line). As a short-cut you can combine these by simply prefixing your user tag with a `.` - e.g. `[.integration]`.
* `[!throws]` - lets Catch know that this test is likely to throw an exception even if successful. This causes the test to be excluded when running with `-e` or `--nothrow`.
@ -55,7 +72,8 @@ All tag names beginning with non-alphanumeric characters are reserved by Catch.
* `[@<alias>]` - tag aliases all begin with `@` (see below).
* `[!benchmark]` - this test case is actually a benchmark. This is an experimental feature, and currently has no documentation. If you want to try it out, look at `projects/SelfTest/Benchmark.tests.cpp` for details.
* `[!benchmark]` - this test case is actually a benchmark. Currently this only serves to hide the test case by default, to avoid the execution time costs.
## Tag aliases
@ -81,12 +99,65 @@ This macro maps onto ```TEST_CASE``` and works in the same way, except that the
* **WHEN(** _something_ **)**
* **THEN(** _something_ **)**
These macros map onto ```SECTION```s except that the section names are the _something_s prefixed by "given: ", "when: " or "then: " respectively.
These macros map onto ```SECTION```s except that the section names are the _something_ texts prefixed by
"given: ", "when: " or "then: " respectively. These macros also map onto the AAA or A<sup>3</sup> test pattern
(standing either for [Assemble-Activate-Assert](http://wiki.c2.com/?AssembleActivateAssert) or
[Arrange-Act-Assert](http://wiki.c2.com/?ArrangeActAssert)), and in this context, the macros provide both code
documentation and reporting of these parts of a test case without the need for extra comments or code to do so.
Semantically, a `GIVEN` clause may have multiple _independent_ `WHEN` clauses within it. This allows a test
to have, e.g., one set of "given" objects and multiple subtests using those objects in various ways in each
of the `WHEN` clauses without repeating the initialisation from the `GIVEN` clause. When there are _dependent_
clauses -- such as a second `WHEN` clause that should only happen _after_ the previous `WHEN` clause has been
executed and validated -- there are additional macros starting with `AND_`:
* **AND_GIVEN(** _something_ **)**
* **AND_WHEN(** _something_ **)**
* **AND_THEN(** _something_ **)**
Similar to ```WHEN``` and ```THEN``` except that the prefixes start with "and ". These are used to chain ```WHEN```s and ```THEN```s together.
These are used to chain ```GIVEN```s, ```WHEN```s and ```THEN```s together. The `AND_*` clause is placed
_inside_ the clause on which it depends. There can be multiple _independent_ clauses that are all _dependent_
on a single outer clause.
```cpp
SCENARIO( "vector can be sized and resized" ) {
GIVEN( "An empty vector" ) {
auto v = std::vector<std::string>{};
// Validate assumption of the GIVEN clause
THEN( "The size and capacity start at 0" ) {
REQUIRE( v.size() == 0 );
REQUIRE( v.capacity() == 0 );
}
// Validate one use case for the GIVEN object
WHEN( "push_back() is called" ) {
v.push_back("hullo");
THEN( "The size changes" ) {
REQUIRE( v.size() == 1 );
REQUIRE( v.capacity() >= 1 );
}
}
}
}
```
This code will result in two runs through the scenario:
```
Scenario : vector can be sized and resized
Given : An empty vector
Then : The size and capacity start at 0
Scenario : vector can be sized and resized
Given : An empty vector
When : push_back() is called
Then : The size changes
```
See also [runnable example on godbolt](https://godbolt.org/z/eY5a64r99),
with a more complicated (and failing) example.
> `AND_GIVEN` was [introduced](https://github.com/catchorg/Catch2/issues/1360) in Catch2 2.4.0.
When any of these macros are used the console reporter recognises them and formats the test case header such that the Givens, Whens and Thens are aligned to aid readability.
@ -95,11 +166,13 @@ Other than the additional prefixes and the formatting in the console reporter th
## Type parametrised test cases
In addition to `TEST_CASE`s, Catch2 also supports test cases parametrised
by types, in the form of `TEMPLATE_TEST_CASE` and
`TEMPLATE_PRODUCT_TEST_CASE`.
by types, in the form of `TEMPLATE_TEST_CASE`,
`TEMPLATE_PRODUCT_TEST_CASE` and `TEMPLATE_LIST_TEST_CASE`.
* **TEMPLATE_TEST_CASE(** _test name_ , _tags_, _type1_, _type2_, ..., _typen_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1437) in Catch2 2.5.0.
_test name_ and _tag_ are exactly the same as they are in `TEST_CASE`,
with the difference that the tag string must be provided (however, it
can be empty). _type1_ through _typen_ is the list of types for which
@ -150,6 +223,8 @@ TEMPLATE_TEST_CASE( "vectors can be sized and resized", "[vector][template]", in
* **TEMPLATE_PRODUCT_TEST_CASE(** _test name_ , _tags_, (_template-type1_, _template-type2_, ..., _template-typen_), (_template-arg1_, _template-arg2_, ..., _template-argm_) **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch2 2.6.0.
_template-type1_ through _template-typen_ is list of template template
types which should be combined with each of _template-arg1_ through
_template-argm_, resulting in _n * m_ test cases. Inside the test case,
@ -187,9 +262,76 @@ TEMPLATE_PRODUCT_TEST_CASE("Product with differing arities", "[template][product
}
```
_While there is an upper limit on the number of types you can specify
in single `TEMPLATE_TEST_CASE` or `TEMPLATE_PRODUCT_TEST_CASE`, the limit
is very high and should not be encountered in practice._
* **TEMPLATE_LIST_TEST_CASE(** _test name_, _tags_, _type list_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1627) in Catch2 2.9.0.
_type list_ is a generic list of types on which test case should be instantiated.
List can be `std::tuple`, `boost::mpl::list`, `boost::mp11::mp_list` or anything with
`template <typename...>` signature.
This allows you to reuse the _type list_ in multiple test cases.
Example:
```cpp
using MyTypes = std::tuple<int, char, float>;
TEMPLATE_LIST_TEST_CASE("Template test case with test types specified inside std::tuple", "[template][list]", MyTypes)
{
REQUIRE(sizeof(TestType) > 0);
}
```
## Signature based parametrised test cases
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0.
In addition to [type parametrised test cases](#type-parametrised-test-cases) Catch2 also supports
signature base parametrised test cases, in form of `TEMPLATE_TEST_CASE_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_SIG`.
These test cases have similar syntax like [type parametrised test cases](#type-parametrised-test-cases), with one
additional positional argument which specifies the signature.
### Signature
Signature has some strict rules for these tests cases to work properly:
* signature with multiple template parameters e.g. `typename T, size_t S` must have this format in test case declaration
`((typename T, size_t S), T, S)`
* signature with variadic template arguments e.g. `typename T, size_t S, typename...Ts` must have this format in test case declaration
`((typename T, size_t S, typename...Ts), T, S, Ts...)`
* signature with single non type template parameter e.g. `int V` must have this format in test case declaration `((int V), V)`
* signature with single type template parameter e.g. `typename T` should not be used as it is in fact `TEMPLATE_TEST_CASE`
Currently Catch2 support up to 11 template parameters in signature
### Examples
* **TEMPLATE_TEST_CASE_SIG(** _test name_ , _tags_, _signature_, _type1_, _type2_, ..., _typen_ **)**
Inside `TEMPLATE_TEST_CASE_SIG` test case you can use the names of template parameters as defined in _signature_.
```cpp
TEMPLATE_TEST_CASE_SIG("TemplateTestSig: arrays can be created from NTTP arguments", "[vector][template][nttp]",
((typename T, int V), T, V), (int,5), (float,4), (std::string,15), ((std::tuple<int, float>), 6)) {
std::array<T, V> v;
REQUIRE(v.size() > 1);
}
```
* **TEMPLATE_PRODUCT_TEST_CASE_SIG(** _test name_ , _tags_, _signature_, (_template-type1_, _template-type2_, ..., _template-typen_), (_template-arg1_, _template-arg2_, ..., _template-argm_) **)**
```cpp
template<typename T, size_t S>
struct Bar {
size_t size() { return S; }
};
TEMPLATE_PRODUCT_TEST_CASE_SIG("A Template product test case with array signature", "[template][product][nttp]", ((typename T, size_t S), T, S), (std::array, Bar), ((int, 9), (float, 42))) {
TestType x;
REQUIRE(x.size() > 0);
}
```
---

View File

@ -1,6 +1,8 @@
<a id="top"></a>
# Test fixtures
## Defining test fixtures
Although Catch allows you to group tests together as sections within a test case, it can still be convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure:
```c++
@ -57,7 +59,10 @@ struct Template_Fixture {
T m_a;
};
TEMPLATE_TEST_CASE_METHOD(Template_Fixture,"A TEMPLATE_TEST_CASE_METHOD based test run that succeeds", "[class][template]", int, float, double) {
TEMPLATE_TEST_CASE_METHOD(Template_Fixture,
"A TEMPLATE_TEST_CASE_METHOD based test run that succeeds",
"[class][template]",
int, float, double) {
REQUIRE( Template_Fixture<TestType>::m_a == 1 );
}
@ -75,7 +80,11 @@ struct Foo_class {
}
};
TEMPLATE_PRODUCT_TEST_CASE_METHOD(Template_Template_Fixture, "A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test succeeds", "[class][template]", (Foo_class, std::vector), int) {
TEMPLATE_PRODUCT_TEST_CASE_METHOD(Template_Template_Fixture,
"A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test succeeds",
"[class][template]",
(Foo_class, std::vector),
int) {
REQUIRE( Template_Template_Fixture<TestType>::m_a.size() == 0 );
}
```
@ -84,6 +93,70 @@ _While there is an upper limit on the number of types you can specify
in single `TEMPLATE_TEST_CASE_METHOD` or `TEMPLATE_PRODUCT_TEST_CASE_METHOD`,
the limit is very high and should not be encountered in practice._
## Signature-based parametrised test fixtures
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0.
Catch2 also provides `TEMPLATE_TEST_CASE_METHOD_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG` to support
fixtures using non-type template parameters. These test cases work similar to `TEMPLATE_TEST_CASE_METHOD` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD`,
with additional positional argument for [signature](test-cases-and-sections.md#signature-based-parametrised-test-cases).
Example:
```cpp
template <int V>
struct Nttp_Fixture{
int value = V;
};
TEMPLATE_TEST_CASE_METHOD_SIG(
Nttp_Fixture,
"A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds",
"[class][template][nttp]",
((int V), V),
1, 3, 6) {
REQUIRE(Nttp_Fixture<V>::value > 0);
}
template<typename T>
struct Template_Fixture_2 {
Template_Fixture_2() {}
T m_a;
};
template< typename T, size_t V>
struct Template_Foo_2 {
size_t size() { return V; }
};
TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG(
Template_Fixture_2,
"A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds",
"[class][template][product][nttp]",
((typename T, size_t S), T, S),
(std::array, Template_Foo_2),
((int,2), (float,6))) {
REQUIRE(Template_Fixture_2<TestType>{}.m_a.size() >= 2);
}
```
## Template fixtures with types specified in template type lists
Catch2 also provides `TEMPLATE_LIST_TEST_CASE_METHOD` to support template fixtures with types specified in
template type lists like `std::tuple`, `boost::mpl::list` or `boost::mp11::mp_list`. This test case works the same as `TEMPLATE_TEST_CASE_METHOD`,
only difference is the source of types. This allows you to reuse the template type list in multiple test cases.
Example:
```cpp
using MyTypes = std::tuple<int, char, double>;
TEMPLATE_LIST_TEST_CASE_METHOD(Template_Fixture,
"Template test case method with test types specified inside std::tuple",
"[class][template][list]",
MyTypes) {
REQUIRE( Template_Fixture<TestType>::m_a == 1 );
}
```
---
[Home](Readme.md#top)

View File

@ -6,6 +6,9 @@
[Catch::StringMaker specialisation](#catchstringmaker-specialisation)<br>
[Catch::is_range specialisation](#catchis_range-specialisation)<br>
[Exceptions](#exceptions)<br>
[Enums](#enums)<br>
[Floating point precision](#floating-point-precision)<br>
Catch needs to be able to convert types you use in assertions and logging expressions into strings (for logging and reporting purposes).
Most built-in or std types are supported out of the box but there are two ways that you can tell Catch how to convert your own types (or other, third-party types) into strings.
@ -14,7 +17,7 @@ Most built-in or std types are supported out of the box but there are two ways t
This is the standard way of providing string conversions in C++ - and the chances are you may already provide this for your own purposes. If you're not familiar with this idiom it involves writing a free function of the form:
```
```cpp
std::ostream& operator << ( std::ostream& os, T const& value ) {
os << convertMyTypeToString( value );
return os;
@ -28,7 +31,7 @@ You should put this function in the same namespace as your type, or the global n
## Catch::StringMaker specialisation
If you don't want to provide an ```operator <<``` overload, or you want to convert your type differently for testing purposes, you can provide a specialization for `Catch::StringMaker<T>`:
```
```cpp
namespace Catch {
template<>
struct StringMaker<T> {
@ -60,12 +63,70 @@ namespace Catch {
By default all exceptions deriving from `std::exception` will be translated to strings by calling the `what()` method. For exception types that do not derive from `std::exception` - or if `what()` does not return a suitable string - use `CATCH_TRANSLATE_EXCEPTION`. This defines a function that takes your exception type, by reference, and returns a string. It can appear anywhere in the code - it doesn't have to be in the same translation unit. For example:
```
CATCH_TRANSLATE_EXCEPTION( MyType& ex ) {
```cpp
CATCH_TRANSLATE_EXCEPTION( MyType const& ex ) {
return ex.message();
}
```
## Enums
> Introduced in Catch2 2.8.0.
Enums that already have a `<<` overload for `std::ostream` will convert to strings as expected.
If you only need to convert enums to strings for test reporting purposes you can provide a `StringMaker` specialisations as any other type.
However, as a convenience, Catch provides the `REGISTER_ENUM` helper macro that will generate the `StringMaker` specialiation for you with minimal code.
Simply provide it the (qualified) enum name, followed by all the enum values, and you're done!
E.g.
```cpp
enum class Fruits { Banana, Apple, Mango };
CATCH_REGISTER_ENUM( Fruits, Fruits::Banana, Fruits::Apple, Fruits::Mango )
TEST_CASE() {
REQUIRE( Fruits::Mango == Fruits::Apple );
}
```
... or if the enum is in a namespace:
```cpp
namespace Bikeshed {
enum class Colours { Red, Green, Blue };
}
// Important!: This macro must appear at top level scope - not inside a namespace
// You can fully qualify the names, or use a using if you prefer
CATCH_REGISTER_ENUM( Bikeshed::Colours,
Bikeshed::Colours::Red,
Bikeshed::Colours::Green,
Bikeshed::Colours::Blue )
TEST_CASE() {
REQUIRE( Bikeshed::Colours::Red == Bikeshed::Colours::Blue );
}
```
## Floating point precision
> [Introduced](https://github.com/catchorg/Catch2/issues/1614) in Catch2 2.8.0.
Catch provides a built-in `StringMaker` specialization for both `float`
and `double`. By default, it uses what we think is a reasonable precision,
but you can customize it by modifying the `precision` static variable
inside the `StringMaker` specialization, like so:
```cpp
Catch::StringMaker<float>::precision = 15;
const float testFloat1 = 1.12345678901234567899f;
const float testFloat2 = 1.12345678991234567899f;
REQUIRE(testFloat1 == testFloat2);
```
This assertion will fail and print out the `testFloat1` and `testFloat2`
to 15 decimal places.
---
[Home](Readme.md#top)

View File

@ -3,32 +3,20 @@
**Contents**<br>
[Getting Catch2](#getting-catch2)<br>
[Where to put it?](#where-to-put-it)<br>
[Writing tests](#writing-tests)<br>
[Test cases and sections](#test-cases-and-sections)<br>
[BDD-Style](#bdd-style)<br>
[Scaling up](#scaling-up)<br>
[Type parametrised test cases](#type-parametrised-test-cases)<br>
[BDD style testing](#bdd-style-testing)<br>
[Data and Type driven tests](#data-and-type-driven-tests)<br>
[Next steps](#next-steps)<br>
## Getting Catch2
The simplest way to get Catch2 is to download the latest [single header version](https://raw.githubusercontent.com/catchorg/Catch2/master/single_include/catch2/catch.hpp). The single header is generated by merging a set of individual headers but it is still just normal source code in a header file.
Ideally you should be using Catch2 through its [CMake integration](cmake-integration.md#top).
Catch2 also provides pkg-config files and single TU distribution, but this
documentation will assume you are using CMake. If you are using single-TU
distribution instead, remember to replace the included header with `catch_amalgamated.hpp`.
Alternative ways of getting Catch2 include using your system package
manager, or installing it using [its CMake package](cmake-integration.md#installing-catch2-from-git-repository).
The full source for Catch2, including test projects, documentation, and other things, is hosted on GitHub. [http://catch-lib.net](http://catch-lib.net) will redirect you there.
## Where to put it?
Catch2 is header only. All you need to do is drop the file somewhere reachable from your project - either in some central location you can set your header search path to find, or directly into your project tree itself! This is a particularly good option for other Open-Source projects that want to use Catch for their test suite. See [this blog entry for more on that](http://www.levelofindirection.com/journal/2011/5/27/unit-testing-in-c-and-objective-c-just-got-ridiculously-easi.html).
The rest of this tutorial will assume that the Catch2 single-include header (or the include folder) is available unqualified - but you may need to prefix it with a folder name if necessary.
_If you have installed Catch2 from system package manager, or CMake
package, you need to include the header as `#include <catch2/catch.hpp>`_
## Writing tests
@ -40,11 +28,8 @@ unsigned int Factorial( unsigned int number ) {
}
```
To keep things simple we'll put everything in a single file (<a href="#scaling-up">see later for more on how to structure your test files</a>).
```c++
#define CATCH_CONFIG_MAIN // This tells Catch to provide a main() - only do this in one cpp file
#include "catch.hpp"
#include <catch2/catch_test_macros.hpp>
unsigned int Factorial( unsigned int number ) {
return number <= 1 ? number : Factorial(number-1)*number;
@ -60,13 +45,10 @@ TEST_CASE( "Factorials are computed", "[factorial]" ) {
This will compile to a complete executable which responds to [command line arguments](command-line.md#top). If you just run it with no arguments it will execute all test cases (in this case there is just one), report any failures, report a summary of how many tests passed and failed and return the number of failed tests (useful for if you just want a yes/ no answer to: "did it work").
If you run this as written it will pass. Everything is good. Right?
Well, there is still a bug here. In fact the first version of this tutorial I posted here genuinely had the bug in! So it's not completely contrived (thanks to Daryle Walker (```@CTMacUser```) for pointing this out).
What is the bug? Well what is the factorial of zero?
[The factorial of zero is one](http://mathforum.org/library/drmath/view/57128.html) - which is just one of those things you have to know (and remember!).
Let's add that to the test case:
Anyway, as the tests above as written will pass, but there is a bug.
The problem is that `Factorial(0)` should return 1 (due to [its
definition](https://en.wikipedia.org/wiki/Factorial#Factorial_of_zero)).
Let's add that as an assertion to the test case:
```c++
TEST_CASE( "Factorials are computed", "[factorial]" ) {
@ -78,7 +60,8 @@ TEST_CASE( "Factorials are computed", "[factorial]" ) {
}
```
Now we get a failure - something like:
After another compile & run cycle, we will see a test failure. The output
will look something like:
```
Example.cpp:9: FAILED:
@ -87,37 +70,51 @@ with expansion:
0 == 1
```
Note that we get the actual return value of Factorial(0) printed for us (0) - even though we used a natural expression with the == operator. That lets us immediately see what the problem is.
Let's change the factorial function to:
Note that the output contains both the original expression,
`REQUIRE( Factorial(0) == 1 )` and the actual value returned by the call
to the `Factorial` function: `0`.
We can fix this bug by slightly modifying the `Factorial` function to:
```c++
unsigned int Factorial( unsigned int number ) {
return number > 1 ? Factorial(number-1)*number : 1;
}
```
Now all the tests pass.
Of course there are still more issues to deal with. For example we'll hit problems when the return value starts to exceed the range of an unsigned int. With factorials that can happen quite quickly. You might want to add tests for such cases and decide how to handle them. We'll stop short of doing that here.
### What did we do here?
Although this was a simple test it's been enough to demonstrate a few things about how Catch is used. Let's take moment to consider those before we move on.
Although this was a simple test it's been enough to demonstrate a few
things about how Catch2 is used. Let's take a moment to consider those
before we move on.
* We introduce test cases with the `TEST_CASE` macro. This macro takes
one or two string arguments - a free form test name and, optionally,
one or more tags (for more see [Test cases and Sections](#test-cases-and-sections)).
* The test automatically self-registers with the test runner, and user
does not have do anything more to ensure that it is picked up by the test
framework. _Note that you can run specific test, or set of tests,
through the [command line](command-line.md#top)._
* The individual test assertions are written using the `REQUIRE` macro.
It accepts a boolean expression, and uses expression templates to
internally decompose it, so that it can be individually stringified
on test failure.
On the last point, note that there are more testing macros available,
because not all useful checks can be expressed as a simple boolean
expression. As an example, checking that an expression throws an exception
is done with the `REQUIRE_THROWS` macro. More on that later.
1. All we did was ```#define``` one identifier and ```#include``` one header and we got everything - even an implementation of ```main()``` that will [respond to command line arguments](command-line.md#top). You can only use that ```#define``` in one implementation file, for (hopefully) obvious reasons. Once you have more than one file with unit tests in you'll just ```#include "catch.hpp"``` and go. Usually it's a good idea to have a dedicated implementation file that just has ```#define CATCH_CONFIG_MAIN``` and ```#include "catch.hpp"```. You can also provide your own implementation of main and drive Catch yourself (see [Supplying-your-own-main()](own-main.md#top)).
2. We introduce test cases with the ```TEST_CASE``` macro. This macro takes one or two arguments - a free form test name and, optionally, one or more tags (for more see <a href="#test-cases-and-sections">Test cases and Sections</a>, ). The test name must be unique. You can run sets of tests by specifying a wildcarded test name or a tag expression. See the [command line docs](command-line.md#top) for more information on running tests.
3. The name and tags arguments are just strings. We haven't had to declare a function or method - or explicitly register the test case anywhere. Behind the scenes a function with a generated name is defined for you, and automatically registered using static registry classes. By abstracting the function name away we can name our tests without the constraints of identifier names.
4. We write our individual test assertions using the ```REQUIRE``` macro. Rather than a separate macro for each type of condition we express the condition naturally using C/C++ syntax. Behind the scenes a simple set of expression templates captures the left-hand-side and right-hand-side of the expression so we can display the values in our test report. As we'll see later there _are_ other assertion macros - but because of this technique the number of them is drastically reduced.
<a id="test-cases-and-sections"></a>
## Test cases and sections
Most test frameworks have a class-based fixture mechanism. That is, test cases map to methods on a class and common setup and teardown can be performed in ```setup()``` and ```teardown()``` methods (or constructor/ destructor in languages, like C++, that support deterministic destruction).
Like most test frameworks, Catch2 supports a class-based fixture mechanism,
where individual tests are methods on class and setup/teardown can be
done in constructor/destructor of the type.
While Catch fully supports this way of working there are a few problems with the approach. In particular the way your code must be split up, and the blunt granularity of it, may cause problems. You can only have one setup/ teardown pair across a set of methods, but sometimes you want slightly different setup in each method, or you may even want several levels of setup (a concept which we will clarify later on in this tutorial). It was <a href="http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html">problems like these</a> that led James Newkirk, who led the team that built NUnit, to start again from scratch and <a href="http://jamesnewkirk.typepad.com/posts/2007/09/announcing-xuni.html">build xUnit</a>).
Catch takes a different approach (to both NUnit and xUnit) that is a more natural fit for C++ and the C family of languages. This is best explained through an example ([code](../examples/100-Fix-Section.cpp)):
However, their use in Catch2 is rare, because idiomatic Catch2 tests
instead use _sections_ to share setup and teardown code between test code.
This is best explained through an example ([code](../examples/100-Fix-Section.cpp)):
```c++
TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
@ -154,125 +151,75 @@ TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
}
```
For each ```SECTION``` the ```TEST_CASE``` is executed from the start - so as we enter each section we know that size is 5 and capacity is at least 5. We enforced those requirements with the ```REQUIRE```s at the top level so we can be confident in them.
This works because the ```SECTION``` macro contains an if statement that calls back into Catch to see if the section should be executed. One leaf section is executed on each run through a ```TEST_CASE```. The other sections are skipped. Next time through the next section is executed, and so on until no new sections are encountered.
For each `SECTION` the `TEST_CASE` is executed from the start. This means
that each section is entered with a freshly constructed vector `v`, that
we know has size 5 and capacity at least 5, because the two assertions
are also checked before the section is entered. Each run through a test
case will execute one, and only one, leaf section.
So far so good - this is already an improvement on the setup/teardown approach because now we see our setup code inline and use the stack.
Section can also be nested, in which case the parent section can be
entered multiple times, once for each leaf section. Nested sections are
most useful when you have multiple tests that share part of the set up.
To continue on the vector example above, you could add a check that
`std::vector::reserve` does not remove unused excess capacity, like this:
The power of sections really shows, however, when we need to execute a sequence of, checked, operations. Continuing the vector example, we might want to verify that attempting to reserve a capacity smaller than the current capacity of the vector changes nothing. We can do that, naturally, like so:
```c++
```cpp
SECTION( "reserving bigger changes capacity but not size" ) {
v.reserve( 10 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
SECTION( "reserving smaller again does not change capacity" ) {
SECTION( "reserving down unused capacity does not change capacity" ) {
v.reserve( 7 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
}
}
```
Sections can be nested to an arbitrary depth (limited only by your stack size). Each leaf section (i.e. a section that contains no nested sections) will be executed exactly once, on a separate path of execution from any other leaf section (so no leaf section can interfere with another). A failure in a parent section will prevent nested sections from running - but then that's the idea.
Another way to look at sections is that they are a way to define a tree
of paths through the test. Each section represents a node, and the final
tree is walked in depth-first manner, with each path only visiting only
one leaf node.
## BDD-Style
If you name your test cases and sections appropriately you can achieve a BDD-style specification structure. This became such a useful way of working that first class support has been added to Catch. Scenarios can be specified using ```SCENARIO```, ```GIVEN```, ```WHEN``` and ```THEN``` macros, which map on to ```TEST_CASE```s and ```SECTION```s, respectively. For more details see [Test cases and sections](test-cases-and-sections.md#top).
The vector example can be adjusted to use these macros like so ([example code](../examples/120-Bdd-ScenarioGivenWhenThen.cpp)):
```c++
SCENARIO( "vectors can be sized and resized", "[vector]" ) {
GIVEN( "A vector with some items" ) {
std::vector<int> v( 5 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 5 );
WHEN( "the size is increased" ) {
v.resize( 10 );
THEN( "the size and capacity change" ) {
REQUIRE( v.size() == 10 );
REQUIRE( v.capacity() >= 10 );
}
}
WHEN( "the size is reduced" ) {
v.resize( 0 );
THEN( "the size changes but not capacity" ) {
REQUIRE( v.size() == 0 );
REQUIRE( v.capacity() >= 5 );
}
}
WHEN( "more capacity is reserved" ) {
v.reserve( 10 );
THEN( "the capacity changes but not the size" ) {
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
}
}
WHEN( "less capacity is reserved" ) {
v.reserve( 0 );
THEN( "neither size nor capacity are changed" ) {
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 5 );
}
}
}
}
```
Conveniently, these tests will be reported as follows when run:
```
Scenario: vectors can be sized and resized
Given: A vector with some items
When: more capacity is reserved
Then: the capacity changes but not the size
```
<a id="scaling-up"></a>
## Scaling up
To keep the tutorial simple we put all our code in a single file. This is fine to get started - and makes jumping into Catch even quicker and easier. As you write more real-world tests, though, this is not really the best approach.
The requirement is that the following block of code ([or equivalent](own-main.md#top)):
```c++
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
```
appears in _exactly one_ source file. Use as many additional cpp files (or whatever you call your implementation files) as you need for your tests, partitioned however makes most sense for your way of working. Each additional file need only ```#include "catch.hpp"``` - do not repeat the ```#define```!
In fact it is usually a good idea to put the block with the ```#define``` [in its own source file](slow-compiles.md#top) (code example [main](../examples/020-TestCase-1.cpp), [tests](../examples/020-TestCase-2.cpp)).
Do not write your tests in header files!
There is no practical limit on nesting sections, as long as your compiler
can handle them, but keep in mind that overly nested sections can become
unreadable. From experience, having section nest more than 3 levels is
usually very hard to follow and not worth the removed duplication.
## Type parametrised test cases
## BDD style testing
Test cases in Catch2 can be also parametrised by type, via the
`TEMPLATE_TEST_CASE` and `TEMPLATE_PRODUCT_TEST_CASE` macros,
which behave in the same way the `TEST_CASE` macro, but are run for
every type or type combination.
Catch2 also provides some basic support for BDD-style testing. There are
macro aliases for `TEST_CASE` and `SECTIONS` that you can use so that
the resulting tests read as BDD spec. `SCENARIO` acts as a `TEST_CASE`
with "Scenario: " name prefix. Then there are `GIVEN`, `WHEN`, `THEN`
(and their variants with `AND_` prefix), which act as a `SECTION`,
similarly prefixed with the macro name.
For more details, see our documentation on [test cases and
sections](test-cases-and-sections.md#type-parametrised-test-cases).
For more details on the macros look at the [test cases and
sections](test-cases-and-sections.md#top) part of the reference docs,
or at the [vector example done with BDD macros](../examples/120-Bdd-ScenarioGivenWhenThen.cpp).
## Data and Type driven tests
Test cases in Catch2 can also be driven by types, input data, or both
at the same time.
For more details look into the Catch2 reference, either at the
[type parametrized test cases](test-cases-and-sections.md#type-parametrised-test-cases),
or [data generators](generators.md#top).
## Next steps
This has been a brief introduction to get you up and running with Catch, and to point out some of the key differences between Catch and other frameworks you may already be familiar with. This will get you going quite far already and you are now in a position to dive in and write some tests.
This page is a brief introduction to get you up and running with Catch2,
and to show the basic features of Catch2. The features mentioned here
can get you quite far, but there are many more. However, you can read
about these as you go, in the ever-growing [reference section](Readme.md#top)
of the documentation.
Of course there is more to learn - most of which you should be able to page-fault in as you go. Please see the ever-growing [Reference section](Readme.md#top) for what's available.
---

95
docs/usage-tips.md Normal file
View File

@ -0,0 +1,95 @@
<a id="top"></a>
# Best practices and other tips on using Catch2
## Running tests
Your tests should be run in a manner roughly equivalent with:
```
./tests --order rand --warn NoAssertions
```
Notice that all the tests are run in a large batch, their relative order
is randomized, and that you ask Catch2 to fail test whose leaf-path
does not contain an assertion.
The reason I recommend running all your tests in the same process is that
this exposes your tests to interference from their runs. This can be both
positive interference, where the changes in global state from previous
test allow later tests to pass, but also negative interference, where
changes in global state from previous test causes later tests to fail.
In my experience, interference, especially destructive interference,
usually comes from errors in the code under test, rather than the tests
themselves. This means that by allowing interference to happen, our tests
can find these issues. Obviously, to shake out interference coming from
different orderings of tests, the test order also need to be shuffled
between runs.
However, running all tests in a single batch eventually becomes impractical
as they will take too long to run, and you will want to run your tests
in parallel.
<a id="parallel-tests"></a>
## Running tests in parallel
There are multiple ways of running tests in parallel, with various level
of structure. If you are using CMake and CTest, then we provide a helper
function [`catch_discover_tests`](cmake-integration.md#automatic-test-registration)
that registers each Catch2 `TEST_CASE` as a single CTest test, which
is then run in a separate process. This is an easy way to set up parallel
tests if you are already using CMake & CTest to run your tests, but you
will lose the advantage of running tests in batches.
Catch2 also supports [splitting tests in a binary into multiple
shards](command-line.md#test-sharding). This can be used by any test
runner to run batches of tests in parallel. Do note that when selecting
on the number of shards, you should have more shards than there are cores,
to avoid issues with long running tests getting accidentally grouped in
the same shard, and causing long-tailed execution time.
**Note that naively composing sharding and random ordering of tests will break.**
Invoking Catch2 test executable like this
```text
./tests --order rand --shard-index 0 --shard-count 3
./tests --order rand --shard-index 1 --shard-count 3
./tests --order rand --shard-index 2 --shard-count 3
```
does not guarantee covering all tests inside the executable, because
each invocation will have its own random seed, thus it will have its own
random order of tests and thus the partitioning of tests into shards will
be different as well.
To do this properly, you need the individual shards to share the random
seed, e.g.
```text
./tests --order rand --shard-index 0 --shard-count 3 --rng-seed 0xBEEF
./tests --order rand --shard-index 1 --shard-count 3 --rng-seed 0xBEEF
./tests --order rand --shard-index 2 --shard-count 3 --rng-seed 0xBEEF
```
## Organizing tests into binaries
Both overly large and overly small test binaries can cause issues. Overly
large test binaries have to be recompiled and relinked often, and the
link times are usually also long. Overly small test binaries in turn pay
significant overhead from linking against Catch2 more often per compiled
test case, and also make it hard/impossible to run tests in batches.
Because there is no hard and fast rule for the right size of a test binary,
I recommend having 1:1 correspondence between libraries in project and test
binaries. (At least if it is possible, in some cases it is not.) Having
a test binary for each library in project keeps related tests together,
and makes tests easy to navigate by reflecting the project's organizational
structure.
---
[Home](Readme.md#top)

View File

@ -6,40 +6,53 @@ including (but not limited to),
[Google Test](http://code.google.com/p/googletest/),
[Boost.Test](http://www.boost.org/doc/libs/1_49_0/libs/test/doc/html/index.html),
[CppUnit](http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=Main_Page),
[Cute](http://www.cute-test.com),
[Cute](http://www.cute-test.com), and
[many, many more](http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B).
So what does Catch bring to the party that differentiates it from these? Apart from a Catchy name, of course.
So what does Catch2 bring to the party that differentiates it from these? Apart from the catchy name, of course.
## Key Features
* Quick and Really easy to get started. Just download catch.hpp, `#include` it and you're away.
* No external dependencies. As long as you can compile C++11 and have a C++ standard library available.
* Quick and easy to get started. Just download two files, add them into your project and you're away.
* No external dependencies. As long as you can compile C++14 and have the C++ standard library available.
* Write test cases as, self-registering, functions (or methods, if you prefer).
* Divide test cases into sections, each of which is run in isolation (eliminates the need for fixtures).
* Use BDD-style Given-When-Then sections as well as traditional unit test cases.
* Only one core assertion macro for comparisons. Standard C/C++ operators are used for the comparison - yet the full expression is decomposed and lhs and rhs values are logged.
* Tests are named using free-form strings - no more couching names in legal identifiers.
## Other core features
* Tests can be tagged for easily running ad-hoc groups of tests.
* Failures can (optionally) break into the debugger on Windows and Mac.
* Failures can (optionally) break into the debugger on common platforms.
* Output is through modular reporter objects. Basic textual and XML reporters are included. Custom reporters can easily be added.
* JUnit xml output is supported for integration with third-party tools, such as CI servers.
* A default main() function is provided, but you can supply your own for complete control (e.g. integration into your own test runner GUI).
* A command line parser is provided and can still be used if you choose to provided your own main() function.
* Catch can test itself.
* Alternative assertion macro(s) report failures but don't abort the test case
* Floating point tolerance comparisons are built in using an expressive Approx() syntax.
* Good set of facilities for floating point comparisons (`Catch::Approx` and full set of matchers)
* Internal and friendly macros are isolated so name clashes can be managed
* Matchers
* Data generators (data driven test support)
* Hamcrest-style Matchers for testing complex properties
* Microbenchmarking support
## Who else is using Catch?
See the list of [open source projects using Catch](opensource-users.md#top).
## Who else is using Catch2?
See the [tutorial](tutorial.md#top) to get more of a taste of using Catch in practice
A whole lot of people. According to the 2021 Jetbrains C++ ecosystem survey,
about 11% of C++ programmers use Catch2 for unit testing, making it the
second most popular unit testing framework.
You can also take a look at the (incomplete) list of [open source projects](opensource-users.md#top)
or the (very incomplete) list of [commercial users of Catch2](commercial-users.md#top)
for some idea on who else also uses Catch2.
---
See the [tutorial](tutorial.md#top) to get more of a taste of using
Catch2 in practice.
---

View File

@ -1,15 +0,0 @@
// 000-CatchMain.cpp
// In a Catch project with multiple files, dedicate one file to compile the
// source code of Catch itself and reuse the resulting object file for linking.
// Let Catch provide main():
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
// That's it
// Compile implementation of Catch for use with files that do contain tests:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -c 000-CatchMain.cpp
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% -c 000-CatchMain.cpp

View File

@ -1,11 +1,8 @@
// 010-TestCase.cpp
// And write tests in the same file:
#include <catch2/catch_test_macros.hpp>
// Let Catch provide main():
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
int Factorial( int number ) {
static int Factorial( int number ) {
return number <= 1 ? number : Factorial( number - 1 ) * number; // fail
// return number <= 1 ? 1 : Factorial( number - 1 ) * number; // pass
}
@ -22,7 +19,7 @@ TEST_CASE( "Factorials of 1 and higher are computed (pass)", "[single-file]" ) {
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 010-TestCase 010-TestCase.cpp && 010-TestCase --success
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 010-TestCase 010-TestCase.cpp && 010-TestCase --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 010-TestCase.cpp && 010-TestCase --success
// Expected compact output (all assertions):

View File

@ -1,12 +1,6 @@
// 020-TestCase-1.cpp
// In a Catch project with multiple files, dedicate one file to compile the
// source code of Catch itself and reuse the resulting object file for linking.
// Let Catch provide main():
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
TEST_CASE( "1: All test cases reside in other .cpp files (empty)", "[multi-file:1]" ) {
}
@ -16,8 +10,8 @@ TEST_CASE( "1: All test cases reside in other .cpp files (empty)", "[multi-file:
// Here just to show there are two source files via option --list-tests.
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -c 020-TestCase-1.cpp
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 020-TestCase TestCase-1.o 020-TestCase-2.cpp && 020-TestCase --success
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -c 020-TestCase-1.cpp
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 020-TestCase TestCase-1.o 020-TestCase-2.cpp && 020-TestCase --success
//
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% -c 020-TestCase-1.cpp
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% -Fe020-TestCase.exe 020-TestCase-1.obj 020-TestCase-2.cpp && 020-TestCase --success

View File

@ -2,9 +2,9 @@
// main() provided by Catch in file 020-TestCase-1.cpp.
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
int Factorial( int number ) {
static int Factorial( int number ) {
return number <= 1 ? number : Factorial( number - 1 ) * number; // fail
// return number <= 1 ? 1 : Factorial( number - 1 ) * number; // pass
}

View File

@ -8,11 +8,11 @@
// - REQUIRE_FALSE() stops at first failure.
// - CHECK_FALSE() continues after failure.
// main() provided in 000-CatchMain.cpp
// main() provided by linkage to Catch2WithMain
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
std::string one() {
static std::string one() {
return "1";
}
@ -53,8 +53,8 @@ TEST_CASE( "Assert that something is false (continue after failure)", "[check-fa
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 030-Asn-Require-Check 030-Asn-Require-Check.cpp 000-CatchMain.o && 030-Asn-Require-Check --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 030-Asn-Require-Check.cpp 000-CatchMain.obj && 030-Asn-Require-Check --success
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 030-Asn-Require-Check 030-Asn-Require-Check.cpp && 030-Asn-Require-Check --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 030-Asn-Require-Check.cpp && 030-Asn-Require-Check --success
// Expected compact output (all assertions):
//

View File

@ -4,9 +4,10 @@
// - Sections (this file)
// - Traditional class-based fixtures
// main() provided in 000-CatchMain.cpp
// main() provided by linkage to Catch2WithMain
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
#include <vector>
TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
@ -44,8 +45,8 @@ TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 100-Fix-Section 100-Fix-Section.cpp 000-CatchMain.o && 100-Fix-Section --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 100-Fix-Section.cpp 000-CatchMain.obj && 100-Fix-Section --success
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 100-Fix-Section 100-Fix-Section.cpp && 100-Fix-Section --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 100-Fix-Section.cpp && 100-Fix-Section --success
// Expected compact output (all assertions):
//

View File

@ -4,9 +4,9 @@
// - Sections
// - Traditional class-based fixtures (this file)
// main() provided in 000-CatchMain.cpp
// main() provided by linkage to Catch2WithMain
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
class DBConnection
{
@ -52,8 +52,11 @@ TEST_CASE_METHOD( UniqueTestsFixture, "Create Employee/Normal", "[create]" ) {
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 110-Fix-ClassFixture 110-Fix-ClassFixture.cpp 000-CatchMain.o && 110-Fix-ClassFixture --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 110-Fix-ClassFixture.cpp 000-CatchMain.obj && 110-Fix-ClassFixture --success
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 110-Fix-ClassFixture 110-Fix-ClassFixture.cpp && 110-Fix-ClassFixture --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 110-Fix-ClassFixture.cpp && 110-Fix-ClassFixture --success
//
// Compile with pkg-config:
// - g++ -std=c++14 -Wall $(pkg-config catch2-with-main --cflags) -o 110-Fix-ClassFixture 110-Fix-ClassFixture.cpp $(pkg-config catch2-with-main --libs)
// Expected compact output (all assertions):
//

View File

@ -1,8 +1,8 @@
// 120-Bdd-ScenarioGivenWhenThen.cpp
// main() provided in 000-CatchMain.cpp
// main() provided by linkage with Catch2WithMain
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
SCENARIO( "vectors can be sized and resized", "[vector]" ) {
@ -48,8 +48,8 @@ SCENARIO( "vectors can be sized and resized", "[vector]" ) {
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 120-Bdd-ScenarioGivenWhenThen 120-Bdd-ScenarioGivenWhenThen.cpp 000-CatchMain.o && 120-Bdd-ScenarioGivenWhenThen --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 120-Bdd-ScenarioGivenWhenThen.cpp 000-CatchMain.obj && 120-Bdd-ScenarioGivenWhenThen --success
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 120-Bdd-ScenarioGivenWhenThen 120-Bdd-ScenarioGivenWhenThen.cpp && 120-Bdd-ScenarioGivenWhenThen --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 120-Bdd-ScenarioGivenWhenThen.cpp && 120-Bdd-ScenarioGivenWhenThen --success
// Expected compact output (all assertions):
//

View File

@ -1,27 +0,0 @@
// 200-Rpt-CatchMain.cpp
// In a Catch project with multiple files, dedicate one file to compile the
// source code of Catch itself and reuse the resulting object file for linking.
// Let Catch provide main():
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
#ifdef CATCH_EXAMPLE_RPT_1
#include CATCH_EXAMPLE_RPT_1
#endif
#ifdef CATCH_EXAMPLE_RPT_2
#include CATCH_EXAMPLE_RPT_2
#endif
#ifdef CATCH_EXAMPLE_RPT_3
#include CATCH_EXAMPLE_RPT_3
#endif
// That's it
// Compile implementation of Catch for use with files that do contain tests:
// - g++ -std=c++11 -Wall -I$(CATCH_ROOT) -DCATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" -o 200-Rpt-CatchMainTeamCity.o -c 200-Rpt-CatchMain.cpp
// cl -EHsc -I%CATCH_ROOT% -DCATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" -Fo200-Rpt-CatchMainTeamCity.obj -c 200-Rpt-CatchMain.cpp

View File

@ -1,171 +0,0 @@
// 207-Rpt-TeamCityReporter.cpp
// Catch has built-in and external reporters:
// Built-in:
// - compact
// - console
// - junit
// - xml
// External:
// - automake
// - tap
// - teamcity (this example)
// main() and reporter code provided in 200-Rpt-CatchMain.cpp
#include <catch2/catch.hpp>
#ifdef _MSC_VER
# pragma warning (disable : 4702) // Disable warning: unreachable code
#endif
TEST_CASE( "TeamCity passes unconditionally succeeding assertion", "[teamcity]" ) {
SUCCEED();
}
TEST_CASE( "TeamCity reports unconditionally failing assertion", "[teamcity]" ) {
FAIL();
}
TEST_CASE( "TeamCity reports failing check", "[teamcity]" ) {
REQUIRE( 3 == 7 );
}
TEST_CASE( "TeamCity reports failing check-false", "[teamcity]" ) {
REQUIRE_FALSE( 3 == 3 );
}
TEST_CASE( "TeamCity reports failing check-that", "[teamcity]" ) {
using namespace Catch;
REQUIRE_THAT( "hello", Contains( "world" ) );
}
TEST_CASE( "TeamCity reports unexpected exception", "[teamcity]" ) {
REQUIRE( (throw std::runtime_error("surprise!"), true) );
}
TEST_CASE( "TeamCity reports undesired exception", "[teamcity]" ) {
REQUIRE_NOTHROW( (throw std::runtime_error("surprise!"), true) );
}
TEST_CASE( "TeamCity reports missing expected exception", "[teamcity]" ) {
REQUIRE_THROWS( true );
}
TEST_CASE( "TeamCity reports missing specific expected exception", "[teamcity]" ) {
REQUIRE_THROWS_AS( throw std::bad_alloc(), std::runtime_error );
}
TEST_CASE( "TeamCity reports unexpected message in expected exception", "[teamcity]" ) {
using namespace Catch;
CHECK_THROWS_WITH( throw std::runtime_error("hello"), "world" );
CHECK_THROWS_WITH( throw std::runtime_error("hello"), Contains("world") );
}
struct MyException: public std::runtime_error
{
MyException( char const * text )
: std::runtime_error( text ) {}
~MyException() override;
};
// prevent -Wweak-vtables:
MyException::~MyException() = default;
struct MyExceptionMatcher : Catch::MatcherBase< std::runtime_error >
{
std::string m_text;
MyExceptionMatcher( char const * text )
: m_text( text )
{}
~MyExceptionMatcher() override;
bool match( std::runtime_error const & arg ) const override
{
return m_text == arg.what() ;
}
std::string describe() const override
{
return "it's me";
}
};
// prevent -Wweak-vtables:
MyExceptionMatcher::~MyExceptionMatcher() = default;
TEST_CASE( "TeamCity failing check-throws-matches", "[teamcity]" ) {
CHECK_THROWS_MATCHES( throw MyException("hello"), MyException, MyExceptionMatcher("world") );
}
// [!throws] - lets Catch know that this test is likely to throw an exception even if successful.
// This causes the test to be excluded when running with -e or --nothrow.
// No special effects for the reporter.
TEST_CASE( "TeamCity throwing exception with tag [!throws]", "[teamcity][!throws]" ) {
REQUIRE_THROWS( throw std::runtime_error("unsurprisingly") );
}
// [!mayfail] - doesn't fail the test if any given assertion fails (but still reports it). This can be useful to flag a work-in-progress, or a known issue that you don't want to immediately fix but still want to track in your tests.
TEST_CASE( "TeamCity failing assertion with tag [!mayfail]", "[teamcity][!mayfail] " ) {
REQUIRE( 3 == 7 ); // doesn't fail test case this time, reports: testIgnored
REQUIRE( 3 == 3 );
}
// [!shouldfail] - like [!mayfail] but fails the test if it passes.
// This can be useful if you want to be notified of accidental, or third-party, fixes.
TEST_CASE( "TeamCity succeeding assertion with tag [!shouldfail]", "[teamcity][!shouldfail]" ) {
SUCCEED( "Marked [!shouldfail]" );
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_ROOT) -DCATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" -o 200-Rpt-CatchMainTeamCity.o -c 200-Rpt-CatchMain.cpp
// - g++ -std=c++11 -Wall -I$(CATCH_ROOT) -o 207-Rpt-TeamCityReporter 207-Rpt-TeamCityReporter.cpp 200-Rpt-CatchMainTeamCity.o && 207-Rpt-TeamCityReporter --list-reporters
//
// - cl -EHsc -I%CATCH_ROOT% -DCATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" -Fo200-Rpt-CatchMainTeamCity.obj -c 200-Rpt-CatchMain.cpp
// - cl -EHsc -I%CATCH_ROOT% 207-Rpt-TeamCityReporter.cpp 200-Rpt-CatchMainTeamCity.o && 207-Rpt-TeamCityReporter --list-reporters
// Compilation output (--list-reporters):
// Available reporters:
// compact: Reports test results on a single line, suitable for IDEs
// console: Reports test results as plain lines of text
// junit: Reports test results in an XML format that looks like Ant's
// junitreport target
// teamcity: Reports test results as TeamCity service messages
// xml: Reports test results as an XML document
// Expected output (abbreviated and broken into shorter lines):
//
// prompt> 207-Rpt-TeamCityReporter.exe --reporter teamcity
// ##teamcity[testSuiteStarted name='207-Rpt-TeamCityReporter.exe']
// ##teamcity[testStarted name='TeamCity passes unconditionally succeeding assertion']
// ##teamcity[testFinished name='TeamCity passes unconditionally succeeding assertion' duration='1']
// ##teamcity[testStarted name='TeamCity reports unconditionally failing assertion']
// ##teamcity[testFailed name='TeamCity reports unconditionally failing assertion' /
// message='.../examples/207-Rpt-TeamCityReporter.cpp:23|n/
// ...............................................................................|n|n/
// .../examples/207-Rpt-TeamCityReporter.cpp:25|nexplicit failure']
// ##teamcity[testFinished name='TeamCity reports unconditionally failing assertion' duration='3']
// ...

View File

@ -5,30 +5,33 @@
// 2. My listener and registration
// 3. Test cases
// main() provided in 000-CatchMain.cpp
// Let Catch provide the required interfaces:
#define CATCH_CONFIG_EXTERNAL_INTERFACES
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
#include <catch2/reporters/catch_reporter_event_listener.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
#include <catch2/catch_test_case_info.hpp>
#include <iostream>
// -----------------------------------------------------------------------
// 1. Printing of listener data:
//
namespace {
std::string ws(int const level) {
return std::string( 2 * level, ' ' );
}
std::ostream& operator<<(std::ostream& out, Catch::Tag t) {
return out << "original: " << t.original;
}
template< typename T >
std::ostream& operator<<( std::ostream& os, std::vector<T> const& v ) {
os << "{ ";
for ( auto x : v )
for ( const auto& x : v )
os << x << ", ";
return os << "}";
}
// struct SourceLineInfo {
// char const* file;
// std::size_t line;
@ -57,7 +60,7 @@ void print( std::ostream& os, int const level, Catch::MessageInfo const& info )
void print( std::ostream& os, int const level, std::string const& title, std::vector<Catch::MessageInfo> const& v ) {
os << ws(level ) << title << ":\n";
for ( auto x : v )
for ( const auto& x : v )
{
os << ws(level+1) << "{\n";
print( os, level+2, x );
@ -119,32 +122,36 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
os << ws(level+1) << "- aborting: " << info.aborting << "\n";
}
// struct TestCaseInfo {
// enum SpecialProperties{
// None = 0,
// IsHidden = 1 << 1,
// ShouldFail = 1 << 2,
// MayFail = 1 << 3,
// Throws = 1 << 4,
// NonPortable = 1 << 5,
// Benchmark = 1 << 6
// };
// struct Tag {
// StringRef original, lowerCased;
// };
//
// bool isHidden() const;
// bool throws() const;
// bool okToFail() const;
// bool expectedToFail() const;
//
// std::string tagsAsString() const;
// enum class TestCaseProperties : uint8_t {
// None = 0,
// IsHidden = 1 << 1,
// ShouldFail = 1 << 2,
// MayFail = 1 << 3,
// Throws = 1 << 4,
// NonPortable = 1 << 5,
// Benchmark = 1 << 6
// };
//
// std::string name;
// std::string className;
// std::string description;
// std::vector<std::string> tags;
// std::vector<std::string> lcaseTags;
// SourceLineInfo lineInfo;
// SpecialProperties properties;
// };
//
// struct TestCaseInfo : NonCopyable {
//
// bool isHidden() const;
// bool throws() const;
// bool okToFail() const;
// bool expectedToFail() const;
//
//
// std::string name;
// std::string className;
// std::vector<Tag> tags;
// SourceLineInfo lineInfo;
// TestCaseProperties properties = TestCaseProperties::None;
// };
void print( std::ostream& os, int const level, std::string const& title, Catch::TestCaseInfo const& info ) {
os << ws(level ) << title << ":\n"
@ -155,11 +162,9 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
<< ws(level+1) << "- tagsAsString(): '" << info.tagsAsString() << "'\n"
<< ws(level+1) << "- name: '" << info.name << "'\n"
<< ws(level+1) << "- className: '" << info.className << "'\n"
<< ws(level+1) << "- description: '" << info.description << "'\n"
<< ws(level+1) << "- tags: " << info.tags << "\n"
<< ws(level+1) << "- lcaseTags: " << info.lcaseTags << "\n";
<< ws(level+1) << "- tags: " << info.tags << "\n";
print( os, level+1 , "- lineInfo", info.lineInfo );
os << ws(level+1) << "- properties (flags): 0x" << std::hex << info.properties << std::dec << "\n";
os << ws(level+1) << "- properties (flags): 0x" << std::hex << static_cast<uint32_t>(info.properties) << std::dec << "\n";
}
// struct TestCaseStats {
@ -172,7 +177,7 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
void print( std::ostream& os, int const level, std::string const& title, Catch::TestCaseStats const& info ) {
os << ws(level ) << title << ":\n";
print( os, level+1 , "- testInfo", info.testInfo );
print( os, level+1 , "- testInfo", *info.testInfo );
print( os, level+1 , "- totals" , info.totals );
os << ws(level+1) << "- stdOut: " << info.stdOut << "\n"
<< ws(level+1) << "- stdErr: " << info.stdErr << "\n"
@ -273,8 +278,8 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
print( os, level+1 , "- getSourceInfo(): ", info.getSourceInfo() );
os << ws(level+1) << "- getTestMacroName(): '" << info.getTestMacroName() << "'\n";
// print( os, level+1 , "- *** m_info (AssertionInfo)", info.m_info );
// print( os, level+1 , "- *** m_resultData (AssertionResultData)", info.m_resultData );
print( os, level+1 , "- *** m_info (AssertionInfo)", info.m_info );
print( os, level+1 , "- *** m_resultData (AssertionResultData)", info.m_resultData );
}
// struct AssertionStats {
@ -297,15 +302,16 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
char const * dashed_line =
"--------------------------------------------------------------------------";
struct MyListener : Catch::TestEventListenerBase {
using TestEventListenerBase::TestEventListenerBase; // inherit constructor
struct MyListener : Catch::EventListenerBase {
using EventListenerBase::EventListenerBase; // inherit constructor
// Get rid of Wweak-tables
~MyListener();
~MyListener() override;
// The whole test run starting
virtual void testRunStarting( Catch::TestRunInfo const& testRunInfo ) override {
void testRunStarting( Catch::TestRunInfo const& testRunInfo ) override {
std::cout
<< std::boolalpha
<< "\nEvent: testRunStarting:\n";
@ -313,7 +319,7 @@ struct MyListener : Catch::TestEventListenerBase {
}
// The whole test run ending
virtual void testRunEnded( Catch::TestRunStats const& testRunStats ) override {
void testRunEnded( Catch::TestRunStats const& testRunStats ) override {
std::cout
<< dashed_line
<< "\nEvent: testRunEnded:\n";
@ -321,7 +327,7 @@ struct MyListener : Catch::TestEventListenerBase {
}
// A test is being skipped (because it is "hidden")
virtual void skipTest( Catch::TestCaseInfo const& testInfo ) override {
void skipTest( Catch::TestCaseInfo const& testInfo ) override {
std::cout
<< dashed_line
<< "\nEvent: skipTest:\n";
@ -329,7 +335,7 @@ struct MyListener : Catch::TestEventListenerBase {
}
// Test cases starting
virtual void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
std::cout
<< dashed_line
<< "\nEvent: testCaseStarting:\n";
@ -337,36 +343,37 @@ struct MyListener : Catch::TestEventListenerBase {
}
// Test cases ending
virtual void testCaseEnded( Catch::TestCaseStats const& testCaseStats ) override {
void testCaseEnded( Catch::TestCaseStats const& testCaseStats ) override {
std::cout << "\nEvent: testCaseEnded:\n";
print( std::cout, 1, "testCaseStats", testCaseStats );
}
// Sections starting
virtual void sectionStarting( Catch::SectionInfo const& sectionInfo ) override {
void sectionStarting( Catch::SectionInfo const& sectionInfo ) override {
std::cout << "\nEvent: sectionStarting:\n";
print( std::cout, 1, "- sectionInfo", sectionInfo );
}
// Sections ending
virtual void sectionEnded( Catch::SectionStats const& sectionStats ) override {
void sectionEnded( Catch::SectionStats const& sectionStats ) override {
std::cout << "\nEvent: sectionEnded:\n";
print( std::cout, 1, "- sectionStats", sectionStats );
}
// Assertions before/ after
virtual void assertionStarting( Catch::AssertionInfo const& assertionInfo ) override {
void assertionStarting( Catch::AssertionInfo const& assertionInfo ) override {
std::cout << "\nEvent: assertionStarting:\n";
print( std::cout, 1, "- assertionInfo", assertionInfo );
}
virtual bool assertionEnded( Catch::AssertionStats const& assertionStats ) override {
void assertionEnded( Catch::AssertionStats const& assertionStats ) override {
std::cout << "\nEvent: assertionEnded:\n";
print( std::cout, 1, "- assertionStats", assertionStats );
return true;
}
};
} // end anonymous namespace
CATCH_REGISTER_LISTENER( MyListener )
// Get rid of Wweak-tables
@ -413,8 +420,8 @@ TEST_CASE_METHOD( Fixture, "3: Testcase with class-based fixture", "[tag-C][tag-
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 210-Evt-EventListeners 210-Evt-EventListeners.cpp 000-CatchMain.o && 210-Evt-EventListeners --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 210-Evt-EventListeners.cpp 000-CatchMain.obj && 210-Evt-EventListeners --success
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 210-Evt-EventListeners 210-Evt-EventListeners.cpp && 210-Evt-EventListeners --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 210-Evt-EventListeners.cpp && 210-Evt-EventListeners --success
// Expected compact output (all assertions):
//

View File

@ -5,17 +5,17 @@
// semantic, because it buffers the output. For most uses however,
// there is no important difference between having `std::cerr` buffered
// or unbuffered.
#include <catch2/catch_test_macros.hpp>
#define CATCH_CONFIG_NOSTDOUT
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
#include <sstream>
#include <cstdio>
class out_buff : public std::stringbuf {
std::FILE* m_stream;
public:
out_buff(std::FILE* stream) :m_stream(stream) {}
~out_buff() { pubsync(); }
int sync() {
out_buff(std::FILE* stream):m_stream(stream) {}
~out_buff();
int sync() override {
int ret = 0;
for (unsigned char c : str()) {
if (putc(c, m_stream) == EOF) {
@ -29,6 +29,12 @@ public:
}
};
out_buff::~out_buff() { pubsync(); }
#if defined(__clang__)
#pragma clang diagnostic ignored "-Wexit-time-destructors" // static variables in cout/cerr/clog
#endif
namespace Catch {
std::ostream& cout() {
static std::ostream ret(new out_buff(stdout));

View File

@ -4,10 +4,14 @@
// Specifically we will implement a random number generator for integers
// It will have infinite capacity and settable lower/upper bound
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
#include <random>
namespace {
// This class shows how to implement a simple generator for Catch tests
class RandomIntGenerator : public Catch::Generators::IGenerator<int> {
std::minstd_rand m_rand;
@ -38,9 +42,15 @@ int const& RandomIntGenerator::get() const {
// Notice that it returns an instance of GeneratorWrapper<int>, which
// is a value-wrapper around std::unique_ptr<IGenerator<int>>.
Catch::Generators::GeneratorWrapper<int> random(int low, int high) {
return Catch::Generators::GeneratorWrapper<int>(std::unique_ptr<Catch::Generators::IGenerator<int>>(new RandomIntGenerator(low, high)));
return Catch::Generators::GeneratorWrapper<int>(
new RandomIntGenerator(low, high)
// Another possibility:
// Catch::Detail::make_unique<RandomIntGenerator>(low, high)
);
}
} // end anonymous namespaces
// The two sections in this test case are equivalent, but the first one
// is much more readable/nicer to use
TEST_CASE("Generating random ints", "[example][generator]") {
@ -50,7 +60,7 @@ TEST_CASE("Generating random ints", "[example][generator]") {
REQUIRE(i <= 100);
}
SECTION("Creating the random generator directly") {
auto i = GENERATE(take(100, GeneratorWrapper<int>(std::unique_ptr<IGenerator<int>>(new RandomIntGenerator(-100, 100)))));
auto i = GENERATE(take(100, GeneratorWrapper<int>(Catch::Detail::make_unique<RandomIntGenerator>(-100, 100))));
REQUIRE(i >= -100);
REQUIRE(i <= 100);
}

View File

@ -0,0 +1,60 @@
// 301-Gen-MapTypeConversion.cpp
// Shows how to use map to modify generator's return type.
// Specifically we wrap a std::string returning generator with a generator
// that converts the strings using stoi, so the returned type is actually
// an int.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
#include <string>
#include <sstream>
namespace {
// Returns a line from a stream. You could have it e.g. read lines from
// a file, but to avoid problems with paths in examples, we will use
// a fixed stringstream.
class LineGenerator : public Catch::Generators::IGenerator<std::string> {
std::string m_line;
std::stringstream m_stream;
public:
LineGenerator() {
m_stream.str("1\n2\n3\n4\n");
if (!next()) {
Catch::Generators::Detail::throw_generator_exception("Couldn't read a single line");
}
}
std::string const& get() const override;
bool next() override {
return !!std::getline(m_stream, m_line);
}
};
std::string const& LineGenerator::get() const {
return m_line;
}
// This helper function provides a nicer UX when instantiating the generator
// Notice that it returns an instance of GeneratorWrapper<std::string>, which
// is a value-wrapper around std::unique_ptr<IGenerator<std::string>>.
Catch::Generators::GeneratorWrapper<std::string> lines(std::string /* ignored for example */) {
return Catch::Generators::GeneratorWrapper<std::string>(
new LineGenerator()
);
}
} // end anonymous namespace
TEST_CASE("filter can convert types inside the generator expression", "[example][generator]") {
auto num = GENERATE(map<int>([](std::string const& line) { return std::stoi(line); },
lines("fake-file")));
REQUIRE(num > 0);
}
// Compiling and running this file will result in 4 successful assertions

View File

@ -0,0 +1,55 @@
// 302-Gen-Table.cpp
// Shows how to use table to run a test many times with different inputs. Lifted from examples on
// issue #850.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators.hpp>
#include <string>
struct TestSubject {
// this is the method we are going to test. It returns the length of the
// input string.
size_t GetLength( const std::string& input ) const { return input.size(); }
};
TEST_CASE("Table allows pre-computed test inputs and outputs", "[example][generator]") {
using std::make_tuple;
// do setup here as normal
TestSubject subj;
SECTION("This section is run for each row in the table") {
std::string test_input;
size_t expected_output;
std::tie( test_input, expected_output ) =
GENERATE( table<std::string, size_t>(
{ /* In this case one of the parameters to our test case is the
* expected output, but this is not required. There could be
* multiple expected values in the table, which can have any
* (fixed) number of columns.
*/
make_tuple( "one", 3 ),
make_tuple( "two", 3 ),
make_tuple( "three", 5 ),
make_tuple( "four", 4 ) } ) );
// run the test
auto result = subj.GetLength(test_input);
// capture the input data to go with the outputs.
CAPTURE(test_input);
// check it matches the pre-calculated data
REQUIRE(result == expected_output);
} // end section
}
/* Possible simplifications where less legacy toolchain support is needed:
*
* - With libstdc++6 or newer, the make_tuple() calls can be ommitted
* (technically C++17 but does not require -std in GCC/Clang). See
* https://stackoverflow.com/questions/12436586/tuple-vector-and-initializer-list
*
* - In C++17 mode std::tie() and the preceding variable delcarations can be
* replaced by structured bindings: auto [test_input, expected] = GENERATE(
* table<std::string, size_t>({ ...
*/
// Compiling and running this file will result in 4 successful assertions

View File

@ -6,42 +6,9 @@
// _WILL_ outlive the variables -- thus they should be either captured
// by value directly, or copied by the generators during construction.
#include <catch2/catch.hpp>
#include <random>
// Lets start by implementing a parametrizable double generator
class RandomDoubleGenerator : public Catch::Generators::IGenerator<double> {
std::minstd_rand m_rand;
std::uniform_real_distribution<> m_dist;
double current_number;
public:
RandomDoubleGenerator(double low, double high):
m_rand(std::random_device{}()),
m_dist(low, high)
{
static_cast<void>(next());
}
double const& get() const override;
bool next() override {
current_number = m_dist(m_rand);
return true;
}
};
// Avoids -Wweak-vtables
double const& RandomDoubleGenerator::get() const {
return current_number;
}
// Also provide a nice shortcut for creating the generator
Catch::Generators::GeneratorWrapper<double> random(double low, double high) {
return Catch::Generators::GeneratorWrapper<double>(std::unique_ptr<Catch::Generators::IGenerator<double>>(new RandomDoubleGenerator(low, high)));
}
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
#include <catch2/generators/catch_generators_random.hpp>
TEST_CASE("Generate random doubles across different ranges",
"[generator][example][advanced]") {
@ -55,16 +22,12 @@ TEST_CASE("Generate random doubles across different ranges",
}));
// This will not compile (intentionally), because it accesses a variable
// auto number = GENERATE(take(50, random(r.first, r.second)));
// We have to manually register the generators instead
// Notice that we are using value capture in the lambda, to avoid lifetime issues
auto number = Catch::Generators::generate( CATCH_INTERNAL_LINEINFO,
[=]{
using namespace Catch::Generators;
return makeGenerators(take(50, random(std::get<0>(r), std::get<1>(r))));
}
);
// auto number = GENERATE(take(50, random(std::get<0>(r), std::get<1>(r))));
// GENERATE_COPY copies all variables mentioned inside the expression
// thus this will work.
auto number = GENERATE_COPY(take(50, random(std::get<0>(r), std::get<1>(r))));
REQUIRE(std::abs(number) > 0);
}

View File

@ -0,0 +1,43 @@
// 311-Gen-CustomCapture.cpp
// Shows how to provide custom capture list to the generator expression
// Note that using variables inside generators is dangerous and should
// be done only if you know what you are doing, because the generators
// _WILL_ outlive the variables. Also, even if you know what you are
// doing, you should probably use GENERATE_COPY or GENERATE_REF macros
// instead. However, if your use case requires having a
// per-variable custom capture list, this example shows how to achieve
// that.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
#include <catch2/generators/catch_generators_random.hpp>
TEST_CASE("Generate random doubles across different ranges",
"[generator][example][advanced]") {
// Workaround for old libstdc++
using record = std::tuple<double, double>;
// Set up 3 ranges to generate numbers from
auto r1 = GENERATE(table<double, double>({
record{3, 4},
record{-4, -3},
record{10, 1000}
}));
auto r2(r1);
// This will take r1 by reference and r2 by value.
// Note that there are no advantages for doing so in this example,
// it is done only for expository purposes.
auto number = Catch::Generators::generate( "custom capture generator", CATCH_INTERNAL_LINEINFO,
[&r1, r2]{
using namespace Catch::Generators;
return makeGenerators(take(50, random(std::get<0>(r1), std::get<1>(r2))));
}
);
REQUIRE(std::abs(number) > 0);
}
// Compiling and running this file will result in 150 successful assertions

View File

@ -1,155 +1,63 @@
#
# Build examples.
#
# Requires CATCH_BUILD_EXAMPLES to be defined 'true', see ../CMakeLists.txt.
#
cmake_minimum_required( VERSION 3.5 )
cmake_minimum_required( VERSION 3.0 )
project( CatchExamples CXX )
project( Catch2Examples LANGUAGES CXX )
message( STATUS "Examples included" )
# define folders used:
set( EXAMPLES_DIR ${CATCH_DIR}/examples )
set( HEADER_DIR ${CATCH_DIR}/single_include )
set( REPORTER_HEADER_DIR ${CATCH_DIR}/include/reporters )
# Some one-offs first:
# 1) Tests and main in one file
add_executable( 010-TestCase
010-TestCase.cpp
)
# single-file sources:
# 2) Tests and main across two files
add_executable( 020-MultiFile
020-TestCase-1.cpp
020-TestCase-2.cpp
)
set( SOURCES_SINGLE_FILE
010-TestCase.cpp
add_executable(231-Cfg_OutputStreams
231-Cfg-OutputStreams.cpp
)
target_link_libraries(231-Cfg_OutputStreams Catch2_buildall_interface)
target_compile_definitions(231-Cfg_OutputStreams PUBLIC CATCH_CONFIG_NOSTDOUT)
# multiple-file modules:
set( SOURCES_020
020-TestCase-1.cpp
020-TestCase-2.cpp
)
# main for idiomatic test sources:
set( SOURCES_IDIOMATIC_MAIN
000-CatchMain.cpp
)
# sources to combine with 000-CatchMain.cpp:
set( SOURCES_IDIOMATIC_TESTS
# These examples use the standard separate compilation
set( SOURCES_IDIOMATIC_EXAMPLES
030-Asn-Require-Check.cpp
100-Fix-Section.cpp
110-Fix-ClassFixture.cpp
120-Bdd-ScenarioGivenWhenThen.cpp
210-Evt-EventListeners.cpp
300-Gen-OwnGenerator.cpp
301-Gen-MapTypeConversion.cpp
302-Gen-Table.cpp
310-Gen-VariablesInGenerators.cpp
311-Gen-CustomCapture.cpp
)
# main-s for reporter-specific test sources:
string( REPLACE ".cpp" "" BASENAMES_IDIOMATIC_EXAMPLES "${SOURCES_IDIOMATIC_EXAMPLES}" )
set( TARGETS_IDIOMATIC_EXAMPLES ${BASENAMES_IDIOMATIC_EXAMPLES} )
set( SOURCES_REPORTERS_MAIN
200-Rpt-CatchMain.cpp
)
string( REPLACE ".cpp" "" BASENAMES_REPORTERS_MAIN 200-Rpt-CatchMain.cpp )
set( NAMES_REPORTERS TeamCity )
foreach( reporter ${NAMES_REPORTERS} )
list( APPEND SOURCES_SPECIFIC_REPORTERS_MAIN ${BASENAMES_REPORTERS_MAIN}${reporter}.cpp )
foreach( name ${TARGETS_IDIOMATIC_EXAMPLES} )
add_executable( ${name}
${EXAMPLES_DIR}/${name}.cpp )
endforeach()
# sources to combine with 200-Rpt-CatchMain{Reporter}.cpp:
set( SOURCES_REPORTERS_TESTS
207-Rpt-TeamCityReporter.cpp
set(ALL_EXAMPLE_TARGETS
${TARGETS_IDIOMATIC_EXAMPLES}
010-TestCase
020-MultiFile
)
# check if all sources are listed, warn if not:
set( SOURCES_ALL
${SOURCES_020}
${SOURCES_SINGLE_FILE}
${SOURCES_IDIOMATIC_MAIN}
${SOURCES_IDIOMATIC_TESTS}
${SOURCES_REPORTERS_MAIN}
${SOURCES_REPORTERS_TESTS}
)
foreach( name ${SOURCES_ALL} )
list( APPEND SOURCES_ALL_PATH ${EXAMPLES_DIR}/${name} )
endforeach()
CheckFileList( SOURCES_ALL_PATH ${EXAMPLES_DIR} )
# create target names:
string( REPLACE ".cpp" "" BASENAMES_SINGLE_FILE "${SOURCES_SINGLE_FILE}" )
string( REPLACE ".cpp" "" BASENAMES_IDIOMATIC_TESTS "${SOURCES_IDIOMATIC_TESTS}" )
string( REPLACE ".cpp" "" BASENAMES_REPORTERS_TESTS "${SOURCES_REPORTERS_TESTS}" )
string( REPLACE ".cpp" "" BASENAMES_REPORTERS_MAIN "${SOURCES_REPORTERS_MAIN}" )
set( TARGETS_SINGLE_FILE ${BASENAMES_SINGLE_FILE} )
set( TARGETS_IDIOMATIC_TESTS ${BASENAMES_IDIOMATIC_TESTS} )
set( TARGETS_REPORTERS_TESTS ${BASENAMES_REPORTERS_TESTS} )
set( TARGETS_REPORTERS_MAIN ${BASENAMES_REPORTERS_MAIN} )
set( TARGETS_ALL
${TARGETS_SINGLE_FILE}
020-TestCase
${TARGETS_IDIOMATIC_TESTS} CatchMain
${TARGETS_REPORTERS_TESTS} CatchMainTeamCity
)
# define program targets:
add_library( CatchMain OBJECT ${EXAMPLES_DIR}/${SOURCES_IDIOMATIC_MAIN} ${HEADER_DIR}/catch2/catch.hpp )
#add_library( CatchMainAutomake OBJECT ${EXAMPLES_DIR}/200-Rpt-CatchMain.cpp ${HEADER_DIR}/catch2/catch.hpp )
#add_library( CatchMainTap OBJECT ${EXAMPLES_DIR}/200-Rpt-CatchMain.cpp ${HEADER_DIR}/catch2/catch.hpp )
add_library( CatchMainTeamCity OBJECT ${EXAMPLES_DIR}/200-Rpt-CatchMain.cpp ${HEADER_DIR}/catch2/catch.hpp )
#target_compile_definitions( CatchMainAutomake PRIVATE CATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_automake.hpp\" )
#target_compile_definitions( CatchMainTap PRIVATE CATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_tap.hpp\" )
target_compile_definitions( CatchMainTeamCity PRIVATE CATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" )
foreach( name ${TARGETS_SINGLE_FILE} )
add_executable( ${name} ${EXAMPLES_DIR}/${name}.cpp ${HEADER_DIR}/catch2/catch.hpp )
endforeach()
foreach( name ${TARGETS_IDIOMATIC_TESTS} )
add_executable( ${name} ${EXAMPLES_DIR}/${name}.cpp $<TARGET_OBJECTS:CatchMain> ${HEADER_DIR}/catch2/catch.hpp )
endforeach()
add_executable( 020-TestCase ${EXAMPLES_DIR}/020-TestCase-1.cpp ${EXAMPLES_DIR}/020-TestCase-2.cpp ${HEADER_DIR}/catch2/catch.hpp )
#add_executable( 207-Rpt-AutomakeReporter ${EXAMPLES_DIR}/207-Rpt-AutomakeReporter.cpp $<TARGET_OBJECTS:CatchMainAutomake> ${HEADER_DIR}/catch2/catch.hpp )
#add_executable( 207-Rpt-TapReporter ${EXAMPLES_DIR}/207-Rpt-TapReporter.cpp $<TARGET_OBJECTS:CatchMainTap> ${HEADER_DIR}/catch2/catch.hpp )
add_executable( 207-Rpt-TeamCityReporter ${EXAMPLES_DIR}/207-Rpt-TeamCityReporter.cpp $<TARGET_OBJECTS:CatchMainTeamCity> ${HEADER_DIR}/catch2/catch.hpp )
#foreach( name ${TARGETS_REPORTERS_TESTS} )
# add_executable( ${name} ${EXAMPLES_DIR}/${name}.cpp $<TARGET_OBJECTS:CatchMain> ${HEADER_DIR}/catch2/catch.hpp )
#endforeach()
foreach( name ${TARGETS_ALL} )
target_include_directories( ${name} PRIVATE ${HEADER_DIR} ${CATCH_DIR} )
set_property(TARGET ${name} PROPERTY CXX_STANDARD 11)
foreach( name ${ALL_EXAMPLE_TARGETS} )
target_link_libraries( ${name} Catch2 Catch2WithMain )
set_property(TARGET ${name} PROPERTY CXX_STANDARD 14)
set_property(TARGET ${name} PROPERTY CXX_EXTENSIONS OFF)
# Add desired warnings
if ( CMAKE_CXX_COMPILER_ID MATCHES "Clang|AppleClang|GNU" )
target_compile_options( ${name} PRIVATE -Wall -Wextra -Wunreachable-code )
endif()
# Clang specific warning go here
if ( CMAKE_CXX_COMPILER_ID MATCHES "Clang" )
# Actually keep these
target_compile_options( ${name} PRIVATE -Wweak-vtables -Wexit-time-destructors -Wglobal-constructors -Wmissing-noreturn )
endif()
if ( CMAKE_CXX_COMPILER_ID MATCHES "MSVC" )
target_compile_options( ${name} PRIVATE /W4 /w44265 /WX )
endif()
endforeach()
list(APPEND CATCH_WARNING_TARGETS ${ALL_EXAMPLE_TARGETS})
set(CATCH_WARNING_TARGETS ${CATCH_WARNING_TARGETS} PARENT_SCOPE)

View File

@ -33,6 +33,10 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
[TEST_SUFFIX suffix]
[PROPERTIES name1 value1...]
[TEST_LIST var]
[REPORTER reporter]
[OUTPUT_DIR dir]
[OUTPUT_PREFIX prefix}
[OUTPUT_SUFFIX suffix]
)
``catch_discover_tests`` sets up a post-build command on the test executable
@ -90,6 +94,35 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
executable is being used in multiple calls to ``catch_discover_tests()``.
Note that this variable is only available in CTest.
``REPORTER reporter``
Use the specified reporter when running the test case. The reporter will
be passed to the Catch executable as ``--reporter reporter``.
``OUTPUT_DIR dir``
If specified, the parameter is passed along as
``--out dir/<test_name>`` to Catch executable. The actual file name is the
same as the test name. This should be used instead of
``EXTRA_ARGS --out foo`` to avoid race conditions writing the result output
when using parallel test execution.
``OUTPUT_PREFIX prefix``
May be used in conjunction with ``OUTPUT_DIR``.
If specified, ``prefix`` is added to each output file name, like so
``--out dir/prefix<test_name>``.
``OUTPUT_SUFFIX suffix``
May be used in conjunction with ``OUTPUT_DIR``.
If specified, ``suffix`` is added to each output file name, like so
``--out dir/<test_name>suffix``. This can be used to add a file extension to
the output e.g. ".xml".
``DL_PATHS path...``
Specifies paths that need to be set for the dynamic linker to find shared
libraries/DLLs when running the test executable (PATH/LD_LIBRARY_PATH respectively).
These paths will both be set when retrieving the list of test cases from the
test executable and when the tests are executed themselves. This requires
cmake/ctest >= 3.22.
#]=======================================================================]
#------------------------------------------------------------------------------
@ -97,8 +130,8 @@ function(catch_discover_tests TARGET)
cmake_parse_arguments(
""
""
"TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST"
"TEST_SPEC;EXTRA_ARGS;PROPERTIES"
"TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;REPORTER;OUTPUT_DIR;OUTPUT_PREFIX;OUTPUT_SUFFIX"
"TEST_SPEC;EXTRA_ARGS;PROPERTIES;DL_PATHS"
${ARGN}
)
@ -109,8 +142,14 @@ function(catch_discover_tests TARGET)
set(_TEST_LIST ${TARGET}_TESTS)
endif()
if (_DL_PATHS)
if(${CMAKE_VERSION} VERSION_LESS "3.22.0")
message(FATAL_ERROR "The DL_PATHS option requires at least cmake 3.22")
endif()
endif()
## Generate a unique name based on the extra arguments
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS}")
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS} ${_REPORTER} ${_OUTPUT_DIR} ${_OUTPUT_PREFIX} ${_OUTPUT_SUFFIX}")
string(SUBSTRING ${args_hash} 0 7 args_hash)
# Define rule to generate test list for aforementioned test executable
@ -134,6 +173,11 @@ function(catch_discover_tests TARGET)
-D "TEST_PREFIX=${_TEST_PREFIX}"
-D "TEST_SUFFIX=${_TEST_SUFFIX}"
-D "TEST_LIST=${_TEST_LIST}"
-D "TEST_REPORTER=${_REPORTER}"
-D "TEST_OUTPUT_DIR=${_OUTPUT_DIR}"
-D "TEST_OUTPUT_PREFIX=${_OUTPUT_PREFIX}"
-D "TEST_OUTPUT_SUFFIX=${_OUTPUT_SUFFIX}"
-D "TEST_DL_PATHS=${_DL_PATHS}"
-D "CTEST_FILE=${ctest_tests_file}"
-P "${_CATCH_DISCOVER_TESTS_SCRIPT}"
VERBATIM
@ -172,4 +216,5 @@ endfunction()
set(_CATCH_DISCOVER_TESTS_SCRIPT
${CMAKE_CURRENT_LIST_DIR}/CatchAddTests.cmake
CACHE INTERNAL "Catch2 full path to CatchAddTests.cmake helper file"
)

154
extras/CatchAddTests.cmake Normal file
View File

@ -0,0 +1,154 @@
# Distributed under the OSI-approved BSD 3-Clause License. See accompanying
# file Copyright.txt or https://cmake.org/licensing for details.
set(prefix "${TEST_PREFIX}")
set(suffix "${TEST_SUFFIX}")
set(spec ${TEST_SPEC})
set(extra_args ${TEST_EXTRA_ARGS})
set(properties ${TEST_PROPERTIES})
set(reporter ${TEST_REPORTER})
set(output_dir ${TEST_OUTPUT_DIR})
set(output_prefix ${TEST_OUTPUT_PREFIX})
set(output_suffix ${TEST_OUTPUT_SUFFIX})
set(dl_paths ${TEST_DL_PATHS})
set(script)
set(suite)
set(tests)
if(WIN32)
set(dl_paths_variable_name PATH)
else()
set(dl_paths_variable_name LD_LIBRARY_PATH)
endif()
function(add_command NAME)
set(_args "")
# use ARGV* instead of ARGN, because ARGN splits arrays into multiple arguments
math(EXPR _last_arg ${ARGC}-1)
foreach(_n RANGE 1 ${_last_arg})
set(_arg "${ARGV${_n}}")
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]") # form a bracket_argument
else()
set(_args "${_args} ${_arg}")
endif()
endforeach()
set(script "${script}${NAME}(${_args})\n" PARENT_SCOPE)
endfunction()
# Run test executable to get list of available tests
if(NOT EXISTS "${TEST_EXECUTABLE}")
message(FATAL_ERROR
"Specified test executable '${TEST_EXECUTABLE}' does not exist"
)
endif()
if(dl_paths)
cmake_path(CONVERT "${dl_paths}" TO_NATIVE_PATH_LIST paths)
set(ENV{${dl_paths_variable_name}} "${paths}")
endif()
execute_process(
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-tests --verbosity quiet
OUTPUT_VARIABLE output
RESULT_VARIABLE result
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
)
if(NOT ${result} EQUAL 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${result}\n"
" Output: ${output}\n"
)
endif()
string(REPLACE "\n" ";" output "${output}")
# Run test executable to get list of available reporters
execute_process(
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-reporters
OUTPUT_VARIABLE reporters_output
RESULT_VARIABLE reporters_result
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
)
if(NOT ${reporters_result} EQUAL 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${reporters_result}\n"
" Output: ${reporters_output}\n"
)
endif()
string(FIND "${reporters_output}" "${reporter}" reporter_is_valid)
if(reporter AND ${reporter_is_valid} EQUAL -1)
message(FATAL_ERROR
"\"${reporter}\" is not a valid reporter!\n"
)
endif()
# Prepare reporter
if(reporter)
set(reporter_arg "--reporter ${reporter}")
endif()
# Prepare output dir
if(output_dir AND NOT IS_ABSOLUTE ${output_dir})
set(output_dir "${TEST_WORKING_DIR}/${output_dir}")
if(NOT EXISTS ${output_dir})
file(MAKE_DIRECTORY ${output_dir})
endif()
endif()
if(dl_paths)
foreach(path ${dl_paths})
cmake_path(NATIVE_PATH path native_path)
list(APPEND environment_modifications "${dl_paths_variable_name}=path_list_prepend:${native_path}")
endforeach()
endif()
# Parse output
foreach(line ${output})
set(test ${line})
# Escape characters in test case names that would be parsed by Catch2
set(test_name ${test})
foreach(char , [ ])
string(REPLACE ${char} "\\${char}" test_name ${test_name})
endforeach(char)
# ...add output dir
if(output_dir)
string(REGEX REPLACE "[^A-Za-z0-9_]" "_" test_name_clean ${test_name})
set(output_dir_arg "--out ${output_dir}/${output_prefix}${test_name_clean}${output_suffix}")
endif()
# ...and add to script
add_command(add_test
"${prefix}${test}${suffix}"
${TEST_EXECUTOR}
"${TEST_EXECUTABLE}"
"${test_name}"
${extra_args}
"${reporter_arg}"
"${output_dir_arg}"
)
add_command(set_tests_properties
"${prefix}${test}${suffix}"
PROPERTIES
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
${properties}
)
if(environment_modifications)
add_command(set_tests_properties
"${prefix}${test}${suffix}"
PROPERTIES
ENVIRONMENT_MODIFICATION "${environment_modifications}")
endif()
list(APPEND tests "${prefix}${test}${suffix}")
endforeach()
# Create a list of all discovered tests, which users may use to e.g. set
# properties on the tests
add_command(set ${TEST_LIST} ${tests})
# Write CTest script
file(WRITE "${CTEST_FILE}" "${script}")

View File

@ -0,0 +1,66 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
# Supported optional args:
# * SHARD_COUNT - number of shards to split target's tests into
# * REPORTER - reporter spec to use for tests
# * TEST_SPEC - test spec used for filtering tests
function(catch_add_sharded_tests TARGET)
if (${CMAKE_VERSION} VERSION_LESS "3.10.0")
message(FATAL_ERROR "add_sharded_catch_tests only supports CMake versions 3.10.0 and up")
endif()
cmake_parse_arguments(
""
""
"SHARD_COUNT;REPORTER;TEST_SPEC"
""
${ARGN}
)
if (NOT DEFINED _SHARD_COUNT)
set(_SHARD_COUNT 2)
endif()
# Generate a unique name based on the extra arguments
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS} ${_REPORTER} ${_OUTPUT_DIR} ${_OUTPUT_PREFIX} ${_OUTPUT_SUFFIX} ${_SHARD_COUNT}")
string(SUBSTRING ${args_hash} 0 7 args_hash)
set(ctest_include_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}-sharded-tests-include-${args_hash}.cmake")
set(ctest_tests_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}-sharded-tests-impl-${args_hash}.cmake")
file(WRITE "${ctest_include_file}"
"if(EXISTS \"${ctest_tests_file}\")\n"
" include(\"${ctest_tests_file}\")\n"
"else()\n"
" add_test(${TARGET}_NOT_BUILT-${args_hash} ${TARGET}_NOT_BUILT-${args_hash})\n"
"endif()\n"
)
set_property(DIRECTORY
APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
)
set(shard_impl_script_file "${CMAKE_CURRENT_LIST_DIR}/CatchShardTestsImpl.cmake")
add_custom_command(
TARGET ${TARGET} POST_BUILD
BYPRODUCTS "${ctest_tests_file}"
COMMAND "${CMAKE_COMMAND}"
-D "TARGET_NAME=${TARGET}"
-D "TEST_BINARY=$<TARGET_FILE:${TARGET}>"
-D "CTEST_FILE=${ctest_tests_file}"
-D "SHARD_COUNT=${_SHARD_COUNT}"
-D "REPORTER_SPEC=${_REPORTER}"
-D "TEST_SPEC=${_TEST_SPEC}"
-P "${shard_impl_script_file}"
VERBATIM
)
endfunction()

View File

@ -0,0 +1,52 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
# Indirection for CatchShardTests that allows us to delay the script
# file generation until build time.
# Expected args:
# * TEST_BINARY - full path to the test binary to run sharded
# * CTEST_FILE - full path to ctest script file to write to
# * TARGET_NAME - name of the target to shard (used for test names)
# * SHARD_COUNT - number of shards to split the binary into
# Optional args:
# * REPORTER_SPEC - reporter specs to be passed down to the binary
# * TEST_SPEC - test spec to pass down to the test binary
if(NOT EXISTS "${TEST_BINARY}")
message(FATAL_ERROR
"Specified test binary '${TEST_BINARY}' does not exist"
)
endif()
set(other_args "")
if (TEST_SPEC)
set(other_args "${other_args} ${TEST_SPEC}")
endif()
if (REPORTER_SPEC)
set(other_args "${other_args} --reporter ${REPORTER_SPEC}")
endif()
# foreach RANGE in cmake is inclusive of the end, so we have to adjust it
math(EXPR adjusted_shard_count "${SHARD_COUNT} - 1")
file(WRITE "${CTEST_FILE}"
"string(RANDOM LENGTH 8 ALPHABET \"0123456789abcdef\" rng_seed)\n"
"\n"
"foreach(shard_idx RANGE ${adjusted_shard_count})\n"
" add_test(${TARGET_NAME}-shard-" [[${shard_idx}]] "/${adjusted_shard_count}\n"
" ${TEST_BINARY}"
" --shard-index " [[${shard_idx}]]
" --shard-count ${SHARD_COUNT}"
" --rng-seed " [[0x${rng_seed}]]
" --order rand"
"${other_args}"
"\n"
" )\n"
"endforeach()\n"
)

View File

@ -1,9 +1,11 @@
#==================================================================================================#
# supported macros #
# - TEST_CASE, #
# - TEMPLATE_TEST_CASE #
# - SCENARIO, #
# - TEST_CASE_METHOD, #
# - CATCH_TEST_CASE, #
# - CATCH_TEMPLATE_TEST_CASE #
# - CATCH_SCENARIO, #
# - CATCH_TEST_CASE_METHOD. #
# #
@ -44,9 +46,19 @@
# set(OptionalCatchTestLauncher ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} ${NUMPROC}) #
# just before calling this ParseAndAddCatchTests function #
# #
# The AdditionalCatchParameters optional variable can be used to pass extra argument to the test #
# command. For example, to include successful tests in the output, one can write #
# set(AdditionalCatchParameters --success) #
# #
# After the script, the ParseAndAddCatchTests_TESTS property for the target, and for each source #
# file in the target is set, and contains the list of the tests extracted from that target, or #
# from that file. This is useful, for example to add further labels or properties to the tests. #
# #
#==================================================================================================#
cmake_minimum_required(VERSION 2.8.8)
if (CMAKE_MINIMUM_REQUIRED_VERSION VERSION_LESS 2.8.8)
message(FATAL_ERROR "ParseAndAddCatchTests requires CMake 2.8.8 or newer")
endif()
option(PARSE_CATCH_TESTS_VERBOSE "Print Catch to CTest parser debug messages" OFF)
option(PARSE_CATCH_TESTS_NO_HIDDEN_TESTS "Exclude tests with [!hide], [.] or [.foo] tags" OFF)
@ -54,7 +66,7 @@ option(PARSE_CATCH_TESTS_ADD_FIXTURE_IN_TEST_NAME "Add fixture class name to the
option(PARSE_CATCH_TESTS_ADD_TARGET_IN_TEST_NAME "Add target name to the test name" ON)
option(PARSE_CATCH_TESTS_ADD_TO_CONFIGURE_DEPENDS "Add test file to CMAKE_CONFIGURE_DEPENDS property" OFF)
function(PrintDebugMessage)
function(ParseAndAddCatchTests_PrintDebugMessage)
if(PARSE_CATCH_TESTS_VERBOSE)
message(STATUS "ParseAndAddCatchTests: ${ARGV}")
endif()
@ -65,7 +77,7 @@ endfunction()
# - full line comments (i.e. // ... )
# contents have been read into '${CppCode}'.
# !keep partial line comments
function(RemoveComments CppCode)
function(ParseAndAddCatchTests_RemoveComments CppCode)
string(ASCII 2 CMakeBeginBlockComment)
string(ASCII 3 CMakeEndBlockComment)
string(REGEX REPLACE "/\\*" "${CMakeBeginBlockComment}" ${CppCode} "${${CppCode}}")
@ -77,24 +89,30 @@ function(RemoveComments CppCode)
endfunction()
# Worker function
function(ParseFile SourceFile TestTarget)
function(ParseAndAddCatchTests_ParseFile SourceFile TestTarget)
# If SourceFile is an object library, do not scan it (as it is not a file). Exit without giving a warning about a missing file.
if(SourceFile MATCHES "\\\$<TARGET_OBJECTS:.+>")
ParseAndAddCatchTests_PrintDebugMessage("Detected OBJECT library: ${SourceFile} this will not be scanned for tests.")
return()
endif()
# According to CMake docs EXISTS behavior is well-defined only for full paths.
get_filename_component(SourceFile ${SourceFile} ABSOLUTE)
if(NOT EXISTS ${SourceFile})
message(WARNING "Cannot find source file: ${SourceFile}")
return()
endif()
PrintDebugMessage("parsing ${SourceFile}")
ParseAndAddCatchTests_PrintDebugMessage("parsing ${SourceFile}")
file(STRINGS ${SourceFile} Contents NEWLINE_CONSUME)
# Remove block and fullline comments
RemoveComments(Contents)
ParseAndAddCatchTests_RemoveComments(Contents)
# Find definition of test names
string(REGEX MATCHALL "[ \t]*(CATCH_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)[ \t]*\\([^\)]+\\)+[ \t\n]*{+[ \t]*(//[^\n]*[Tt][Ii][Mm][Ee][Oo][Uu][Tt][ \t]*[0-9]+)*" Tests "${Contents}")
# https://regex101.com/r/JygOND/1
string(REGEX MATCHALL "[ \t]*(CATCH_)?(TEMPLATE_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)[ \t]*\\([ \t\n]*\"[^\"]*\"[ \t\n]*(,[ \t\n]*\"[^\"]*\")?(,[ \t\n]*[^\,\)]*)*\\)[ \t\n]*\{+[ \t]*(//[^\n]*[Tt][Ii][Mm][Ee][Oo][Uu][Tt][ \t]*[0-9]+)*" Tests "${Contents}")
if(PARSE_CATCH_TESTS_ADD_TO_CONFIGURE_DEPENDS AND Tests)
PrintDebugMessage("Adding ${SourceFile} to CMAKE_CONFIGURE_DEPENDS property")
ParseAndAddCatchTests_PrintDebugMessage("Adding ${SourceFile} to CMAKE_CONFIGURE_DEPENDS property")
set_property(
DIRECTORY
APPEND
@ -102,13 +120,21 @@ function(ParseFile SourceFile TestTarget)
)
endif()
# check CMP0110 policy for new add_test() behavior
if(POLICY CMP0110)
cmake_policy(GET CMP0110 _cmp0110_value) # new add_test() behavior
else()
# just to be thorough explicitly set the variable
set(_cmp0110_value)
endif()
foreach(TestName ${Tests})
# Strip newlines
string(REGEX REPLACE "\\\\\n|\n" "" TestName "${TestName}")
# Get test type and fixture if applicable
string(REGEX MATCH "(CATCH_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)[ \t]*\\([^,^\"]*" TestTypeAndFixture "${TestName}")
string(REGEX MATCH "(CATCH_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)" TestType "${TestTypeAndFixture}")
string(REGEX MATCH "(CATCH_)?(TEMPLATE_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)[ \t]*\\([^,^\"]*" TestTypeAndFixture "${TestName}")
string(REGEX MATCH "(CATCH_)?(TEMPLATE_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)" TestType "${TestTypeAndFixture}")
string(REGEX REPLACE "${TestType}\\([ \t]*" "" TestFixture "${TestTypeAndFixture}")
# Get string parts of test definition
@ -129,7 +155,7 @@ function(ParseFile SourceFile TestTarget)
if("${TestType}" STREQUAL "SCENARIO")
set(Name "Scenario: ${Name}")
endif()
if(PARSE_CATCH_TESTS_ADD_FIXTURE_IN_TEST_NAME AND TestFixture)
if(PARSE_CATCH_TESTS_ADD_FIXTURE_IN_TEST_NAME AND "${TestType}" MATCHES "(CATCH_)?TEST_CASE_METHOD" AND TestFixture )
set(CTestName "${TestFixture}:${Name}")
else()
set(CTestName "${Name}")
@ -155,7 +181,6 @@ function(ParseFile SourceFile TestTarget)
list(APPEND Labels ${Tags})
list(FIND Labels "!hide" IndexOfHideLabel)
set(HiddenTagFound OFF)
foreach(label ${Labels})
string(REGEX MATCH "^!hide|^\\." result ${label})
@ -165,23 +190,49 @@ function(ParseFile SourceFile TestTarget)
endif(result)
endforeach(label)
if(PARSE_CATCH_TESTS_NO_HIDDEN_TESTS AND ${HiddenTagFound} AND ${CMAKE_VERSION} VERSION_LESS "3.9")
PrintDebugMessage("Skipping test \"${CTestName}\" as it has [!hide], [.] or [.foo] label")
ParseAndAddCatchTests_PrintDebugMessage("Skipping test \"${CTestName}\" as it has [!hide], [.] or [.foo] label")
else()
PrintDebugMessage("Adding test \"${CTestName}\"")
ParseAndAddCatchTests_PrintDebugMessage("Adding test \"${CTestName}\"")
if(Labels)
PrintDebugMessage("Setting labels to ${Labels}")
ParseAndAddCatchTests_PrintDebugMessage("Setting labels to ${Labels}")
endif()
# Escape commas in the test spec
string(REPLACE "," "\\," Name ${Name})
# Work around CMake 3.18.0 change in `add_test()`, before the escaped quotes were necessary,
# only with CMake 3.18.0 the escaped double quotes confuse the call. This change is reverted in 3.18.1
# And properly introduced in 3.19 with the CMP0110 policy
if(_cmp0110_value STREQUAL "NEW" OR ${CMAKE_VERSION} VERSION_EQUAL "3.18")
ParseAndAddCatchTests_PrintDebugMessage("CMP0110 set to NEW, no need for add_test(\"\") workaround")
else()
ParseAndAddCatchTests_PrintDebugMessage("CMP0110 set to OLD adding \"\" for add_test() workaround")
set(CTestName "\"${CTestName}\"")
endif()
# Handle template test cases
if("${TestTypeAndFixture}" MATCHES ".*TEMPLATE_.*")
set(Name "${Name} - *")
endif()
# Add the test and set its properties
add_test(NAME "\"${CTestName}\"" COMMAND ${OptionalCatchTestLauncher} ${TestTarget} ${Name} ${AdditionalCatchParameters})
add_test(NAME "${CTestName}" COMMAND ${OptionalCatchTestLauncher} $<TARGET_FILE:${TestTarget}> ${Name} ${AdditionalCatchParameters})
# Old CMake versions do not document VERSION_GREATER_EQUAL, so we use VERSION_GREATER with 3.8 instead
if(PARSE_CATCH_TESTS_NO_HIDDEN_TESTS AND ${HiddenTagFound} AND ${CMAKE_VERSION} VERSION_GREATER "3.8")
PrintDebugMessage("Setting DISABLED test property")
set_tests_properties("\"${CTestName}\"" PROPERTIES DISABLED ON)
ParseAndAddCatchTests_PrintDebugMessage("Setting DISABLED test property")
set_tests_properties("${CTestName}" PROPERTIES DISABLED ON)
else()
set_tests_properties("\"${CTestName}\"" PROPERTIES FAIL_REGULAR_EXPRESSION "No tests ran"
set_tests_properties("${CTestName}" PROPERTIES FAIL_REGULAR_EXPRESSION "No tests ran"
LABELS "${Labels}")
endif()
set_property(
TARGET ${TestTarget}
APPEND
PROPERTY ParseAndAddCatchTests_TESTS "${CTestName}")
set_property(
SOURCE ${SourceFile}
APPEND
PROPERTY ParseAndAddCatchTests_TESTS "${CTestName}")
endif()
@ -190,11 +241,12 @@ endfunction()
# entry point
function(ParseAndAddCatchTests TestTarget)
PrintDebugMessage("Started parsing ${TestTarget}")
message(DEPRECATION "ParseAndAddCatchTest: function deprecated because of possibility of missed test cases. Consider using 'catch_discover_tests' from 'Catch.cmake'")
ParseAndAddCatchTests_PrintDebugMessage("Started parsing ${TestTarget}")
get_target_property(SourceFiles ${TestTarget} SOURCES)
PrintDebugMessage("Found the following sources: ${SourceFiles}")
ParseAndAddCatchTests_PrintDebugMessage("Found the following sources: ${SourceFiles}")
foreach(SourceFile ${SourceFiles})
ParseFile(${SourceFile} ${TestTarget})
ParseAndAddCatchTests_ParseFile(${SourceFile} ${TestTarget})
endforeach()
PrintDebugMessage("Finished parsing ${TestTarget}")
ParseAndAddCatchTests_PrintDebugMessage("Finished parsing ${TestTarget}")
endfunction()

10360
extras/catch_amalgamated.cpp Normal file

File diff suppressed because it is too large Load Diff

12233
extras/catch_amalgamated.hpp Normal file

File diff suppressed because it is too large Load Diff

20
fuzzing/CMakeLists.txt Normal file
View File

@ -0,0 +1,20 @@
# License: Boost 1.0
# By Paul Dreik 2020
# add a library that brings in the main() function from libfuzzer
# and has all the dependencies, so the individual fuzzers can be
# added one line each.
add_library(fuzzhelper NullOStream.h NullOStream.cpp)
target_link_libraries(fuzzhelper PUBLIC Catch2::Catch2)
# use C++17 so we can get string_view
target_compile_features(fuzzhelper PUBLIC cxx_std_17)
# This should be possible to set from the outside to be oss-fuzz compatible,
# fix later. For now, target libFuzzer only.
target_link_options(fuzzhelper PUBLIC "-fsanitize=fuzzer")
foreach(fuzzer TestSpecParser XmlWriter textflow)
add_executable(fuzz_${fuzzer} fuzz_${fuzzer}.cpp)
target_link_libraries(fuzz_${fuzzer} PRIVATE fuzzhelper)
endforeach()

10
fuzzing/NullOStream.cpp Normal file
View File

@ -0,0 +1,10 @@
#include "NullOStream.h"
void NullOStream::avoidOutOfLineVirtualCompilerWarning()
{
}
int NullStreambuf::overflow(int c){
setp(dummyBuffer, dummyBuffer + sizeof(dummyBuffer));
return (c == traits_type::eof()) ? '\0' : c;
}

20
fuzzing/NullOStream.h Normal file
View File

@ -0,0 +1,20 @@
#pragma once
#include <ostream>
#include <streambuf>
// from https://stackoverflow.com/a/8244052
class NullStreambuf : public std::streambuf {
char dummyBuffer[64];
protected:
virtual int overflow(int c) override final;
};
class NullOStream final : private NullStreambuf, public std::ostream {
public:
NullOStream() : std::ostream(this) {}
NullStreambuf *rdbuf() { return this; }
virtual void avoidOutOfLineVirtualCompilerWarning();
};

Some files were not shown because too many files have changed in this diff Show More