diff --git a/docs/en/contribute/esp-idf-tests-with-pytest.rst b/docs/en/contribute/esp-idf-tests-with-pytest.rst index 39797d10f4..92eb879202 100644 --- a/docs/en/contribute/esp-idf-tests-with-pytest.rst +++ b/docs/en/contribute/esp-idf-tests-with-pytest.rst @@ -272,6 +272,27 @@ This code example is taken from :idf_file:`pytest_esp_eth.py `__ with a user-friendly readable reason. + +This code example is taken from :idf_file:`pytest_panic.py ` + +.. code:: python + + + @pytest.mark.xfail('config.getvalue("target") == "esp32s2"', reason='raised IllegalInstruction instead') + def test_cache_error(dut: PanicTestDut, config: str, test_func_name: str) -> None: + +This marker means that if the test would be a known failure one on esp32s2. + + Run the Tests in CI ------------------- @@ -368,6 +389,44 @@ We’re using two types of custom markers, target markers which indicate that th You can add new markers by adding one line under the ``${IDF_PATH}/pytest.ini`` ``markers =`` section. The grammar should be: ``: `` +Generate JUnit Report +~~~~~~~~~~~~~~~~~~~~~ + +You can call pytest with ``--junitxml `` to generate the JUnit report. In ESP-IDF, the test case name would be unified as "..". + +Skip Auto Flash Binary +~~~~~~~~~~~~~~~~~~~~~~ + +Skipping auto-flash binary everytime would be useful when you're debugging your test script. + +You can call pytest with ``--skip-autoflash y`` to achieve it. + +Record Statistics +~~~~~~~~~~~~~~~~~ + +Sometimes you may need to record some statistics while running the tests, like the performance test statistics. + +You can use `record_xml_attribute `__ fixture in your test script, and the statistics would be recorded as attributes in the JUnit report. + +Logging System +~~~~~~~~~~~~~~ + +Sometimes you may need to add some extra logging lines while running the test cases. + +You can use `python logging module `__ to achieve this. + +Known Limitations and Workarounds +--------------------------------- + +Avoid Using ``Thread`` for Performance Test +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``pytest-embedded`` is using some threads internally to help gather all stdout to the pexpect process. Due to the limitation of `Global Interpreter Lock `__, if you're using threads to do performance tests, these threads would block each other and there would be great performance loss. + +**workaround** + +Use `Process `__ instead, the APIs should be almost the same as ``Thread``. + Further Readings ---------------- diff --git a/pytest.ini b/pytest.ini index 7e3c8ac276..d7c768797a 100644 --- a/pytest.ini +++ b/pytest.ini @@ -3,10 +3,12 @@ python_files = pytest_*.py # ignore PytestExperimentalApiWarning for record_xml_attribute +# set traceback to "short" to prevent the overwhelming tracebacks addopts = -s --embedded-services esp,idf -W ignore::_pytest.warning_types.PytestExperimentalApiWarning + --tb short markers = esp32: support esp32 target