espressif/esp-idf
1
0
Fork 1
mirror of https://github.com/espressif/esp-idf.git synced 2025-08-07 14:44:32 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'test/panic_add_riscv' into 'master'

Browse Source 
tests: panic: add esp32s3, esp32c3, esp32c2 support

Closes IDF-5692

See merge request espressif/esp-idf!21349
This commit is contained in:
Ivan Grokhotkov
2023-01-04 18:29:01 +08:00
parent 708a9047e8 6495810c5d
commit 8020407d3d
4 changed files with 351 additions and 217 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
tools/test_apps/.build-test-rules.yml
View File

@@ -165,9 +165,9 @@ tools/test_apps/system/panic:
enable: enable:
- if: INCLUDE_DEFAULT == 1 or IDF_TARGET == "esp32h4" - if: INCLUDE_DEFAULT == 1 or IDF_TARGET == "esp32h4"
disable_test: disable_test:
- if: IDF_TARGET not in ["esp32", "esp32s2"] - if: IDF_TARGET not in ["esp32", "esp32s2", "esp32c3", "esp32s3", "esp32c2"]
temporary: true temporary: true
reason: lack of runners reason: test app not ported to this target yet
tools/test_apps/system/startup: tools/test_apps/system/startup:
enable: enable:

57
tools/test_apps/system/panic/README.md
View File

@@ -1,27 +1,64 @@
| Supported Targets | ESP32 | ESP32-C2 | ESP32-C3 | ESP32-C6 | ESP32-H4 | ESP32-S2 | ESP32-S3 | | Supported Targets | ESP32 | ESP32-C2 | ESP32-C3 | ESP32-C6 | ESP32-H4 | ESP32-S2 | ESP32-S3 |
| ----------------- | ----- | -------- | -------- | -------- | -------- | -------- | -------- | | ----------------- | ----- | -------- | -------- | -------- | -------- | -------- | -------- |
# Introduction
The panic test app checks the behavior of ESP-IDF Panic Handler.
This test app is relatively complex because it has to check many possible combinations of:
- Failure scenario: abort, assertion, interrupt watchdog, illegal instruction, ...
- Chip target: esp32, esp32c3, ...
- Configuration: default, GDB Stub, Core Dump to UART, ...
Failure scenarios are implemented in [test_panic_main.c](main/test_panic_main.c). The test application receives the name of the scenario from console (e.g. `test_illegal_instruction` ). The failure scenario is executed and the app panics. Once the panic output is printed, the pytest-based test case parses the output and verifies that the behavior of the panic handler was correct.
In [pytest_panic.py](pytest_panic.py), there typically is one test function for each failure scenario. Each test function is then parametrized by `config` parameter. This creates "copies" of the test case for each of the configurations (default, GDB Stub, etc.) Tests are also parametrized with target-specific markers. Most tests can run on every target, but there are a few exceptions, such as failure scenarios specific to the dual-core chips.
The test cases use a customized DUT class `PanicTestDut`, defined in [panic_dut.py](test_panic_util/panic_dut.py). This class is derived from [`IdfDut`](https://docs.espressif.com/projects/pytest-embedded/en/latest/references/pytest_embedded_idf/#pytest_embedded_idf.dut.IdfDut). It defines several helper functions to make the test cases easier to read.
# Building # Building
Several configurations are provided as `sdkconfig.ci.XXX` and serve as a template. Several configurations are provided as `sdkconfig.ci.XXX` and serve as a template.
## Example with configuration "panic" for target ESP32 For example, to build the test app with configuration `panic` for ESP32-C3, run:
``` ```bash
idf.py set-target esp32 idf.py set-target esp32c3
cat sdkconfig.defaults sdkconfig.ci.panic > sdkconfig cat sdkconfig.defaults sdkconfig.ci.panic > sdkconfig
idf.py build idf.py build
``` ```
# Running # Building multiple configurations side by side
All the setup needs to be done as described in the [test apps README](../../README.md), except that the test cases need to be specified when running the app:
``` If you need to work with multiple configurations at the same time it can be useful to keep each build in a separate directory. For example, to build the `panic` configuration for ESP32-C3 in a separate directory, run:
python app_test.py test_panic_illegal_instruction ```bash
idf.py -DIDF_TARGET=esp32c3 -DSDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.ci.panic" -DSDKCONFIG=build_esp32c3_panic/sdkconfig -B build_esp32c3_panic build
``` ```
Multiple test cases are passed as additional arguments: This way, all the build products and the sdkconfig file are kept in the directory `build_esp32c3_gdbstub`. pytest-embedded will search for binaries in this directory if you run tests as shown in the section below.
This approach allows switching between different build configurations and targets without deleting the build directories.
# Running the app manually
```bash
idf.py flash monitor
``` ```
python app_test.py test_panic_illegal_instruction test_panic_int_wdt test_panic_storeprohibited (don't forget the -B argument if you have built the app in a directory other than `build`)
Once the app is running, input the name of the test (e.g. `test_abort`) and press Enter.
# Running tests
Suppose you have built the app for a specific target and with a certain `sdkconfig.ci.CONFIG` config. You need to run the tests just for this config and the target:
```bash
pytest --target TARGET -k '[CONFIG]'
``` ```
*Note that you need to pick the correct test cases at run time according to the configuration you built before. The above examples are for configuration "panic"* For example, if you have built the `panic` config for ESP32-C3, run:
```bash
pytest --target esp32c3 -k '[panic]'
```
Or, to run a single test for the given config, e.g. `test_abort`:
```bash
pytest --target esp32c3 -k 'test_abort[panic]'
```

415
tools/test_apps/system/panic/pytest_panic.py
View File

@@ -1,39 +1,47 @@
# SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD # SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: CC0-1.0 # SPDX-License-Identifier: CC0-1.0
import re import re
from pprint import pformat
from typing import List, Optional from typing import List, Optional
import pexpect import pexpect
import pytest import pytest
from test_panic_util import PanicTestDut from test_panic_util import PanicTestDut
# Markers for all the targets this test currently runs on
TARGETS_TESTED = [pytest.mark.esp32, pytest.mark.esp32s2, pytest.mark.esp32c3, pytest.mark.esp32s3, pytest.mark.esp32c2]
# Most tests run on all targets and with all configs.
# This list is passed to @pytest.mark.parametrize for each of the test cases.
# It creates an outer product of the sets: [configs] x [targets],
# with some exceptions.
CONFIGS = [ CONFIGS = [
pytest.param('coredump_flash_bin_crc', marks=[pytest.mark.esp32, pytest.mark.esp32s2]), pytest.param('coredump_flash_bin_crc', marks=TARGETS_TESTED),
pytest.param('coredump_flash_elf_sha', marks=[pytest.mark.esp32]), # sha256 only supported on esp32 pytest.param('coredump_flash_elf_sha', marks=[pytest.mark.esp32]), # sha256 only supported on esp32, IDF-1820
pytest.param('coredump_uart_bin_crc', marks=[pytest.mark.esp32, pytest.mark.esp32s2]), pytest.param('coredump_uart_bin_crc', marks=TARGETS_TESTED),
pytest.param('coredump_uart_elf_crc', marks=[pytest.mark.esp32, pytest.mark.esp32s2]), pytest.param('coredump_uart_elf_crc', marks=TARGETS_TESTED),
pytest.param('gdbstub', marks=[pytest.mark.esp32, pytest.mark.esp32s2]), pytest.param('gdbstub', marks=TARGETS_TESTED),
pytest.param('panic', marks=[pytest.mark.esp32, pytest.mark.esp32s2]), pytest.param('panic', marks=TARGETS_TESTED),
] ]
# An ESP32-only config, used for tests requiring two cores # Some tests only run on dual-core targets, they use the config below.
CONFIGS_ESP32 = [ TARGETS_DUAL_CORE = [pytest.mark.esp32, pytest.mark.esp32s3]
pytest.param('coredump_flash_bin_crc', marks=[pytest.mark.esp32]), CONFIGS_DUAL_CORE = [
pytest.param('coredump_flash_elf_sha', marks=[pytest.mark.esp32]), pytest.param('coredump_flash_bin_crc', marks=TARGETS_DUAL_CORE),
pytest.param('coredump_uart_bin_crc', marks=[pytest.mark.esp32]), pytest.param('coredump_flash_elf_sha', marks=[pytest.mark.esp32]), # sha256 only supported on esp32, IDF-1820
pytest.param('coredump_uart_elf_crc', marks=[pytest.mark.esp32]), pytest.param('coredump_uart_bin_crc', marks=TARGETS_DUAL_CORE),
pytest.param('gdbstub', marks=[pytest.mark.esp32]), pytest.param('coredump_uart_elf_crc', marks=TARGETS_DUAL_CORE),
pytest.param('panic', marks=[pytest.mark.esp32]), pytest.param('gdbstub', marks=TARGETS_DUAL_CORE),
pytest.param('panic', marks=TARGETS_DUAL_CORE),
] ]
# IDF-5692: Uncomment the marks related to ESP32-S3 and quad_psram once ESP32-S3 runners are available # Some tests run on all targets but need to behave differently on the dual-core ones.
CONFIG_EXTRAM_STACK = [ # This list is used to check if the target is a dual-core one.
pytest.param('coredump_extram_stack', TARGETS_DUAL_CORE_NAMES = [x.mark.name for x in TARGETS_DUAL_CORE]
marks=[pytest.mark.esp32, pytest.mark.esp32s2, pytest.mark.psram,
# pytest.mark.esp32s3, pytest.mark.quad_psram # The tests which panic on external stack require PSRAM capable runners
]) CONFIGS_EXTRAM_STACK = [
pytest.param('coredump_extram_stack', marks=[pytest.mark.esp32, pytest.mark.esp32s2, pytest.mark.psram, pytest.mark.esp32s3, pytest.mark.quad_psram])
] ]
@@ -46,17 +54,10 @@ def common_test(dut: PanicTestDut, config: str, expected_backtrace: Optional[Lis
dut.expect_exact('Entering gdb stub now.') dut.expect_exact('Entering gdb stub now.')
dut.start_gdb() dut.start_gdb()
frames = dut.gdb_backtrace() frames = dut.gdb_backtrace()
# Make sure frames and the expected_backtrace have the same size, else, an exception will occur
if expected_backtrace is not None: if expected_backtrace is not None:
size = min(len(frames), len(expected_backtrace)) dut.verify_gdb_backtrace(frames, expected_backtrace)
frames = frames[0:size] dut.revert_log_level()
expected_backtrace = expected_backtrace[0:size] return # don't expect "Rebooting" output below
if not dut.match_backtrace(frames, expected_backtrace):
raise AssertionError(
'Unexpected backtrace in test {}:\n{}'.format(config, pformat(frames))
)
dut.revert_log_level()
return
if 'uart' in config: if 'uart' in config:
dut.process_coredump_uart() dut.process_coredump_uart()
@@ -71,86 +72,91 @@ def common_test(dut: PanicTestDut, config: str, expected_backtrace: Optional[Lis
@pytest.mark.parametrize('config', CONFIGS, indirect=True) @pytest.mark.parametrize('config', CONFIGS, indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_task_wdt_cpu0(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_task_wdt_cpu0(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_exact( dut.expect_exact(
'Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:' 'Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:'
) )
dut.expect_exact('CPU 0: main') dut.expect_exact('CPU 0: main')
dut.expect_none('register dump:') if dut.is_xtensa:
dut.expect_exact('Print CPU 0 (current core) backtrace') # on Xtensa, dumping registers on abort is not necessary, we only need to dump the backtrace
dut.expect_backtrace() dut.expect_none('register dump:')
dut.expect_exact('Print CPU 0 (current core) backtrace')
dut.expect_backtrace()
else:
# on RISC-V, need to dump both registers and stack memory to reconstruct the backtrace
dut.expect_reg_dump(core=0)
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_none('Guru Meditation')
if config == 'gdbstub': common_test(
common_test( dut,
dut, config,
config, expected_backtrace=get_default_backtrace(test_func_name),
expected_backtrace=[ )
'test_task_wdt_cpu0',
'app_main'
],
)
else:
common_test(dut, config)
@pytest.mark.parametrize('config', CONFIGS_ESP32, indirect=True) @pytest.mark.parametrize('config', CONFIGS_DUAL_CORE, indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_task_wdt_cpu1(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_task_wdt_cpu1(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_exact( dut.expect_exact(
'Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:' 'Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:'
) )
dut.expect_exact('CPU 1: Infinite loop') dut.expect_exact('CPU 1: Infinite loop')
dut.expect_none('register dump:') if dut.is_xtensa:
dut.expect_exact('Print CPU 1 backtrace') # see comment in test_task_wdt_cpu0
dut.expect_backtrace() dut.expect_none('register dump:')
dut.expect_exact('Print CPU 1 backtrace')
dut.expect_backtrace()
# On Xtensa, we get incorrect backtrace from GDB in this test
expected_backtrace = ['infinite_loop', 'vPortTaskWrapper']
else:
assert False, 'No dual-core RISC-V chips yet, check this test case later'
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_none('Guru Meditation')
if config == 'gdbstub': common_test(
common_test( dut,
dut, config,
config, expected_backtrace=expected_backtrace,
expected_backtrace=[ )
'infinite_loop'
],
)
else:
common_test(dut, config)
@pytest.mark.parametrize('config', CONFIGS_ESP32, indirect=True) @pytest.mark.parametrize('config', CONFIGS_DUAL_CORE, indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_task_wdt_both_cpus(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_task_wdt_both_cpus(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) if dut.target == 'esp32s3':
pytest.xfail(reason='Only prints "Print CPU 1 backtrace", IDF-6560')
dut.run_test_func(test_func_name)
dut.expect_exact( dut.expect_exact(
'Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:' 'Task watchdog got triggered. The following tasks/users did not reset the watchdog in time:'
) )
dut.expect_exact('CPU 0: Infinite loop') dut.expect_exact('CPU 0: Infinite loop')
dut.expect_exact('CPU 1: Infinite loop') dut.expect_exact('CPU 1: Infinite loop')
dut.expect_none('register dump:') if dut.is_xtensa:
dut.expect_exact('Print CPU 0 (current core) backtrace') # see comment in test_task_wdt_cpu0
dut.expect_backtrace() dut.expect_none('register dump:')
dut.expect_exact('Print CPU 1 backtrace') dut.expect_exact('Print CPU 0 (current core) backtrace')
dut.expect_backtrace() dut.expect_backtrace()
dut.expect_exact('Print CPU 1 backtrace')
dut.expect_backtrace()
# On Xtensa, we get incorrect backtrace from GDB in this test
expected_backtrace = ['infinite_loop', 'vPortTaskWrapper']
else:
assert False, 'No dual-core RISC-V chips yet, check this test case later'
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_none('Guru Meditation')
if config == 'gdbstub': common_test(
common_test( dut,
dut, config,
config, expected_backtrace=expected_backtrace,
expected_backtrace=[ )
'infinite_loop'
],
)
else:
common_test(dut, config)
@pytest.mark.parametrize('config', CONFIG_EXTRAM_STACK, indirect=True) @pytest.mark.parametrize('config', CONFIGS_EXTRAM_STACK, indirect=True)
def test_panic_extram_stack(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_panic_extram_stack(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.expect_test_func_name(test_func_name)
dut.expect_none('Allocated stack is not in external RAM') dut.expect_none('Allocated stack is not in external RAM')
@@ -172,19 +178,21 @@ def test_panic_extram_stack(dut: PanicTestDut, config: str, test_func_name: str)
def test_int_wdt( def test_int_wdt(
dut: PanicTestDut, target: str, config: str, test_func_name: str dut: PanicTestDut, target: str, config: str, test_func_name: str
) -> None: ) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_gme('Interrupt wdt timeout on CPU0') dut.expect_gme('Interrupt wdt timeout on CPU0')
dut.expect_reg_dump(0) dut.expect_reg_dump(0)
dut.expect_backtrace() if dut.is_xtensa:
if target == 'esp32s2': dut.expect_backtrace()
dut.expect_elf_sha256() else:
dut.expect_none('Guru Meditation') dut.expect_stack_dump()
if target != 'esp32s2': # esp32s2 is single-core if target in TARGETS_DUAL_CORE_NAMES:
assert dut.is_xtensa, 'No dual-core RISC-V chips yet, check the test case'
dut.expect_reg_dump(1) dut.expect_reg_dump(1)
dut.expect_backtrace() dut.expect_backtrace()
dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_elf_sha256()
dut.expect_none('Guru Meditation')
common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name)) common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name))
@@ -194,46 +202,68 @@ def test_int_wdt(
def test_int_wdt_cache_disabled( def test_int_wdt_cache_disabled(
dut: PanicTestDut, target: str, config: str, test_func_name: str dut: PanicTestDut, target: str, config: str, test_func_name: str
) -> None: ) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_gme('Interrupt wdt timeout on CPU0') dut.expect_gme('Interrupt wdt timeout on CPU0')
dut.expect_reg_dump(0) dut.expect_reg_dump(0)
dut.expect_backtrace() if dut.is_xtensa:
if target == 'esp32s2': dut.expect_backtrace()
dut.expect_elf_sha256() else:
dut.expect_none('Guru Meditation') dut.expect_stack_dump()
if target != 'esp32s2': # esp32s2 is single-core if target in TARGETS_DUAL_CORE_NAMES:
assert dut.is_xtensa, 'No dual-core RISC-V chips yet, check the test case'
dut.expect_reg_dump(1) dut.expect_reg_dump(1)
dut.expect_backtrace() dut.expect_backtrace()
dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_elf_sha256()
dut.expect_none('Guru Meditation')
common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name)) common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name))
@pytest.mark.parametrize('config', CONFIGS, indirect=True) @pytest.mark.parametrize('config', CONFIGS, indirect=True)
@pytest.mark.xfail('config.getvalue("target") == "esp32s2"', reason='raised IllegalInstruction instead')
@pytest.mark.generic @pytest.mark.generic
def test_cache_error(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_cache_error(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_gme('Cache disabled but cached memory region accessed') if dut.target in ['esp32c3', 'esp32c2']:
# Cache error interrupt is not raised, IDF-6398
dut.expect_gme('Illegal instruction')
elif dut.target in ['esp32s2']:
# Cache error interrupt is not enabled, IDF-1558
dut.expect_gme('IllegalInstruction')
else:
dut.expect_gme('Cache disabled but cached memory region accessed')
dut.expect_reg_dump(0) dut.expect_reg_dump(0)
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_none('Guru Meditation')
expected_backtrace = ['die'] + get_default_backtrace(test_func_name)
if dut.target in ['esp32s2', 'esp32s3']:
# 'test_cache_error' missing from GDB backtrace on ESP32-S2 and ESP-S3, IDF-6561
expected_backtrace = ['die', 'app_main', 'main_task', 'vPortTaskWrapper']
common_test( common_test(
dut, config, expected_backtrace=['die'] + get_default_backtrace(test_func_name) dut, config, expected_backtrace=expected_backtrace
) )
@pytest.mark.parametrize('config', CONFIGS, indirect=True) @pytest.mark.parametrize('config', CONFIGS, indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_stack_overflow(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_stack_overflow(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_gme('Unhandled debug exception') if dut.is_xtensa:
dut.expect_exact('Stack canary watchpoint triggered (main)') dut.expect_gme('Unhandled debug exception')
dut.expect_exact('Stack canary watchpoint triggered (main)')
else:
# Stack watchpoint handling missing on RISC-V, IDF-6397
dut.expect_gme('Breakpoint')
dut.expect_reg_dump(0) dut.expect_reg_dump(0)
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_none('Guru Meditation')
common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name)) common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name))
@@ -244,16 +274,26 @@ def test_stack_overflow(dut: PanicTestDut, config: str, test_func_name: str) ->
def test_instr_fetch_prohibited( def test_instr_fetch_prohibited(
dut: PanicTestDut, config: str, test_func_name: str dut: PanicTestDut, config: str, test_func_name: str
) -> None: ) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_gme('InstrFetchProhibited') if dut.is_xtensa:
dut.expect_reg_dump(0) dut.expect_gme('InstrFetchProhibited')
dut.expect_backtrace() dut.expect_reg_dump(0)
dut.expect_backtrace()
expected_backtrace = ['_init'] + get_default_backtrace(test_func_name)
else:
dut.expect_gme('Instruction access fault')
dut.expect_reg_dump(0)
dut.expect_stack_dump()
# On RISC-V, GDB is not able to determine the correct backtrace after
# a jump to an invalid address.
expected_backtrace = ['??']
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_none('Guru Meditation')
common_test( common_test(
dut, dut,
config, config,
expected_backtrace=['_init'] + get_default_backtrace(test_func_name), expected_backtrace=expected_backtrace,
) )
@@ -262,10 +302,16 @@ def test_instr_fetch_prohibited(
def test_illegal_instruction( def test_illegal_instruction(
dut: PanicTestDut, config: str, test_func_name: str dut: PanicTestDut, config: str, test_func_name: str
) -> None: ) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_gme('IllegalInstruction') if dut.is_xtensa:
dut.expect_gme('IllegalInstruction')
else:
dut.expect_gme('Illegal instruction')
dut.expect_reg_dump(0) dut.expect_reg_dump(0)
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_none('Guru Meditation')
common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name)) common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name))
@@ -274,10 +320,16 @@ def test_illegal_instruction(
@pytest.mark.parametrize('config', CONFIGS, indirect=True) @pytest.mark.parametrize('config', CONFIGS, indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_storeprohibited(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_storeprohibited(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect_gme('StoreProhibited') if dut.is_xtensa:
dut.expect_gme('StoreProhibited')
else:
dut.expect_gme('Store access fault')
dut.expect_reg_dump(0) dut.expect_reg_dump(0)
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none('Guru Meditation') dut.expect_none('Guru Meditation')
common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name)) common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name))
@@ -286,107 +338,130 @@ def test_storeprohibited(dut: PanicTestDut, config: str, test_func_name: str) ->
@pytest.mark.parametrize('config', CONFIGS, indirect=True) @pytest.mark.parametrize('config', CONFIGS, indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_abort(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_abort(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect(r'abort\(\) was called at PC [0-9xa-f]+ on core 0') dut.expect(r'abort\(\) was called at PC [0-9xa-f]+ on core 0')
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none(['Guru Meditation', 'Re-entered core dump']) dut.expect_none(['Guru Meditation', 'Re-entered core dump'])
if config == 'gdbstub': common_test(
common_test( dut,
dut, config,
config, expected_backtrace=[
expected_backtrace=[ 'panic_abort',
'panic_abort', 'esp_system_abort',
'esp_system_abort', 'abort'
'abort' ] + get_default_backtrace(test_func_name),
] + get_default_backtrace(test_func_name), )
)
else:
common_test(dut, config)
@pytest.mark.parametrize('config', CONFIGS, indirect=True) @pytest.mark.parametrize('config', CONFIGS, indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_ub(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_ub(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect('Undefined behavior of type out_of_bounds') dut.expect('Undefined behavior of type out_of_bounds')
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none(['Guru Meditation', 'Re-entered core dump']) dut.expect_none(['Guru Meditation', 'Re-entered core dump'])
if config == 'gdbstub': common_test(
common_test( dut,
dut, config,
config, expected_backtrace=[
expected_backtrace=[ 'panic_abort',
'panic_abort', 'esp_system_abort',
'esp_system_abort', '__ubsan_default_handler',
'__ubsan_default_handler', '__ubsan_handle_out_of_bounds'
'__ubsan_handle_out_of_bounds' ] + get_default_backtrace(test_func_name),
] + get_default_backtrace(test_func_name), )
)
else:
common_test(dut, config)
######################### @pytest.mark.parametrize('config', CONFIGS, indirect=True)
# for config panic only #
#########################
@pytest.mark.esp32
@pytest.mark.esp32s2
@pytest.mark.xfail('config.getvalue("target") == "esp32s2"', reason='raised IllegalInstruction instead')
@pytest.mark.parametrize('config', ['panic'], indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_abort_cache_disabled( def test_abort_cache_disabled(
dut: PanicTestDut, config: str, test_func_name: str dut: PanicTestDut, config: str, test_func_name: str
) -> None: ) -> None:
dut.expect_test_func_name(test_func_name) if dut.target == 'esp32s2':
pytest.xfail(reason='Crashes in itoa which is not in ROM, IDF-3572')
dut.run_test_func(test_func_name)
dut.expect(r'abort\(\) was called at PC [0-9xa-f]+ on core 0') dut.expect(r'abort\(\) was called at PC [0-9xa-f]+ on core 0')
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none(['Guru Meditation', 'Re-entered core dump']) dut.expect_none(['Guru Meditation', 'Re-entered core dump'])
common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name)) common_test(
dut,
config,
expected_backtrace=[
'panic_abort',
'esp_system_abort',
'abort'
] + get_default_backtrace(test_func_name),
)
@pytest.mark.esp32 @pytest.mark.parametrize('config', CONFIGS, indirect=True)
@pytest.mark.esp32s2
@pytest.mark.parametrize('config', ['panic'], indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_assert(dut: PanicTestDut, config: str, test_func_name: str) -> None: def test_assert(dut: PanicTestDut, config: str, test_func_name: str) -> None:
dut.expect_test_func_name(test_func_name) dut.run_test_func(test_func_name)
dut.expect( dut.expect(
re.compile( re.compile(
rb'assert failed:[\s\w()]*?\s[.\w/]*\.(?:c|cpp|h|hpp):\d.*$', re.MULTILINE rb'assert failed:[\s\w()]*?\s[.\w/]*\.(?:c|cpp|h|hpp):\d.*$', re.MULTILINE
) )
) )
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none(['Guru Meditation', 'Re-entered core dump']) dut.expect_none(['Guru Meditation', 'Re-entered core dump'])
common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name)) common_test(
dut,
config,
expected_backtrace=[
'panic_abort',
'esp_system_abort',
'__assert_func'
] + get_default_backtrace(test_func_name))
@pytest.mark.esp32 @pytest.mark.parametrize('config', CONFIGS, indirect=True)
@pytest.mark.esp32s2
@pytest.mark.xfail('config.getvalue("target") == "esp32s2"', reason='raised IllegalInstruction instead')
@pytest.mark.parametrize('config', ['panic'], indirect=True)
@pytest.mark.generic @pytest.mark.generic
def test_assert_cache_disabled( def test_assert_cache_disabled(
dut: PanicTestDut, config: str, test_func_name: str dut: PanicTestDut, config: str, test_func_name: str
) -> None: ) -> None:
dut.expect_test_func_name(test_func_name) if dut.target == 'esp32s2':
pytest.xfail(reason='Crashes in itoa which is not in ROM, IDF-3572')
dut.run_test_func(test_func_name)
dut.expect(re.compile(rb'assert failed: [0-9xa-fA-F]+.*$', re.MULTILINE)) dut.expect(re.compile(rb'assert failed: [0-9xa-fA-F]+.*$', re.MULTILINE))
dut.expect_backtrace() if dut.is_xtensa:
dut.expect_backtrace()
else:
dut.expect_stack_dump()
dut.expect_elf_sha256() dut.expect_elf_sha256()
dut.expect_none(['Guru Meditation', 'Re-entered core dump']) dut.expect_none(['Guru Meditation', 'Re-entered core dump'])
common_test(dut, config, expected_backtrace=get_default_backtrace(test_func_name)) common_test(
dut,
config,
expected_backtrace=[
'panic_abort',
'esp_system_abort',
'__assert_func'
] + get_default_backtrace(test_func_name))
@pytest.mark.esp32 @pytest.mark.esp32
@pytest.mark.parametrize('config', ['panic_delay'], indirect=True) @pytest.mark.parametrize('config', ['panic_delay'], indirect=True)
@pytest.mark.generic
def test_panic_delay(dut: PanicTestDut) -> None: def test_panic_delay(dut: PanicTestDut) -> None:
dut.expect_test_func_name('test_storeprohibited') dut.run_test_func('test_storeprohibited')
try: try:
dut.expect_exact('Rebooting...', timeout=4) dut.expect_exact('Rebooting...', timeout=4)
except pexpect.TIMEOUT: except pexpect.TIMEOUT:

92
tools/test_apps/system/panic/test_panic_util/panic_dut.py
View File

@@ -1,10 +1,10 @@
# SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD # SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Unlicense OR CC0-1.0 # SPDX-License-Identifier: Unlicense OR CC0-1.0
import logging import logging
import os import os
import subprocess import subprocess
import sys import sys
from typing import Any, Dict, List, TextIO from typing import Any, Dict, List, Optional, TextIO
import pexpect import pexpect
from panic_utils import NoGdbProcessError, attach_logger, quote_string, sha256, verify_valid_gdb_subprocess from panic_utils import NoGdbProcessError, attach_logger, quote_string, sha256, verify_valid_gdb_subprocess
@@ -24,28 +24,33 @@ class PanicTestDut(IdfDut):
app: IdfApp app: IdfApp
serial: IdfSerial serial: IdfSerial
def __init__(self, *args, **kwargs) -> None: # type: ignore def __init__(self, *args: Any, **kwargs: Any) -> None:
super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
self.gdb: GdbController = None # type: ignore self.gdbmi: Optional[GdbController] = None
# record this since pygdbmi is using logging.debug to generate some single character mess # record this since pygdbmi is using logging.debug to generate some single character mess
self.log_level = logging.getLogger().level self.log_level = logging.getLogger().level
# pygdbmi is using logging.debug to generate some single character mess # pygdbmi is using logging.debug to generate some single character mess
if self.log_level <= logging.DEBUG: if self.log_level <= logging.DEBUG:
logging.getLogger().setLevel(logging.INFO) logging.getLogger().setLevel(logging.INFO)
self.coredump_output: TextIO = None # type: ignore self.coredump_output: Optional[TextIO] = None
def close(self) -> None: def close(self) -> None:
if self.gdb: if self.gdbmi:
self.gdb.exit() logging.info('Waiting for GDB to exit')
self.gdbmi.exit()
super().close() super().close()
def revert_log_level(self) -> None: def revert_log_level(self) -> None:
logging.getLogger().setLevel(self.log_level) logging.getLogger().setLevel(self.log_level)
def expect_test_func_name(self, test_func_name: str) -> None: @property
def is_xtensa(self) -> bool:
return self.target in self.XTENSA_TARGETS
def run_test_func(self, test_func_name: str) -> None:
self.expect_exact('Enter test name:') self.expect_exact('Enter test name:')
self.write(test_func_name) self.write(test_func_name)
self.expect_exact('Got test name: ' + test_func_name) self.expect_exact('Got test name: ' + test_func_name)
@@ -62,8 +67,13 @@ class PanicTestDut(IdfDut):
pass pass
def expect_backtrace(self) -> None: def expect_backtrace(self) -> None:
self.expect_exact('Backtrace:') assert self.is_xtensa, 'Backtrace can be printed only on Xtensa'
self.expect_none('CORRUPTED') match = self.expect(r'Backtrace:( 0x[0-9a-fA-F]{8}:0x[0-9a-fA-F]{8})+(?P<corrupted> \|<-CORRUPTED)?')
assert not match.group('corrupted')
def expect_stack_dump(self) -> None:
assert not self.is_xtensa, 'Stack memory dump is only printed on RISC-V'
self.expect_exact('Stack memory:')
def expect_gme(self, reason: str) -> None: def expect_gme(self, reason: str) -> None:
"""Expect method for Guru Meditation Errors""" """Expect method for Guru Meditation Errors"""
@@ -137,25 +147,29 @@ class PanicTestDut(IdfDut):
Wrapper to write to gdb with a longer timeout, as test runner Wrapper to write to gdb with a longer timeout, as test runner
host can be slow sometimes host can be slow sometimes
""" """
return self.gdb.write(command, timeout_sec=10) assert self.gdbmi, 'This function should be called only after start_gdb'
return self.gdbmi.write(command, timeout_sec=10)
def start_gdb(self) -> None: def start_gdb(self) -> None:
""" """
Runs GDB and connects it to the "serial" port of the DUT. Runs GDB and connects it to the "serial" port of the DUT.
After this, the DUT expect methods can no longer be used to capture output. After this, the DUT expect methods can no longer be used to capture output.
""" """
gdb_path = self.toolchain_prefix + 'gdb' if self.is_xtensa:
gdb_path = f'xtensa-{self.target}-elf-gdb'
else:
gdb_path = 'riscv32-esp-elf-gdb'
try: try:
from pygdbmi.constants import GdbTimeoutError from pygdbmi.constants import GdbTimeoutError
default_gdb_args = ['--nx', '--quiet', '--interpreter=mi2'] default_gdb_args = ['--nx', '--quiet', '--interpreter=mi2']
gdb_command = [gdb_path] + default_gdb_args gdb_command = [gdb_path] + default_gdb_args
self.gdb = GdbController(command=gdb_command) self.gdbmi = GdbController(command=gdb_command)
pygdbmi_logger = attach_logger() pygdbmi_logger = attach_logger()
except ImportError: except ImportError:
# fallback for pygdbmi<0.10.0.0. # fallback for pygdbmi<0.10.0.0.
from pygdbmi.gdbcontroller import GdbTimeoutError from pygdbmi.gdbcontroller import GdbTimeoutError
self.gdb = GdbController(gdb_path=gdb_path) self.gdbmi = GdbController(gdb_path=gdb_path)
pygdbmi_logger = self.gdb.logger pygdbmi_logger = self.gdbmi.logger
# pygdbmi logs to console by default, make it log to a file instead # pygdbmi logs to console by default, make it log to a file instead
pygdbmi_log_file_name = os.path.join(self.logdir, 'pygdbmi_log.txt') pygdbmi_log_file_name = os.path.join(self.logdir, 'pygdbmi_log.txt')
@@ -166,22 +180,23 @@ class PanicTestDut(IdfDut):
log_handler.setFormatter( log_handler.setFormatter(
logging.Formatter('%(asctime)s %(levelname)s: %(message)s') logging.Formatter('%(asctime)s %(levelname)s: %(message)s')
) )
logging.info(f'Saving pygdbmi logs to {pygdbmi_log_file_name}')
pygdbmi_logger.addHandler(log_handler) pygdbmi_logger.addHandler(log_handler)
try: try:
gdb_command = self.gdb.command gdb_command = self.gdbmi.command
except AttributeError: except AttributeError:
# fallback for pygdbmi < 0.10 # fallback for pygdbmi < 0.10
gdb_command = self.gdb.cmd gdb_command = self.gdbmi.cmd
logging.info(f'Running command: "{" ".join(quote_string(c) for c in gdb_command)}"') logging.info(f'Running command: "{" ".join(quote_string(c) for c in gdb_command)}"')
for _ in range(10): for _ in range(10):
try: try:
# GdbController creates a process with subprocess.Popen(). Is it really running? It is probable that # GdbController creates a process with subprocess.Popen(). Is it really running? It is probable that
# an RPI under high load will get non-responsive during creating a lot of processes. # an RPI under high load will get non-responsive during creating a lot of processes.
if not hasattr(self.gdb, 'verify_valid_gdb_subprocess'): if not hasattr(self.gdbmi, 'verify_valid_gdb_subprocess'):
# for pygdbmi >= 0.10.0.0 # for pygdbmi >= 0.10.0.0
verify_valid_gdb_subprocess(self.gdb.gdb_process) verify_valid_gdb_subprocess(self.gdbmi.gdb_process)
resp = self.gdb.get_gdb_response( resp = self.gdbmi.get_gdb_response(
timeout_sec=10 timeout_sec=10
) # calls verify_valid_gdb_subprocess() internally for pygdbmi < 0.10.0.0 ) # calls verify_valid_gdb_subprocess() internally for pygdbmi < 0.10.0.0
# it will be interesting to look up this response if the next GDB command fails (times out) # it will be interesting to look up this response if the next GDB command fails (times out)
@@ -207,17 +222,25 @@ class PanicTestDut(IdfDut):
self.gdb_write('-file-exec-and-symbols {}'.format(self.app.elf_file)) self.gdb_write('-file-exec-and-symbols {}'.format(self.app.elf_file))
# Connect GDB to UART # Connect GDB to UART
self.serial.proc.close() self.serial.close()
logging.info('Connecting to GDB Stub...') logging.info('Connecting to GDB Stub...')
self.gdb_write('-gdb-set serial baud 115200') self.gdb_write('-gdb-set serial baud 115200')
responses = self.gdb_write('-target-select remote ' + self.serial.port)
if sys.platform == 'darwin':
assert '/dev/tty.' not in self.serial.port, \
'/dev/tty.* ports can\'t be used with GDB on macOS. Use with /dev/cu.* instead.'
# Make sure we get the 'stopped' notification # Make sure we get the 'stopped' notification
responses = self.gdb_write('-target-select remote ' + self.serial.port)
stop_response = self.find_gdb_response('stopped', 'notify', responses) stop_response = self.find_gdb_response('stopped', 'notify', responses)
if not stop_response:
retries = 3
while not stop_response and retries > 0:
logging.info('Sending -exec-interrupt')
responses = self.gdb_write('-exec-interrupt') responses = self.gdb_write('-exec-interrupt')
stop_response = self.find_gdb_response('stopped', 'notify', responses) stop_response = self.find_gdb_response('stopped', 'notify', responses)
assert stop_response retries -= 1
frame = stop_response['payload']['frame'] frame = stop_response['payload']['frame']
if 'file' not in frame: if 'file' not in frame:
frame['file'] = '?' frame['file'] = '?'
@@ -226,33 +249,32 @@ class PanicTestDut(IdfDut):
logging.info('Stopped in {func} at {addr} ({file}:{line})'.format(**frame)) logging.info('Stopped in {func} at {addr} ({file}:{line})'.format(**frame))
# Drain remaining responses # Drain remaining responses
self.gdb.get_gdb_response(raise_error_on_timeout=False) self.gdbmi.get_gdb_response(raise_error_on_timeout=False)
def gdb_backtrace(self) -> Any: def gdb_backtrace(self) -> Any:
""" """
Returns the list of stack frames for the current thread. Returns the list of stack frames for the current thread.
Each frame is a dictionary, refer to pygdbmi docs for the format. Each frame is a dictionary, refer to pygdbmi docs for the format.
""" """
assert self.gdb assert self.gdbmi
responses = self.gdb_write('-stack-list-frames') responses = self.gdb_write('-stack-list-frames')
return self.find_gdb_response('done', 'result', responses)['payload']['stack'] return self.find_gdb_response('done', 'result', responses)['payload']['stack']
@staticmethod @staticmethod
def match_backtrace( def verify_gdb_backtrace(
gdb_backtrace: List[Any], expected_functions_list: List[Any] gdb_backtrace: List[Any], expected_functions_list: List[Any]
) -> bool: ) -> None:
""" """
Returns True if the function names listed in expected_functions_list match the backtrace Raises an assert if the function names listed in expected_functions_list do not match the backtrace
given by gdb_backtrace argument. The latter is in the same format as returned by gdb_backtrace() given by gdb_backtrace argument. The latter is in the same format as returned by gdb_backtrace()
function. function.
""" """
return all( actual_functions_list = [frame['func'] for frame in gdb_backtrace]
[ if actual_functions_list != expected_functions_list:
frame['func'] == expected_functions_list[i] logging.error(f'Expected backtrace: {expected_functions_list}')
for i, frame in enumerate(gdb_backtrace) logging.error(f'Actual backtrace: {actual_functions_list}')
] assert False, 'Got unexpected backtrace'
)
@staticmethod @staticmethod
def find_gdb_response( def find_gdb_response(