The eager upgreade policy should upgrade the full dependency tree and
not just the packages listed in the requirements file. The disadvantage
is some extra time for package resolution.
This upgrade policy is applied to CI-related packages only because we
don't want this to be used for customer-facing packages and/or provide
the customers with a different environment to that tested in the CI.
When the eFuse field consists of single descriptor, its size can be
taken from struct member 'bit_count'. However, when the field consists of
multiple descriptors, the size should be obtained via esp_efuse_get_field_size()
The build system keeps track of each component source. Currently
there are four types of sources:
1. "project_components" - project components
2. "project_extra_components" - components from EXTRA_COMPONENT_DIRS
3. "project_managed_components" - custom project dependencies managed by the IDF Component Manager
4. "idf_components" - ESP-IDF built-in components, typically under /components
This can be used to identify the component libraries that are likely to
change during application development and pass them to ldgen as mutable
libraries. Add all components with "project_components" as their source
as mutable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a helper function `__should_generate_sdkconfig()` that checks
whether the sdkconfig file should be generated. The
`__generate_sdkconfig()` function can be called multiple times, such as
when the initial sdkconfig is generated at the start of the build
process and later after additional components are fetched by the
component manager. There might be no components fetched by the
component manager, for example, in the hello_world example, or the
downloaded components may not contain any configuration files. In such
cases, there is no need to regenerate the sdkconfig. This helper
function stores the list of configuration files in the
`__PREV_KCONFIGS`, `__PREV_KCONFIG_PROJBUILDS`, and
`__PREV_SDKCONFIG_RENAMES` build properties at its end, and at the
beginning, it compares them with the current lists of configuration
files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add checks to validate client_verify_data pointer and length before
processing in handle_session_command1. Prevents NULL pointer dereference
when client omits verifier data in Session_Command1, which could cause
device crash during provisioning (remote DoS attack).
For CONFIG_BOOTLOADER_SKIP_VALIDATE_IN_DEEP_SLEEP enabled and exit from
deep sleep case the secure boot signature verification must be skipped
to improve the wakeup performance.
Closes https://github.com/espressif/esp-idf/issues/15590
The idf_create_size_report function allows for the creation of size
report targets based on the generated link map file. The size report
targets are created using the TARGET option name: "<target>",
"<target>-files", and "<target>-components". These size report targets
are added to the idf_default_project with the TARGET set to "size",
resulting in the creation of "size", "size-files", and "size-components"
targets for the default project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_project_init macro is evaluated within the global variable
scope, and currently, there are a few variables still set. Let's ensure
all variables are properly unset to avoid polluting the global variable
namespace.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The MINIMAL_BUILD property is not relevant in cmakev2, as only
components explicitly linked through targets are included in the build
by design. Display a warning when this outdated build property is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The link flag for generating the map file is currently added globally to
the link_options. If multiple executables are generated, the link map
file is overwritten by the last created executable. Since cmakev2
supports building multiple executables, the link map file options need
to be set for each executable individually. To address this, add a new
MAPFILE_TARGET option to the idf_build_executable function. When set,
the link map file will be generated by the linker, and a target
specified in the MAPFILE_TARGET option will be created for the map file.
This also splits the idf_project_default macro. Only the
idf_project_init macro needs be called within the global scope, as it
includes the project_include.cmake files and the cmake version of the
configuration. The remaining functionality of the idf_project_default
macro is implemented in a __project_default helper function to avoid
polluting the global variable space.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the `idf_build_generate_metadata` function takes an
executable target as an argument, and the binary name is stored in the
`EXECUTABLE_BINARY` property. This approach is inconvenient because a
single executable might generate multiple binary images, making it
unreliable to store the binary image name in the executable property due
to the N:1 relationship. To address this, pass the binary image target
instead of the executable target to the `idf_build_generate_metadata`
function. This change is facilitated by linking the binary target,
executable target, and library interface targets with the following
properties:
binary:EXECUTABLE_TARGET -> executable:LIBRARY_INTERFACE -> library
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
With commit b719292b75 ("refactor(build-system): Simplify flash target creation"),
partition table and bootloader components are correctly adding their
dependency on the flash target, so there is no need for this to be done
in the build system. This removes the temporary workaround.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
refactor(example/bt): Refactor controller_hci_uart example for ESP32-C3/S3 to use UHCI driver
Closes IDF-14345
See merge request espressif/esp-idf!43270
This commit adds support for uf2 targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/uf2.cmake.
- Adds the idf_create_uf2() function to create the uf2 targets. This
function now takes the executable as an argument thus allowing the uf2
target to be created per-executable.
- idf_project_default() is updated to create the uf2 targets.
This commit adds support for dfu targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/dfu.cmake file on similar lines as the
tools/cmake/dfu.cmake file.
- Adds the idf_create_dfu() function to create the dfu targets. This
function now takes the executable as an argument thus allowing the dfu
target to be created per-executable.
- idf_project_default() is updated to create the dfu target.
fix(uart): watchdog got triggered due to uart sclk not exist
Closes IDFGH-15851, IDFGH-16685, IDFGH-16687, IDFCI-2846, and TIDF-228846
See merge request espressif/esp-idf!43279
The test_rebuild_no_changes test verifies that running idf.py build
successively without any file changes results in identical build
artifacts on the second run (i.e., nothing gets rebuilt).
This test was failing in buildv2 because it expected kconfig_menus.json
to be present in build/config/ after a normal build. However, in
cmakev2, kconfig_menus.json is not generated during regular builds.
In cmakev1, kconfig_menus.json was generated globally during every
build alongside other config files (sdkconfig.h, sdkconfig.cmake, etc).
In cmakev2, kconfig_menus.json generation does not happend for
normal builds because it depends on the Kconfig menu hierarchy
and cannot be generated globally. It must be generated per-executable.
Hence, this commit updates the artefacts list for cmakev2 to not expect
the kconfig_menus.json file during a build/re-build action.
Interrupt wdt would get triggered on uart_driver_install if uart driver was deleted before
Closes https://github.com/espressif/esp-idf/issues/17779
RTC wdt would get triggered on esp_restart if uart driver was deleted before
MIN() macro is only an expansion of a conditional operator.
xRingbufferGetCurFreeSize was called twice in the original code,
which may return different values in two calls, leading to incorrect
send_size calculation and eventually could trigger task watchdog.
feat(parlio_rx): support to force trigger eof
Closes IDF-14143, IDFCI-2958, IDFCI-6027, IDFCI-2929, and IDFCI-6242
See merge request espressif/esp-idf!43116
- Update the Key Manager key types to be generic
- Define a new enum to determine the length of the keys
- Refactor the Key Manager driver support generic key types and key lengths
- Also store key deployment mode in the key recovery info
fix(nvs_sec_provider): replace abort() when nvs_keys is missing with a warrnig log (GitHub PR)
Closes IDFGH-16597
See merge request espressif/esp-idf!43337
Applicable for chips affected by interrupt issue:
- ESP32C5
- ESP32C61
- ESP32H4
For all other chips that support the ZCMP extension without issues,
it will be enabled unconditionally.
This change improves build consistency across external projects integrated
through CMake by ensuring that compiler flags defined in configuration files
are passed correctly to the toolchain. It covers the majority of use cases,
as external projects are typically also CMake-based. For projects that use
a custom build system, users will still need to specify the required flags
manually.
Add new APIs to retrieve statistics about power management locks:
- esp_pm_get_lock_stats(): Get statistics for all PM lock types
- esp_pm_get_lock_instance_stats(): Get statistics for a single PM lock instance
Also update the test cases to verify the new functionality and
enable PM profiling in test configuration.
Closes https://github.com/espressif/esp-idf/issues/17770
Changes generated by pre-commit related to ruff-format and an update to
the copyright date.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduce ESPTOOLPY_FAST_REFLASHING configuration option. It instructs
ldgen to group entity mappings for libraries deemed mutable (prone to
change) separately from those considered immutable (unlikely to change).
Organizing mutable and immutable libraries separately in the linker
script allows the linker to form larger contiguous blocks of data for
immutable libraries in the application's output sections. These blocks
are likely to stay mostly unchanged between application recompilations,
enabling them to be skipped during reflashing.
Separating mutable and immutable libraries in the linker script to
minimize changes in the output sections is insufficient. Padding is
added after the input sections of mutable libraries in the default data
and text output sections. This creates a buffer for the mutable
libraries, allowing additional changes to be made without altering the
layout of the binary image.
Additionally two optimizations currently in use can still mix data from
these libraries, leading to significant changes even within the grouped
immutable libraries.
1. constant merging
Linker will try to merge input sections that have the MERGE and
STRING flags from different libraries (object files) to perform
optimizations like tail merging. For example, adding a string
literal in a mutable library will also change the addresses of
string literals from immutable libraries in such a merged section,
causing changes in the generated code when those literals are
referenced.
Disabled with COMPILER_DISABLE_MERGE_CONSTANTS(-fno-merge-constants)
2. literal pools on Xtensa
As optimization, the linker may merge literal pools from different
libraries (object files) to improve the generated code size. This
has the same effect as constant merging, and changes in mutable
libraries may cause changes in the generated code for immutable
libraries. To get larger unchanged continuous blocks in the text
output sections for immutable libraries, we need to ensure that the
Xtensa literal pools remain close to their references and are not
merged.
Disabled with CONFIG_COMPILER_ENABLE_TEXT_SECTION_LITERALS(-mtext-section-literals)
When ESPTOOLPY_FAST_REFLASHING is enabled, these two optimizations are
disabled to achieve larger unchaged continuous blocks for the grouped
immutable libraries, even though disabling these optimizations results
in slightly larger code.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The build system keeps track of each component source. Currently
there are four types of sources:
1. "project_components" - project components
2. "project_extra_components" - components from EXTRA_COMPONENT_DIRS
3. "project_managed_components" - custom project dependencies managed by the IDF Component Manager
4. "idf_components" - ESP-IDF built-in components, typically under /components
This can be used to identify the component libraries that are likely to
change during application development and pass them to ldgen as mutable
libraries. Add all components with "project_components" as their source
as mutable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the `--debug` option to print the entity tree to stdout. This can be
useful for viewing the entire entity tree state to identify potential
issues or incorrect placements.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduce a new `--mutable-libraries-file` option that accepts a file
containing the filenames of mutable libraries, each listed on a separate
line. Mutable libraries are component libraries expected to change
during development. In contrast, immutable component libraries are not
expected to change. In the generated linker script, the input sections
of mutable libraries are grouped together rather than being mixed with
those of immutable libraries. The goal is to create large continuous
areas in the ELF file's output sections that remain unchanged for
immutable libraries during application recompilation, allowing these
areas to be skipped during flashing.
The build system identifies the mutable libraries and passes them to
ldgen using the `--mutable-libraries-file` option. It maintains
information about component sources, one of which is
`project_components`. This source type identifies components that are
directly related to the project being developed and are very likely to
change.
Mappings for mutable libraries are explicitly created for all sections
in the default scheme. This happens before the entity
`(archive:object_file:symbol/input_section)` node tree with placements is
generated and is equivalent to having these mappings in the mapping
linker fragment. All placements for mappings, whether newly added or
already existing as defined in linker fragments, associated with mutable
libraries are flagged as `mutable` in the entity node tree. This flag
ensures that these placements are included in the final linker script.
Currently, ldgen only emits placements that are either significant or
forced. A placement is considered significant if, for example, it is not
already covered by a placement in parent node. For instance, `*(.iram1
.iram1.*)` placement already includes `*libapp_trace.a:(.iram1
.iram1.*)`, so the latter is not emitted by default. The `mutable` flag
ensures that placements for mutable libraries are emitted in the linker
script and placed at dedicated location.
The locations where placements for mutable libraries are specified in
the linker script are identified by a new `mutable` marker, for example,
`mutable[flash_text]`. The placements for immutable libraries remain in
the existing `mapping` marker, for example, `mapping[flash_text]`. The
`mutable` marker for each target is placed after the `mapping` marker.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Entity mappings for immutable libraries are placed in the existing
`mapping[target]` marker, while those for mutable libraries are placed
in the new `mutable[target]` marker, which comes after the `mapping`
marker. Additionally, include padding after the input sections of
mutable libraries in the default data and text output sections,
providing a headroom for the mutable libraries to grow. Padding is
currently not added, for example, in the `.iram0.data` output section,
which is not expected to change frequently. Padding for other mutable
input sections may be added in the future.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Encrypted flash write operation sometimes result in random corruption in
certain bytes. Root cause points to sudden current surge due to involvement of
encryption block overwhelming LDO supply. More details will be provided
in the ESP32-C5 SoC Errata document.
This fix limits the CPU clock to 160MHz for flash encryption enabled
case. Failing encrypted flash write tests could successfully pass in
this configuration. Going ahead, a dynamic clock adjustment in flash
driver will be considered to mitigate this issue.
fix(wifi): fix issue that disarm send wake null timer donot take effect and sta bolck scan timeout
Closes WIFI-7020, WIFIBUG-1523, and WIFIBUG-1379
See merge request espressif/esp-idf!43158
fix(efuse): Disallow enabling SB using SHA-384 when SB using SHA-256 is enabled (ESP32-P4 ECO5)
Closes IDF-14063
See merge request espressif/esp-idf!42884
"RTC memory should be lowest priority and its free size should be big enough"
was failing because the expected size of RTC heap left after init was bigger
than the total RTC size itself.
The test was updated to:
- check that the minimum free RTC heap size matches the current size
- the current RTC heap size is bigger than 80% of the total RTC memory region
Add a new example showcasing how to obtain and interpret statistics
about the usage of an NVS partition.
Also demonstrates how stats change after writing some key-value pairs
in 2 separate namespaces.
fix: Update esptool command in GitHub template to use legacy command for backward compatibility
Closes IDFGH-16670
See merge request espressif/esp-idf!43179
The cmakev2 build system may evaluate components recursively and all
components has to make sure all their variables are properly
initialized.
Fixes: a7115b6b36 ("refactor(jpeg): Split hal layer for jpeg")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
ci(hal/tee): Fix intermittent hang-ups in `PERI_APM` tests on ESP32-C5
Closes IDFCI-3274, IDFCI-4807, and IDFCI-4808
See merge request espressif/esp-idf!42937
This is both a feature and an optimization.
Feature:
Adjustable size of the internal storage in esp_event queue, currently
used by ISR posting, as they wont be able to make a malloc.
Optimization:
When non-isr is posting an event, use the inernal storage in the struct
instead of always allocating a new heap for the data. Most events in
esp-idf only contains a few bytes event information, and we have that
allocation payed for anyway.
This solved in a big part our memory fragmentation issue, as events
happens freqvently and used to create small memory allocations for just
4 bytes, and then in the event handler we usually allocated a bigger
chunk of heap for our feature. When returning from the event handler,
the 4 byte allocation was freed, leaving a hole in the heap.
Merges: https://github.com/espressif/esp-idf/pull/17797
The `tools/test_build_system/buildv2_test_app` is a cmakev2 build system
testing application, same as `build_test_app`, which should be removed
from the readme check performed by check_build_test_rules.py.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
feat(ble): Support controller log compression for ESP32-C3 and ESP32-S3(0871069)
Closes BLERP-2287, BLERP-2288, BLERP-2286, and BLERP-2285
See merge request espressif/esp-idf!42293
The COMPONENTS and EXCLUDE_COMPONENTS variables are unused in cmakev2. A
deprecation warning will be printed if they are set. This change also
includes a helper functions for printing deprecation warnings.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 dropped generation of prefix_map_gdbinit, reflect this
change also in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently both menu configs, for component's project configuration and
component's configuration, for components not included in the build have
the same name: "Configuration for components not included in the build".
This might be confusing. Let's use "Project configuration for components
not included in the build" for component Kconfig.projbuild files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 added esp_stdio to common requires, add it in cmakev2 too
for backward compatibility.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
refactor(parlio_rx): use gdma_link for better gdma link management
Closes IDF-13369, IDF-13627, and IDFGH-16449
See merge request espressif/esp-idf!40743
feat(openthread): optimize fail cases in CI test
Closes IDFCI-2537, IDFCI-2642, IDFCI-3576, IDFCI-3623, IDFCI-3955, IDFCI-5136, IDFCI-5072, IDFCI-5065, IDFCI-4997, IDFCI-3026, IDFCI-3019, and IDFCI-3020
See merge request espressif/esp-idf!42971
Currently, the toolchain CMake files use the remove_duplicated_flags
function from utilities.cmake. The cmakev2 implementation also includes
this function for backward compatibility. Move the
remove_duplicated_flags function to a separate file,
deduplicate_flags.cmake, so it can be shared between cmakev1 and
cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Enable check-kconfig-files. It was temporarily disabled because it was
not possible to add a new sourced file through an environment variable,
as kconfcheck had a fixed list of allowed environment variable names.
This issue should now be resolved.
Currently, the buildv2 tests are initiated using the same patterns as
the tests for the current build system. This means that any change in
the current build system will also trigger the buildv2 tests. Initially,
it might be wise not to block the CI for changes in the current build
system in case there is an issue with buildv2. Therefore, let's
explicitly start the buildv2 tests only when the buildv2 label is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The esp_libc relies on the stdio implementation of the vfs component if
CONFIG_VFS_SUPPORT_IO is enabled. This is not an issue in cmakev1
because if the vfs component is not included in the project build, its
configuration is not available, and CONFIG_VFS_SUPPORT_IO is not set.
However, in cmakev2, the configuration for all components is available,
and the presence of some component configuration options does not
necessarily mean that the component is included in the project build.
When esp_libc is compiled without the vfs dependency but with
CONFIG_VFS_SUPPORT_IO enabled, the libc initialization will encounter a
panic due to a NULL pointer dereference (fp->_flags) in __swsetup_r, as
fopen in esp_libc_init_global_stdio will return NULL.
Although the current documentation exposes only limited functionality,
which is unlikely to change, add an explicit note that the build system
v2 is currently a technical preview intended for testing and early
evaluation.
Ensure consistency in referring to v2 throughout the documentation. In
some places, it is referred to as V2 or Version 2; unify this to v2.
Additionally, update the hello_world example for v2 to use the
idf::spi_flash alias instead of the interface variable to maintain
consistency with the rest of the documentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Extend the current documentation to recommend using the
idf_component_register function as the preferred method for creating a
new component for v2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Using idf_component_register is the preferred method for creating new
components for cmakev2. This approach ensures compatibility with both
versions of the build system. The KCONFIG and KCONFIG_PROJBUILD options
have been removed from the API documentation, but are retained in code
in case a cmakev1 component uses these options to warn about
incompatibility. Also remove a note about `project_include.cmake`,
because cmakev2 includes all project_include files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Enhance the `esp_target_info` component example by adding some of
the missing features that the component might require. This includes
adding a linker script, a linker fragment, linking the component archive
as a whole library, and providing an example of how the linker options
can be set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Components designed exclusively for cmakev2 are not compatible with
cmakev1 due to the lack of forward compatibility. It is important to
clarify this at the very beginning of the `Creating a New Component`
section. Additionally recommended a guidance on how to create a
component that will be compatible with both versions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the archive file name is set only in the cmakev1
idf_component_register shim. The predictable component archive file name
is important, for example, for usage in linker fragments. Ensure that
the cmakev2 component also has the archive file name set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev2 component sources are set, the SRCS target
property is used. This is obviously wrong because the correct CMake
property is SOURCES.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Rename the LINKER_SCRIPTS_STATIC component property to LINKER_SCRIPTS.
This property stores linker scripts that are not processed by ldgen,
which essentially includes all of them. The only linker script processed
by ldgen is sections.ld, which is handled by the esp_system component.
This implies that there is likely no practical use case for other
components to utilize ldgen processed linker scripts. This change is
purely cosmetic to allow components to add linker scripts with:
idf_component_set_property(${COMPONENT_TARGET} LINKER_SCRIPTS linker_script.ld APPEND)
instead of
idf_component_set_property(${COMPONENT_TARGET} LINKER_SCRIPTS_STATIC linker_script.ld APPEND)
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add very basic documentation. It is currently written with the
expectation that the reader is already familiar with the existing build
system and focuses more on the changes that might be necessary to adapt
existing components to work with cmakev2. Additionally, it provides
basic information on how to create a new component and project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, we include numerous functions in the automatically generated
documentation for the build system API. Let's begin with only the
essential functions and gradually add more to the API based on requests
and actual needs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit adds a workaround in idf_build_component shim to avoid
initialization of a duplicate managed component. The component manager
when running on a Windows system returns duplicate managed components
upon successive invocations.
The current approach involves wrapping the library target within the
library interface target with the whole-archive flags, such as for the
GCC linker. However, this does not work as expected because the library
target is expanded, and the whole-archive flags are also applied to
other library targets that the wrapped library target depends on. IOW
the whole-archive flags surround multiple archives, not just the one
requrested. Generally, using linker flags like whole-archive in the
library INTERFACE(INTERFACE_LINK_LIBRARIES) does not seem to work as
CMake may perform deduplication and rearrange the flags [1].
The proof of concept used `CMAKE_LINK_LIBRARY_USING_<FEATURE>` with the
WHOLE_ARCHIVE feature, which was introduced in CMake 3.24 to specify how
the library target should be linked. There are two issues with this.
First, this feature is only available from CMake 3.24 onwards, while our
minimum CMake version is set to 3.22. More importantly, all occurrences
of a library on the link line are wrapped. For example, if a library
like `vfs` appears multiple times on the link line due to dependencies,
each occurrence will be wrapped with whole-archive, causing the linker
to complain about multiple symbol definitions. Therefore, even though
WHOLE_ARCHIVE is recommended for handling whole-archive linkage, it does
not seem suitable for our purposes. This was overlooked in the PoC
because only a simple testing component was used to check the
WHOLE_ARCHIVE behavior.
One way to address this issue is to adopt the same approach used in
cmakev1, which involves specifying the whole-archive flags when linking
component library targets to the final executable. In this case, CMake
retains the flags without any alteration, unlike when the flags are
specified in INTERFACE_LINK_LIBRARIES for the component interface
targets. While this approach is feasible, it would alter the current
logic, where we have a single library interface for the entire idf
library that can be linked to the executable. This change would also
complicate the direct use of the idf library, as it would no longer be
possible to simply link it to the executable. Instead, the executable
would need to correctly link the component libraries with the
whole-archive flags. We could encapsulate this process within the
idf_build_executable function, but projects that only use
idf_build_library would need to implement the same solution.
It appears possible to address this issue by using target_link_options
instead. This allows to specify exactly what should appear on the link
command line. One side effect of this approach is that the library
appears on the link command line multiple times: first when used with
target_link_options and second when used in target_link_libraries.
However, this does not seem to pose a problem. The flags specified with
target_link_options appear on the command line first, ensuring that the
whole-archive is prioritized, and the component archive libraries may be
repeated on the link line anyway due to dependencies. Essentially, the
final link command line is very similar to the one from cmakev1. We may
revisit this approach in the future, but for now, it seems to work as
expected.
[1] https://gitlab.kitware.com/cmake/cmake/-/issues/20078
[2] https://cmake.org/cmake/help/latest/variable/CMAKE_LINK_LIBRARY_USING_FEATURE.html
[3] https://discourse.cmake.org/t/automatically-wrapping-a-static-library-
in-whole-archive-no-whole-archive-when-used-during-linking/5883
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev1 properties for INCLUDE_DIRS and
PRIV_INCLUDE_DIRS are constructed in the
__set_component_cmakev1_properties function, the return values from the
get_target_property function for INCLUDE_DIRECTORIES and
INTERFACE_INCLUDE_DIRECTORIES properties are not checked. If a component
target does not set e.g. INCLUDE_DIRECTORIES property,
get_target_property will return a value such as `include_dirs-NOTFOUND`.
This value is subsequently passed to __get_relative_paths in the PATHS
argument, causing the file(RELATIVE_PATH) call in __get_relative_paths
to fail with an error.
```
CMake Error at /home/fhrbata/work/esp-idf/tools/cmakev2/utilities.cmake:235 (file):
file RELATIVE_PATH must be passed a full path to the file:
include_dirs-NOTFOUND
```
Fix this by explicitly set the PATHS to an empty list before passing it
to the __get_relative_paths function if the property is not set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
idf_build_executable() creates a dummy source file to create the
executable target. Since this file is created unconditionally everytime
CMake configuration happens, a source file change is registered which
causes CMake to execute other targets even though no other source file
changed.
The commit adb2d5deee ("feat(cmake): Produce warnings when component dependen..")
introduced additional checks for source files and include directories
used by a component that are located outside the component's directory.
If these files and directories belong to another component, a warning is
issued. This feature has not yet been implemented in cmakev2, so related
tests are temporary disabled.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit renames the buildv2_test_app.c file to build_test_app.c in
the buildv2_test_app template to enable tests to refer to the same
source file name when writing tests for both cmakev1 and cmakev2.
This commit enables the following tests for cmakev2 in CI which do not
need any updates:
- test_bootloader.py
- test_git.py
- test_kconfig.py
- test_partition.py
- test_reproducible_build.py
- test_sdkconfig.py
In cmakev1, certain arguments provided to the idf_component_register
function are stored as component properties. These properties are used
to generate the project_description.json file, which other tools rely
on. Since the idf_component_register function is obsolete in cmakev2, we
need to recreate component properties such as INCLUDE_DIRS,
PRIV_INCLUDE_DIRS, REQUIRES, and PRIV_REQUIRES, which were previously
provided by the idf_component_register function in cmakev1. To achieve
this, let's examine the component's real target cmake properties and
reconstruct them to maintain compatibility with cmakev1.
The real target cmake properties may include generator expressions,
which are ignored. This should be acceptable, as they were not addressed
in cmakev1 either, and handling them is likely not feasible.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Return the difference between two lists, meaning the elements that are
present in the first list but not in the second.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This helper function removes entries containing generator expressions
from the given list. The list is modified in place.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a helper function that returns a list of paths relative to a
given base directory for a list of input paths.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When a linker script file with a .in extension is added using the
target_linker_script function, it is processed with the C preprocessor.
The linker scripts are preprocessed only once, even if they are used in
multiple libraries, because they are the same.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, add the COMPONENT_BUILD_DIR property. This can be used to
store component-specific generated files, such as preprocessed linker
scripts.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `includes` list variable is used without initialization. Initialize
it explicitly.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component has been renamed, update the name in the common
requirements for the cmakev1 components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component can be referenced by multiple identifiers such as the
component name, which is derived from the component directory name,
component target, interface, or aliases. All component properties are
attached to the component interface target, which is also used for
declaring component dependencies. The cmakev2 build system has a
function called __get_component_interface, which is responsible for
identifying the component interface based on the given component
identifier. Since this function is called frequently, it needs to be
reasonably fast.
This introduces a new __idf_component_interface_cache INTERFACE target,
which serves as a mapping cache between component identifiers and the
component interface. The cache for each component is initialized in the
__init_component function, which introduces a component to the build
system. Currently, the component interface search is conducted by
examining the COMPONENTS_DISCOVERED and COMPONENT_INTERFACES lists
stored as build properties. Since the build system is aware of most
component identifiers, such as component name, target, and alias, during
component initialization, it can add mappings between component
identifiers and the component interface to a cache, which is built as
the components are initialized. This cache is used in the
__get_component_interface function instead of looking into the
COMPONENTS_DISCOVERED or COMPONENT_INTERFACES lists. This significantly
speeds up the component interface search and also makes the code much
simpler and more readable.
The component interface cache also completely replaces the existing
component name resolution, which was introduced because of the component
manager, and the cache used for resolved component names. This is
possible because all the necessary information is available during
component initialization when the component interface cache is
populated. The ambiguity of components is resolved based on component
source/priority.
Here is an example of the component interface mapping for the
espressif__led_strip component to the idf_espressif__led_strip interface
target. The component name, without the namespace, is referred to as the
short name. In this example, it is led_strip.
- espressif__led_strip -> idf_espressif__led_strip # name(directory name)
- idf_espressif__led_strip -> idf_espressif__led_strip # interface
- idf::espressif__led_strip -> idf_espressif__led_strip # alias
- _idf_espressif__led_strip -> idf_espressif__led_strip # real target
- led_strip -> idf_espressif__led_strip # short name
When another component with the same espressif__led_strip name is
initialized with a different priority, there is no need to change
anything in the cache, because the mapping stays the same.
The cache must be updated when two components share the same short name
but belong to different namespaces. This situation is likely uncommon.
For instance, consider espressif__led_strip and my__led_strip. If
my__led_strip has a higher priority, the cache is updated to reflect the
short name as follows:
- led_strip -> idf_my__led_strip
If both components have the same priority, the short name mapping for
led_strip is entirely removed. Conversely, if my__led_strip has a lower
priority, no short name mapping is added for it.
The short name is also added to the COMPONENT_SHORT_NAME property of the
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a revised version of the existing __create_confserver_target
function. It creates a specified confserver target for a given
executable. The kconfig_menus.json file, used by the IDEs, is generated
when the confserver starts. This differs from the previous behavior,
where kconfig_menus.json was created globally along with other sdkconfig
formats. The reason for this change is that kconfig_menus.json contains
the Kconfig menu hierarchy and it is not just a flat option-value
format. It needs to accurately reflect which configurations for which
components are included or excluded. The kconfig_menus.json is generated
at `build/kconfig_menus.json`, where IDEs expect it. This means the file
is overwritten every time the server starts by kconfig_menus.json
version for given executable, so only one confserver can run at a time.
This is likely acceptable, as I don't believe it's possible to safely
run multiple instances of confserver due to the potential race
conditions when the sdkconfig file are generated.
In the future, we may include the location of kconfig_menus.json in
project_description.json so it can be easily identified by IDEs for
each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the initial sdkconfig is generated, the
kconfig_menus.json format is also created alongside other formats like
CMake, JSON, or C header. The kconfig_menus.json depends on the Kconfig
hierarchy and cannot be generated globally. It must be generated for
each executable to ensure that included and excluded components are
correctly positioned within the Kconfig menu hierarchy. Remove the
automatic generation of kconfig_menus.json.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current base kconfgen command is stored in the __BASE_KCONFGEN_CMD
build property, and it includes the --env-file option pointing to the
`build/config.env` file. The `build/config.env` file does not separate
component configuration files into included and excluded components,
because it is created before any executable is added and we need it for
the initial sdkconfig generation. The global `build/config.env` can be
used for every sdkconfig format (cmake, json, txt), but not for the
kconfig_menus.json format, because kconfig_menus.json relies on the
Kconfig layout and hierarchy. Let's allow specifying the --env-file for
each kconfgen invocation and remove it from the base kconfgen command.
This allows you to specify an --env-file for each executable, each with
its own config.env file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __run_kconfgen function does not take any arguments. The output
formats and related files are stored in the __KCONFGEN_OUTPUTS_CMD build
property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is basically a renamed version of __create_save_defconfig_target.
The creation of the save-defconfig target is done in
`idf_project_default()`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For a specified executable, create a menuconfig target using the name
provided in the TARGET option. The function generates a config.env file
specific to the executable, where Kconfig files for components linked to
the executable and Kconfig files for components not linked to it are
separated. This separation allows for a clear visual distinction between
the configuration of components that are linked and those that are not
linked to the executable within the menuconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Generate the config.env file for the specified executable. The
configuration file will be stored in the build directory, within a
directory named after the executable target name. The kconfigs*.in
files, which are generated by prepare_kconfig_files.py, will also be
stored in this directory.
This function primarily prepares the arguments for the
__create_config_env_file function based on the components linked to the
executable, ensuring that component Kconfig files are stored in the
appropriate kconfigs*.in files, depending on whether the component is
linked to the executable.
The directory where the generated files are stored is added to the
executable CONFIG_ENV_DIR property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __create_config_env_file function generates a configuration file for
the prepare_kconfig_files.py script. Currently, it specifies the paths
for the kconfigs*.in files, which are generated by
prepare_kconfig_files.py, to be located in the build directory. The
kconfigs*.in files, along with the configuration file, must also be
generated for each executable, as each executable may have a
different set of linked components. Set the paths for the
kconfigs*.in files to be generated by the prepare_kconfig_files.py
script in the same directory as the configuration file.
Also allow explicit specification of which component configuration files
should be placed in different kconfigs*.in files. For sdkconfig
generation, it is not relevant whether a component's Kconfig file is
placed in the kconfigs.in or kconfigs_excluded.in file. This distinction
is only important for visually separating the configuration of included
and excluded components in the menuconfig. For the main sdkconfig, all
component Kconfig files are placed in the kconfigs.in file, and the
generated kconfigs*.in files are stored in the build directory. For
executables, the generated kconfigs*.in files will be stored in separate
directories for each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The excluded Kconfig files cannot be collected globally, they must be
collected per executable. The distinction between included and excluded
Kconfig files is relevant only for menuconfig, which is specific to each
executable, as each executable may have a different set of linked
components. For the global sdkconfig file and its CMake and JSON
variants, whether the Kconfig files are excluded or not is irrelevant.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Commit 46b41dfec187 ("test(cmakev2): Enable CI tests for component manager ..")
introduced a temporary workaround for the cmakev2 testing
application by including the cmakev1 project.cmake file after the
return() statement. The issue is that some tests, such as
test_build_fail_on_build_time, append commands to the project's
CMakeLists.txt. These commands are placed after the newly added return()
statement, causing them to be ignored and resulting in test failures. To
resolve this, include the cmakev1 project.cmake within the cmake block
comment.
Fixes: 46b41dfec187 ("test(cmakev2): Enable CI tests for component manager ..")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Three tests are disabled for cmakev2:
* test_build_dfu
* test_build_uf2
* test_build_loadable_elf
and one is modified:
* test_build_skdconfig_phy_init_data
the other 14 tests should pass.
The tests, test_build_dfu and test_build_uf2, are temporarily disabled
because cmakev2 currently does not include the dfu and uf2 targets.
These tests should be re-enabled once support is added to cmakev2.
The test_build_loadable_elf is disabled because the bootloader_support
component determines its requirements based on the sdkconfig values,
specifically the CONFIG_APP_BUILD_TYPE_RAM used in this test. When
CONFIG_APP_BUILD_TYPE_RAM is set, bootloader_support declares a
dependency on micro-ecc. This is wrong even in cmakev1 and should be
fixed.
The test_build_skdconfig_phy_init_data test requires the esp_phy
component but does not specify it in its requirements. It relies on the
cmakev1 behavior, where all components are included in the build by
default. This approach does not work for cmakev2. To resolve this, make
the dependency on esp_phy explicit, ensuring compatibility with both
versions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the cmakev2 implementation of Kconfig name consistency has a
slightly different warning message compared to cmakev2, causing the test
`test_build.py::test_build_with_misspelled_kconfig` to fail. Make the
warning messages consistent with cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The test_build_with_misspelled_kconfig test expects a KConfig.projbuild
file in the main component. However, the file is correctly named
Kconfig.projbuild, which causes the test to fail for cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, cmakev2 evaluates each component using add_subdirectory with
the EXCLUDE_FROM_ALL option. The intention was to exclude all components
from being built by default unless they are explicitly linked to an
executable based on dependencies. This approach aims to avoid building
components that are included but not actually linked to the executable.
However, this has the side effect of preventing components from adding
their custom targets to the "all" target, which is used, for example, by
esp_phy. Generally, we should not restrict components from adding
targets to "all". Since components are only included if explicitly
requested by default, removing the EXCLUDE_FROM_ALL option should be
acceptable.
The downside is that if IDF_INCLUDE_ALL_COMPONENTS is set and the user
runs `idf.py build`, all components will be built, even if they are not
linked to the executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the check is based on the existence of sdkconfig, but there
may be situations where sdkconfig exists without containing
CONFIG_IDF_TARGET. Perform a target consistency check only if the target
is identified in the main sdkconfig file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The function arguments in the documentation comments were using field
list, which caused text overflow in the generated documentation and
generally resulted in poor formatting. Let's use paragraphs for
the argument descriptions instead.
The documentation comments are written in reStructuredText, but
currently, they use inconsistent indentation. Standardize all the
documentation comments to use a four-character indentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add stump files for the build system v2 documentation with automatic API
generation from CMake files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Load the esp_docs_cmakev2_extension to process documentation comments
from CMake files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a Sphinx extension that introduces a new `cmakev2` domain with
multiple directives, allowing for the automatic extraction of
documentation comments from CMake files and their inclusion in the
Sphinx-generated documentation.
Directives:
- `cmakev2:include`: The included CMake file is processed for
documentation comments within the `#[[api` and `#]]` marks, which
should contain valid reStructuredText markup.
- `cmakev2:function`: Creates a CMake function node. All function nodes
are sorted by name and placed into the `_cmakev2_functions` section.
- `cmakev2:macro`: Creates a CMake macro node. All macro nodes are
sorted by name and placed into the `_cmakev2_macros` section.
- `cmakev2:variable`: Describes a CMake variable node. All variable
nodes are sorted by name and placed into the `_cmakev2_variables`
section.
Each node can be referenced with `` :cmakev2:ref:`<node name>` ``, where
the node name is the function, macro, or variable name as used in the
related directive.
Example:
CMake file:
```
#[[api
.. cmakev2:function:: idf_flash_binary
#]]
```
This function can be referenced with `` :cmakev2:ref:`idf_flash_binary` ``
and will be placed in the `.. _cmakev2_functions:` section.
The extension is currently located in esp-idf, but in the future, we
should consider moving it to esp-docs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The build system should include the file generated by the component
manager after downloading components even if the component manager
requested a re-run due to a missing Kconfig.
This commit updates the order of initialization for build system v2. All
non-project initializations are handled in idf.cmake and all
project-specific initializations are now moved to idf_project_init().
Generate the flasher_args.json file for the global flash target for
tools that require it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Generate binary images and add flash targets for hello_world_example and
fatfs_example.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
All other functions, such as idf_build_binary, use OUTPUT_FILE. This is
more descriptive and provides more flexibility to add additional
file-related options in the future if necessary. Make
idf_build_generate_metadata consistent with the other function
interfaces by using OUTPUT_FILE.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Use idf_build_binary, idf_sign_binary, and idf_flash_binary to generate
and flash the default project application.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Given the binary target generated by the idf_build_binary or
idf_sign_binary function, add a new POST_BUILD dependency that displays
a message indicating the binary image must be manually signed before
flashing. This situation arises when CONFIG_SECURE_SIGNED_APPS is
enabled, but CONFIG_SECURE_BOOT_BUILD_SIGNED_BINARIES is not set. As a
result, the binary image is not signed during the build process and
requires manual signing.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Given the binary target created by the idf_build_binary or
idf_sign_binary function, add a new dependency that verifies whether the
binary image fits within the smallest application partition.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a new flash target for a previously created binary target using
the idf_build_binary or idf_sign_binary function. Specify the new target
name with the TARGET option. The binary image path is stored in the
BINARY_PATH property of the binary target and is set when the binary
target is created using the idf_build_binary or idf_sign_binary
function. Use the NAME option as a prefix for the generated esptool
argument file, and the FLASH option can be used to include the binary
image flashing in the global flash target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a signed binary image from a previously generated binary using
the `idf_build_binary` function. This image can be signed with an
optional key from a KEYFILE or, by default, with the key specified in
the sdkconfig. A new custom command is added to generate the signed
binary, which is stored at the location specified by the OUTPUT_FILE
option and a new custom target is created for it, with the name provided
in the TARGET option. The BINARY_PATH property is added to this new
TARGET, containing the path to the signed binary image.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `idf_build_binary` function generates a binary image for a specified
executable target, which may or may not have been created using the
`idf_build_executable` function. It adds a custom command to generate
the binary image file at the location specified by the `OUTPUT_FILE`
option and creates a new target for it, with the name specified by the
`TARGET` option. The path of the generated binary file is also stored in
the `BINARY_PATH` property of the newly created binary target.
Additionally, this binary path is stored in the `EXECUTABLE_BINARY`
property of the given executable. This information is used by the
`idf_build_generate_metadata` function, which takes the executable
target as an argument and includes the binary name in the
`project_description.json`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The global flash targets are created using
__create_project_flash_targets, but these are merely placeholders that
allow components to declare their dependencies on the global flash or
encrypted-flash target in their project_include.cmake. The actual logic
for flash targets is located in the project_include.cmake of the
esptool_py component. Therefore, the targets must be initialized
after this file has been included. To address this, a new function,
__init_project_flash_targets, has been added for this purpose.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a phony target that intentionally fails the build, displaying the
error messages specified in the message arguments. An empty CMake file is
created and included, which is then deleted when the target is executed.
This forces CMake to rerun, as the file is added to the CMake rerun
dependencies.
This has been carried over from cmakev1 as it is used by several
components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the lwip is linked to mbedtls if CONFIG_LWIP_ENABLE is set.
This approach works in cmakev1 because only the configuration for
BUILD_COMPONENTS is available during component evaluation. However, this
is not the case in cmakev2, where the configuration for all components
is available. This means that even if CONFIG_LWIP_ENABLE is set, it does
not guarantee that the lwip component is included in the build. For
cmakev2, modify the check to use a generator expression to ensure that
lwip is linked to mbedtls only if lwip is actually included in the
build.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Some components rely on the existence of flash targets to declare
dependencies. Add flash and encrypted-flash targets during project
initialization to ensure they are available before including
project_include.cmake files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Since the idf_project_default is already available, let's use it in the
buildv2 testing application. Also change the test app name from
buildv2_test_app to build_test_app, as some tests require the ELF and
bin images to be created under this name.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The optional dependency is currently added if the optional component
interface target exists, which is always the case unless a non-existent
component is requested. Instead, base the optional dependency on the
component interface target alias, as it is created only when the
component is included in the project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
If the IDF_INCLUDE_ALL_COMPONENTS variable is set, all discovered
components will be included. By default, components are included based
on their requirements. When IDF_INCLUDE_ALL_COMPONENTS is enabled, the
all_component_info dictionary in the project_description.json file will
be fully populated.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `idf_build_generate_metadata()` function is used to generate project
metadata, specifically the `project_description.json` file, for a given
executable target. The output file path can be specified as an argument
to the function, allowing metadata generation for multiple executables.
It identifies the library, created by the `idf_build_library` function,
that is linked to the executable based on the `LINK_LIBRARIES`
executable target property. The `LIBRARY_INTERFACES` build property
tracks all library interfaces created with the `idf_build_library`
function, so the targets in the `LINK_LIBRARIES` executable property are
checked against this list to find the library. The library contains
information about components linked to it in the
`LIBRARY_COMPONENTS_LINKED` library property. These components are
processed and added to the `build_component_info` dictionary in the
`project_description.json` along with other information.
Note that, compared to cmakev1, the `all_component_info` dictionary in
`project_description` does not include some information by default, such
as included directories. This information is available in cmakev1 due to
early component evaluation, which is not done in cmakev2. This will
likely require some adjustments on the hints side.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 sets __PREFIX as a component property. To maintain backward
compatibility, set it in cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Search for the library interface target created with the
idf_build_library() function and linked to the executable, examine the
LINK_LIBRARIES for the executable and the LIBRARY_INTERFACES build
property, which stores all library interface targets created by the
idf_build_library() function.
This helper is intended for use by functions, such as the
idf_build_generate_metadata() function, which accepts the executable
target as an argument to find the library linked to it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_project_default macro is intended to be used by default for
ESP-IDF projects. Currently, it handles the necessary project
initialization that must occur after CMake's project() is called. It
creates the default executable based on the project name and links a
library to it, based on the main component and its transitive
dependencies. This macro will be further extended to also generate
binary image, project metadata, and add other expected targets. It
primarily serves as a replacement for the project() macro used in
cmakev1.
Expected typical usage:
```
cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmakev2/idf.cmake)
project(hello_world LANGUAGES C CXX ASM)
idf_project_default()
```
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create an empty source file for the add_executable target, along with a
library containing specified components, and then create an executable.
This approach allows for the easy creation of multiple executables by
specifying the component with the app_main function, as demonstrated in
the test.
```
idf_build_executable(fatfs_example COMPONENTS fatfs_example)
idf_build_executable(hello_world_example COMPONENTS hello_world_example)
```
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The option is unnecessary and complicates the interface. Additionally,
it does not adhere to the previously established naming convention. The
target name for the library interface is provided as a standard function
argument.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmakev2, the component CMakeLists.txt files are evaluated recursively
based on the component dependencies. This means that variables from
previously evaluated components are set and available in the scope of
the currently evaluated component. It is necessary to initialize every
variable that is used.
Fixes: 56e0c11bb6 ("feat(usb_cdc_console): moved usb-cdc ROM console to n..")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add two components: fatfs_example and hello_world_example. These are
simply copies of the main components from the existing examples.
Enhance the test_executable test to create targets for two
executables, which can be built after CMake configuration.
```
cmake --build build/ --target hello_world_example
cmake --build build/ --target fatfs_example
```
Additionally, move the test_component_priority to the end of the test
chain, as it modifies the esp_system component, which interferes with
the linkage of the executables.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Integrate the ldgen into cmakev2. With this change, it becomes possible
to actually link the project executables.
In cmakev2, the handling of linker scripts is deferred to
idf_build_library, unlike in cmakev1, where linker scripts were added
and generated during the target_linker_script call. In cmakev2, the
target_linker_script only adds the linker scripts and templates, along
with the output filenames for the linker scripts generated from the
templates, to the component property. When idf_build_library is called
and all the requested components are included, it uses the
__get_target_dependencies function to obtain all transitively linked
targets to the library interface target. These targets are mapped to the
components, and the LIBRARY_COMPONENTS_LINKED library property is set.
It contains all components linked to the library interface target. The
components from LIBRARY_COMPONENTS_LINKED are used to collect linker
fragments and linker scripts utilized in the library. Additionally, all
targets transitively linked to the library are used to identify archive
files used in the library. This includes component archives and archives
added with the add_prebuilt_library function. The archives and
ldfragments related to the components linked to the library are used
when ldgen generates the linker scripts from templates.
The linker scripts, both static and generated by ldgen, are added to the
library interface link options and INTERFACE_LINK_DEPENDS property. For
generated linker scripts, a custom target is created and added as a
dependency for the library interface to ensure they are generated before
the link.
The difference compared to cmakev1 is that the generated linker scripts,
currently only sections.ld, are not global in the project but are
generated per library. This means there might be multiple versions of
sections.ld depending on the components included in the library. For
example, a component like esp_system may be linked to multiple library
interface targets, each with a different set of components. This results
in different sets of fragment files and library archives and different
versions of the sections.ld linker script. This should ensure proper
dependencies between targets. In other words, if a component changes its
linker fragment, only executables linked to libraries using this
component should be re-linked. As a consequence of this approach, the
generated linker scripts for different libraries need to have different
names or be stored in different directories to avoid overwriting the
linker script for one library with the linker script for another library
using the same component. This is handled with a suffix, which is based
on the library interface target name and appended to the generated
linker script. So, for example, there is no sections.ld, but instead
sections.ld_fatfs_lib or sections.ld_hello_world_lib. As a next step, we
can add a DEFAULT option to idf_build_library and avoid adding the
suffix for the default library.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
There may be multiple libraries, each linking a different set of
components. Introduce a new library property, LIBRARY_COMPONENTS_LINKED,
to track the components linked with a specific library. This property
can be used, for example, by menuconfig to distinguish between included
and excluded components on a per-library or per-executable basis, or by
ldgen to identify linker scripts for the linked components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the INTERFACE_LINK_LIBRARIES and LINK_LIBRARIES properties,
recursively retrieve the list of dependencies for the specified target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the common components are initialized in the idf_project_init
macro, which means they are included even for cmakev2 components.
However, cmakev2 components are expected to explicitly specify all their
dependencies instead of relying on common components being automatically
linked. Therefore, common components are only relevant within the
context of cmakev1 components. To address this, move the
__init_common_components function call to idf_component_register and
include common components only when they are truly needed for a cmakev1
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a simple test_executable that links to all discovered components.
This is intended to verify if the component sources can be compiled
using the command `cmake --build build/ --target test_executable`. The
ldgen is not yet integrated, so the test_executable is not actually
created or linked. However, this process can be used to check if the
components are compiled.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Set the ESP_PLATFORM macro to indicate to component sources that they
are being compiled with ESP-IDF.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the generic compile_options are ignored because they are
unset after retrieving the build property. To fix this, avoid explicitly
setting compile_options, as this variable is initialized with the
idf_build_get_property(compile_options ...) call.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Automatically link commonly required components to the component target
created in the idf_component_register function. This is still necessary
even in cmakev2, as existing components depend on this behavior.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Identify the commonly required components based on the target and
architecture, and store them in the __COMPONENT_REQUIRES_COMMON build
property. These components are automatically included, and their
interfaces are stored in the __COMMON_COMPONENT_INTERFACES build
property. The commonly required component interfaces are automatically
linked to each component.
Existing components for cmakev1 rely on the automatic linking of common
components. Therefore, at least initially, we need to implement the same
functionality in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmakev2, the project_include.cmake files for every component are
included. This means that even when working with RISC-V, the
project_include.cmake file for Xtensa is still included. Ensure that the
architecture is verified, and exit if it is not Xtensa.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the configuration, set the IDF_TARGET build property. This is
also set by cmakev1 and can be used to easily determine the architecture
for the project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add embedded source files to the component target sources. This is
intentionally managed in idf_component_include instead of
idf_component_register, allowing even cmakev2 components to set the
component's EMBED_FILES and EMBED_TXTFILES properties. Although it might
be more convenient to call the target_add_binary_data helper function
directly in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a backward-compatible idf_component_register function. This is
necessary to enable existing components to integrate with cmakev2. Note
that, by design, when the configuration is evaluated first, the KCONFIG
and KCONFIG_PROJBUILD options become obsolete, and an error is reported
if non-default configuration file names are specified.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
IDF_BUILD_V2 does not have BUILD_COMPONENTS build property. Therefore,
when IDF_BUILD_V2 is defined, use a generator expressions instead.
The inclusion of esp_eth_netif_glue.c, which relies on the presence of
the esp_netif component, should ideally be managed by a configuration
option. The same applies to the requirements for esp_eth. Currently,
only a basic adjustment has been made to enable the inclusion of esp_eth
in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
IDF_BUILD_V2 does not have BUILD_COMPONENTS build property. Therefore,
when IDF_BUILD_V2 is defined, use idf_component_optional_requires and
generator expressions instead. I believe that the one-liner should also
work with IDF_BUILD_V1, but the change is kept separate to clearly show
the difference between IDF_BUILD_V1 and IDF_BUILD_V2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
IDF_BUILD_V2 does not have BUILD_COMPONENTS build property. Therefore,
when IDF_BUILD_V2 is defined, use a generator expressions instead. I
believe that the one-liner should also work with IDF_BUILD_V1, but the
change is kept separate to clearly show the difference between
IDF_BUILD_V1 and IDF_BUILD_V2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a component interface alias only after the actual target is created,
meaning the component is included. The alias has a well-defined name and
can be used in generator expressions like
`$<$<TARGET_EXISTS:idf::esp_netif>:>` without needing to retrieve the
COMPONENT_ALIAS property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add binary data into the build target by converting it into a generated
source file, which is then compiled into a binary object as part of the
build process.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The add_deprecated_target_alias function is used by some existing
components, so it should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The file_generate function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The fail_target function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The add_prefix function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The spaces2list function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the specified component as a dependency only if it is included in
the build. This functions the same way as in cmakev1, but it uses
generator expressions because cmake2 does not maintain the
BUILD_COMPONENTS build property, which would list all components
included in the build.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, this is just a placeholder without implementation. The
implementation will be completed in a subsequent patch set that
integrates ldgen. For now, this is enough for ESP-IDF components can be
included by cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __DEPENDENCY_CHAIN was not properly created because the component
names were not added to it. Fix this by maintaining it correctly.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This removes the temporary restriction that only allowed components with
project_components as their source to be evaluated.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This helper function collects component sources within the component's
directory. It will be used by the idf_component_register.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add an optional BASE_DIR option to specify a directory that will serve
as the base directory for evaluating input paths. This can be useful,
for example, when collecting component sources.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The toolchain files are using the remove_duplicated_flags function from
utilities.cmake. However, we want to avoid mixing utilities from cmakev1
and cmakev2. Use `IDF_BUILD_VER_TAG` to include utilities from the
currently used build system version.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the IDF_BUILD_VER_TAG variables to include the build system version
tag. These variables can be used in files, such as CMake toolchain
files, which are shared between cmakev1 and cmakev2. This allows for the
inclusion of files from the currently running build system version. For
example:
include($ENV{IDF_PATH}/tools/cmake${IDF_BUILD_VER_TAG}/utilities.cmake).
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
It is not possible to add a new sourced file through an environment
variable because kconfcheck has a fixed list of allowed environment
variable names. This issue is tracked under IDF-13847.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The compat.cmake file is intended to include functions necessary for
backward compatibility with cmakev1.
Add the check_expected_tool_version function, which is used by some
project_include.cmake files to verify if the tool's version matches the
installed tool. This functionality is used, for example, by esp_common.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Some idf.py commands, such as menuconfig, use build_target and call
ensure_build_directory, which requires the project_description.json file
to be generated. To enable the use of idf.py commands with the testing
example, call __generate_project_info() to generate the
project_description.json file. This allows you to execute idf.py
menuconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current config.env.in template cannot be easily extended and shared
between cmakev1 and cmakev2. Although the root Kconfig is shared and
sources COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE and
COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE, these environment
variables cannot be set to empty strings. This issue arises unless we
also adjust cmakev1 and esp-docs, which use prepare_kconfig_files.py, to
properly set COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE and
COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE.
If an environment variable is set to an empty string and sourced in
Kconfig, kconfgen will expand the variable with expandvars(), resulting
in an empty value. This expanded variable is then appended to the root
Kconfig or the currently processed Kconfig directory path, depending on
whether osource or orsource is used, and passed to iglob. The iglog will
return the directory as a result, and when Kconfig attempts to open the
file, it will encounter an error because it is a directory.
To resolve this issue and enable sharing of the root Kconfig file, a new
config_buildv2.env.in template has been added, which includes
environment variables for the excluded components.
For cmakev1, kconfgen will encounter the osource command for
COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE. Since this variable is not
defined in config.env.in, expandvars() will return the environment
variable name instead of expanding it, and iglob will not match
anything. Because osource is used for excluded components, the
non-existent file for osource will be skipped.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Pass the environmental variables COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE
and COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE to menuconfig.
This will display the configuration for excluded components under new
menus in menuconfig. Additionally, set the IDF_BUILD_V2 environment
variable to make these new menus visible. The IDF_BUILD_V2 variable is
also passed to all kconfgen commands.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
To generate a complete configuration for all discovered components, we
need to ensure that even kconfig files for the excluded components are
seen by Kconfig. Modify the prepare_kconfig_files.py script to generate
files that list the Kconfig files for excluded components. These files
are sourced in the root Kconfig using the
COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE and
COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE environment variables.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduce new configuration menus for components that are not part of
the build. These menus will only be visible if the IDF_BUILD_V2
environment variable is defined and it will be set only in cmakev2. This
enables sharing of the root Kconfig file between cmakev1 and cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For backward compatibility, since externalproject_add is used by
project_include.cmake in the bootloader component. The ExternalProject
should probably be included there instead.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, store the full path of the project_include.cmake file in the
component's __PROJECT_INCLUDE property, if it exists. Include the
project_include.cmake files for all discovered components at the global
scope within the idf_project_init macro.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Set the global variables IDF_BUILD_V2 and IDF_BUILD_VER, along with the
build properties and environmental variables. These are intended to be
used by components to determine which build system version they are
evaluated under and to make adjustments based on this information.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current approach involves transparently calling __init_build within
idf_build_library and performing all post-project() initialization
there. The issue with this method is that project_include.cmake files
also need to be included during post-project() initialization, but they
should be included in the global scope. This cannot be achieved within
the __init_build function unless it is converted into a macro. Although
using a macro is a potential solution, it risks causing global scope
pollution. Another complication is the location where project() can be
invoked; it must be explicitly[1] stated in the project's
CMakeLists.txt. This requirement conflicts with our intention to wrap
it within the idf_project_default helper. Given these challenges, it
makes sense to introduce an explicit idf_project_init macro, where all
post-project() initialization occurs, including the inclusion of
project_include.cmake files. While we can still encapsulate this within
idf_project_default, for the plumbing commands (idf_build_library,
idf_build_executable, ...), it will need to be explicitly called after
the project() invocation.
Usage for default project:
cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmakev2/idf.cmake)
project()
idf_project_default()
Usage for plumbing commands:
cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmakev2/idf.cmake)
project()
idf_project_init()
idf_build_library()
...
[1] https://cmake.org/cmake/help/latest/command/project.html#usage
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
A new pytest_buildv2_system job has been added to execute selected tests
from the existing test_build_system test suite. Currently, only the
test_non_default_target.py is enabled. The ultimate goal is to run most
of the build system tests with buildv2. This should help ensure
compatibility between IDF build system v1 and v2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This marker enables the skipping of tests that, for any reason, cannot
be executed with the IDF build system version 2. It accepts an optional
string argument that explains why the test cannot be run with version 2.
If no explanation is provided, a default message is used. This marker is
used in the `pytest_collection_modifyitems` hook to skip tests marked
with it when the `--buildv2` pytest command line option is used.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The IDF build system v2 should be backward compatible with IDF build
system v1 in most situations. Therefore, it makes sense to reuse the
existing v1 tests and run them for v2 as well. This approach will help
ensure that v2 maintains backward compatibility. Introduce a new
--buildv2 option, which switches the existing tests to use the newly
added buildv2_test_app for v2. The goal is to enable the existing v1
tests incrementally in CI as the v2 implementation progresses.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the target consistency check with sdkconfig is performed even
if sdkconfig is not present, by comparing the selected target with the
target defined in sdkconfig.defaults. This results in an inconsistency
report when a different target is set in the IDF_TARGET environment
or cmake variable compared to sdkconfig.defaults. The behavior in
cmakev1 is to prioritize IDF_TARGET and ignore values in
sdkconfig.defaults. Perform the consistency check only if the
sdkconfig file exists.
Additionally, skip this consistency check if a set-target action is
taking place, as the old target in sdkconfig is being replaced with a
new value anyway.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The test for guessing the target from the SDKCONFIG_DEFAULTS environment
variable is not cleaning up after itself, leaving the SDKCONFIG_DEFAULTS
environment variable set. The subsequent test performs target guessing
from SDKCONFIG_DEFAULTS passed to cmake with the -D option. The
SDKCONFIG_DEFAULTS environment variable should take precedence over the
SDKCONFIG_DEFAULTS cmake variable. This is correctly handled in cmakev2
, but in cmakev1, the SDKCONFIG_DEFAULTS cmake variable is used even if
the SDKCONFIG_DEFAULTS environment variable is set. This appears to be a
bug or at least an inconsistency in cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, this generates a very basic project_description.json file,
allowing idf.py commands that call ensure_build_directory, such as
reconfigure, to be used with cmakev2. The ensure_build_directory
function sets a global context, which is presently used only for hints,
and the project_description.json file is loaded as part of this context.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add two project components, component1 and component2, and verify that
their real targets are created when they are included.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Call idf_component_include for all components requested in the library
and link them into it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a core function of the build system, responsible for including the
specified component identified by name, into the build
process.
Currently, the non-project components written in cmakev1 are ignored and
not evaluated with the add_subdirectory command because there are no
shims for the cmakev1 API yet. However, this allows for the evaluation
of project components written using the cmakev2 approach, which is
closer to the native CMake usage.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Gather the compilation options from COMPILE_OPTIONS, C_COMPILE_OPTIONS,
CXX_COMPILE_OPTIONS, and ASM_COMPILE_OPTIONS build properties into a
single list using generator expressions. This list can then be used with
the target_compile_options call.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Set the LINKER_TYPE build property. Different linkers may have varying
options, so it's important to identify the linker type to configure the
options correctly. Currently, LINKER_TYPE is used to set the appropriate
linker options for linking the entire archive, which differs between the
GNU and Apple linkers when building on the host.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Simple test for PROJECT_NAME and PROJECT_VER build properties to ensure
they are set according to the values provided in the project() call.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, only the value stored in CMake's cache is considered when
setting the default value. Allow regular variables as well, which may be
set in the project's CMakeLists.txt file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Initialize the PROJECT_VER build property based on the following
precedence.
1. The PROJECT_VER environment or CMake variable.
2. The version.txt file located in the top-level project directory.
3. The VERSION argument, if provided, in the project() macro.
4. The output of git describe if the project is within a Git repository.
5. Defaults to 1 if none of the above conditions are met.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the __init_build_configuration() function to set all C, CXX, ASM,
and linker options, along with compilation defines, in a single
location. In cmakev1, these settings are scattered across multiple
places and it seems logical to group them together.
This function is called from the __init_build(), which is invoked after
project(). This sequence is necessary because some options depend on the
CMake information about the compiler, which is only available once the
toolchain has been initialized in project().
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Most of the global settings are configured in idf.cmake, but certain
elements can only be initialized after the project() function is called
and the toolchain is set up. For example, this includes the compilation
options, which may depend on identified compiler or the project name and
version set in the project() call. The __init_build function is called
at the beginning of the idf_build_library() function, meaning it should
be executed after the project() call. It ensures that initialization
occurs only once by setting the __BUILD_INITIALIZED build property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Initialize submodules that are not yet initialized, and issue a warning
for submodules that do not match the recorded hash in the git tree.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When working with command output, it is sometimes necessary to handle it
line by line. Add a `__split` helper function that can split an input
string based on a specified separator, which defaults to a newline, and
store the results in a list. This function also offers modifiers to
strip whitespace and remove empty entries.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Initialize the IDF_VER build property using the version.txt file, or use
git-describe. If neither is available, fall back to the value of the
IDF_VERSION environment variable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The Git tool will be used in various parts of the build system, such as
retrieving the IDF version using git-describe or checking submodules.
Similar to the Python interpreter, add an initialization function to
identify the Git executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The following functions are added to tools/cmakev2/kconfig.cmake:
- __generate_kconfig_outputs(): Generates Kconfig output files.
- __create_base_kconfgen_command(): Creates command for running
kconfgen.
- __run_kconfgen(): Invokes kconfgen tool.
The following functions are added to tools/cmakev2/kconfig.cmake:
- __setup_kconfig_environment(): Creates Kconfig environment for Kconfig
output generation.
- __get_init_config_version(): Fetch ESP-IDF version and set the
environment variable.
- __create_config_env_file(): Create the config.env file for kconfgen.
The following functions are added to tools/cmakev2/kconfig.cmake:
- __collect_kconfig_files_from_components(): Collect Kconfig files from
components.
- __collect_kconfig_files_from_bootloader_components(): Collect Kconfig
files from bootloader components.
- __collect_kconfig_files_from_directory(): Collect Kconfig files from
a directory.
This commit adds the tools/cmakev2/kconfig.cmake file for Kconfig
processing. The file provides the following functions:
- __init_kconfig(): Placeholder to initialize Kconfig build properties,
collect kconfig files and generate the initial sdkconfig.
- __generate_sdkconfig(): Placeholder to generate sdkconfig output files.
Add idf_build_library function and enable setting and getting its
properties. This lays the groundwork for future initialization that
needs to occur after the project() call, such as determining compilation
options and others.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `idf_build_(set|get)_property` and
`idf_component_(set|get)_property` functions share a lot of similar
code. Move these common parts into new `__(set|get)_property` helper
functions. With the upcoming `idf_build_library` API function, we might
need to add properties for the interface target created for the library,
which would otherwise lead to yet another code duplication for setting
and getting library interface properties.
Update the current implementations of `idf_build_(set|get)_property` and
`idf_component_(set|get)_property` to utilize these new helper
functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Determine the IDF_TOOLCHAIN from the following sources in this order:
environmental variable and CMake cache variable. Ensure there are no
inconsistencies between the values set in these different locations.
Set the IDF_TOOLCHAIN and IDF_TOOLCHAIN_FILE build properties. Also,
configure the IDF_TOOLCHAIN CMake cache variable and set the
CMAKE_TOOLCHAIN_FILE global variable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Determine the IDF_TARGET from the following sources in this order:
environmental variable, CMake cache variable, and sdkconfig files.
Ensure there are no inconsistencies between the values set in these
different locations.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the environmental variables, CMake cache variables, or default
values, set the DKCONFIG and SDKCONFIG_DEFAULTS build properties.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Determine the Python interpreter and verify package dependencies if the
CMake cache variable PYTHON_DEPS_CHECKED is not set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The version.cmake file should be the sole file used from the cmakev1
build system. There's no need to maintain the IDF_VERSION information in
two separate locations.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Helper for obtaining the default value of a variable. It returns the
value of the specified variable based on the following order of
precedence, with the highest precedence first:
1. Environmental variable
2. CMake cache variable
3. Provided default value
This can be used to retrieve the value of variables that might also be
set in the environment or cache, such as PYTHON or SDKCONFIG.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
With the default signature of `cmake_parse_arguments`, without using
`PARSE_ARGV`, it's not possible to determine if options were not
specified or set as an empty string (or empty list)[1]. If an empty
string is passed to the `PATHS` option, the variable parsed by
`cmake_parse_arguments` is not defined. This issue can be addressed by
using `PARSE_ARGV`, but this approach only works for functions and
requires CMake version 3.31 or newer. Additionally, when `PARSE_ARGV` is
used for multiple value option, the values are not concatenated into a
single list, which is inconvenient, as the lists are instead escaped. If
the `PATHS` option is not defined, set it to an empty string. This
allows passing an empty string, as well as a mix of lists and
individual strings, through PATHS.
The behaviour can be seen with a simple example:
$ cmake -P test.cmake
```test.cmake
cmake_minimum_required(VERSION 3.22)
function(test)
set(options)
set(one_value)
set(multi_value PATHS)
cmake_parse_arguments(ARG "${options}" "${one_value}" "${multi_value}" ${ARGN})
#cmake_parse_arguments(PARSE_ARGV 0 ARG "${options}" "${one_value}" "${multi_value}")
message("PATHS: ${ARG_PATHS}")
endfunction()
test(PATHS "one;two;three" "four" "five;six" "seven")
```
```
PATHS: one;two;three;four;five;six;seven
vs
PATHS: one\;two\;three;four;five\;six;seven
```
Also update the current usage of __get_absolute_paths, as the check for
empty PATHS is no longer necessary.
[1] https://cmake.org/cmake/help/latest/policy/CMP0174.html#policy:CMP0174
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a basic CMake project that currently includes tests for
displaying build and component properties, as well as testing component
priority. These tests should be removed once proper CI testing is
in place.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
There are currently no CI tests for CMakeV2, so add it to the
exclude_check_tools_files.txt file. Once CI testing is implemented, this
change should be reverted.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Discover component directories and initialize components within them.
This process does not include managed components, which should be added
separately at a later stage. To facilitate this, some minimal
functionalities are introduced, such as build properties, component
properties, and other helper functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
fix(test_rtc_wdt): Fixed CI test failures for RTC WDT by increasing margin
Closes IDFCI-4816, IDFCI-4835, IDFCI-5863, IDFCI-4799, and IDFCI-4817
See merge request espressif/esp-idf!42959
This commit adds a new idf.py --preset NAME argument, which allows
selecting the CMake configuration preset with a given name.
For idf.py, selecting the preset does two things:
- sets the build directory, which is specified in the preset itself
using "binaryDir" field
- passes --preset argument to CMake configuration phase
The multi_config example is updated to illustrate how this feature is
expected to be used.
The CI tests for RTC WDT test app were failing because we allowed only
20% margin between configured vs actual timeout of wdt. However, the
RC oscillator accuracy on some boards is very poor and it results in
almost 25% difference between configured and actual timeout. Hence,
increased the failure margin from 20% to 30%. Also some boards like
ESP32P4 have known problem and gives timeout after twice the configured
time. Hence, increased the wdt interrupt wait time to three times the
configured value, so we don't miss the interrupts for such boards.
In case of ESP32-C5 and ESP32-C61, mmu_ids for PSRAM and Flash
MMU are the same due to their shared memory space. Thus, instead of
mmu_id we should use mmu_target_t.
fix: Reorder Ethernet initialization in VLAN example
Closes IDFCI-3146, IDFCI-3145, IDFCI-3144, IDFCI-5092, IDFCI-5091, and IDFCI-5090
See merge request espressif/esp-idf!42401
This prevents premature matching on partial data when the output is
being streamed, which was causing the test to capture 100000 instead of
the full 1000002.
Refactored esp_lcd_rgb_panel_get_frame_buffer and
esp_lcd_dpi_panel_get_frame_buffer
to follow the standard printf/vprintf pattern:
- Created static _v helper functions that take va_list as parameter
- Moved parameter validation to the public API functions
- Public functions now handle va_start/va_end and call the helper
This reverts earlier commit d2858847 and thereby making SPIRAM_USE_MEMMAP
available for all targets.
This config option has no direct usage but just avoids adding the PSRAM
to heap but still keeps it memory mapped.
Closes https://github.com/espressif/esp-idf/issues/17764
For the dual core chips like ESP32H4 that are running on slow CPU
clock frequency (96MHz), the number of events getting dispatched in
the test case called "performance test - no dedicated task" were
falling short of expectation because of extra overhead with locks
etc when runing with a non dedicated task. Even for other chips like
ESP32S3, if the main CPU core is running at 80MHz, the result is same.
Hence, added a consideration for CPU clock frequency when deciding
pass criteria for this test case.
Add a new test case to verify that USJ printing doesn't block CPU on chip
wake-up from light sleep. The test measures the average time per print
operation and ensures it's below 5000 microseconds.
- Introduced new HAL components for LCD and MIPI DSI, including:
- `lcd_hal.c` and `lcd_hal.h` for LCD hardware abstraction layer.
- `mipi_dsi_hal.c` and `mipi_dsi_hal.h` for MIPI DSI hardware
abstraction layer.
- Peripheral connection definitions in `lcd_periph.c` and
`lcd_periph.h`.
- Added necessary types and utility functions in `lcd_types.h` and
`mipi_dsi_types.h`.
- Updated CMakeLists to include the new HAL components in the build
system.
- Removed legacy references to LCD and MIPI DSI in the HAL and SOC
CMakeLists.
- When `esp_flash_read()` is invoked from REE, it internally enables WB mode
via `spi_flash_ll_wb_mode_enable()`. However, the ROM flash APIs used by TEE
do not support WB mode, resulting in failures when TEE attempts to access
flash after this call.
- This commit adds a workaround in the TEE flash layer by saving WB mode state,
temporarily disabling it for ROM API calls, and restoring it afterward.
Currently, REE SPI flash HAL operations are routed as service calls to TEE,
but the TEE implementation incorrectly uses ROM APIs instead of HAL APIs.
This leads to issues and is not the recommended approach.
fix(ble_mesh): resolve issues in long packet mode
Closes BLERP-2324, BLERP-2323, BLERP-2322, BLERP-2321, BLERP-2320, and BLERP-2319
See merge request espressif/esp-idf!42745
fix(ble/bluedroid): fix memory leak during deinit when service table is created but not started
Closes BCI-565
See merge request espressif/esp-idf!42456
1. fix(wifi): Rename old NAN configuration to NAN-Sync
- Rename CONFIG_ESP_WIFI_NAN_ENABLE to CONFIG_ESP_WIFI_NAN_SYNC_ENABLE to indicate
the support for Synchronized NAN (Wi-Fi Aware).
- Because the original flag really controls the synchronized feature set, rename it
to CONFIG_ESP_WIFI_NAN_SYNC_ENABLE so the NAN-Sync and NAN-USD paths can be
selected independently without confusion.
2. Document esp_wifi_start requirement and fix USD examples
3. Rename nan_callbacks to nan_sync_callbacks
4. Remove the discovery_flag, clarify docs for sync vs USD flows, and add USD start/stop APIs
5. Require esp_wifi_start() before USD start
6. docs(nan): add NAN-USD application examples
7. add migration guide and hints for NAN-USD proto field
8. Improve allow_broadcast documentation
9. Add attention to the API esp_wifi_remain_on_channel
10. fix(wifi): align NAN API renames and docs for v6.0
- keep shared APIs under esp_wifi_nan_* while reserving
sync/usd names for mode-specific entry points
- clarify synchronized-cluster scope in headers, docs, and migration notes (EN/zh-CN)
- update examples for renamed helpers and WIFI_NAN_SYNC_CONFIG_DEFAULT()
- rename `wifi_nan_config_t` to `wifi_nan_sync_config_t`
11. Mark NAN-USD as esp-idf experimental feature
1. Remove redundant fixes in upstream wpa_supplicant for USD
- Upstream supplicant has mostly fixed the issues regarding the
unnecessary resetting pauseStateTimeout.
- Upstream supplicant still needs one fix to avoid resetting the
pauseStateTimeout when subscribe message is received from the peer
which had triggered the pauseStateTimeout previously.
2. Replace array-based channel list with bitmap for NaN-USD
Use `wifi_scan_channel_bitmap_t` to represent the channel list for NaN-USD
publisher and subscriber configurations. This replaces the previous approach
that used a large array to store allowed channels.
Also aligns with internal scan bitmap conventions across Wi-Fi stack.
3. call esp_wifi_nan_stop() after USD exchange or STA stop
Ensure esp_wifi_nan_stop() is called after publish/subscribe activity
completes or when WIFI_EVENT_STA_STOP is received. This prevents NAN stop
errors due to inactive interface. NaN-USD currently uses STA interface
for Tx/Rx.
4. Fix task watchdog timer triggered in active USD subscriber:
As both USD supplicant and offchan TX component gets executed
in the wifi task, it created a deadlock like scenario where offchan TX
done callback was never getting executed and supplicant 'nan_de_timer'
keeps running but failing to send any subscribe frame.
5. Make sure that device is able to recieve action management frames
of size more than 1400 bytes.
6. Update proto field in SSI to match Wi-Fi Aware (NaN) spec format
The 'proto' field in the 'wifi_nan_wfa_ssi_t' structure previously used an
enum (wifi_nan_svc_proto_t), resulting in a 32-bit field. According to
the Wi-Fi NAN Specification (Table 5.7), this field must be a single
octet (8 bits). This commit updates the type to uint8_t to ensure
compliance with the specification.
This mismatch previously triggered warnings but did not cause functional
errors.
7. Set `allow_broadcast` to true in USD Remain on channel
This enables the peer discovery as USD uses NAN-Network Multicast BSSID
as A3 address in publish frames.
8. Implement custom channel<->frequency conversion for NAN-USD
NaN-USD only permits 20 MHz bandwidth channels in the 2.4 GHz and 5 GHz bands
(as per section 4.5.3 of the Wi-Fi Aware Specification v4.0). To avoid linking
a large portion of upstream supplicant code for frequency-to-channel and
channel-to-frequency conversion, implement minimal custom helper functions.
9. Limit NAN_DE_MAX_SERVICE to 2 for ESP_SUPPLICANT
10. Note: Upstream clamps negative `os_reltime` deltas to zero in `nan_de_srv_time_to_next()`,
but our ESP_SUPPLICANT port keeps `os_time_t` unsigned, so that guard just provokes a
compiler warning. We skip it for now because the scheduling loop validates past timestamps
before computing the diff.
Co-authored-by: Shyamal Khachane <shyamal.khachane@espressif.com>
This commit adds the support to receive broadcast frames in ROC Rx
callback. Some use cases may need to receive broadcast packets for
discovery of other peers.
- This can be enabled/disabled through 'allow__broadcast' flag in
'wifi_roc_req_t' structure.
- In default configration, broadcast packets will always be filtered out and
will not be passed to the Rx callback to save CPU consumption.
1. Make sure that wpa_supplicant does not encapsulate user-given SSI info
with WFA OUI.
2. USD component shall transfer the SSI given by user as it is, without
any modification or encapsulation.
3. Note that this will break the compatibility of ESP USD implementation
with a device running upstream wpa_supplicant as wpa_supplicant
encapsulates the SSI information with WfA oui.
For compatibility with wpa_supplicant, user needs to handle the
SSI encapsulation/decapsulation.
According to the Wi-Fi Direct spec draft, address 3 in unicast USD
frames shall be set to the P2P Device Address of the sender (i.e., this
differs from the Wi-Fi Aware specification). Fix it.
Signed-off-by: Andrei Otcheretianski <andrei.otcheretianski@intel.com>
According to the Wi-Fi Direct spec draft, address 3 in multicast USD
frames shall be set to wildcard BSSID (i.e., this differs from the Wi-Fi
Aware specification). Fix it.
Signed-off-by: Andrei Otcheretianski <andrei.otcheretianski@intel.com>
P2P2 uses USD with a different group address than the NAN Network ID
defined in the Wi-Fi Aware specification. Select the group address based
on whether USD is used with P2P2 or something else. This changes
behavior only for the P2P2 cases.
Signed-off-by: Shivani Baranwal <quic_shivbara@quicinc.com>
This behavior is not described in the Wi-Fi Aware specification, so
remove it and instead, expect services to terminate USD explicitly when
no further Follow-up messages are needed.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Remain waiting for additional Follow-up messages on a publisher that is
in pauseState for a service that requires further service discovery.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
If a publish service is configured with an explicit lifetime (ttl), do
not allow pauseState duration to exceed that.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
This can be used to optimize listen operations to be as long as the
driver supports instead of having to use a small enough value to work
with all drivers.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Do not unpause publisher if more than one second has passed from the
last Follow-up message TX or RX. There is no such behavior described in
the Wi-Fi Aware specification and it is possible for a service to need
more time to generate Follow-up messages. Leave it to the service itself
to force timeout, if desired, or terminate pauseState after the 60
second overall timeout.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
There is no need to copy the A3 value for follow-up frames through the
control interface events and commands since it can be handled internally
in the service with sufficient accuracy. More parallel operations with
multiple peers might need per-peer information, but that can be extended
in the future, if that level of complexity is really needed in practice.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Wi-Fi Aware spec v4.0 was not clear on all cases and used a bit unclear
definition of A3 use in Table 5 (Address field definiton for NAN SDF
frames in USD). That resulted in the initial implementation using
Wildcard BSSID to comply with the IEEE 802.11 rules on Public Action
frame addressing.
For USD to have chances of working with synchronized NNA devices, A3
needs to be set to the NAN Cluster ID when replying to a frame received
from a synchronized NAN device. While there is no cluster ID for USD,
this can be done by copying the A3 from the received frame. For the
cases where sending out an unsolicited multicast frame, the NAN Network
ID should be used instead of the Wildcard BSSID.
While this behavior is not strictly speaking compliant with the IEEE
802.11 standard, this is the expected behavior for NAN devices, so
update the USD implementation to match.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
hostapd did not accept NAN SDFs that used NAN Network ID instead of
Wildcard BSSID in A3. Extend this to process NAN Network ID just like
Wildcard BSSID for these frames to allow the specific group address to
be used.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Add frequency list to active NAN USD Subscriber to search for a
Publisher on multiple channels. This is the publish channel list used by
the Subscriber to periodically search for a service on these channels.
publishChannelList was already supported in the Publisher and this
commit extends that to the Subscriber.
This is needed for a P2P2 seeker that is an active subscriber looking
for an advertiser on a list of publish channels.
Signed-off-by: Shivani Baranwal <quic_shivbara@quicinc.com>
Extend the NAN Subscribe and Publish methods to allow p2p=1 to be
specified to include P2P attributes in a P2P IE to be added into the NAN
Element Container attribute. In addition, use the P2P Device Address for
the NAN SDFs for the P2P cases.
Signed-off-by: Shivani Baranwal <quic_shivbara@quicinc.com>
Support implementation architecture where the NAN discovery engine is
located in the driver/firmware instead of wpa_supplicant.
Signed-off-by: Shivani Baranwal <quic_shivbara@quicinc.com>
The Wi-Fi Aware Specification v4.0 only defines unsolicited
transmissions only, solicited transmissions only and both unsolicited
and solicited transmissions publish. The other possibility is undefined
so we should reject it.
Signed-off-by: Chien Wong <m@xv97.com>
Add NAN discovery engine and wpa_supplicant interface to use it for the
subset of NAN functionality that is needed for USD.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
When DMA keep writing the memory, some data may be corrupted after reset. For example, the stack of bootloader may be overwritten and failed to boot until a higher scope of reset (Core).
Also removed the DPORT_PERIP_RST_EN_REG alias on ESP32S2. Now it's the same as some following chips (EN0).
feat(dma): graduate the dma driver into a single component: esp_hal_dma
Closes IDF-14080, IDF-13675, and IDF-13673
See merge request espressif/esp-idf!42250
- Destination selection is unified for SystemView and app_trace
- Destination param is removed from app_trace APIs
- Destinaiton and related configuration now can be override from users
with a callback from system_init_fn
esp_stdio contains everything the old esp_vfs_console contained (the vfs stdio glue layer)
as well as other functionality related to stdio (previously referred to as console)
ci(network): fixed dependencies for misc network and wifi related test apps
Closes IDF-14193, IDF-14201, IDF-14205, IDF-14204, IDF-14192, and IDF-14196
See merge request espressif/esp-idf!42592
PPA SRM engine added YUV422 and GRAY8 color mode support
PPA SRM engine macro block size increased to 32x32
PPA Blending engine added YUV420, YUV422 and GRAY8 color mode support
Added one more pair of 2DDMA channels
Priority bit width increased corespondingly
Added three new CSC modes for RX channel 0
Also cleaned up DMA2D capability definitions in soc_caps.h
- Previously, only the U-mode interrupt threshold was raised in REE critical sections,
leaving M-mode at the lowest level.
- As a result, when a service call transitioned to M-mode, all interrupts were still
allowed to fire, including those that should have been masked.
Unity based test app configures RTC WDT to verify following features:
1. Verify all 4 stages of RTC WDT are able to trigger timeout and
invoke the pre-configured interrupt service routine
2. Verify RTC WDT can reset the main SYSTEM or main SYSTEM + RTC
fix(ble/bluedroid): prevent crash when deinit/disable host during scan
Closes BLERP-2266, BLERP-2267, BLERP-2268, BLERP-2270, and BLERP-2274
See merge request espressif/esp-idf!42134
- Remove GPTIMER and TIMG related definitions from soc_caps_full.h files
- Move timer peripheral definitions to appropriate HAL layer files
- Update references across components to use proper HAL abstractions
- Consolidate timer group and GPTIMER capabilities organization
- Ensure consistent timer configuration across all ESP32 variants
This refactoring improves the separation of concerns between SOC
capabilities and HAL implementations for timer-related functionality.
- Add extended advertising configuration with customizable PHY selection (1M, 2M, Coded)
- Implement long packet mode to overcome standard BLE Mesh packet length limitations
- Add configurable advertising parameters: interval, count, channel map, TX power
- Introduce enhanced message context structure for fine-grained advertising control
- Add Kconfig options for buffer management and segment count configuration
- Support both forced and preferred long packet transmission modes
- Update advertising buffer pools to accommodate extended and long packet types
- Add validation for enhanced advertising parameters and packet length limits
- Maintain backward compatibility with existing BLE Mesh functionality
fix(example/classic_bt): Fix the stack overflow of the SPPAppT and mouse_move_task tasks
Closes IDFCI-3985 and IDFCI-3631
See merge request espressif/esp-idf!42474
The commit 51f3c021158d ("Add quotes around node ID in dependency
graph") introduced proper quoting for node IDs in the generated dot
file. Since the dot format does not allow dashes in node names without
quoting, and dashes are used in component names, the component names in
the generated dot file are now correctly quoted. Adjust the
g1_components test to accommodate this change by stripping the quotes.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Graphviz node ID's are represented as strings, that can only use a
restricted set of characters (characters, digits, and underscores), and
must not match a reserved keyword. These restrictions do not apply when
the string is wrapped in double quotes. This allows for component names
with dashes in them, for example.
Closes https://github.com/espressif/esp-idf/pull/17594
This changes how available ports are detected. It removes code duplication
for MacOS filtering which already is in esptool and prepares for port
prioritization to be used from esptool.
fix: allow larger buffer size for commands (especially for idf.py confserver)
Closes IDFCI-3569 and IDF-13919
See merge request espressif/esp-idf!42312
The new version of esptool has a quite interactive output that supports
folding steps and overriding progress bar. Using interactive mode to run
esptool scripts will make the output propagate to the terminal immediately.
fix(esp_tls_conn_new_sync): Fix `esp_tls_conn_new_sync`'s doc to reflect its implementation (GitHub PR)
Closes IDFGH-16528 and IDFGH-16475
See merge request espressif/esp-idf!42263
Tools/esp-idf-size: Remove --legacy argument and replace JSON format with JSON2
Closes IDF-8772 and DOC-12437
See merge request espressif/esp-idf!41743
- Ensure bound checks correctly handle all scenarios, including
when a requested operation's (SPI0/1) range fully contains the
TEE-protected region.
- Disable delegation of INTWDT timeout and Cache error interrupts as they reset
the device after the panic handler
The component validation script was using an incorrect property name
'__COMPONENT_TARGETS' when retrieving component targets. This should be
'__BUILD_COMPONENT_TARGETS' to match the actual property name used
throughout the build system.
This fix ensures the component validation can properly access the list
of component targets and perform validation checks correctly.
Add a condition on the definition of the MALLOC_CAP_EXEC macro to
prevent it from being defined if ESP_SYSTEM_MEMPROT_FEATURE or
ESP_SYSTEM_PMP_IDRAM_SPLIT is enabled, thus throwing a compile time
error when using it.
Closes https://github.com/espressif/esp-idf/issues/14837
- Add NDP Peer inactivity detection and termination support
- Add NDP Action frames internal retries
- Fix NAN API return status to match with Tx status
- Fix false wait events during NAN datapath setup
- Replace NAN default event handlers with callbacks to
avoid race conditions between wifi and user tasks
- Fix parsing Availability of third party devices
Closes https://github.com/espressif/esp-idf/issues/17529
Co-authored-by: akshat <akshat.agrawal@espressif.com>
fix(bt/controller): fixed the sniff anchor point can be mismatch from central and peripheral
Closes BTQABR2023-553
See merge request espressif/esp-idf!42240
This commit introduces a new feature to the roaming logic. If the
currently connected AP has the 'transition disable' bit set in its
RSN IE, the roaming logic will now ignore any scanned APs that only
support WPA2-PSK. This prevents a security downgrade when roaming in a
mixed WPA2/WPA3 environment.
A new Kconfig option, CONFIG_ESP_WIFI_IGNORE_WPA2_ONLY_ON_TRANSITION_DISABLE,
has been added to control this feature. It is disabled by default.
This commit addresses several issues in the BSSID blacklisting
feature of the roaming application:
- Merged duplicate functions into a single,
unified function, resolving a compilation error.
- Corrected and
to properly access the member of the
struct, fixing invalid memory access.
- Introduced in Kconfig to enable
the manual blacklisting feature and made auto-blacklisting
dependent on it.
- Updated to use the
correct BSSID from .
- Optimized the removal of expired blacklist entries by using
for better efficiency.
With RSN overriding enabled, AP can be configured to set MFPC to 0 and
MFPR to 0 in the RSNE and MFPC to 1 and MFPR to 1 in the RSNOE and
RSNO2E. IGTK generation, configuration to the driver, and inclusion of
the IGTK KDE in 4-way handshake should also take into account the
management frame protection settings in the override variants.
Signed-off-by: Sai Pratyusha Magam <quic_smagam@quicinc.com>
Integer promotion converts u8 rsnxe[i] to an int which is not
sufficiently large to be able to handle the maximum shift left of 24
bits here. Type cast rsnxe[i] to u32 explicitly to get rid of the sign
bit and avoid this undefined behavior from the shift operation.
Credit to OSS-Fuzz: https://issues.oss-fuzz.com/issues/376786400
Fixes: d675d3b15b40 ("Add helper functions for parsing RSNXE capabilities")
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
When rsn_override_omit_rsnxe=1 is used to omit the RSNXE from Beacon and
Probe Response frames, it should also be omitted from (Re)Association
Response frames since there is a general expectation on the RSNXE being
used consistently between these frames. This is unlikely to have much of
a difference for most use cases in practice, but this could impact FILS
association if the non-AP STA were to confirm that the unprotected and
protected version of the RSNXE from the AP were identical.
Fixes: 8b2ddfdbb688 ("RSNO: Allow RSNXE to be omitted")
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Always set SNonce cookie and enable RSN Override elements validation
irrespective of the RSN Selection element usage in (Re)Association
Request frame when RSN overriding supported.
Signed-off-by: Veerendranath Jakkam <quic_vjakkam@quicinc.com>
A STA that supports RSN overriding will always use the SNonce cookie. An
AP that does not advertise RSN overriding elements must not enforce that
SNonce cookie is used with RSN Selection element since a STA includes
the latter only when it sees the AP advertising RSN overriding elements.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
This allows all variants to be verified based on a protected frame to
achieve robust downgrade protection.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
This provides an implicitly protected (SNonce is used as an input to PTK
derivation) mechanism for a STA to indicate support for RSN overriding
in a manner that does not cause interopability issues with deployed APs.
In addition, update sm->SNonce on the Authenticator only based on
message 2/4 since that is the only EAPOL-Key message that is defined to
provide the actual SNonce value. While clearing of this internal buffer
on message 4/4 might not cause issues, it is better to keep the actual
SNonce value here since the SNonce cookie can be used at a later point
in the sequence.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
This replaces the use of the RSNE Override and RSNE Override 2 elements
with empty payload to indicate which RSNE variant was used.
In addition, this adds stricter validation of the RSNE in
(Re)Association Request frame to allow only the pairwise cipher suites
and AKMs listed in the indicated RSNE variant to be used.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
The separate RSNOE/RSNO2E/RSNXOE buffers were not actually used on the
Authenticator, so remove them.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Use else-if check for better clarity regarding usage of wpa_ie_buf3 to
make it explicit that memory is allocated for this pointer only once.
Signed-off-by: Adil Saeed Musthafa <quic_adilm@quicinc.com>
The RSNXE generation function was extended to support this earlier, but
that update was missed from the RSNXOE variant.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Add an explicit rsn_override_omit_rsnxe=1 configuration parameter to
allow the RSNXE to be omitted when using the RSNXOE and wanting to
minimize interoperability issues with STAs that might recognize the
RSNXE, but not handle it correctly, e.g., when multiple octets of
payload is included.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Add support for RSNE/RSNXE Override elements. Use these elements to
determine AP's extended RSN parameters.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Allow hostapd to be configured to advertised two separate sets of
RSNE/RSNXE parameters so that RSNE/RSNXE can use a reduced set of
capabilities (e.g., WPA2-Personal only) for supporting deployed STAs
that have issues with transition modes while the new override elements
can use a newer security option (e.g., WPA3-Personal only) for STAs that
support the new mechanism.
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
Simplify the implementation by using shared functions for parsing the
capabilities instead of using various similar but not exactly identical
checks throughout the implementation.
Signed-off-by: Jouni Malinen <j@w1.fi>
Added a pulse counter example code for ulp riscv chips. The example
works by HP core generating high frequency pulses on a GPIO, which
are counted by ULP core to find out the highest possible frequency
of pulses that can be achieved without missing any edges.
There are idf.py hints for helping the user to set component
dependencies properly instead of building sources out-of-component or
including headers from outside the component directory. These are
produced with
tools/idf_py_actions/hint_modules/component_requirements.py.
However, idf.py hints are printed only when the build fails. If the user
starts with a buildable solution then the suggestions to add component
dependencies are not printed.
This commit introduces cmake-level warnings for building source files
from outside the component and including header files without setting up
proper component dependencies.
feat(tools): Load idf.py extensions from project's component directories and python packages
Closes IDF-5736 and IDF-4121
See merge request espressif/esp-idf!39875
Allow to disable implicit inlining of constexpr functions from libstdc++.
This is a known GCC issue https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93008
that may affect C++ application size depending on its structure.
Extend search for idf_ext.py beyond the project directory to include
all build components involved in the build. Also discover idf_ext.py
modules via Python entrypoints.
This commit deprecates xPortGetTickRateHz() from the FreeRTOS port API
as the FreeRTOS tickrate is constant for an application and can be
inferred using the CONFIG_FREERTOS_HZ config option.
This commit creates a new port layer API xPortGetRunTimeCounterValue()
in port.c files. This helps to remove inclusion of header files such as
esp_timer.h and xtensa/hal.h from portmacro.h
Previously it would return ESP_ERR_INVALID_STATE, which meant that if called from
user-code before the system tries to initialize the timer then esp-idf would
fail to boot.
This could happen if a user wanted to use esp-timer from a cpp constructor.
Closes https://github.com/espressif/esp-idf/issues/9679
- Dynamically switch the AES operation modes based on the buffer operating length
- Shorter AES and SHA operations can now run faster and concurrently as well
Closes https://github.com/espressif/esp-idf/issues/15914
- The API esp_flash_encryption_set_release_mode() by defualt programs
the XTS-AES pseudo round level efuse to level low but did not considered
any existing value that would have been programmed in the efuse bit.
fix: get rid of the hardware workarounds for the latest ESP32P4 revision
Closes IDF-13409, IDF-13781, and IDF-13782
See merge request espressif/esp-idf!41719
- When NVS encryption is enabled on SoCs with the HMAC peripheral that have flash encryption
enabled, the HMAC-based NVS encryption scheme is now selected as default instead of the
flash encryption-based scheme.
- If your application previously used the flash encryption-based scheme, you need to manually
configure the NVS encryption scheme to flash encryption from HMAC through ``menuconfig``
or your project's ``sdkconfig`` (i.e., setting ``CONFIG_NVS_SEC_KEY_PROTECT_USING_FLASH_ENC=y``).
sdmmc: supported sdmmc on p4 ECO5 real chip, and UHS-I SDR104 (200MHz)
Closes IDF-12222, IDF-13731, and IDF-14050
See merge request espressif/esp-idf!41563
bugfix: Fix array compared against 0 in btc_manage.c and the issue of uninitialized pointer reading in coexist_printf
Closes IDF-13868
See merge request espressif/esp-idf!41455
feat(ledc): Add LEDC support for ESP32H21 and ESP32H4
Closes IDF-12343, IDF-12344, IDF-12920, IDF-11568, IDF-11569, IDF-12115, and IDF-13672
See merge request espressif/esp-idf!41172
1. In async requests, if the two or more requests are made on same
socket then it used to block the second request.
2. The main thread is used to block on select call. And there done
no FD_SET for particular fd.
Closes https://github.com/espressif/esp-idf/issues/16998
- Fix incorrect setting in the edge interrupt acknowledgement API
- Avoid executing the service call dispatcher in the U-mode ecall,
rather execute `mret` to jump it
- Avoid `t1` register corruption when processing `ecall`
- Switch back to the bootloader stack from TEE stack after the
execution of the entire TEE initialization routine
- Rename `tee_test_fw` app configs for better CI tracking
- Decrease the lower bound of TEE I/DRAM config options
- Trim the TEE test-apps build
- Improve the TEE/REE OTA pytest script with additional checks
- Fix build issues when `tee_sec_storage`/`tee_ota_ops` are a
a part of the project build but ESP-TEE is disabled
For bin log, reserve the first 4 bytes as zero for variables that are pointed to NULL
and should not be printed in the log. So the esp-idf-monitor will skip printing
those variables.
lp_core_i2c.h header file has sda and scl pins hardcoded to GPIO6 and
GPIO7 which works only for ESP32C6. ESP32C5 uses GPIO2 and GPIO3 for
I2C SDA and SCL. Hence, added LP_I2C_SCL_IO and LP_I2C_SDA_IO macros
under conditional compilation in library header file, so there is no
need to hardcode I2C GPIO pins and any other test apps or examples
that are including the LP I2C header file can also use Macro directly.
For secure update without secure boot case, the encryption length for
app image must consider signature block length as well. This was
correctly handled for secure boot case but not for secure update without
secure boot.
This commit removes support for the vPortCleanUpTCB() user hook as well
as drops support for associated Kconfig option,
CONFIG_FREERTOS_STATIC_TASK_CLEAN_UP.
Previously the RTC timer entropy was being mixed with the RNG timer
in a wrong way. Which led to the overwriting of the LSB with rtc
timer value.
This change fixes that behaviour
intr_types.h has been replaced by esp_intr_types.h and the deprecated esprv_intc_*
from interrupt_deprecated.h have been replaced by the more generic
esprv_* functions.
The latest cJSON vulnerabilities appear to list "davegamble" as the
vendor instead of "cjson_project." Add a new CPE with "davegamble" as
the vendor.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit refactors the flash target creation. Now bootloader and
partition table components add dependencies to the flash target directly
from their component CMakeLists.txt files instead of it being done in
the esptool_py component. The commit also removes the redundant
__esptool_py_setup_main_flash_target() function.
Support cpu & modem & top domain and minimum system peripherals retention in esp32h4 beta5
Closes PM-449, PM-484, PM-448, PM-451, PM-497, IDF-12283, and IDF-12286
See merge request espressif/esp-idf!40605
removed unused console cli.
removed unnecessary dependencies.
behaviour changed: user must run the i2cconfig command before others
updated pytest as well
test(app_trace): add SystemView UART tracing tests for all targets
Closes IDF-11919, IDF-11920, IDF-11921, DOC-12212, and IDF-7660
See merge request espressif/esp-idf!41552
The `-mtune=esp-base` option is identical to the default tuning profile,
except that `slow_unaligned_access` is set to false.
This reduces the instruction count for built-in `memcpy` and improves
performance, since our chips can handle misaligned access with minimal
penalty (without triggering exceptions).
Example:
void load(uint32_t *r, char* x) {
memcpy(r, x, sizeof(uint32_t));
}
void store(char* x, uint32_t v) {
memcpy(x, &v, sizeof(uint32_t));
}
Previously generated code:
load:
lbu a5,2(a1)
lbu a3,0(a1)
lbu a4,1(a1)
sb a5,2(a0)
sb a3,0(a0)
sb a4,1(a0)
lbu a5,3(a1)
sb a5,3(a0)
ret
store:
srli a3,a1,8
srli a4,a1,16
srli a5,a1,24
addi sp,sp,-16
sb a1,0(a0)
sb a3,1(a0)
sb a4,2(a0)
sb a5,3(a0)
addi sp,sp,16
jr ra
With `-mtune=esp-base`:
load:
lw a5,0(a1)
sw a5,0(a0)
ret
store:
sw a1,0(a0)
ret
Inlining behavior
=================
Without `-mtune=esp-base`:
- `memcpy()` is inlined only when the compile-time size is ≤ 12 bytes.
- Maximum cost: ~25 instructions
With `-mtune=esp-base`:
- `memcpy()` is inlined for all compile-time constant sizes.
- Maximum cost: ~14 instructions
As a result, some applications may see reduced code size, while others
may increase slightly. However, performance always improves because
extra `memcpy` calls are eliminated.
Performance results
===================
esp32p4 (Ethernet iperf):
- No noticeable difference
esp32c61 (Wi-Fi iperf):
- ~2 Mb/s increase for TCP and UDP TX (may be within measurement error)
NOTE
====
Applies only to RISC-V chips that do not have the hardware issue marked
by the SOC_CPU_MISALIGNED_ACCESS_ON_PMP_MISMATCH_ISSUE macro.
- Fixes the authmode sent in WIFI_EVENT_STA_CONNECTED for WPA3-Enterprise, which
was previously taken directly from the AKM field. This caused incorrect values in some cases.
- Updates the "security" log to show the correct wifi_auth_mode_t.
- Fixes wrong authmode strength used in the connection path.
'running' - task that was running when execution stopped
'selected' - task that was selected by used in GDB (command "thread <id>")
Note that initially, after the program is interrupted 'selected' == 'running'
Update the QEMU instructions for the security_features_app example (GitHub PR)
Closes IDFGH-16318 and IDFGH-16311
See merge request espressif/esp-idf!41438
feat(sdspi): Add an option to modify the waiting time for MISO before sending next command
Closes IDFGH-15977 and IDFGH-9089
See merge request espressif/esp-idf!41175
In event of a corner case, where connection is not completed,
connection reattempt is disabled, so need to post connection
failure event to application to make it aware of the failure
PMP configurations for load and store addresses may
have different permissions (e.g., "R" vs. "RW").
Due to the timing alignment of internal signals, the address
permission check may be incorrectly applied during the second
part of a misaligned access transaction.
As a workaround, insert two instructions (e.g. ADDI/NOP) between
accessing to different memory regions. This spacing avoids the
false permission check caused by signal timing overlap.
refactor(ulp_riscv): Modify i2c read/write API for better logging and return error code
Closes IDFGH-16269 and IDFGH-15237
See merge request espressif/esp-idf!41342
fix(driver_twai): modify the calculation description of quanta_resolution_hz (GitHub PR)
Closes IDFGH-16055 and IDFGH-16061
See merge request espressif/esp-idf!41405
Updated the i2c read/write APIs ulp_riscv_i2c_master_read_from_device
and ulp_riscv_i2c_master_write_to_device in ulp_riscv component to
return error codes back to the application
Closes https://github.com/espressif/esp-idf/issues/15904
Moved the error logs outside critical section for i2c communication errors
like READ fail, WRITE fail etc. in the ulp_riscv_i2c component
Also changed the error log API from ESP_EARLY_LOG to ESP_LOG, so we can support
tag based filtering and enabling/disabling of logs
Closes https://github.com/espressif/esp-idf/issues/17425
change(i2c_master): Change the I2C return value to ESP_ERR_INVALID_RESPONSE when NACK detected
Closes IDF-13751
See merge request espressif/esp-idf!41251
feat(examples): improved example for Ethernet SPI polling mode without interrupt (GitHub PR)
Closes IDFGH-15933
See merge request espressif/esp-idf!41071
struct tm contains the number of years since 1900, but the printing code
assumed it contained actual current year.
It would also print it as YYYY, while the documentation and code implies
it should be YY.
Closes https://github.com/espressif/esp-idf/issues/17451
This commit adds new CMake APIs for the build system, viz.,
idf_build_add_post_elf_dependency() and
idf_build_get_post_elf_dependencies().
These APIs allow components to add post-ELF processing hooks before the
binary file is generated.
Closes https://github.com/espressif/esp-idf/issues/17251
fix(esp_hw_support): fix dead code and out-of-bounds in esp_sleep_clock_config function
Closes IDF-13119 and IDF-13103
See merge request espressif/esp-idf!40989
- Introduced TWAI utility commands for sending, dumping, and managing TWAI frames.
- Added configuration options for TWAI GPIO pins and support for TWAI-FD.
- Created necessary CMake and Kconfig files for building the TWAI utilities.
This enhancement provides a comprehensive interface for TWAI operations.
feat(esp_http_client): added new HTTP state HTTP_STATE_CONNECTING and change state flow
Closes IDFGH-15423 and IDFGH-15362
See merge request espressif/esp-idf!40507
Add top-level section for configuring and initializing external USB PHY
on ESP32-S3. Includes GPIO mapping, code example, and guidance on
suspend and speed selection pins.
Link to ESP USB PHY guide for SP5301 setup to avoid duplicating content.
Add dedicated schematic reference for STUSB03E with analog switch.
This commit updated the client closing condition to avoid spurious
dispatching of event HTTP_EVENT_DISCONNECTED while closing closeed connection.
Closes https://github.com/espressif/esp-idf/issues/16070
This commit added new http state HTTP_STATE_CONNECTING, and made states public.
Also added public API to get current state.
Updated state flow in esp_http_client_perform()
Closes https://github.com/espressif/esp-idf/issues/16019
Change default values for protocomm security configuration options to improve
security:
- CONFIG_ESP_PROTOCOMM_SUPPORT_SECURITY_VERSION_0 now defaults to 'n'
- CONFIG_ESP_PROTOCOMM_SUPPORT_SECURITY_VERSION_1 now defaults to 'n'
This reduces code size by default and encourages use of more secure protocomm
implementations.
Projects using these security versions must explicitly enable them in their
configuration.
Refactor: Use enum values when assigning "pull_[up|down]_en" fields of "gpio_config" (GitHub PR)
Closes IDFGH-16192
See merge request espressif/esp-idf!41058
fix(wifi): fix some esp-now issues and lightsleep issues
Closes WIFIBUG-1124, WIFIBUG-1302, WIFIBUG-1303, WIFIBUG-1312, and WIFIBUG-1304
See merge request espressif/esp-idf!40454
The following updates have been made in this commit:
- The commit places ring buffer code in flash memory by default.
- CONFIG_RINGBUF_PLACE_FUNCTIONS_INTO_FLASH has been removed.
- CONFIG_RINGBUF_IN_IRAM is added and can be used to restore the
previous memory placement.
The following updates have been made in this commit:
- The commit places FreeRTOS code in flash memory by default.
- CONFIG_FREERTOS_PLACE_FUNCTIONS_INTO_FLASH has been removed.
- CONFIG_FREERTOS_IN_IRAM is unhidden and can be used to restore the
previous memory placement.
- A test has been added for users to conduct performance impact testing
based on memory placement configurations.
Ensure that enum values are used When assigning `pull_up_en` and `pull_down_en`
fields of `gpio_config_t`. Helps avoid `invalid conversion` errors when
building those code snippets in C++.
1. fix tbtt issues when wifi_slp_iram_opt is not enabled and light sleep
enabled
2. support espnow rx all the time at coexistence default mode, fix some
coexist perfermance issue with coexistence default mode.
3. support connectionless ps with enhanced_light_sleep
4. fix espnow coexist issue when switching to coexistence mode
5. fix concurrent issue of connectionless_interval_set API
The previous workaround does not work, the backup/restore should apply to RTC IO registers.
Also moved the workaround to sleep_gpio.c to avoid gpio hal using kconfig.
The freertos_compatibility.c file has been removed and appropriate
guidelines have been added to hints.yml and the migration guide to use
alternatives for the backward compatibility functions that have been
removed.
change(esp32): Calling gpio_reset_pin on an input only pin should not produce an error message. (GitHub PR)
Closes IDFGH-16106
See merge request espressif/esp-idf!40975
The `gpio_reset_pin` function attempted to enable internal pullup on pins which does not have one. This change adds a guard to `gpio_reset_pin` for calling `gpio_pullup_en` - the same guard that makes `gpio_pullup_en` print the error.
Initially, ESP-IDF used the do_global_ctors() function to run global
constructors. This was done to accommodate Xtensa targets that emit
.ctors.* sections, which are ordered in descending order.
For RISC-V, compilation used .init_array.* sections, which are designed
to have ascending order. Priority constructors in .init_array.* sections
were correctly processed in ascending order. However, non-priority
.init_array section was processed in descending order, as it was done
for Xtensa .ctors.
Starting with ESP-IDF v6.0, the implementation switched to the standard
LibC behavior (__libc_init_array()), which processes both priority and
non-priority constructors in ascending order.
To achieve this, a breaking changes were introduced:
- Xtensa .ctors.* priority entries converted to .init_array.* format
(ascending), to be passed to __libc_init_array().
- Processing order of non-priority .init_array and .ctors sections was
changed from descending to ascending.
Also, this change introduces .preinit_array for linking. This may be
needed for some C++ or sanitizer features.
Related to https://github.com/espressif/esp-idf/issues/15529
Fixed bt hci event was not report to host when hci command was sent(master)
Closes BTQABR2023-538 and BTQABR2023-548
See merge request espressif/esp-idf!40992
When a WPS handshake is already in progress and the enrollee sends another EAPOL-Start
(e.g., due to missed packets or timeout), the registrar resets its state by calling
'eap_wsc_reset()'. This function frees 'sm->eap_method_priv' and then calls
'esp_wifi_ap_wps_disable()', which internally triggers another call to 'eap_wsc_reset()'.
This results in a double reset where the second invocation accesses the already freed
'sm->eap_method_priv', leading to a crash.
This fix sets 'sm->eap_method_priv' to NULL immediately after freeing it to ensure
any subsequent calls to eap_wsc_reset() do not access an invalid pointer.
1. If the user wants authenticate the request, then user needs to do
this before upgrading the protocol to websocket.
2. To achieve this, added pre_handshake_callack, which will execute
before handshake, i.e. before switching protocol.
fix(esp_wifi): Fix incorrect values for negotiated bandwidth for 2040 coex cases
Closes WIFIBUG-1199 and WIFIBUG-1201
See merge request espressif/esp-idf!39577
fix(lcd): reserve the GPIOs used by RGB LCD and disconnect the LCD signals when the dirver is uninstalled.
Closes AUD-6545
See merge request espressif/esp-idf!40734
ci(hal): Extend the PMS hal test-app for verifying TEE-based interrupt scenarios
Closes IDF-12642, IDF-12645, IDF-12873, and IDF-12876
See merge request espressif/esp-idf!40322
Fixed some BLE bugs 250628 on esp32(d9a3de0)
Closes BLERP-1697, BLERP-1965, BLERP-2075, BLERP-2077, and BLERP-1991
See merge request espressif/esp-idf!40421
- Support BLE vendor HCI related params reset
- Support multi conn optimization vendor hci command
- Fixed the issue where rssi of conn is incorrect when latency is no zero
- Fixed prio setting for BLE events
- Fixed disconnection issue when coexisting with other BLE events
- Fixed crash due to bandwidth full when updating conn params
- add ESP32-P4 and ESP32-H2 as Supported Targets to following
examples, using ESP-Hosted and Wi-Fi Remote as components:
- iperf (H2)
- getting_started/softAP
- getting_started/station
- scan
- fast_scan
- softap_sta
- updated `.build-test-rules.yml` to enable pre-commit to accept ESP-P4
and ESP-H2 as Supported Targets
- updated iperf sdkconfig.defaults for P4 to use parameters from wifi-remote
The esp_secure_boot_verify_sbv2_signature_block() and esp_secure_boot_verify_rsa_signature_block()
APIs need and use the verified_digest argument only for BOOTLOADER_BUILD, but the argument is
not used in the application code, and the value present in verified_digest is considered invalid.
Thus, allow passing NULL as the verified_digest parameter to help some save space.
docs: remove touch_pad.rst from C61 not_updated list
Closes IDF-10146, IDF-10136, IDF-10128, IDF-10122, IDF-10139, and IDF-10131
See merge request espressif/esp-idf!40681
Fixed memory leak in esp_http_client_cleanup() where current_header_value
buffer was not being freed when ESP_ERR_HTTP_FETCH_HEADER is returned
during header parsing failures.
The bootloader build script incorrectly used the `DEPENDS` keyword with the
`add_custom_command(TARGET ...)` signature. This is unsupported, causes
warnings with modern CMake versions enforcing the CMP0175 policy.
This commit also updates the POST_BUILD messages generated during the
bootloader build to depend on more precise CMake targets rather than
depending on the generic bootloader.elf target.
The following changes have been made:
- Add new xTaskGetStackStart() function with proper StackType_t* return type
- Deprecate pxTaskGetStackStart() with wrapper implementation for backward compatibility
Moved USB PHY source and headers from `usb/` to `esp_hw_support/`
to better reflect their hardware-specific nature.
Replaced use of public GPIO driver APIs with internal LL (Low-Level) APIs
for more precise hardware control and alignment with ESP-IDF HAL best practices.
Deleted deprecated PHY helper functions.
Consolidated all SoC-specific declarations under `usb_periph.h`.
BREAKING CHANGE: deprecated USB PHY APIs have been removed; update
callers to use the new interfaces in esp_hw_support/usb_phy.h
This commit trims the idf_build_executable() logic and refactors flash
target creation, utility target creation and post-build target creation
into their own separate cmake files/or functions.
This commit global variables such as ESPTOOLPY, ESPSECUREPY, ESPEFUSEPY,
ESPMONITOR and ESPTOOLPY_CHIP from the project_include.cmake file of
esptool_py component. All other components which use these variables
have been updated to fetch the same from esptool_py component's
properties.
This commit refactors the esptool_py component to provide utility
functions for creating utility targets such as erase_flash, merge-bin
and monitor.
The following changes were done in this commit:
- Added __esptool_py_setup_utility_targets() to create utility targets.
- Utility target creation now happens in idf_build_executable() in
build.cmake.
- Removed more global scope processing and variables from esptool_py
component project_include.cmake.
This commit refactors the esptool_py component to move the
flasher_args.json file generation to the main project level cmake file
when idf_build_executable() runs.
This commit refactors the esptool_py component to provide utility
functions for binary file generation targets instead of creating the
targets. Binary generation targets are now moved to the respective
projects.
The following changes were done in this commit:
- Added __idf_build_binary() function to esptool_py to create the binary
file generation target.
- Added __idf_build_secure_binary() as the secure boot equivalent of the
above function.
- Top level project build now creates its own binary targets in
idf_build_executable() in build.cmake.
- Bootloader and esp_tee subprojects create their binary file generation
targets in their respective CMakeLists.txt files.
- All post-build targets such as the app_size_check target are now
created by the respective projects and not esptool_py.
- General clean-up of the esptool_py cmake files.
This commit refactors the esptool_py component to provide utility
functions for flash target management instead of creating the targets
directly. Flash target creation is now moved to the project level in
build.cmake file when idf_build_executable() runs.
The following changes were done in this commit:
- Added __esptool_py_setup_tools(), __esptool_py_setup_estool_py_args()
and __ensure_esptool_py_setup() functions to centralize esptool_py
setup.
- Added __esptool_py_setup_main_flash_target() which is called by
idf_build_executable() to create the flash targets.
- Updated esptool_py_flash_target(), esptool_py_custom_target() to
accept an optional FILENAME_PREFIX argument to enable creation of
build artifacts based on custom names.
- Create placeholder flash targets early in the build process when
idf_build_process() is called for components to add dependencies on
these targets.
- Moved app-flash target creation from esptool_py/CMakeLists.txt to
build.cmake.
- Added function description to esptool_py functions.
This commit establishes the foundation for making the esptool_py
component idempotent.
The following changes are made in this commit:
- Removes unnecessary dependency of esp_wifi component on esptool_py.
- Add missing esptool_py dependencies to components which directly use
esptool_py specific functions or variables but do not declare a public
or private dependency.
feat(esp_http_client): Add HTTP_EVENT_ON_STATUS_CODE to notify early status code acquisition
Closes IDF-13452
See merge request espressif/esp-idf!40244
Removed
- target markers. Now must use target as parametrization in esp-idf
- host test markers. Now will be automatically added with linux target and qemu marker
feat(esp_driver_cam): DVP driver supports only initializing the clock and XCLK pin to generate a clock for the external device
Closes IDF-13385
See merge request espressif/esp-idf!39837
BLE Log SPI Out Dev Phase 4
Closes BLERP-2033, BLERP-2032, BLERP-2014, BLERP-2019, BLERP-2016, and BLERP-2015
See merge request espressif/esp-idf!40190
Fixed some BLE bugs 250220 on esp32c3(2edb0b0)
Closes BLERP-1961, BLERP-1963, BLERP-1967, BLERP-1986, and BLERP-1973
See merge request espressif/esp-idf!39798
cpu_utility_ll_stall_cpu() used to busy-wait check stalled status
after stalling the core. On H4 it turns out that this status will
not be set if the core happens to be in WFI state when stalled.
If this happened the stall function would just wait forever for
the status.
ci(hal): Add HAL/LL-based test app for the TEE and APM peripherals
Closes IDF-8614, IDF-8615, IDF-9229, IDF-9230, IDF-10422, IDF-12646, IDF-12647, IDF-12648, IDF-12649, and IDF-12877
See merge request espressif/esp-idf!39873
- Fixed TX issue when the event is aborted
- Fixed BLE assert llc_llcp.c 487
- Fixed BLE assert sch_prog.c 304
- Fixed anonymous extended adv reporting
- Support vendor HCI related params reset
feat(storage): Update emmc and perf_benchmark example to work with ESP32-P4, fix SDMMC slot deinit bug in legacy driver
Closes IDF-13394 and IDF-13483
See merge request espressif/esp-idf!39826
This commit fixes an issue encountered during OTA when partial download
is enabled using an AWS signed URL restricted to GET requests.
It also adds an API to retrieve the OTA image size from the Content-Range header.
Added the hidden config in the esp-tls component for the dynamic buffer
strategy configuration feature. So that external components like ota
can findout whether this feature is supported or not
fix(wifi/mesh): fixed the mesh crash issue when changing mesh AP's authmode
Closes WIFIBUG-1274 and IDFGH-15387
See merge request espressif/esp-idf!40165
feat(lwip/config): add menuconfig option to enable ND6 RIO support and set the RIO cache size
Closes IDFGH-13888
See merge request espressif/esp-idf!39488
When connecting USB OTG to an external FSLS PHY,
the FSLS Serial Interface signals can be routed to any GPIO via the GPIO
matrix. Thus, these macros are meaningless.
GCC defines that malloc(0) returns a non-null pointer (glibc behavior).
This can break logic for some cases, because our implementation of
malloc(0) returns null pointer.
POSIX and the ISO C standard allow both behaviors.
GCC 15 introduces a regression in guaranteed zero-initialization of
padding bits. The suggested solution is to add the
-fzero-init-padding-bits=unions compile option.
To prevent similar issues in the future, this change adds the
-fzero-init-padding-bits=all build option. It ensures that padding bits
in unions and structs are properly zeroed, avoiding regressions.
xtensa linker throws warning
dangerous relocation: windowed longcall crosses 1GB boundary; return may fail
create separate function to call function by pointer to avoid warnings
If a port is specified for the idf.py command, pass it to esp-idf-diag.
This ensures that the esptools used for gathering target information
connect to the explicitly specified port rather than relying on
autodetection.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
description:Onwhich Espressif SoC revision does your application run on? Run `esptool chip_id` to find it.
description:Onwhich Espressif SoC revision does your application run on? Run `esptool chip-id` (or `esptool.py chip_id` for ESP-IDF v5.5 and older) to find it.
- if [ -n "$LOCAL_GITLAB_HTTPS_HOST" ]; then export LOCAL_CI_REPOSITORY_URL="https://gitlab-ci-token:${CI_JOB_TOKEN}@${LOCAL_GITLAB_HTTPS_HOST}/${CI_PROJECT_PATH}"; fi
- if [ -n "$LOCAL_GIT_MIRROR" ]; then export LOCAL_CI_REPOSITORY_URL="${LOCAL_GIT_MIRROR}/${CI_PROJECT_PATH}"; fi
- echo "Using repository at $LOCAL_CI_REPOSITORY_URL"
- git diff --exit-code -- ${IDF_TARGET}/esp_efuse_table.c || { echo 'Differences found for ${IDF_TARGET} target. Please run idf.py efuse-common-table and commit the changes.'; exit 1; }
- cd ${IDF_PATH}/components/efuse/test_efuse_host
- ./efuse_tests.py
test_efuse_table_on_host_esp32:
extends:.test_efuse_table_on_host_template
test_efuse_table_on_host_esp32s2:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32s2
test_efuse_table_on_host_esp32s3:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32s3
test_efuse_table_on_host_esp32c3:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32c3
test_efuse_table_on_host_esp32h2:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32h2
test_efuse_table_on_host_esp32c6:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32c6
- pytest_for_ut ./efuse_tests.py
test_logtrace_proc:
extends:.host_test_template
@@ -201,20 +163,21 @@ test_tools:
INSTALL_EXTRA_TOOLS:"qemu-xtensa qemu-riscv32"# for test_idf_qemu.py
@@ -39,27 +39,5 @@ _For other small/non-public changes, which are not expected to be in the release
* Mention submodule MR, if there is
* Mention backport(ed) MR, if there is
_Don't touch the subsection titles below, they will be parsed by scripts._
## Release notes <!-- Mandatory -->
_Changes made in this MR that should go into the **Release Notes** should be listed here. Please use **past tense** and *specify the area (see maintainers page of IDF internal wiki)*. If there is a subscope, include it and separate with slash (`/`). Minor changes can go to the descriptions above without a release notes entry._
_Write all the changes in a **list** (Start at the beginning of the line with `-` or `*`). If multiple changes are made, each of them should take a single line. If there is only one change to list, it should still be the only line of a list. If this MR does not need any release notes, write "No release notes" here without the `-` or `*`. e.g._
_If there are any breaking changes, please mention it here. Talking about (1) what is not accepted any more, (2) the alternative solution and (3) the benefits/reason. e.g._
_Please strictly follow the breaking change restriction, which means, if there is a breaking change but you are merging to non-major versions, you have to separate the breaking part out to another MR for a major version. The breaking change subsection is only accepted in MRs merging to major versions._
* [VFS/UART] Now vfs_uart_set_rts_cts accept one more instance argument, to support configuration to different ports.
<!-- Don't remove the next line - assigns the MR author as the assignee -->
@@ -10,7 +10,7 @@ NOTE: This document on release branches may be out-of-date. Check the [Compatibi
See [Compatibility Advisory for Chip Revision Numbering Scheme](https://www.espressif.com/sites/default/files/advisory_downloads/AR2022-005%20Compatibility%20Advisory%20for%20Chip%20Revision%20Numbering%20%20Scheme.pdf) on the versioning of Espressif SoC revisions.
You can run `esptool chip_id` to detect the series and revision of an SoC. See [SoC Errata](https://www.espressif.com/en/support/documents/technical-documents?keys=errata) for more on how to distinguish between chip revisions, and the improvements provided by chip revisions. And run `idf.py --version` to know the version of current ESP-IDF.
You can run `esptool chip-id` to detect the series and revision of an SoC. See [SoC Errata](https://www.espressif.com/en/support/documents/technical-documents?keys=errata) for more on how to distinguish between chip revisions, and the improvements provided by chip revisions. And run `idf.py --version` to know the version of current ESP-IDF.
## ESP-IDF Support for Different Chip Revisions
@@ -140,6 +140,12 @@ Supported since ESP-IDF v5.1.
| release/v5.4 | v5.4.1+ | v5.4.1 |
| release/v5.5 and above | v5.5+ | v5.5 |
### ESP32-P4
#### v1.0, v1.3
Supported since ESP-IDF v5.3.
## What If the ESP-IDF Version Is Lower than the `Required` Version?
Latest ESP-IDF versions can prevent from downloading to, or even execute binaries on unsupported chips. ESP-IDF of versions v4.4.5+, v5.0.1+, v5.1 and above have both esptool download check and bootloader loading check against the chip revision. While ESP-IDF v4.3.5 has only esptool downloading check.
@@ -15,18 +15,19 @@ ESP-IDF is the development framework for Espressif SoCs supported on Windows, Li
The following table shows ESP-IDF support of Espressif SoCs where ![alt text][preview] and ![alt text][supported] denote preview status and support, respectively. The preview support is usually limited in time and intended for beta versions of chips. Please use an ESP-IDF release where the desired SoC is already supported.
@@ -10,7 +10,7 @@ The core components are organized into two groups.
The first group (referred to as `G0`) includes `hal`, `arch` (where `arch` is either `riscv` or `xtensa` depending on the chip), `esp_rom`, `esp_common`, and `soc`. This group contains information about and provides low-level access to the underlying hardware. In the case of `esp_common`, it contains hardware-agnostic code and utilities. These components may have dependencies on each other within the group, but outside dependencies should be minimized. The reason for this approach is that these components are fundamental, and many other components may require them. Ideally, the dependency relationship only goes one way, making it easier for this group to be usable in other projects.
The second group (referred to as `G1`) operates at a higher level than the first group. `G1` includes the components `esp_hw_support`, `esp_system`, `newlib`, `spi_flash`, `freertos`, `log`, and `heap`. Like the first group, circular dependencies within this group are allowed, and these components can have dependencies on the first group. G1 components represent essential software mechanisms for building other components.
The second group (referred to as `G1`) operates at a higher level than the first group. `G1` includes the components `esp_hw_support`, `esp_system`, `esp_libc`, `spi_flash`, `freertos`, `log`, and `heap`. Like the first group, circular dependencies within this group are allowed, and these components can have dependencies on the first group. G1 components represent essential software mechanisms for building other components.
## Descriptions
@@ -40,7 +40,7 @@ Example:
#### `esp_common`
Contains hardware-agnostic definitions, constants, macros, utilities, 'pure' and/or algorithmic functions that is useable by all other components (that is, barring there being a more appropriate component to put them in).
Contains hardware-agnostic definitions, constants, macros, utilities, 'pure' and/or algorithmic functions that is usable by all other components (that is, barring there being a more appropriate component to put them in).
Example:
@@ -85,7 +85,7 @@ Logging library.
Heap implementation.
#### `newlib`
#### `esp_libc`
Some functions n the standard library are implemented here, especially those needing other `G1` components.
// In general tracing goes in the following way. User application requests tracing module to send some data by calling esp_apptrace_buffer_get(),
// module allocates necessary buffer in current input trace block. Then user fills received buffer with data and calls esp_apptrace_buffer_put().
// When current input trace block is filled with app data it is exposed to host and the second block becomes input one and buffer filling restarts.
// While target application fills one TRAX block host reads another one via JTAG.
// This module also allows communication in the opposite direction: from host to target. As it was said ESP32 and host can access different TRAX blocks
// simultaneously, so while target writes trace data to one block host can write its own data (e.g. tracing commands) to another one then when
// blocks are switched host receives trace data and target receives data written by host application. Target user application can read host data
// by calling esp_apptrace_read() API.
// To control buffer switching and for other communication purposes this implementation uses some TRAX registers. It is safe since HW TRAX tracing
// can not be used along with application tracing feature so these registers are freely readable/writeable via JTAG from host and via ERI from ESP32 cores.
// Overhead of this implementation on target CPU is produced only by allocating/managing buffers and copying of data.
// On the host side special OpenOCD command must be used to read trace data.
// 2. TRAX Registers layout
// ========================
// This module uses two TRAX HW registers and one Performance Monitor register to communicate with host SW (OpenOCD).
// - Control register uses TRAX_DELAYCNT as storage. Only lower 24 bits of TRAX_DELAYCNT are writable. Control register has the following bitfields:
// CRC indicator (0xA55A) is used to distinguish valid CRC values from other data that might be in the register.
// CRC16 is calculated over the entire exposed block and is updated every time a block is exposed to the host.
// This allows the host to verify data integrity of the received trace data.
// 3. Modes of operation
// =====================
// This module supports two modes of operation:
// - Post-mortem mode. This is the default mode. In this mode application tracing module does not check whether host has read all the data from block
// exposed to it and switches block in any case. The mode does not need host interaction for operation and so can be useful when only the latest
// trace data are necessary, e.g. for analyzing crashes. On panic the latest data from current input block are exposed to host and host can read them.
// It can happen that system panic occurs when there are very small amount of data which are not exposed to host yet (e.g. crash just after the
// TRAX block switch). In this case the previous 16KB of collected data will be dropped and host will see the latest, but very small piece of trace.
// It can be insufficient to diagnose the problem. To avoid such situations there is menuconfig option
// CONFIG_APPTRACE_POSTMORTEM_FLUSH_THRESH
// which controls the threshold for flushing data in case of panic.
// - Streaming mode. Tracing module enters this mode when host connects to target and sets respective bits in control registers (per core).
// In this mode before switching the block tracing module waits for the host to read all the data from the previously exposed block.
// On panic tracing module also waits (timeout is configured via menuconfig via CONFIG_APPTRACE_ONPANIC_HOST_FLUSH_TMO) for the host to read all data.
// 4. Communication Protocol
// =========================
// 4.1 Trace Memory Blocks
// -----------------------
// Communication is controlled via special register. Host periodically polls control register on each core to find out if there are any data available.
// When current input memory block is filled it is exposed to host and 'block_len' and 'block_id' fields are updated in the control register.
// Host reads new register value and according to it's value starts reading data from exposed block. Meanwhile target starts filling another trace block.
// When host finishes reading the block it clears 'block_len' field in control register indicating to the target that it is ready to accept the next one.
// If the host has some data to transfer to the target it writes them to trace memory block before clearing 'block_len' field. Then it sets
// 'host_data_present' bit and clears 'block_len' field in control register. Upon every block switch target checks 'host_data_present' bit and if it is set
// reads them to down buffer before writing any trace data to switched TRAX block.
// 4.2 User Data Chunks Level
// --------------------------
// Since trace memory block is shared between user data chunks and data copying is performed on behalf of the API user (in its normal context) in
// multithreading environment it can happen that task/ISR which copies data is preempted by another high prio task/ISR. So it is possible situation
// that task/ISR will fail to complete filling its data chunk before the whole trace block is exposed to the host. To handle such conditions tracing
// module prepends all user data chunks with header which contains allocated buffer size and actual data length within it. OpenOCD command
// which reads application traces reports error when it reads incomplete user data block.
// Data which are transffered from host to target are also prepended with a header. Down channel data header is simple and consists of one two bytes field
// containing length of host data following the header.
// 4.3 Data Buffering
// ------------------
// It takes some time for the host to read TRAX memory block via JTAG. In streaming mode it can happen that target has filled its TRAX block, but host
// has not completed reading of the previous one yet. So in this case time critical tracing calls (which can not be delayed for too long time due to
// the lack of free memory in TRAX block) can be dropped. To avoid such scenarios tracing module implements data buffering. Buffered data will be sent
// to the host later when TRAX block switch occurs. The maximum size of the buffered data is controlled by menuconfig option
// CONFIG_APPTRACE_PENDING_DATA_SIZE_MAX.
// 4.4 Target Connection/Disconnection
// -----------------------------------
// When host is going to start tracing in streaming mode it needs to put both ESP32 cores into initial state when 'host connected' bit is set
// on both cores. To accomplish this host halts both cores and sets this bit in TRAX registers. But target code can be halted in state when it has read control
// register but has not updated its value. To handle such situations target code indicates to the host that it is updating control register by writing
// non-zero value to status register. Actually it writes address of the instruction which it will execute when it finishes with
// the registers update. When target is halted during control register update host sets breakpoint at the address from status register and resumes CPU.
// After target code finishes with register update it is halted on breakpoint, host detects it and safely sets 'host connected' bit. When both cores
// are set up they are resumed. Tracing starts without further intrusion into CPUs work.
// When host is going to stop tracing in streaming mode it needs to disconnect targets. Disconnection process is done using the same algorithm
// as for connecting, but 'host connected' bits are cleared on ESP32 cores.
// 5. Module Access Synchronization
// ================================
// Access to internal module's data is synchronized with custom mutex. Mutex is a wrapper for portMUX_TYPE and uses almost the same sync mechanism as in
// vPortCPUAcquireMutex/vPortCPUReleaseMutex. The mechanism uses S32C1I Xtensa instruction to implement exclusive access to module's data from tasks and
// ISRs running on both cores. Also custom mutex allows specifying timeout for locking operation. Locking routine checks underlying mutex in cycle until
// it gets its ownership or timeout expires. The differences of application tracing module's mutex implementation from vPortCPUAcquireMutex/vPortCPUReleaseMutex are:
// - Support for timeouts.
// - Local IRQs for CPU which owns the mutex are disabled till the call to unlocking routine. This is made to avoid possible task's prio inversion.
// When low prio task takes mutex and enables local IRQs gets preempted by high prio task which in its turn can try to acquire mutex using infinite timeout.
// So no local task switch occurs when mutex is locked. But this does not apply to tasks on another CPU.
// WARNING: Priority inversion can happen when low prio task works on one CPU and medium and high prio tasks work on another.
// WARNING: Care must be taken when selecting timeout values for trace calls from ISRs. Tracing module does not care about watchdogs when waiting
// on internal locks and for host to complete previous block reading, so if timeout value exceeds watchdog's one it can lead to the system reboot.
// 6. Timeouts
// ===========
// Timeout mechanism is based on xthal_get_ccount() routine and supports timeout values in microseconds.
// There are two situations when task/ISR can be delayed by tracing API call. Timeout mechanism takes into account both conditions:
// - Trace data are locked by another task/ISR. When waiting on trace data lock.
// - Current TRAX memory input block is full when working in streaming mode (host is connected). When waiting for host to complete previous block reading.
// When waiting for any of above conditions xthal_get_ccount() is called periodically to calculate time elapsed from trace API routine entry. When elapsed
// time exceeds specified timeout value operation is canceled and ESP_ERR_TIMEOUT code is returned.
// In general tracing goes in the following way. User application requests tracing module to send some data by calling esp_apptrace_buffer_get(),
// module allocates necessary buffer in current input trace block. Then user fills received buffer with data and calls esp_apptrace_buffer_put().
// When current input trace block is filled with app data it is exposed to host and the second block becomes input one and buffer filling restarts.
// While target application fills one TRAX block host reads another one via JTAG.
// This module also allows communication in the opposite direction: from host to target. As it was said ESP32 and host can access different TRAX blocks
// simultaneously, so while target writes trace data to one block host can write its own data (e.g. tracing commands) to another one then when
// blocks are switched host receives trace data and target receives data written by host application. Target user application can read host data
// by calling esp_apptrace_read() API.
// To control buffer switching and for other communication purposes this implementation uses some TRAX registers. It is safe since HW TRAX tracing
// can not be used along with application tracing feature so these registers are freely readable/writeable via JTAG from host and via ERI from ESP32 cores.
// Overhead of this implementation on target CPU is produced only by allocating/managing buffers and copying of data.
// On the host side special OpenOCD command must be used to read trace data.
// 2. TRAX Registers layout
// ========================
// This module uses two TRAX HW registers and one Performance Monitor register to communicate with host SW (OpenOCD).
// - Control register uses TRAX_DELAYCNT as storage. Only lower 24 bits of TRAX_DELAYCNT are writable. Control register has the following bitfields:
// CRC indicator (0xA55A) is used to distinguish valid CRC values from other data that might be in the register.
// CRC16 is calculated over the entire exposed block and is updated every time a block is exposed to the host.
// This allows the host to verify data integrity of the received trace data.
// 3. Modes of operation
// =====================
// This module supports two modes of operation:
// - Post-mortem mode. This is the default mode. In this mode application tracing module does not check whether host has read all the data from block
// exposed to it and switches block in any case. The mode does not need host interaction for operation and so can be useful when only the latest
// trace data are necessary, e.g. for analyzing crashes. On panic the latest data from current input block are exposed to host and host can read them.
// It can happen that system panic occurs when there are very small amount of data which are not exposed to host yet (e.g. crash just after the
// TRAX block switch). In this case the previous 16KB of collected data will be dropped and host will see the latest, but very small piece of trace.
// It can be insufficient to diagnose the problem. To avoid such situations there is menuconfig option
// CONFIG_APPTRACE_POSTMORTEM_FLUSH_THRESH
// which controls the threshold for flushing data in case of panic.
// - Streaming mode. Tracing module enters this mode when host connects to target and sets respective bits in control registers (per core).
// In this mode before switching the block tracing module waits for the host to read all the data from the previously exposed block.
// On panic tracing module also waits (timeout is configured via menuconfig via CONFIG_APPTRACE_ONPANIC_HOST_FLUSH_TMO) for the host to read all data.
// 4. Communication Protocol
// =========================
// 4.1 Trace Memory Blocks
// -----------------------
// Communication is controlled via special register. Host periodically polls control register on each core to find out if there are any data available.
// When current input memory block is filled it is exposed to host and 'block_len' and 'block_id' fields are updated in the control register.
// Host reads new register value and according to it's value starts reading data from exposed block. Meanwhile target starts filling another trace block.
// When host finishes reading the block it clears 'block_len' field in control register indicating to the target that it is ready to accept the next one.
// If the host has some data to transfer to the target it writes them to trace memory block before clearing 'block_len' field. Then it sets
// 'host_data_present' bit and clears 'block_len' field in control register. Upon every block switch target checks 'host_data_present' bit and if it is set
// reads them to down buffer before writing any trace data to switched TRAX block.
// 4.2 User Data Chunks Level
// --------------------------
// Since trace memory block is shared between user data chunks and data copying is performed on behalf of the API user (in its normal context) in
// multithreading environment it can happen that task/ISR which copies data is preempted by another high prio task/ISR. So it is possible situation
// that task/ISR will fail to complete filling its data chunk before the whole trace block is exposed to the host. To handle such conditions tracing
// module prepends all user data chunks with header which contains allocated buffer size and actual data length within it. OpenOCD command
// which reads application traces reports error when it reads incomplete user data block.
// Data which are transffered from host to target are also prepended with a header. Down channel data header is simple and consists of one two bytes field
// containing length of host data following the header.
// 4.3 Target Connection/Disconnection
// -----------------------------------
// When host is going to start tracing in streaming mode it needs to put both ESP32 cores into initial state when 'host connected' bit is set
// on both cores. To accomplish this host halts both cores and sets this bit in TRAX registers. But target code can be halted in state when it has read control
// register but has not updated its value. To handle such situations target code indicates to the host that it is updating control register by writing
// non-zero value to status register. Actually it writes address of the instruction which it will execute when it finishes with
// the registers update. When target is halted during control register update host sets breakpoint at the address from status register and resumes CPU.
// After target code finishes with register update it is halted on breakpoint, host detects it and safely sets 'host connected' bit. When both cores
// are set up they are resumed. Tracing starts without further intrusion into CPUs work.
// When host is going to stop tracing in streaming mode it needs to disconnect targets. Disconnection process is done using the same algorithm
// as for connecting, but 'host connected' bits are cleared on ESP32 cores.
// 5. Module Access Synchronization
// ================================
// Access to internal module's data is synchronized with custom mutex. Mutex is a wrapper for portMUX_TYPE and uses almost the same sync mechanism as in
// vPortCPUAcquireMutex/vPortCPUReleaseMutex. The mechanism uses S32C1I Xtensa instruction to implement exclusive access to module's data from tasks and
// ISRs running on both cores. Also custom mutex allows specifying timeout for locking operation. Locking routine checks underlying mutex in cycle until
// it gets its ownership or timeout expires. The differences of application tracing module's mutex implementation from vPortCPUAcquireMutex/vPortCPUReleaseMutex are:
// - Support for timeouts.
// - Local IRQs for CPU which owns the mutex are disabled till the call to unlocking routine. This is made to avoid possible task's prio inversion.
// When low prio task takes mutex and enables local IRQs gets preempted by high prio task which in its turn can try to acquire mutex using infinite timeout.
// So no local task switch occurs when mutex is locked. But this does not apply to tasks on another CPU.
// WARNING: Priority inversion can happen when low prio task works on one CPU and medium and high prio tasks work on another.
// WARNING: Care must be taken when selecting timeout values for trace calls from ISRs. Tracing module does not care about watchdogs when waiting
// on internal locks and for host to complete previous block reading, so if timeout value exceeds watchdog's one it can lead to the system reboot.
// 6. Timeouts
// ===========
// Timeout mechanism is based on xthal_get_ccount() routine and supports timeout values in microseconds.
// There are two situations when task/ISR can be delayed by tracing API call. Timeout mechanism takes into account both conditions:
// - Trace data are locked by another task/ISR. When waiting on trace data lock.
// - Current TRAX memory input block is full when working in streaming mode (host is connected). When waiting for host to complete previous block reading.
// When waiting for any of above conditions xthal_get_ccount() is called periodically to calculate time elapsed from trace API routine entry. When elapsed
// time exceeds specified timeout value operation is canceled and ESP_ERR_TIMEOUT code is returned.
#ifndef SEGGER_USE_PARA // Some compiler complain about unused parameters.
#define SEGGER_USE_PARA(Para) (void)Para // This works for most compilers.
#endif
#define SEGGER_ADDR2PTR(Type, Addr) (/*lint -e(923) -e(9078)*/((Type*)((PTR_ADDR)(Addr)))) // Allow cast from address to pointer.
#define SEGGER_PTR2ADDR(p) (/*lint -e(923) -e(9078)*/((PTR_ADDR)(p))) // Allow cast from pointer to address.
#define SEGGER_PTR2PTR(Type, p) (/*lint -e(740) -e(826) -e(9079) -e(9087)*/((Type*)(p))) // Allow cast from one pointer type to another (ignore different size).
SEGGER_BUFFER_DESC*pBufferDesc;// Buffer descriptor to use for output.
void(*pfFlush)(SEGGER_SNPRINTF_CONTEXT*pContext);// Callback executed once the buffer is full. Callback decides if the buffer gets cleared to store more or not.
// Description for a circular buffer (also called "ring buffer")
// which is used as up-buffer (T->H)
//
typedefstruct{
constchar*sName;// Optional name. Standard names so far are: "Terminal", "SysView", "J-Scope_t4i4"
char*pBuffer;// Pointer to start of buffer
unsignedSizeOfBuffer;// Buffer size in bytes. Note that one byte is lost, as this implementation does not fill up the buffer in order to avoid the problem of being unable to distinguish between full and empty.
unsignedWrOff;// Position of next item to be written by either target.
volatileunsignedRdOff;// Position of next item to be read by host. Must be volatile since it may be modified by host.
unsignedFlags;// Contains configuration flags. Flags[31:24] are used for validity check and must be zero. Flags[23:2] are reserved for future use. Flags[1:0] = RTT operating mode.
}SEGGER_RTT_BUFFER_UP;
//
// Description for a circular buffer (also called "ring buffer")
// which is used as down-buffer (H->T)
//
typedefstruct{
constchar*sName;// Optional name. Standard names so far are: "Terminal", "SysView", "J-Scope_t4i4"
char*pBuffer;// Pointer to start of buffer
unsignedSizeOfBuffer;// Buffer size in bytes. Note that one byte is lost, as this implementation does not fill up the buffer in order to avoid the problem of being unable to distinguish between full and empty.
volatileunsignedWrOff;// Position of next item to be written by host. Must be volatile since it may be modified by host.
unsignedRdOff;// Position of next item to be read by target (down-buffer).
unsignedFlags;// Contains configuration flags. Flags[31:24] are used for validity check and must be zero. Flags[23:2] are reserved for future use. Flags[1:0] = RTT operating mode.
}SEGGER_RTT_BUFFER_DOWN;
//
// RTT control block which describes the number of buffers available
// as well as the configuration for each buffer
//
//
typedefstruct{
characID[16];// Initialized to "SEGGER RTT"
intMaxNumUpBuffers;// Initialized to SEGGER_RTT_MAX_NUM_UP_BUFFERS (type. 2)
intMaxNumDownBuffers;// Initialized to SEGGER_RTT_MAX_NUM_DOWN_BUFFERS (type. 2)
SEGGER_RTT_BUFFER_UPaUp[SEGGER_RTT_MAX_NUM_UP_BUFFERS];// Up buffers, transferring information up from target via debug probe to host
SEGGER_RTT_BUFFER_DOWNaDown[SEGGER_RTT_MAX_NUM_DOWN_BUFFERS];// Down buffers, transferring information down from host via debug probe to target
#define SEGGER_RTT_HASDATA_UP(n) (((SEGGER_RTT_BUFFER_UP*)((uintptr_t)&_SEGGER_RTT.aUp[n] + SEGGER_RTT_UNCACHED_OFF))->WrOff - ((SEGGER_RTT_BUFFER_UP*)((uintptr_t)&_SEGGER_RTT.aUp[n] + SEGGER_RTT_UNCACHED_OFF))->RdOff) // Access uncached to make sure we see changes made by the J-Link side and all of our changes go into HW directly
#define SEGGER_SYSVIEW_INFO_SIZE 9 // Minimum size, which has to be reserved for a packet. 1-2 byte of message type, 0-2 byte of payload length, 1-5 bytes of timestamp.
#define SEGGER_SYSVIEW_QUANTA_U32 5 // Maximum number of bytes to encode a U32, should be reserved for each 32-bit value in a packet.
#define SEGGER_SYSVIEW_LOG (0u)
#define SEGGER_SYSVIEW_WARNING (1u)
#define SEGGER_SYSVIEW_ERROR (2u)
#define SEGGER_SYSVIEW_FLAG_APPEND (1u << 6)
#define SEGGER_SYSVIEW_PREPARE_PACKET(p) (p) + 4
//
// SystemView events. First 32 IDs from 0 .. 31 are reserved for these
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() ((*(U32*)(0xE000ED04)) & 0x1FF) // Get the currently active interrupt Id. (i.e. read Cortex-M ICSR[8:0] = active vector)
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() (__get_IPSR()) // Workaround for IAR, which might do a byte-access to 0xE000ED04. Read IPSR instead.
#else
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() ((*(U32*)(0xE000ED04)) & 0x3F) // Older versions of IAR do not include __get_IPSR, but might also not optimize to byte-access.
#endif
#else
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() ((*(U32*)(0xE000ED04)) & 0x3F) // Get the currently active interrupt Id. (i.e. read Cortex-M ICSR[5:0] = active vector)
#endif
#else
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() SEGGER_SYSVIEW_X_GetInterruptId() // Get the currently active interrupt Id from the user-provided function.
* Unlike esp_ota_begin(), this function does not erase the partition which receives the OTA update, but rather expects that part of the image
* has already been written correctly, and it resumes writing from the given offset.
*
* @note When flash encryption is enabled, data writes must be 16-byte aligned.
* Any leftover (non-aligned) data is temporarily cached and may be lost after reboot.
* Therefore, during resumption, ensure that image offset is always 16-byte aligned.
*
* @param partition Pointer to info for the partition which is receiving the OTA update. Required.
* @param erase_size Specifies how much flash memory to erase before resuming OTA, depending on whether a sequential write or a bulk erase is being used.
* @param image_offset Offset from where to resume the OTA process. Should be set to the number of bytes already written.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
