The eager upgreade policy should upgrade the full dependency tree and
not just the packages listed in the requirements file. The disadvantage
is some extra time for package resolution.
This upgrade policy is applied to CI-related packages only because we
don't want this to be used for customer-facing packages and/or provide
the customers with a different environment to that tested in the CI.
When the eFuse field consists of single descriptor, its size can be
taken from struct member 'bit_count'. However, when the field consists of
multiple descriptors, the size should be obtained via esp_efuse_get_field_size()
The build system keeps track of each component source. Currently
there are four types of sources:
1. "project_components" - project components
2. "project_extra_components" - components from EXTRA_COMPONENT_DIRS
3. "project_managed_components" - custom project dependencies managed by the IDF Component Manager
4. "idf_components" - ESP-IDF built-in components, typically under /components
This can be used to identify the component libraries that are likely to
change during application development and pass them to ldgen as mutable
libraries. Add all components with "project_components" as their source
as mutable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a helper function `__should_generate_sdkconfig()` that checks
whether the sdkconfig file should be generated. The
`__generate_sdkconfig()` function can be called multiple times, such as
when the initial sdkconfig is generated at the start of the build
process and later after additional components are fetched by the
component manager. There might be no components fetched by the
component manager, for example, in the hello_world example, or the
downloaded components may not contain any configuration files. In such
cases, there is no need to regenerate the sdkconfig. This helper
function stores the list of configuration files in the
`__PREV_KCONFIGS`, `__PREV_KCONFIG_PROJBUILDS`, and
`__PREV_SDKCONFIG_RENAMES` build properties at its end, and at the
beginning, it compares them with the current lists of configuration
files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add checks to validate client_verify_data pointer and length before
processing in handle_session_command1. Prevents NULL pointer dereference
when client omits verifier data in Session_Command1, which could cause
device crash during provisioning (remote DoS attack).
For CONFIG_BOOTLOADER_SKIP_VALIDATE_IN_DEEP_SLEEP enabled and exit from
deep sleep case the secure boot signature verification must be skipped
to improve the wakeup performance.
Closes https://github.com/espressif/esp-idf/issues/15590
The idf_create_size_report function allows for the creation of size
report targets based on the generated link map file. The size report
targets are created using the TARGET option name: "<target>",
"<target>-files", and "<target>-components". These size report targets
are added to the idf_default_project with the TARGET set to "size",
resulting in the creation of "size", "size-files", and "size-components"
targets for the default project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_project_init macro is evaluated within the global variable
scope, and currently, there are a few variables still set. Let's ensure
all variables are properly unset to avoid polluting the global variable
namespace.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The MINIMAL_BUILD property is not relevant in cmakev2, as only
components explicitly linked through targets are included in the build
by design. Display a warning when this outdated build property is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The link flag for generating the map file is currently added globally to
the link_options. If multiple executables are generated, the link map
file is overwritten by the last created executable. Since cmakev2
supports building multiple executables, the link map file options need
to be set for each executable individually. To address this, add a new
MAPFILE_TARGET option to the idf_build_executable function. When set,
the link map file will be generated by the linker, and a target
specified in the MAPFILE_TARGET option will be created for the map file.
This also splits the idf_project_default macro. Only the
idf_project_init macro needs be called within the global scope, as it
includes the project_include.cmake files and the cmake version of the
configuration. The remaining functionality of the idf_project_default
macro is implemented in a __project_default helper function to avoid
polluting the global variable space.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the `idf_build_generate_metadata` function takes an
executable target as an argument, and the binary name is stored in the
`EXECUTABLE_BINARY` property. This approach is inconvenient because a
single executable might generate multiple binary images, making it
unreliable to store the binary image name in the executable property due
to the N:1 relationship. To address this, pass the binary image target
instead of the executable target to the `idf_build_generate_metadata`
function. This change is facilitated by linking the binary target,
executable target, and library interface targets with the following
properties:
binary:EXECUTABLE_TARGET -> executable:LIBRARY_INTERFACE -> library
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
With commit b719292b75 ("refactor(build-system): Simplify flash target creation"),
partition table and bootloader components are correctly adding their
dependency on the flash target, so there is no need for this to be done
in the build system. This removes the temporary workaround.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
refactor(example/bt): Refactor controller_hci_uart example for ESP32-C3/S3 to use UHCI driver
Closes IDF-14345
See merge request espressif/esp-idf!43270
This commit adds support for uf2 targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/uf2.cmake.
- Adds the idf_create_uf2() function to create the uf2 targets. This
function now takes the executable as an argument thus allowing the uf2
target to be created per-executable.
- idf_project_default() is updated to create the uf2 targets.
This commit adds support for dfu targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/dfu.cmake file on similar lines as the
tools/cmake/dfu.cmake file.
- Adds the idf_create_dfu() function to create the dfu targets. This
function now takes the executable as an argument thus allowing the dfu
target to be created per-executable.
- idf_project_default() is updated to create the dfu target.
fix(uart): watchdog got triggered due to uart sclk not exist
Closes IDFGH-15851, IDFGH-16685, IDFGH-16687, IDFCI-2846, and TIDF-228846
See merge request espressif/esp-idf!43279
The test_rebuild_no_changes test verifies that running idf.py build
successively without any file changes results in identical build
artifacts on the second run (i.e., nothing gets rebuilt).
This test was failing in buildv2 because it expected kconfig_menus.json
to be present in build/config/ after a normal build. However, in
cmakev2, kconfig_menus.json is not generated during regular builds.
In cmakev1, kconfig_menus.json was generated globally during every
build alongside other config files (sdkconfig.h, sdkconfig.cmake, etc).
In cmakev2, kconfig_menus.json generation does not happend for
normal builds because it depends on the Kconfig menu hierarchy
and cannot be generated globally. It must be generated per-executable.
Hence, this commit updates the artefacts list for cmakev2 to not expect
the kconfig_menus.json file during a build/re-build action.
Interrupt wdt would get triggered on uart_driver_install if uart driver was deleted before
Closes https://github.com/espressif/esp-idf/issues/17779
RTC wdt would get triggered on esp_restart if uart driver was deleted before
MIN() macro is only an expansion of a conditional operator.
xRingbufferGetCurFreeSize was called twice in the original code,
which may return different values in two calls, leading to incorrect
send_size calculation and eventually could trigger task watchdog.
feat(parlio_rx): support to force trigger eof
Closes IDF-14143, IDFCI-2958, IDFCI-6027, IDFCI-2929, and IDFCI-6242
See merge request espressif/esp-idf!43116
- Update the Key Manager key types to be generic
- Define a new enum to determine the length of the keys
- Refactor the Key Manager driver support generic key types and key lengths
- Also store key deployment mode in the key recovery info
fix(nvs_sec_provider): replace abort() when nvs_keys is missing with a warrnig log (GitHub PR)
Closes IDFGH-16597
See merge request espressif/esp-idf!43337
Applicable for chips affected by interrupt issue:
- ESP32C5
- ESP32C61
- ESP32H4
For all other chips that support the ZCMP extension without issues,
it will be enabled unconditionally.
This change improves build consistency across external projects integrated
through CMake by ensuring that compiler flags defined in configuration files
are passed correctly to the toolchain. It covers the majority of use cases,
as external projects are typically also CMake-based. For projects that use
a custom build system, users will still need to specify the required flags
manually.
Add new APIs to retrieve statistics about power management locks:
- esp_pm_get_lock_stats(): Get statistics for all PM lock types
- esp_pm_get_lock_instance_stats(): Get statistics for a single PM lock instance
Also update the test cases to verify the new functionality and
enable PM profiling in test configuration.
Closes https://github.com/espressif/esp-idf/issues/17770
Changes generated by pre-commit related to ruff-format and an update to
the copyright date.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduce ESPTOOLPY_FAST_REFLASHING configuration option. It instructs
ldgen to group entity mappings for libraries deemed mutable (prone to
change) separately from those considered immutable (unlikely to change).
Organizing mutable and immutable libraries separately in the linker
script allows the linker to form larger contiguous blocks of data for
immutable libraries in the application's output sections. These blocks
are likely to stay mostly unchanged between application recompilations,
enabling them to be skipped during reflashing.
Separating mutable and immutable libraries in the linker script to
minimize changes in the output sections is insufficient. Padding is
added after the input sections of mutable libraries in the default data
and text output sections. This creates a buffer for the mutable
libraries, allowing additional changes to be made without altering the
layout of the binary image.
Additionally two optimizations currently in use can still mix data from
these libraries, leading to significant changes even within the grouped
immutable libraries.
1. constant merging
Linker will try to merge input sections that have the MERGE and
STRING flags from different libraries (object files) to perform
optimizations like tail merging. For example, adding a string
literal in a mutable library will also change the addresses of
string literals from immutable libraries in such a merged section,
causing changes in the generated code when those literals are
referenced.
Disabled with COMPILER_DISABLE_MERGE_CONSTANTS(-fno-merge-constants)
2. literal pools on Xtensa
As optimization, the linker may merge literal pools from different
libraries (object files) to improve the generated code size. This
has the same effect as constant merging, and changes in mutable
libraries may cause changes in the generated code for immutable
libraries. To get larger unchanged continuous blocks in the text
output sections for immutable libraries, we need to ensure that the
Xtensa literal pools remain close to their references and are not
merged.
Disabled with CONFIG_COMPILER_ENABLE_TEXT_SECTION_LITERALS(-mtext-section-literals)
When ESPTOOLPY_FAST_REFLASHING is enabled, these two optimizations are
disabled to achieve larger unchaged continuous blocks for the grouped
immutable libraries, even though disabling these optimizations results
in slightly larger code.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The build system keeps track of each component source. Currently
there are four types of sources:
1. "project_components" - project components
2. "project_extra_components" - components from EXTRA_COMPONENT_DIRS
3. "project_managed_components" - custom project dependencies managed by the IDF Component Manager
4. "idf_components" - ESP-IDF built-in components, typically under /components
This can be used to identify the component libraries that are likely to
change during application development and pass them to ldgen as mutable
libraries. Add all components with "project_components" as their source
as mutable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the `--debug` option to print the entity tree to stdout. This can be
useful for viewing the entire entity tree state to identify potential
issues or incorrect placements.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduce a new `--mutable-libraries-file` option that accepts a file
containing the filenames of mutable libraries, each listed on a separate
line. Mutable libraries are component libraries expected to change
during development. In contrast, immutable component libraries are not
expected to change. In the generated linker script, the input sections
of mutable libraries are grouped together rather than being mixed with
those of immutable libraries. The goal is to create large continuous
areas in the ELF file's output sections that remain unchanged for
immutable libraries during application recompilation, allowing these
areas to be skipped during flashing.
The build system identifies the mutable libraries and passes them to
ldgen using the `--mutable-libraries-file` option. It maintains
information about component sources, one of which is
`project_components`. This source type identifies components that are
directly related to the project being developed and are very likely to
change.
Mappings for mutable libraries are explicitly created for all sections
in the default scheme. This happens before the entity
`(archive:object_file:symbol/input_section)` node tree with placements is
generated and is equivalent to having these mappings in the mapping
linker fragment. All placements for mappings, whether newly added or
already existing as defined in linker fragments, associated with mutable
libraries are flagged as `mutable` in the entity node tree. This flag
ensures that these placements are included in the final linker script.
Currently, ldgen only emits placements that are either significant or
forced. A placement is considered significant if, for example, it is not
already covered by a placement in parent node. For instance, `*(.iram1
.iram1.*)` placement already includes `*libapp_trace.a:(.iram1
.iram1.*)`, so the latter is not emitted by default. The `mutable` flag
ensures that placements for mutable libraries are emitted in the linker
script and placed at dedicated location.
The locations where placements for mutable libraries are specified in
the linker script are identified by a new `mutable` marker, for example,
`mutable[flash_text]`. The placements for immutable libraries remain in
the existing `mapping` marker, for example, `mapping[flash_text]`. The
`mutable` marker for each target is placed after the `mapping` marker.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Entity mappings for immutable libraries are placed in the existing
`mapping[target]` marker, while those for mutable libraries are placed
in the new `mutable[target]` marker, which comes after the `mapping`
marker. Additionally, include padding after the input sections of
mutable libraries in the default data and text output sections,
providing a headroom for the mutable libraries to grow. Padding is
currently not added, for example, in the `.iram0.data` output section,
which is not expected to change frequently. Padding for other mutable
input sections may be added in the future.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Encrypted flash write operation sometimes result in random corruption in
certain bytes. Root cause points to sudden current surge due to involvement of
encryption block overwhelming LDO supply. More details will be provided
in the ESP32-C5 SoC Errata document.
This fix limits the CPU clock to 160MHz for flash encryption enabled
case. Failing encrypted flash write tests could successfully pass in
this configuration. Going ahead, a dynamic clock adjustment in flash
driver will be considered to mitigate this issue.
fix(wifi): fix issue that disarm send wake null timer donot take effect and sta bolck scan timeout
Closes WIFI-7020, WIFIBUG-1523, and WIFIBUG-1379
See merge request espressif/esp-idf!43158
fix(efuse): Disallow enabling SB using SHA-384 when SB using SHA-256 is enabled (ESP32-P4 ECO5)
Closes IDF-14063
See merge request espressif/esp-idf!42884
"RTC memory should be lowest priority and its free size should be big enough"
was failing because the expected size of RTC heap left after init was bigger
than the total RTC size itself.
The test was updated to:
- check that the minimum free RTC heap size matches the current size
- the current RTC heap size is bigger than 80% of the total RTC memory region
Add a new example showcasing how to obtain and interpret statistics
about the usage of an NVS partition.
Also demonstrates how stats change after writing some key-value pairs
in 2 separate namespaces.
fix: Update esptool command in GitHub template to use legacy command for backward compatibility
Closes IDFGH-16670
See merge request espressif/esp-idf!43179
The cmakev2 build system may evaluate components recursively and all
components has to make sure all their variables are properly
initialized.
Fixes: a7115b6b36 ("refactor(jpeg): Split hal layer for jpeg")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
ci(hal/tee): Fix intermittent hang-ups in `PERI_APM` tests on ESP32-C5
Closes IDFCI-3274, IDFCI-4807, and IDFCI-4808
See merge request espressif/esp-idf!42937
This is both a feature and an optimization.
Feature:
Adjustable size of the internal storage in esp_event queue, currently
used by ISR posting, as they wont be able to make a malloc.
Optimization:
When non-isr is posting an event, use the inernal storage in the struct
instead of always allocating a new heap for the data. Most events in
esp-idf only contains a few bytes event information, and we have that
allocation payed for anyway.
This solved in a big part our memory fragmentation issue, as events
happens freqvently and used to create small memory allocations for just
4 bytes, and then in the event handler we usually allocated a bigger
chunk of heap for our feature. When returning from the event handler,
the 4 byte allocation was freed, leaving a hole in the heap.
Merges: https://github.com/espressif/esp-idf/pull/17797
The `tools/test_build_system/buildv2_test_app` is a cmakev2 build system
testing application, same as `build_test_app`, which should be removed
from the readme check performed by check_build_test_rules.py.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
feat(ble): Support controller log compression for ESP32-C3 and ESP32-S3(0871069)
Closes BLERP-2287, BLERP-2288, BLERP-2286, and BLERP-2285
See merge request espressif/esp-idf!42293
The COMPONENTS and EXCLUDE_COMPONENTS variables are unused in cmakev2. A
deprecation warning will be printed if they are set. This change also
includes a helper functions for printing deprecation warnings.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 dropped generation of prefix_map_gdbinit, reflect this
change also in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently both menu configs, for component's project configuration and
component's configuration, for components not included in the build have
the same name: "Configuration for components not included in the build".
This might be confusing. Let's use "Project configuration for components
not included in the build" for component Kconfig.projbuild files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 added esp_stdio to common requires, add it in cmakev2 too
for backward compatibility.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
refactor(parlio_rx): use gdma_link for better gdma link management
Closes IDF-13369, IDF-13627, and IDFGH-16449
See merge request espressif/esp-idf!40743
feat(openthread): optimize fail cases in CI test
Closes IDFCI-2537, IDFCI-2642, IDFCI-3576, IDFCI-3623, IDFCI-3955, IDFCI-5136, IDFCI-5072, IDFCI-5065, IDFCI-4997, IDFCI-3026, IDFCI-3019, and IDFCI-3020
See merge request espressif/esp-idf!42971
Currently, the toolchain CMake files use the remove_duplicated_flags
function from utilities.cmake. The cmakev2 implementation also includes
this function for backward compatibility. Move the
remove_duplicated_flags function to a separate file,
deduplicate_flags.cmake, so it can be shared between cmakev1 and
cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Enable check-kconfig-files. It was temporarily disabled because it was
not possible to add a new sourced file through an environment variable,
as kconfcheck had a fixed list of allowed environment variable names.
This issue should now be resolved.
Currently, the buildv2 tests are initiated using the same patterns as
the tests for the current build system. This means that any change in
the current build system will also trigger the buildv2 tests. Initially,
it might be wise not to block the CI for changes in the current build
system in case there is an issue with buildv2. Therefore, let's
explicitly start the buildv2 tests only when the buildv2 label is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The esp_libc relies on the stdio implementation of the vfs component if
CONFIG_VFS_SUPPORT_IO is enabled. This is not an issue in cmakev1
because if the vfs component is not included in the project build, its
configuration is not available, and CONFIG_VFS_SUPPORT_IO is not set.
However, in cmakev2, the configuration for all components is available,
and the presence of some component configuration options does not
necessarily mean that the component is included in the project build.
When esp_libc is compiled without the vfs dependency but with
CONFIG_VFS_SUPPORT_IO enabled, the libc initialization will encounter a
panic due to a NULL pointer dereference (fp->_flags) in __swsetup_r, as
fopen in esp_libc_init_global_stdio will return NULL.
Although the current documentation exposes only limited functionality,
which is unlikely to change, add an explicit note that the build system
v2 is currently a technical preview intended for testing and early
evaluation.
Ensure consistency in referring to v2 throughout the documentation. In
some places, it is referred to as V2 or Version 2; unify this to v2.
Additionally, update the hello_world example for v2 to use the
idf::spi_flash alias instead of the interface variable to maintain
consistency with the rest of the documentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Extend the current documentation to recommend using the
idf_component_register function as the preferred method for creating a
new component for v2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Using idf_component_register is the preferred method for creating new
components for cmakev2. This approach ensures compatibility with both
versions of the build system. The KCONFIG and KCONFIG_PROJBUILD options
have been removed from the API documentation, but are retained in code
in case a cmakev1 component uses these options to warn about
incompatibility. Also remove a note about `project_include.cmake`,
because cmakev2 includes all project_include files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Enhance the `esp_target_info` component example by adding some of
the missing features that the component might require. This includes
adding a linker script, a linker fragment, linking the component archive
as a whole library, and providing an example of how the linker options
can be set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Components designed exclusively for cmakev2 are not compatible with
cmakev1 due to the lack of forward compatibility. It is important to
clarify this at the very beginning of the `Creating a New Component`
section. Additionally recommended a guidance on how to create a
component that will be compatible with both versions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the archive file name is set only in the cmakev1
idf_component_register shim. The predictable component archive file name
is important, for example, for usage in linker fragments. Ensure that
the cmakev2 component also has the archive file name set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev2 component sources are set, the SRCS target
property is used. This is obviously wrong because the correct CMake
property is SOURCES.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Rename the LINKER_SCRIPTS_STATIC component property to LINKER_SCRIPTS.
This property stores linker scripts that are not processed by ldgen,
which essentially includes all of them. The only linker script processed
by ldgen is sections.ld, which is handled by the esp_system component.
This implies that there is likely no practical use case for other
components to utilize ldgen processed linker scripts. This change is
purely cosmetic to allow components to add linker scripts with:
idf_component_set_property(${COMPONENT_TARGET} LINKER_SCRIPTS linker_script.ld APPEND)
instead of
idf_component_set_property(${COMPONENT_TARGET} LINKER_SCRIPTS_STATIC linker_script.ld APPEND)
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add very basic documentation. It is currently written with the
expectation that the reader is already familiar with the existing build
system and focuses more on the changes that might be necessary to adapt
existing components to work with cmakev2. Additionally, it provides
basic information on how to create a new component and project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, we include numerous functions in the automatically generated
documentation for the build system API. Let's begin with only the
essential functions and gradually add more to the API based on requests
and actual needs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit adds a workaround in idf_build_component shim to avoid
initialization of a duplicate managed component. The component manager
when running on a Windows system returns duplicate managed components
upon successive invocations.
The current approach involves wrapping the library target within the
library interface target with the whole-archive flags, such as for the
GCC linker. However, this does not work as expected because the library
target is expanded, and the whole-archive flags are also applied to
other library targets that the wrapped library target depends on. IOW
the whole-archive flags surround multiple archives, not just the one
requrested. Generally, using linker flags like whole-archive in the
library INTERFACE(INTERFACE_LINK_LIBRARIES) does not seem to work as
CMake may perform deduplication and rearrange the flags [1].
The proof of concept used `CMAKE_LINK_LIBRARY_USING_<FEATURE>` with the
WHOLE_ARCHIVE feature, which was introduced in CMake 3.24 to specify how
the library target should be linked. There are two issues with this.
First, this feature is only available from CMake 3.24 onwards, while our
minimum CMake version is set to 3.22. More importantly, all occurrences
of a library on the link line are wrapped. For example, if a library
like `vfs` appears multiple times on the link line due to dependencies,
each occurrence will be wrapped with whole-archive, causing the linker
to complain about multiple symbol definitions. Therefore, even though
WHOLE_ARCHIVE is recommended for handling whole-archive linkage, it does
not seem suitable for our purposes. This was overlooked in the PoC
because only a simple testing component was used to check the
WHOLE_ARCHIVE behavior.
One way to address this issue is to adopt the same approach used in
cmakev1, which involves specifying the whole-archive flags when linking
component library targets to the final executable. In this case, CMake
retains the flags without any alteration, unlike when the flags are
specified in INTERFACE_LINK_LIBRARIES for the component interface
targets. While this approach is feasible, it would alter the current
logic, where we have a single library interface for the entire idf
library that can be linked to the executable. This change would also
complicate the direct use of the idf library, as it would no longer be
possible to simply link it to the executable. Instead, the executable
would need to correctly link the component libraries with the
whole-archive flags. We could encapsulate this process within the
idf_build_executable function, but projects that only use
idf_build_library would need to implement the same solution.
It appears possible to address this issue by using target_link_options
instead. This allows to specify exactly what should appear on the link
command line. One side effect of this approach is that the library
appears on the link command line multiple times: first when used with
target_link_options and second when used in target_link_libraries.
However, this does not seem to pose a problem. The flags specified with
target_link_options appear on the command line first, ensuring that the
whole-archive is prioritized, and the component archive libraries may be
repeated on the link line anyway due to dependencies. Essentially, the
final link command line is very similar to the one from cmakev1. We may
revisit this approach in the future, but for now, it seems to work as
expected.
[1] https://gitlab.kitware.com/cmake/cmake/-/issues/20078
[2] https://cmake.org/cmake/help/latest/variable/CMAKE_LINK_LIBRARY_USING_FEATURE.html
[3] https://discourse.cmake.org/t/automatically-wrapping-a-static-library-
in-whole-archive-no-whole-archive-when-used-during-linking/5883
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev1 properties for INCLUDE_DIRS and
PRIV_INCLUDE_DIRS are constructed in the
__set_component_cmakev1_properties function, the return values from the
get_target_property function for INCLUDE_DIRECTORIES and
INTERFACE_INCLUDE_DIRECTORIES properties are not checked. If a component
target does not set e.g. INCLUDE_DIRECTORIES property,
get_target_property will return a value such as `include_dirs-NOTFOUND`.
This value is subsequently passed to __get_relative_paths in the PATHS
argument, causing the file(RELATIVE_PATH) call in __get_relative_paths
to fail with an error.
```
CMake Error at /home/fhrbata/work/esp-idf/tools/cmakev2/utilities.cmake:235 (file):
file RELATIVE_PATH must be passed a full path to the file:
include_dirs-NOTFOUND
```
Fix this by explicitly set the PATHS to an empty list before passing it
to the __get_relative_paths function if the property is not set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
idf_build_executable() creates a dummy source file to create the
executable target. Since this file is created unconditionally everytime
CMake configuration happens, a source file change is registered which
causes CMake to execute other targets even though no other source file
changed.
The commit adb2d5deee ("feat(cmake): Produce warnings when component dependen..")
introduced additional checks for source files and include directories
used by a component that are located outside the component's directory.
If these files and directories belong to another component, a warning is
issued. This feature has not yet been implemented in cmakev2, so related
tests are temporary disabled.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit renames the buildv2_test_app.c file to build_test_app.c in
the buildv2_test_app template to enable tests to refer to the same
source file name when writing tests for both cmakev1 and cmakev2.
This commit enables the following tests for cmakev2 in CI which do not
need any updates:
- test_bootloader.py
- test_git.py
- test_kconfig.py
- test_partition.py
- test_reproducible_build.py
- test_sdkconfig.py
In cmakev1, certain arguments provided to the idf_component_register
function are stored as component properties. These properties are used
to generate the project_description.json file, which other tools rely
on. Since the idf_component_register function is obsolete in cmakev2, we
need to recreate component properties such as INCLUDE_DIRS,
PRIV_INCLUDE_DIRS, REQUIRES, and PRIV_REQUIRES, which were previously
provided by the idf_component_register function in cmakev1. To achieve
this, let's examine the component's real target cmake properties and
reconstruct them to maintain compatibility with cmakev1.
The real target cmake properties may include generator expressions,
which are ignored. This should be acceptable, as they were not addressed
in cmakev1 either, and handling them is likely not feasible.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Return the difference between two lists, meaning the elements that are
present in the first list but not in the second.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This helper function removes entries containing generator expressions
from the given list. The list is modified in place.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a helper function that returns a list of paths relative to a
given base directory for a list of input paths.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When a linker script file with a .in extension is added using the
target_linker_script function, it is processed with the C preprocessor.
The linker scripts are preprocessed only once, even if they are used in
multiple libraries, because they are the same.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, add the COMPONENT_BUILD_DIR property. This can be used to
store component-specific generated files, such as preprocessed linker
scripts.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `includes` list variable is used without initialization. Initialize
it explicitly.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component has been renamed, update the name in the common
requirements for the cmakev1 components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component can be referenced by multiple identifiers such as the
component name, which is derived from the component directory name,
component target, interface, or aliases. All component properties are
attached to the component interface target, which is also used for
declaring component dependencies. The cmakev2 build system has a
function called __get_component_interface, which is responsible for
identifying the component interface based on the given component
identifier. Since this function is called frequently, it needs to be
reasonably fast.
This introduces a new __idf_component_interface_cache INTERFACE target,
which serves as a mapping cache between component identifiers and the
component interface. The cache for each component is initialized in the
__init_component function, which introduces a component to the build
system. Currently, the component interface search is conducted by
examining the COMPONENTS_DISCOVERED and COMPONENT_INTERFACES lists
stored as build properties. Since the build system is aware of most
component identifiers, such as component name, target, and alias, during
component initialization, it can add mappings between component
identifiers and the component interface to a cache, which is built as
the components are initialized. This cache is used in the
__get_component_interface function instead of looking into the
COMPONENTS_DISCOVERED or COMPONENT_INTERFACES lists. This significantly
speeds up the component interface search and also makes the code much
simpler and more readable.
The component interface cache also completely replaces the existing
component name resolution, which was introduced because of the component
manager, and the cache used for resolved component names. This is
possible because all the necessary information is available during
component initialization when the component interface cache is
populated. The ambiguity of components is resolved based on component
source/priority.
Here is an example of the component interface mapping for the
espressif__led_strip component to the idf_espressif__led_strip interface
target. The component name, without the namespace, is referred to as the
short name. In this example, it is led_strip.
- espressif__led_strip -> idf_espressif__led_strip # name(directory name)
- idf_espressif__led_strip -> idf_espressif__led_strip # interface
- idf::espressif__led_strip -> idf_espressif__led_strip # alias
- _idf_espressif__led_strip -> idf_espressif__led_strip # real target
- led_strip -> idf_espressif__led_strip # short name
When another component with the same espressif__led_strip name is
initialized with a different priority, there is no need to change
anything in the cache, because the mapping stays the same.
The cache must be updated when two components share the same short name
but belong to different namespaces. This situation is likely uncommon.
For instance, consider espressif__led_strip and my__led_strip. If
my__led_strip has a higher priority, the cache is updated to reflect the
short name as follows:
- led_strip -> idf_my__led_strip
If both components have the same priority, the short name mapping for
led_strip is entirely removed. Conversely, if my__led_strip has a lower
priority, no short name mapping is added for it.
The short name is also added to the COMPONENT_SHORT_NAME property of the
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a revised version of the existing __create_confserver_target
function. It creates a specified confserver target for a given
executable. The kconfig_menus.json file, used by the IDEs, is generated
when the confserver starts. This differs from the previous behavior,
where kconfig_menus.json was created globally along with other sdkconfig
formats. The reason for this change is that kconfig_menus.json contains
the Kconfig menu hierarchy and it is not just a flat option-value
format. It needs to accurately reflect which configurations for which
components are included or excluded. The kconfig_menus.json is generated
at `build/kconfig_menus.json`, where IDEs expect it. This means the file
is overwritten every time the server starts by kconfig_menus.json
version for given executable, so only one confserver can run at a time.
This is likely acceptable, as I don't believe it's possible to safely
run multiple instances of confserver due to the potential race
conditions when the sdkconfig file are generated.
In the future, we may include the location of kconfig_menus.json in
project_description.json so it can be easily identified by IDEs for
each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the initial sdkconfig is generated, the
kconfig_menus.json format is also created alongside other formats like
CMake, JSON, or C header. The kconfig_menus.json depends on the Kconfig
hierarchy and cannot be generated globally. It must be generated for
each executable to ensure that included and excluded components are
correctly positioned within the Kconfig menu hierarchy. Remove the
automatic generation of kconfig_menus.json.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current base kconfgen command is stored in the __BASE_KCONFGEN_CMD
build property, and it includes the --env-file option pointing to the
`build/config.env` file. The `build/config.env` file does not separate
component configuration files into included and excluded components,
because it is created before any executable is added and we need it for
the initial sdkconfig generation. The global `build/config.env` can be
used for every sdkconfig format (cmake, json, txt), but not for the
kconfig_menus.json format, because kconfig_menus.json relies on the
Kconfig layout and hierarchy. Let's allow specifying the --env-file for
each kconfgen invocation and remove it from the base kconfgen command.
This allows you to specify an --env-file for each executable, each with
its own config.env file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __run_kconfgen function does not take any arguments. The output
formats and related files are stored in the __KCONFGEN_OUTPUTS_CMD build
property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is basically a renamed version of __create_save_defconfig_target.
The creation of the save-defconfig target is done in
`idf_project_default()`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For a specified executable, create a menuconfig target using the name
provided in the TARGET option. The function generates a config.env file
specific to the executable, where Kconfig files for components linked to
the executable and Kconfig files for components not linked to it are
separated. This separation allows for a clear visual distinction between
the configuration of components that are linked and those that are not
linked to the executable within the menuconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Generate the config.env file for the specified executable. The
configuration file will be stored in the build directory, within a
directory named after the executable target name. The kconfigs*.in
files, which are generated by prepare_kconfig_files.py, will also be
stored in this directory.
This function primarily prepares the arguments for the
__create_config_env_file function based on the components linked to the
executable, ensuring that component Kconfig files are stored in the
appropriate kconfigs*.in files, depending on whether the component is
linked to the executable.
The directory where the generated files are stored is added to the
executable CONFIG_ENV_DIR property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __create_config_env_file function generates a configuration file for
the prepare_kconfig_files.py script. Currently, it specifies the paths
for the kconfigs*.in files, which are generated by
prepare_kconfig_files.py, to be located in the build directory. The
kconfigs*.in files, along with the configuration file, must also be
generated for each executable, as each executable may have a
different set of linked components. Set the paths for the
kconfigs*.in files to be generated by the prepare_kconfig_files.py
script in the same directory as the configuration file.
Also allow explicit specification of which component configuration files
should be placed in different kconfigs*.in files. For sdkconfig
generation, it is not relevant whether a component's Kconfig file is
placed in the kconfigs.in or kconfigs_excluded.in file. This distinction
is only important for visually separating the configuration of included
and excluded components in the menuconfig. For the main sdkconfig, all
component Kconfig files are placed in the kconfigs.in file, and the
generated kconfigs*.in files are stored in the build directory. For
executables, the generated kconfigs*.in files will be stored in separate
directories for each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The excluded Kconfig files cannot be collected globally, they must be
collected per executable. The distinction between included and excluded
Kconfig files is relevant only for menuconfig, which is specific to each
executable, as each executable may have a different set of linked
components. For the global sdkconfig file and its CMake and JSON
variants, whether the Kconfig files are excluded or not is irrelevant.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Commit 46b41dfec187 ("test(cmakev2): Enable CI tests for component manager ..")
introduced a temporary workaround for the cmakev2 testing
application by including the cmakev1 project.cmake file after the
return() statement. The issue is that some tests, such as
test_build_fail_on_build_time, append commands to the project's
CMakeLists.txt. These commands are placed after the newly added return()
statement, causing them to be ignored and resulting in test failures. To
resolve this, include the cmakev1 project.cmake within the cmake block
comment.
Fixes: 46b41dfec187 ("test(cmakev2): Enable CI tests for component manager ..")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Three tests are disabled for cmakev2:
* test_build_dfu
* test_build_uf2
* test_build_loadable_elf
and one is modified:
* test_build_skdconfig_phy_init_data
the other 14 tests should pass.
The tests, test_build_dfu and test_build_uf2, are temporarily disabled
because cmakev2 currently does not include the dfu and uf2 targets.
These tests should be re-enabled once support is added to cmakev2.
The test_build_loadable_elf is disabled because the bootloader_support
component determines its requirements based on the sdkconfig values,
specifically the CONFIG_APP_BUILD_TYPE_RAM used in this test. When
CONFIG_APP_BUILD_TYPE_RAM is set, bootloader_support declares a
dependency on micro-ecc. This is wrong even in cmakev1 and should be
fixed.
The test_build_skdconfig_phy_init_data test requires the esp_phy
component but does not specify it in its requirements. It relies on the
cmakev1 behavior, where all components are included in the build by
default. This approach does not work for cmakev2. To resolve this, make
the dependency on esp_phy explicit, ensuring compatibility with both
versions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the cmakev2 implementation of Kconfig name consistency has a
slightly different warning message compared to cmakev2, causing the test
`test_build.py::test_build_with_misspelled_kconfig` to fail. Make the
warning messages consistent with cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The test_build_with_misspelled_kconfig test expects a KConfig.projbuild
file in the main component. However, the file is correctly named
Kconfig.projbuild, which causes the test to fail for cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, cmakev2 evaluates each component using add_subdirectory with
the EXCLUDE_FROM_ALL option. The intention was to exclude all components
from being built by default unless they are explicitly linked to an
executable based on dependencies. This approach aims to avoid building
components that are included but not actually linked to the executable.
However, this has the side effect of preventing components from adding
their custom targets to the "all" target, which is used, for example, by
esp_phy. Generally, we should not restrict components from adding
targets to "all". Since components are only included if explicitly
requested by default, removing the EXCLUDE_FROM_ALL option should be
acceptable.
The downside is that if IDF_INCLUDE_ALL_COMPONENTS is set and the user
runs `idf.py build`, all components will be built, even if they are not
linked to the executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the check is based on the existence of sdkconfig, but there
may be situations where sdkconfig exists without containing
CONFIG_IDF_TARGET. Perform a target consistency check only if the target
is identified in the main sdkconfig file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The function arguments in the documentation comments were using field
list, which caused text overflow in the generated documentation and
generally resulted in poor formatting. Let's use paragraphs for
the argument descriptions instead.
The documentation comments are written in reStructuredText, but
currently, they use inconsistent indentation. Standardize all the
documentation comments to use a four-character indentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add stump files for the build system v2 documentation with automatic API
generation from CMake files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Load the esp_docs_cmakev2_extension to process documentation comments
from CMake files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a Sphinx extension that introduces a new `cmakev2` domain with
multiple directives, allowing for the automatic extraction of
documentation comments from CMake files and their inclusion in the
Sphinx-generated documentation.
Directives:
- `cmakev2:include`: The included CMake file is processed for
documentation comments within the `#[[api` and `#]]` marks, which
should contain valid reStructuredText markup.
- `cmakev2:function`: Creates a CMake function node. All function nodes
are sorted by name and placed into the `_cmakev2_functions` section.
- `cmakev2:macro`: Creates a CMake macro node. All macro nodes are
sorted by name and placed into the `_cmakev2_macros` section.
- `cmakev2:variable`: Describes a CMake variable node. All variable
nodes are sorted by name and placed into the `_cmakev2_variables`
section.
Each node can be referenced with `` :cmakev2:ref:`<node name>` ``, where
the node name is the function, macro, or variable name as used in the
related directive.
Example:
CMake file:
```
#[[api
.. cmakev2:function:: idf_flash_binary
#]]
```
This function can be referenced with `` :cmakev2:ref:`idf_flash_binary` ``
and will be placed in the `.. _cmakev2_functions:` section.
The extension is currently located in esp-idf, but in the future, we
should consider moving it to esp-docs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The build system should include the file generated by the component
manager after downloading components even if the component manager
requested a re-run due to a missing Kconfig.
This commit updates the order of initialization for build system v2. All
non-project initializations are handled in idf.cmake and all
project-specific initializations are now moved to idf_project_init().
Generate the flasher_args.json file for the global flash target for
tools that require it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Generate binary images and add flash targets for hello_world_example and
fatfs_example.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
All other functions, such as idf_build_binary, use OUTPUT_FILE. This is
more descriptive and provides more flexibility to add additional
file-related options in the future if necessary. Make
idf_build_generate_metadata consistent with the other function
interfaces by using OUTPUT_FILE.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Use idf_build_binary, idf_sign_binary, and idf_flash_binary to generate
and flash the default project application.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Given the binary target generated by the idf_build_binary or
idf_sign_binary function, add a new POST_BUILD dependency that displays
a message indicating the binary image must be manually signed before
flashing. This situation arises when CONFIG_SECURE_SIGNED_APPS is
enabled, but CONFIG_SECURE_BOOT_BUILD_SIGNED_BINARIES is not set. As a
result, the binary image is not signed during the build process and
requires manual signing.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Given the binary target created by the idf_build_binary or
idf_sign_binary function, add a new dependency that verifies whether the
binary image fits within the smallest application partition.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a new flash target for a previously created binary target using
the idf_build_binary or idf_sign_binary function. Specify the new target
name with the TARGET option. The binary image path is stored in the
BINARY_PATH property of the binary target and is set when the binary
target is created using the idf_build_binary or idf_sign_binary
function. Use the NAME option as a prefix for the generated esptool
argument file, and the FLASH option can be used to include the binary
image flashing in the global flash target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a signed binary image from a previously generated binary using
the `idf_build_binary` function. This image can be signed with an
optional key from a KEYFILE or, by default, with the key specified in
the sdkconfig. A new custom command is added to generate the signed
binary, which is stored at the location specified by the OUTPUT_FILE
option and a new custom target is created for it, with the name provided
in the TARGET option. The BINARY_PATH property is added to this new
TARGET, containing the path to the signed binary image.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `idf_build_binary` function generates a binary image for a specified
executable target, which may or may not have been created using the
`idf_build_executable` function. It adds a custom command to generate
the binary image file at the location specified by the `OUTPUT_FILE`
option and creates a new target for it, with the name specified by the
`TARGET` option. The path of the generated binary file is also stored in
the `BINARY_PATH` property of the newly created binary target.
Additionally, this binary path is stored in the `EXECUTABLE_BINARY`
property of the given executable. This information is used by the
`idf_build_generate_metadata` function, which takes the executable
target as an argument and includes the binary name in the
`project_description.json`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The global flash targets are created using
__create_project_flash_targets, but these are merely placeholders that
allow components to declare their dependencies on the global flash or
encrypted-flash target in their project_include.cmake. The actual logic
for flash targets is located in the project_include.cmake of the
esptool_py component. Therefore, the targets must be initialized
after this file has been included. To address this, a new function,
__init_project_flash_targets, has been added for this purpose.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create a phony target that intentionally fails the build, displaying the
error messages specified in the message arguments. An empty CMake file is
created and included, which is then deleted when the target is executed.
This forces CMake to rerun, as the file is added to the CMake rerun
dependencies.
This has been carried over from cmakev1 as it is used by several
components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the lwip is linked to mbedtls if CONFIG_LWIP_ENABLE is set.
This approach works in cmakev1 because only the configuration for
BUILD_COMPONENTS is available during component evaluation. However, this
is not the case in cmakev2, where the configuration for all components
is available. This means that even if CONFIG_LWIP_ENABLE is set, it does
not guarantee that the lwip component is included in the build. For
cmakev2, modify the check to use a generator expression to ensure that
lwip is linked to mbedtls only if lwip is actually included in the
build.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Some components rely on the existence of flash targets to declare
dependencies. Add flash and encrypted-flash targets during project
initialization to ensure they are available before including
project_include.cmake files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Since the idf_project_default is already available, let's use it in the
buildv2 testing application. Also change the test app name from
buildv2_test_app to build_test_app, as some tests require the ELF and
bin images to be created under this name.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The optional dependency is currently added if the optional component
interface target exists, which is always the case unless a non-existent
component is requested. Instead, base the optional dependency on the
component interface target alias, as it is created only when the
component is included in the project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
If the IDF_INCLUDE_ALL_COMPONENTS variable is set, all discovered
components will be included. By default, components are included based
on their requirements. When IDF_INCLUDE_ALL_COMPONENTS is enabled, the
all_component_info dictionary in the project_description.json file will
be fully populated.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `idf_build_generate_metadata()` function is used to generate project
metadata, specifically the `project_description.json` file, for a given
executable target. The output file path can be specified as an argument
to the function, allowing metadata generation for multiple executables.
It identifies the library, created by the `idf_build_library` function,
that is linked to the executable based on the `LINK_LIBRARIES`
executable target property. The `LIBRARY_INTERFACES` build property
tracks all library interfaces created with the `idf_build_library`
function, so the targets in the `LINK_LIBRARIES` executable property are
checked against this list to find the library. The library contains
information about components linked to it in the
`LIBRARY_COMPONENTS_LINKED` library property. These components are
processed and added to the `build_component_info` dictionary in the
`project_description.json` along with other information.
Note that, compared to cmakev1, the `all_component_info` dictionary in
`project_description` does not include some information by default, such
as included directories. This information is available in cmakev1 due to
early component evaluation, which is not done in cmakev2. This will
likely require some adjustments on the hints side.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 sets __PREFIX as a component property. To maintain backward
compatibility, set it in cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Search for the library interface target created with the
idf_build_library() function and linked to the executable, examine the
LINK_LIBRARIES for the executable and the LIBRARY_INTERFACES build
property, which stores all library interface targets created by the
idf_build_library() function.
This helper is intended for use by functions, such as the
idf_build_generate_metadata() function, which accepts the executable
target as an argument to find the library linked to it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_project_default macro is intended to be used by default for
ESP-IDF projects. Currently, it handles the necessary project
initialization that must occur after CMake's project() is called. It
creates the default executable based on the project name and links a
library to it, based on the main component and its transitive
dependencies. This macro will be further extended to also generate
binary image, project metadata, and add other expected targets. It
primarily serves as a replacement for the project() macro used in
cmakev1.
Expected typical usage:
```
cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmakev2/idf.cmake)
project(hello_world LANGUAGES C CXX ASM)
idf_project_default()
```
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Create an empty source file for the add_executable target, along with a
library containing specified components, and then create an executable.
This approach allows for the easy creation of multiple executables by
specifying the component with the app_main function, as demonstrated in
the test.
```
idf_build_executable(fatfs_example COMPONENTS fatfs_example)
idf_build_executable(hello_world_example COMPONENTS hello_world_example)
```
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The option is unnecessary and complicates the interface. Additionally,
it does not adhere to the previously established naming convention. The
target name for the library interface is provided as a standard function
argument.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmakev2, the component CMakeLists.txt files are evaluated recursively
based on the component dependencies. This means that variables from
previously evaluated components are set and available in the scope of
the currently evaluated component. It is necessary to initialize every
variable that is used.
Fixes: 56e0c11bb6 ("feat(usb_cdc_console): moved usb-cdc ROM console to n..")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add two components: fatfs_example and hello_world_example. These are
simply copies of the main components from the existing examples.
Enhance the test_executable test to create targets for two
executables, which can be built after CMake configuration.
```
cmake --build build/ --target hello_world_example
cmake --build build/ --target fatfs_example
```
Additionally, move the test_component_priority to the end of the test
chain, as it modifies the esp_system component, which interferes with
the linkage of the executables.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Integrate the ldgen into cmakev2. With this change, it becomes possible
to actually link the project executables.
In cmakev2, the handling of linker scripts is deferred to
idf_build_library, unlike in cmakev1, where linker scripts were added
and generated during the target_linker_script call. In cmakev2, the
target_linker_script only adds the linker scripts and templates, along
with the output filenames for the linker scripts generated from the
templates, to the component property. When idf_build_library is called
and all the requested components are included, it uses the
__get_target_dependencies function to obtain all transitively linked
targets to the library interface target. These targets are mapped to the
components, and the LIBRARY_COMPONENTS_LINKED library property is set.
It contains all components linked to the library interface target. The
components from LIBRARY_COMPONENTS_LINKED are used to collect linker
fragments and linker scripts utilized in the library. Additionally, all
targets transitively linked to the library are used to identify archive
files used in the library. This includes component archives and archives
added with the add_prebuilt_library function. The archives and
ldfragments related to the components linked to the library are used
when ldgen generates the linker scripts from templates.
The linker scripts, both static and generated by ldgen, are added to the
library interface link options and INTERFACE_LINK_DEPENDS property. For
generated linker scripts, a custom target is created and added as a
dependency for the library interface to ensure they are generated before
the link.
The difference compared to cmakev1 is that the generated linker scripts,
currently only sections.ld, are not global in the project but are
generated per library. This means there might be multiple versions of
sections.ld depending on the components included in the library. For
example, a component like esp_system may be linked to multiple library
interface targets, each with a different set of components. This results
in different sets of fragment files and library archives and different
versions of the sections.ld linker script. This should ensure proper
dependencies between targets. In other words, if a component changes its
linker fragment, only executables linked to libraries using this
component should be re-linked. As a consequence of this approach, the
generated linker scripts for different libraries need to have different
names or be stored in different directories to avoid overwriting the
linker script for one library with the linker script for another library
using the same component. This is handled with a suffix, which is based
on the library interface target name and appended to the generated
linker script. So, for example, there is no sections.ld, but instead
sections.ld_fatfs_lib or sections.ld_hello_world_lib. As a next step, we
can add a DEFAULT option to idf_build_library and avoid adding the
suffix for the default library.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
There may be multiple libraries, each linking a different set of
components. Introduce a new library property, LIBRARY_COMPONENTS_LINKED,
to track the components linked with a specific library. This property
can be used, for example, by menuconfig to distinguish between included
and excluded components on a per-library or per-executable basis, or by
ldgen to identify linker scripts for the linked components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the INTERFACE_LINK_LIBRARIES and LINK_LIBRARIES properties,
recursively retrieve the list of dependencies for the specified target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the common components are initialized in the idf_project_init
macro, which means they are included even for cmakev2 components.
However, cmakev2 components are expected to explicitly specify all their
dependencies instead of relying on common components being automatically
linked. Therefore, common components are only relevant within the
context of cmakev1 components. To address this, move the
__init_common_components function call to idf_component_register and
include common components only when they are truly needed for a cmakev1
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a simple test_executable that links to all discovered components.
This is intended to verify if the component sources can be compiled
using the command `cmake --build build/ --target test_executable`. The
ldgen is not yet integrated, so the test_executable is not actually
created or linked. However, this process can be used to check if the
components are compiled.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Set the ESP_PLATFORM macro to indicate to component sources that they
are being compiled with ESP-IDF.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the generic compile_options are ignored because they are
unset after retrieving the build property. To fix this, avoid explicitly
setting compile_options, as this variable is initialized with the
idf_build_get_property(compile_options ...) call.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Automatically link commonly required components to the component target
created in the idf_component_register function. This is still necessary
even in cmakev2, as existing components depend on this behavior.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Identify the commonly required components based on the target and
architecture, and store them in the __COMPONENT_REQUIRES_COMMON build
property. These components are automatically included, and their
interfaces are stored in the __COMMON_COMPONENT_INTERFACES build
property. The commonly required component interfaces are automatically
linked to each component.
Existing components for cmakev1 rely on the automatic linking of common
components. Therefore, at least initially, we need to implement the same
functionality in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmakev2, the project_include.cmake files for every component are
included. This means that even when working with RISC-V, the
project_include.cmake file for Xtensa is still included. Ensure that the
architecture is verified, and exit if it is not Xtensa.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the configuration, set the IDF_TARGET build property. This is
also set by cmakev1 and can be used to easily determine the architecture
for the project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add embedded source files to the component target sources. This is
intentionally managed in idf_component_include instead of
idf_component_register, allowing even cmakev2 components to set the
component's EMBED_FILES and EMBED_TXTFILES properties. Although it might
be more convenient to call the target_add_binary_data helper function
directly in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a backward-compatible idf_component_register function. This is
necessary to enable existing components to integrate with cmakev2. Note
that, by design, when the configuration is evaluated first, the KCONFIG
and KCONFIG_PROJBUILD options become obsolete, and an error is reported
if non-default configuration file names are specified.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
IDF_BUILD_V2 does not have BUILD_COMPONENTS build property. Therefore,
when IDF_BUILD_V2 is defined, use a generator expressions instead.
The inclusion of esp_eth_netif_glue.c, which relies on the presence of
the esp_netif component, should ideally be managed by a configuration
option. The same applies to the requirements for esp_eth. Currently,
only a basic adjustment has been made to enable the inclusion of esp_eth
in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
IDF_BUILD_V2 does not have BUILD_COMPONENTS build property. Therefore,
when IDF_BUILD_V2 is defined, use idf_component_optional_requires and
generator expressions instead. I believe that the one-liner should also
work with IDF_BUILD_V1, but the change is kept separate to clearly show
the difference between IDF_BUILD_V1 and IDF_BUILD_V2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
IDF_BUILD_V2 does not have BUILD_COMPONENTS build property. Therefore,
when IDF_BUILD_V2 is defined, use a generator expressions instead. I
believe that the one-liner should also work with IDF_BUILD_V1, but the
change is kept separate to clearly show the difference between
IDF_BUILD_V1 and IDF_BUILD_V2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a component interface alias only after the actual target is created,
meaning the component is included. The alias has a well-defined name and
can be used in generator expressions like
`$<$<TARGET_EXISTS:idf::esp_netif>:>` without needing to retrieve the
COMPONENT_ALIAS property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add binary data into the build target by converting it into a generated
source file, which is then compiled into a binary object as part of the
build process.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The add_deprecated_target_alias function is used by some existing
components, so it should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The file_generate function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The fail_target function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The add_prefix function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The spaces2list function is used by some existing components, so it
should be added to cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the specified component as a dependency only if it is included in
the build. This functions the same way as in cmakev1, but it uses
generator expressions because cmake2 does not maintain the
BUILD_COMPONENTS build property, which would list all components
included in the build.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, this is just a placeholder without implementation. The
implementation will be completed in a subsequent patch set that
integrates ldgen. For now, this is enough for ESP-IDF components can be
included by cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __DEPENDENCY_CHAIN was not properly created because the component
names were not added to it. Fix this by maintaining it correctly.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This removes the temporary restriction that only allowed components with
project_components as their source to be evaluated.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This helper function collects component sources within the component's
directory. It will be used by the idf_component_register.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add an optional BASE_DIR option to specify a directory that will serve
as the base directory for evaluating input paths. This can be useful,
for example, when collecting component sources.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The toolchain files are using the remove_duplicated_flags function from
utilities.cmake. However, we want to avoid mixing utilities from cmakev1
and cmakev2. Use `IDF_BUILD_VER_TAG` to include utilities from the
currently used build system version.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the IDF_BUILD_VER_TAG variables to include the build system version
tag. These variables can be used in files, such as CMake toolchain
files, which are shared between cmakev1 and cmakev2. This allows for the
inclusion of files from the currently running build system version. For
example:
include($ENV{IDF_PATH}/tools/cmake${IDF_BUILD_VER_TAG}/utilities.cmake).
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
It is not possible to add a new sourced file through an environment
variable because kconfcheck has a fixed list of allowed environment
variable names. This issue is tracked under IDF-13847.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The compat.cmake file is intended to include functions necessary for
backward compatibility with cmakev1.
Add the check_expected_tool_version function, which is used by some
project_include.cmake files to verify if the tool's version matches the
installed tool. This functionality is used, for example, by esp_common.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Some idf.py commands, such as menuconfig, use build_target and call
ensure_build_directory, which requires the project_description.json file
to be generated. To enable the use of idf.py commands with the testing
example, call __generate_project_info() to generate the
project_description.json file. This allows you to execute idf.py
menuconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current config.env.in template cannot be easily extended and shared
between cmakev1 and cmakev2. Although the root Kconfig is shared and
sources COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE and
COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE, these environment
variables cannot be set to empty strings. This issue arises unless we
also adjust cmakev1 and esp-docs, which use prepare_kconfig_files.py, to
properly set COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE and
COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE.
If an environment variable is set to an empty string and sourced in
Kconfig, kconfgen will expand the variable with expandvars(), resulting
in an empty value. This expanded variable is then appended to the root
Kconfig or the currently processed Kconfig directory path, depending on
whether osource or orsource is used, and passed to iglob. The iglog will
return the directory as a result, and when Kconfig attempts to open the
file, it will encounter an error because it is a directory.
To resolve this issue and enable sharing of the root Kconfig file, a new
config_buildv2.env.in template has been added, which includes
environment variables for the excluded components.
For cmakev1, kconfgen will encounter the osource command for
COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE. Since this variable is not
defined in config.env.in, expandvars() will return the environment
variable name instead of expanding it, and iglob will not match
anything. Because osource is used for excluded components, the
non-existent file for osource will be skipped.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Pass the environmental variables COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE
and COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE to menuconfig.
This will display the configuration for excluded components under new
menus in menuconfig. Additionally, set the IDF_BUILD_V2 environment
variable to make these new menus visible. The IDF_BUILD_V2 variable is
also passed to all kconfgen commands.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
To generate a complete configuration for all discovered components, we
need to ensure that even kconfig files for the excluded components are
seen by Kconfig. Modify the prepare_kconfig_files.py script to generate
files that list the Kconfig files for excluded components. These files
are sourced in the root Kconfig using the
COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE and
COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE environment variables.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Introduce new configuration menus for components that are not part of
the build. These menus will only be visible if the IDF_BUILD_V2
environment variable is defined and it will be set only in cmakev2. This
enables sharing of the root Kconfig file between cmakev1 and cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For backward compatibility, since externalproject_add is used by
project_include.cmake in the bootloader component. The ExternalProject
should probably be included there instead.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, store the full path of the project_include.cmake file in the
component's __PROJECT_INCLUDE property, if it exists. Include the
project_include.cmake files for all discovered components at the global
scope within the idf_project_init macro.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Set the global variables IDF_BUILD_V2 and IDF_BUILD_VER, along with the
build properties and environmental variables. These are intended to be
used by components to determine which build system version they are
evaluated under and to make adjustments based on this information.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current approach involves transparently calling __init_build within
idf_build_library and performing all post-project() initialization
there. The issue with this method is that project_include.cmake files
also need to be included during post-project() initialization, but they
should be included in the global scope. This cannot be achieved within
the __init_build function unless it is converted into a macro. Although
using a macro is a potential solution, it risks causing global scope
pollution. Another complication is the location where project() can be
invoked; it must be explicitly[1] stated in the project's
CMakeLists.txt. This requirement conflicts with our intention to wrap
it within the idf_project_default helper. Given these challenges, it
makes sense to introduce an explicit idf_project_init macro, where all
post-project() initialization occurs, including the inclusion of
project_include.cmake files. While we can still encapsulate this within
idf_project_default, for the plumbing commands (idf_build_library,
idf_build_executable, ...), it will need to be explicitly called after
the project() invocation.
Usage for default project:
cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmakev2/idf.cmake)
project()
idf_project_default()
Usage for plumbing commands:
cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmakev2/idf.cmake)
project()
idf_project_init()
idf_build_library()
...
[1] https://cmake.org/cmake/help/latest/command/project.html#usage
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
A new pytest_buildv2_system job has been added to execute selected tests
from the existing test_build_system test suite. Currently, only the
test_non_default_target.py is enabled. The ultimate goal is to run most
of the build system tests with buildv2. This should help ensure
compatibility between IDF build system v1 and v2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This marker enables the skipping of tests that, for any reason, cannot
be executed with the IDF build system version 2. It accepts an optional
string argument that explains why the test cannot be run with version 2.
If no explanation is provided, a default message is used. This marker is
used in the `pytest_collection_modifyitems` hook to skip tests marked
with it when the `--buildv2` pytest command line option is used.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The IDF build system v2 should be backward compatible with IDF build
system v1 in most situations. Therefore, it makes sense to reuse the
existing v1 tests and run them for v2 as well. This approach will help
ensure that v2 maintains backward compatibility. Introduce a new
--buildv2 option, which switches the existing tests to use the newly
added buildv2_test_app for v2. The goal is to enable the existing v1
tests incrementally in CI as the v2 implementation progresses.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the target consistency check with sdkconfig is performed even
if sdkconfig is not present, by comparing the selected target with the
target defined in sdkconfig.defaults. This results in an inconsistency
report when a different target is set in the IDF_TARGET environment
or cmake variable compared to sdkconfig.defaults. The behavior in
cmakev1 is to prioritize IDF_TARGET and ignore values in
sdkconfig.defaults. Perform the consistency check only if the
sdkconfig file exists.
Additionally, skip this consistency check if a set-target action is
taking place, as the old target in sdkconfig is being replaced with a
new value anyway.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The test for guessing the target from the SDKCONFIG_DEFAULTS environment
variable is not cleaning up after itself, leaving the SDKCONFIG_DEFAULTS
environment variable set. The subsequent test performs target guessing
from SDKCONFIG_DEFAULTS passed to cmake with the -D option. The
SDKCONFIG_DEFAULTS environment variable should take precedence over the
SDKCONFIG_DEFAULTS cmake variable. This is correctly handled in cmakev2
, but in cmakev1, the SDKCONFIG_DEFAULTS cmake variable is used even if
the SDKCONFIG_DEFAULTS environment variable is set. This appears to be a
bug or at least an inconsistency in cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, this generates a very basic project_description.json file,
allowing idf.py commands that call ensure_build_directory, such as
reconfigure, to be used with cmakev2. The ensure_build_directory
function sets a global context, which is presently used only for hints,
and the project_description.json file is loaded as part of this context.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add two project components, component1 and component2, and verify that
their real targets are created when they are included.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Call idf_component_include for all components requested in the library
and link them into it.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a core function of the build system, responsible for including the
specified component identified by name, into the build
process.
Currently, the non-project components written in cmakev1 are ignored and
not evaluated with the add_subdirectory command because there are no
shims for the cmakev1 API yet. However, this allows for the evaluation
of project components written using the cmakev2 approach, which is
closer to the native CMake usage.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Gather the compilation options from COMPILE_OPTIONS, C_COMPILE_OPTIONS,
CXX_COMPILE_OPTIONS, and ASM_COMPILE_OPTIONS build properties into a
single list using generator expressions. This list can then be used with
the target_compile_options call.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Set the LINKER_TYPE build property. Different linkers may have varying
options, so it's important to identify the linker type to configure the
options correctly. Currently, LINKER_TYPE is used to set the appropriate
linker options for linking the entire archive, which differs between the
GNU and Apple linkers when building on the host.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Simple test for PROJECT_NAME and PROJECT_VER build properties to ensure
they are set according to the values provided in the project() call.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, only the value stored in CMake's cache is considered when
setting the default value. Allow regular variables as well, which may be
set in the project's CMakeLists.txt file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Initialize the PROJECT_VER build property based on the following
precedence.
1. The PROJECT_VER environment or CMake variable.
2. The version.txt file located in the top-level project directory.
3. The VERSION argument, if provided, in the project() macro.
4. The output of git describe if the project is within a Git repository.
5. Defaults to 1 if none of the above conditions are met.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the __init_build_configuration() function to set all C, CXX, ASM,
and linker options, along with compilation defines, in a single
location. In cmakev1, these settings are scattered across multiple
places and it seems logical to group them together.
This function is called from the __init_build(), which is invoked after
project(). This sequence is necessary because some options depend on the
CMake information about the compiler, which is only available once the
toolchain has been initialized in project().
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Most of the global settings are configured in idf.cmake, but certain
elements can only be initialized after the project() function is called
and the toolchain is set up. For example, this includes the compilation
options, which may depend on identified compiler or the project name and
version set in the project() call. The __init_build function is called
at the beginning of the idf_build_library() function, meaning it should
be executed after the project() call. It ensures that initialization
occurs only once by setting the __BUILD_INITIALIZED build property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Initialize submodules that are not yet initialized, and issue a warning
for submodules that do not match the recorded hash in the git tree.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When working with command output, it is sometimes necessary to handle it
line by line. Add a `__split` helper function that can split an input
string based on a specified separator, which defaults to a newline, and
store the results in a list. This function also offers modifiers to
strip whitespace and remove empty entries.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Initialize the IDF_VER build property using the version.txt file, or use
git-describe. If neither is available, fall back to the value of the
IDF_VERSION environment variable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The Git tool will be used in various parts of the build system, such as
retrieving the IDF version using git-describe or checking submodules.
Similar to the Python interpreter, add an initialization function to
identify the Git executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The following functions are added to tools/cmakev2/kconfig.cmake:
- __generate_kconfig_outputs(): Generates Kconfig output files.
- __create_base_kconfgen_command(): Creates command for running
kconfgen.
- __run_kconfgen(): Invokes kconfgen tool.
The following functions are added to tools/cmakev2/kconfig.cmake:
- __setup_kconfig_environment(): Creates Kconfig environment for Kconfig
output generation.
- __get_init_config_version(): Fetch ESP-IDF version and set the
environment variable.
- __create_config_env_file(): Create the config.env file for kconfgen.
The following functions are added to tools/cmakev2/kconfig.cmake:
- __collect_kconfig_files_from_components(): Collect Kconfig files from
components.
- __collect_kconfig_files_from_bootloader_components(): Collect Kconfig
files from bootloader components.
- __collect_kconfig_files_from_directory(): Collect Kconfig files from
a directory.
This commit adds the tools/cmakev2/kconfig.cmake file for Kconfig
processing. The file provides the following functions:
- __init_kconfig(): Placeholder to initialize Kconfig build properties,
collect kconfig files and generate the initial sdkconfig.
- __generate_sdkconfig(): Placeholder to generate sdkconfig output files.
Add idf_build_library function and enable setting and getting its
properties. This lays the groundwork for future initialization that
needs to occur after the project() call, such as determining compilation
options and others.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The `idf_build_(set|get)_property` and
`idf_component_(set|get)_property` functions share a lot of similar
code. Move these common parts into new `__(set|get)_property` helper
functions. With the upcoming `idf_build_library` API function, we might
need to add properties for the interface target created for the library,
which would otherwise lead to yet another code duplication for setting
and getting library interface properties.
Update the current implementations of `idf_build_(set|get)_property` and
`idf_component_(set|get)_property` to utilize these new helper
functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Determine the IDF_TOOLCHAIN from the following sources in this order:
environmental variable and CMake cache variable. Ensure there are no
inconsistencies between the values set in these different locations.
Set the IDF_TOOLCHAIN and IDF_TOOLCHAIN_FILE build properties. Also,
configure the IDF_TOOLCHAIN CMake cache variable and set the
CMAKE_TOOLCHAIN_FILE global variable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Determine the IDF_TARGET from the following sources in this order:
environmental variable, CMake cache variable, and sdkconfig files.
Ensure there are no inconsistencies between the values set in these
different locations.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the environmental variables, CMake cache variables, or default
values, set the DKCONFIG and SDKCONFIG_DEFAULTS build properties.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Determine the Python interpreter and verify package dependencies if the
CMake cache variable PYTHON_DEPS_CHECKED is not set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The version.cmake file should be the sole file used from the cmakev1
build system. There's no need to maintain the IDF_VERSION information in
two separate locations.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Helper for obtaining the default value of a variable. It returns the
value of the specified variable based on the following order of
precedence, with the highest precedence first:
1. Environmental variable
2. CMake cache variable
3. Provided default value
This can be used to retrieve the value of variables that might also be
set in the environment or cache, such as PYTHON or SDKCONFIG.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
With the default signature of `cmake_parse_arguments`, without using
`PARSE_ARGV`, it's not possible to determine if options were not
specified or set as an empty string (or empty list)[1]. If an empty
string is passed to the `PATHS` option, the variable parsed by
`cmake_parse_arguments` is not defined. This issue can be addressed by
using `PARSE_ARGV`, but this approach only works for functions and
requires CMake version 3.31 or newer. Additionally, when `PARSE_ARGV` is
used for multiple value option, the values are not concatenated into a
single list, which is inconvenient, as the lists are instead escaped. If
the `PATHS` option is not defined, set it to an empty string. This
allows passing an empty string, as well as a mix of lists and
individual strings, through PATHS.
The behaviour can be seen with a simple example:
$ cmake -P test.cmake
```test.cmake
cmake_minimum_required(VERSION 3.22)
function(test)
set(options)
set(one_value)
set(multi_value PATHS)
cmake_parse_arguments(ARG "${options}" "${one_value}" "${multi_value}" ${ARGN})
#cmake_parse_arguments(PARSE_ARGV 0 ARG "${options}" "${one_value}" "${multi_value}")
message("PATHS: ${ARG_PATHS}")
endfunction()
test(PATHS "one;two;three" "four" "five;six" "seven")
```
```
PATHS: one;two;three;four;five;six;seven
vs
PATHS: one\;two\;three;four;five\;six;seven
```
Also update the current usage of __get_absolute_paths, as the check for
empty PATHS is no longer necessary.
[1] https://cmake.org/cmake/help/latest/policy/CMP0174.html#policy:CMP0174
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a basic CMake project that currently includes tests for
displaying build and component properties, as well as testing component
priority. These tests should be removed once proper CI testing is
in place.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
There are currently no CI tests for CMakeV2, so add it to the
exclude_check_tools_files.txt file. Once CI testing is implemented, this
change should be reverted.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Discover component directories and initialize components within them.
This process does not include managed components, which should be added
separately at a later stage. To facilitate this, some minimal
functionalities are introduced, such as build properties, component
properties, and other helper functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
fix(test_rtc_wdt): Fixed CI test failures for RTC WDT by increasing margin
Closes IDFCI-4816, IDFCI-4835, IDFCI-5863, IDFCI-4799, and IDFCI-4817
See merge request espressif/esp-idf!42959
This commit adds a new idf.py --preset NAME argument, which allows
selecting the CMake configuration preset with a given name.
For idf.py, selecting the preset does two things:
- sets the build directory, which is specified in the preset itself
using "binaryDir" field
- passes --preset argument to CMake configuration phase
The multi_config example is updated to illustrate how this feature is
expected to be used.
The CI tests for RTC WDT test app were failing because we allowed only
20% margin between configured vs actual timeout of wdt. However, the
RC oscillator accuracy on some boards is very poor and it results in
almost 25% difference between configured and actual timeout. Hence,
increased the failure margin from 20% to 30%. Also some boards like
ESP32P4 have known problem and gives timeout after twice the configured
time. Hence, increased the wdt interrupt wait time to three times the
configured value, so we don't miss the interrupts for such boards.
In case of ESP32-C5 and ESP32-C61, mmu_ids for PSRAM and Flash
MMU are the same due to their shared memory space. Thus, instead of
mmu_id we should use mmu_target_t.
fix: Reorder Ethernet initialization in VLAN example
Closes IDFCI-3146, IDFCI-3145, IDFCI-3144, IDFCI-5092, IDFCI-5091, and IDFCI-5090
See merge request espressif/esp-idf!42401
This prevents premature matching on partial data when the output is
being streamed, which was causing the test to capture 100000 instead of
the full 1000002.
Refactored esp_lcd_rgb_panel_get_frame_buffer and
esp_lcd_dpi_panel_get_frame_buffer
to follow the standard printf/vprintf pattern:
- Created static _v helper functions that take va_list as parameter
- Moved parameter validation to the public API functions
- Public functions now handle va_start/va_end and call the helper
This reverts earlier commit d2858847 and thereby making SPIRAM_USE_MEMMAP
available for all targets.
This config option has no direct usage but just avoids adding the PSRAM
to heap but still keeps it memory mapped.
Closes https://github.com/espressif/esp-idf/issues/17764
For the dual core chips like ESP32H4 that are running on slow CPU
clock frequency (96MHz), the number of events getting dispatched in
the test case called "performance test - no dedicated task" were
falling short of expectation because of extra overhead with locks
etc when runing with a non dedicated task. Even for other chips like
ESP32S3, if the main CPU core is running at 80MHz, the result is same.
Hence, added a consideration for CPU clock frequency when deciding
pass criteria for this test case.
Add a new test case to verify that USJ printing doesn't block CPU on chip
wake-up from light sleep. The test measures the average time per print
operation and ensures it's below 5000 microseconds.
- Introduced new HAL components for LCD and MIPI DSI, including:
- `lcd_hal.c` and `lcd_hal.h` for LCD hardware abstraction layer.
- `mipi_dsi_hal.c` and `mipi_dsi_hal.h` for MIPI DSI hardware
abstraction layer.
- Peripheral connection definitions in `lcd_periph.c` and
`lcd_periph.h`.
- Added necessary types and utility functions in `lcd_types.h` and
`mipi_dsi_types.h`.
- Updated CMakeLists to include the new HAL components in the build
system.
- Removed legacy references to LCD and MIPI DSI in the HAL and SOC
CMakeLists.
- When `esp_flash_read()` is invoked from REE, it internally enables WB mode
via `spi_flash_ll_wb_mode_enable()`. However, the ROM flash APIs used by TEE
do not support WB mode, resulting in failures when TEE attempts to access
flash after this call.
- This commit adds a workaround in the TEE flash layer by saving WB mode state,
temporarily disabling it for ROM API calls, and restoring it afterward.
Currently, REE SPI flash HAL operations are routed as service calls to TEE,
but the TEE implementation incorrectly uses ROM APIs instead of HAL APIs.
This leads to issues and is not the recommended approach.
When DMA keep writing the memory, some data may be corrupted after reset. For example, the stack of bootloader may be overwritten and failed to boot until a higher scope of reset (Core).
Also removed the DPORT_PERIP_RST_EN_REG alias on ESP32S2. Now it's the same as some following chips (EN0).
description:Onwhich Espressif SoC revision does your application run on? Run `esptool chip-id` to find it.
description:Onwhich Espressif SoC revision does your application run on? Run `esptool chip-id` (or `esptool.py chip_id` for ESP-IDF v5.5 and older) to find it.
- git diff --exit-code -- ${IDF_TARGET}/esp_efuse_table.c || { echo 'Differences found for ${IDF_TARGET} target. Please run idf.py efuse-common-table and commit the changes.'; exit 1; }
@@ -140,6 +140,12 @@ Supported since ESP-IDF v5.1.
| release/v5.4 | v5.4.1+ | v5.4.1 |
| release/v5.5 and above | v5.5+ | v5.5 |
### ESP32-P4
#### v1.0, v1.3
Supported since ESP-IDF v5.3.
## What If the ESP-IDF Version Is Lower than the `Required` Version?
Latest ESP-IDF versions can prevent from downloading to, or even execute binaries on unsupported chips. ESP-IDF of versions v4.4.5+, v5.0.1+, v5.1 and above have both esptool download check and bootloader loading check against the chip revision. While ESP-IDF v4.3.5 has only esptool downloading check.
#ifndef SEGGER_USE_PARA // Some compiler complain about unused parameters.
#define SEGGER_USE_PARA(Para) (void)Para // This works for most compilers.
#endif
#define SEGGER_ADDR2PTR(Type, Addr) (/*lint -e(923) -e(9078)*/((Type*)((PTR_ADDR)(Addr)))) // Allow cast from address to pointer.
#define SEGGER_PTR2ADDR(p) (/*lint -e(923) -e(9078)*/((PTR_ADDR)(p))) // Allow cast from pointer to address.
#define SEGGER_PTR2PTR(Type, p) (/*lint -e(740) -e(826) -e(9079) -e(9087)*/((Type*)(p))) // Allow cast from one pointer type to another (ignore different size).
SEGGER_BUFFER_DESC*pBufferDesc;// Buffer descriptor to use for output.
void(*pfFlush)(SEGGER_SNPRINTF_CONTEXT*pContext);// Callback executed once the buffer is full. Callback decides if the buffer gets cleared to store more or not.
// Description for a circular buffer (also called "ring buffer")
// which is used as up-buffer (T->H)
//
typedefstruct{
constchar*sName;// Optional name. Standard names so far are: "Terminal", "SysView", "J-Scope_t4i4"
char*pBuffer;// Pointer to start of buffer
unsignedSizeOfBuffer;// Buffer size in bytes. Note that one byte is lost, as this implementation does not fill up the buffer in order to avoid the problem of being unable to distinguish between full and empty.
unsignedWrOff;// Position of next item to be written by either target.
volatileunsignedRdOff;// Position of next item to be read by host. Must be volatile since it may be modified by host.
unsignedFlags;// Contains configuration flags. Flags[31:24] are used for validity check and must be zero. Flags[23:2] are reserved for future use. Flags[1:0] = RTT operating mode.
}SEGGER_RTT_BUFFER_UP;
//
// Description for a circular buffer (also called "ring buffer")
// which is used as down-buffer (H->T)
//
typedefstruct{
constchar*sName;// Optional name. Standard names so far are: "Terminal", "SysView", "J-Scope_t4i4"
char*pBuffer;// Pointer to start of buffer
unsignedSizeOfBuffer;// Buffer size in bytes. Note that one byte is lost, as this implementation does not fill up the buffer in order to avoid the problem of being unable to distinguish between full and empty.
volatileunsignedWrOff;// Position of next item to be written by host. Must be volatile since it may be modified by host.
unsignedRdOff;// Position of next item to be read by target (down-buffer).
unsignedFlags;// Contains configuration flags. Flags[31:24] are used for validity check and must be zero. Flags[23:2] are reserved for future use. Flags[1:0] = RTT operating mode.
}SEGGER_RTT_BUFFER_DOWN;
//
// RTT control block which describes the number of buffers available
// as well as the configuration for each buffer
//
//
typedefstruct{
characID[16];// Initialized to "SEGGER RTT"
intMaxNumUpBuffers;// Initialized to SEGGER_RTT_MAX_NUM_UP_BUFFERS (type. 2)
intMaxNumDownBuffers;// Initialized to SEGGER_RTT_MAX_NUM_DOWN_BUFFERS (type. 2)
SEGGER_RTT_BUFFER_UPaUp[SEGGER_RTT_MAX_NUM_UP_BUFFERS];// Up buffers, transferring information up from target via debug probe to host
SEGGER_RTT_BUFFER_DOWNaDown[SEGGER_RTT_MAX_NUM_DOWN_BUFFERS];// Down buffers, transferring information down from host via debug probe to target
#define SEGGER_RTT_HASDATA_UP(n) (((SEGGER_RTT_BUFFER_UP*)((uintptr_t)&_SEGGER_RTT.aUp[n] + SEGGER_RTT_UNCACHED_OFF))->WrOff - ((SEGGER_RTT_BUFFER_UP*)((uintptr_t)&_SEGGER_RTT.aUp[n] + SEGGER_RTT_UNCACHED_OFF))->RdOff) // Access uncached to make sure we see changes made by the J-Link side and all of our changes go into HW directly
#define SEGGER_SYSVIEW_INFO_SIZE 9 // Minimum size, which has to be reserved for a packet. 1-2 byte of message type, 0-2 byte of payload length, 1-5 bytes of timestamp.
#define SEGGER_SYSVIEW_QUANTA_U32 5 // Maximum number of bytes to encode a U32, should be reserved for each 32-bit value in a packet.
#define SEGGER_SYSVIEW_LOG (0u)
#define SEGGER_SYSVIEW_WARNING (1u)
#define SEGGER_SYSVIEW_ERROR (2u)
#define SEGGER_SYSVIEW_FLAG_APPEND (1u << 6)
#define SEGGER_SYSVIEW_PREPARE_PACKET(p) (p) + 4
//
// SystemView events. First 32 IDs from 0 .. 31 are reserved for these
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() ((*(U32*)(0xE000ED04)) & 0x1FF) // Get the currently active interrupt Id. (i.e. read Cortex-M ICSR[8:0] = active vector)
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() (__get_IPSR()) // Workaround for IAR, which might do a byte-access to 0xE000ED04. Read IPSR instead.
#else
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() ((*(U32*)(0xE000ED04)) & 0x3F) // Older versions of IAR do not include __get_IPSR, but might also not optimize to byte-access.
#endif
#else
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() ((*(U32*)(0xE000ED04)) & 0x3F) // Get the currently active interrupt Id. (i.e. read Cortex-M ICSR[5:0] = active vector)
#endif
#else
#define SEGGER_SYSVIEW_GET_INTERRUPT_ID() SEGGER_SYSVIEW_X_GetInterruptId() // Get the currently active interrupt Id from the user-provided function.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.