Additionally, always enable the partition MD5 check if flash encryption is on in
Release mode. This ensures the partition table ciphertext has not been modified
(CVE-2021-27926).
The exception is pre-V3.1 ESP-IDF bootloaders and partition tables, which
don't have support for the MD5 entry.
* Patched longjmp to be context-switch safe
longjmp modifies the windowbase and windowstart
registers, which isn't safe if a context switch
occurs during the modification. After a context
switch, windowstart and windowbase will be
different, leading to a wrongly set windowstart
bit due to longjmp writing it based on the
windowbase before the context switch. This
corrupts the registers at the next window
overflow reaching that wrongly set bit.
The solution is to disable interrupts during
this code. It is only 6 instructions long,
the impact shouldn't be significant.
The fix is implemented as a wrapper which
replaces the original first instructions of
longjmp which are buggy. Then, it jumps back
to execute the rest of the original longjmp
function.
Added a comparably reliable test to the
test apps.
1. Fix setting channel error after WiFi stop
2. Fixed issue of reason code change from 15 to 204 when provide wrong password
3. Fix set config return value error
4. Fix ampdu age timer memory leak
5. Add beacon timeout event
6. Ignore dirtyBuf check when buffer size is larger than 4092
7. Fix SoftAP not initiating 4-way handshake if max length passphrase is set
8. Fix esptouch find hidden AP fail
9. Clear AP Bss when auth timeout happens.
10.Fix not connect to open AP when WPA3 auth fail.
11.Fix esp_wifi_80211_tx when out of memory unlock mutex issue.
12.Fix esp_wifi_80211_tx can't be used in ap mode
13.Fix hidden AP scans after connecting AP
14.Fix watchdog happens when receiving action frame
This issue is caused by modem sleep.The sleep interrupt will come the second
time before the CLKN interrupt. If we disable the sleep interrupt when it
comes in the first time, the CLKN will never come.
When user forgot to add git.exe or python to the PATH, there was a not
very helpful error message.
This improves the error with which tool is misssing and shows a link to
the espressif installer tool.
Newer versions of doxygen will give warnings for comments in
INPUT lists
Delete all comment lines to fix these warnings, our folder structure
stil gives an OK overview of what we are including
Current code stopped inquiry if major class is not Phone. Modified the
condition to consider for both Phone and Audio to cover majority of real
world devices.
Closes https://github.com/espressif/esp-idf/issues/6548
Fixes issue with DPORT init task, this task uses minimum stack size and may not be
enough if stack smashing detection is set to Overall mode.
Also reworks the way we calculate minimum stack to allow for adding multiple
contributing factors.
Closes https://github.com/espressif/esp-idf/issues/6403
The case sets one pin as output and the other as INPUT_OUTPUT (so output buffer
is enabled in both), then only changes level of one pin - causing a conflict.
Probably this test should be deleted, as it's not good practice for two output
pins to drive each other, but this fix also seems to work.
Sometimes the flash size read from bootloader is not correct. This may
forbid SPI Flash driver from reading the the area larger than the size
in bootloader header.
When the new config option is enabled, the latest configured
ESPTOOLPY_FLAHSIZE in the app header will be used to override the value
read from bootloader header.
Added BD_ADDR comparison in l2cu_lcp_allocate and removed check from
security connection request handler as it's handled in l2cu_lcp_allocate
for both connection request and create connection.
There was no need for this function as there is already a way present to
fetch own mac address. Own mac addr can be fetched by calling
controller_get_interface().
With the introduction of cookies to track a session, it is possible that the
clients restart the provisioning on the same session, specifically when a user
cancels a current provisioning attempt. This can result in an error as the state
on the device side and client side will go out of sync.
This has now been changed such that if SESSION_STATE_CMD0 is received on
an existing session, the state is reset and flow allowed to continue.
Earlier, a "socket connection" was being considered as a protocomm "session".
However, for HTTP, we cannot expect clients to necessarily use the same
socket for all requests of a given transaction. This was indeed causing
some failures for Wi-Fi provisioning from Android phones.
So, we have introduced http cookies to track a session even if the socket
connection does not stay intact throughout the transaction.
This change is backward compatible. If the client does not use cookies, the
socket connection will be considered as the session.
Closes: https://github.com/espressif/esp-idf/issues/6316
Reference: https://github.com/espressif/esp-idf/pull/6330
This reverts commit bf35ef1ce7.
It has been noticed that there are scenarios where even though firmware is not enabled
with flash encryption config feature, it should be able to write to encrypted partitions.
This revert adds the feature back which was removed around v4.0 timelines, and same
change will be backported to all releases (upto v4.0) for consistency.
esp_http_server: Add flag in sock_db to identify httpd_sess_close is called from httpd_session_close_lru (v4.0)
See merge request espressif/esp-idf!12114
- line was truncated because 64 characters were not sufficient
- length passed to snprintf should be full buffer length, not -1
- make the width of lock name field fixed
- fix alignment of lock type column
1. Add check CSA state before CSA timer process
2. Change wifi scan duration from 120ms to 100ms
3. Using deport reg instead of ahb
4. Check TID in ieee80211_recv_bar()
5. Revert to report specific reason code when receiving deauth during 4-way-handshark
6. Fix the bug that tx ampdu parameter is not from peer device
7. Refactor wifi_interface_t
8. Faster WiFi station connect improvement, avoid 100ms passive scan
9. Add FCS failed packets filter
10.Update esp32 phy lib to v4660
11.Fix csa timer issue
12.Fix country code last byte to space instead of NULL
13.Fix softap cannot forward A-MSDU
14.Fix max tx power to 20dBm
15.Fix the issue that the esp_wifi_sta_get_ap_info can't get country
Previously if starting/stopping BLE advertising when BLE Mesh
is enabled, the corresponding events will be notified through
the callback registered by esp_ble_mesh_register_prov_callback().
With this commit, the func esp_ble_mesh_register_ble_callback()
needs to be invoked for resgitering the callback used for BLE
coexistence related functionalities (i.e. ADV/SCAN).
This commit fixes 'Impersonation in Passkey entry protocol'
(CVE-2020-26558) and suggests fixes for other vulnerabilites like
'Impersonation in the Pin Pairing Protocol' (CVE-2020-26555) and
'Authentication of the LE Legacy Pairing Protocol'
CVE-2020-26558 can be easily implemented if the peer device can
impersonate our public key. This commit adds a check by comparing our
and received public key and returns failed pairing if keys are same.
This commit also adds comments suggesting to use secure connection when
supported by all devices.
esp_http_client: Skip check for redirection counter if status code is success, fix issue with digest auth, configurable user agent string. (v4.0)
See merge request espressif/esp-idf!11599
Can still be enabled by passing --cmake-warn-uninitialized on the command line
Prevents CMake warnings printed by default if IDF_PATH is underneath the CMake
project directory.
The reason for this is that CMake --warn-uninitialized only enables checks
inside the project directory (ie top-level CMakeLists.txt directory and
subdirectories), it doesn't enable for files included from other directories.
(The only way to enable warnings in other directories is to pass
--check-system-dirs and this looks like it's only useful for CMake's own
developers as it prints a lot of warnings from inside CMake otherwise - see
https://gitlab.kitware.com/cmake/cmake/-/issues/19645 )
Plan to follow up with a later commit to clean up most of the warnings (which
aren't problems for CMake execution), but we'll also disable this option by
default to avoid this unexpected triggering of IDF warnings.
If esp_restart_noos() is run and the stack address points to external memory (SPIRAM)
then Cache_Read_Disable() raises up the error "Cache disabled but cached memory region accessed"
to fix this we switch stack to internal RAM before disable cache.
Added unit tests.
Closes: https://github.com/espressif/esp-idf/issues/5107
This commit adds the feature where the CAN ISR will continue to
run even if the cache is disabled. Whilst cache is disabled, any
received messages will go into the RX queue, and any pending TX
messages in the TX queue will be transmitted. This feature should
be enabled using the CONFIG_CAN_ISR_IN_IRAM option.
This commit removes any function calls within
the CAN driver that result in a call to ESP_LOG
whilst inside a critical section.
These function calls are either moved outside
critical sections (e.g., intr_alloc and gpio
functions), or substituted (e.g., assert()).
1. Use flag ESP32_WIFI_ENABLE_WPA3_SAE to control WPA3 code, disabling
it code footprint reduces by 7.7kB in libwpa_supplicant.a
2. Fix handling of multiple AP credentials in WPS, apps need update
to handle the new event for the fix to work
Commit 5e8795eebe has changed the partition table offset, which
has resulted in the ciphertext not matching the one expected in the
example test.
Fix by calculating the ciphertext using espsecure.py.
When the flash encryption is enabled then we do not need to change the partition table.
The partition_example.csv should not have fixed offsets for partitions because we want to move the whole table.
The fixed offsets in the table were cleared.
1. Add description for esp_wifi_set_inactive_time()
2. Add documentation to avoid using WiFi modem sleep for WiFi/BT coexistence
3. Remove description of unusable API esp_wifi_restart()
4. Update esp_now_fetch_peer discription
5. Update table format prblm for esp_wifi_set_max_tx_power()
6. Update description for ssid and password
Curretly wpa_supplicant uses internal APIs for RSA operations
which internally uses lots of big num operations.
Big num operations are CPU expensive and can take a lot of time
which can cause watchdog timer to tigger.
This can be optimize by using mbedtls APIs which uses
hardware blocks for big num operations.
To fix this, write new crypto_mbedtls-rsa.c which has APIs
similar to crypto_internal-rsa.c but uses mbedtls APIs.
1. Modify LR for esp32
2. Optimize some debug logs for probe response
3. Optimize phy and soc files
4. Fix the bug that no event for connect
5. Fix the bug that set_channel doesn't work for softAP
6. Add sta connect again ap sent disconnect event
7. Add set/get inactive time api
8. Fix connect hidden AP doesn't update information
9. Add rf test long short support
* Fix the issue that deinit node with "erase_flash"
set to true, but info is not erased from nvs
* Reuse bt_mesh_cfg_reset() when deinit node
* Optimize Provisioner related erase operations
* No store pending timeout will be used when Node
is not provisioned OR Provisioner is disabled
and erase operation is performed
* Change the default timeout for settings operation
to 0, and rpl store rate to 0
use eventgroup to sync spp_vfs_write
each connection has a switch_delay_timer not sharing a one
revert functions like spp_find_slot_by_xxx
fix vfs read bug when peer close
When reset the rx info of transport layer, the
rpl list will always cleared, and rpl stored
in the nvs will only be erased when erase flag
is true and BLE_MESH_SETTINGS is enabled.
Compared with the previous solution, it should
be more clear.
- Previously when a model is initialized or deinitialized, in the
access layer, we need to check the model id with the ids in the
table in order to find the proper model operation function.
- Currently all the operation functions of each model will be set
during the mesh initialization. When the model is found, we can
directly use the corresponding callback for different operations.
- Currently only init/deinit operations are registered, later we
will add more operations.
When `pthread_mutex_destroy` is used to release mutex, `pthread_mutex_lock_internal` is used,
which results in the increase of `uxmutexehold` and no recovery base priority
Add following changes as part of this:
1. EAP client will crash during validation of key size when CA
certs and keys not present. Add changes to validate it first.
2. Free memory allocated in TLS context
For WPA3 connection nearing PMK lifetime expiry, PMK Cache
needs a re-authentication or the cache will expire. After
current expiry of 12 hours Station ends up sending a deauth
to the AP. An SAE re-authentication also cannot occur without
a disconnection with current implementation. So increase the
PMK lifetime to 100 days for now.
1. will post twice disconnect event when read error
2. will block `timeout` times when set disable_auto_connect
3. When `esp_websocket_client_stop` before `esp_websocket_client_send*`,
if the `esp_websocket_client_send*` fails, the status will change to
'WEBSOCKET_STATE_WAIT_TIMEOUT', and the next `esp_websocket_client_start` will fail forever
tools/ldgen: Parse paths with Unicode characters in archives & fix parsing of section names on Windows (v4.0)
See merge request espressif/esp-idf!10966
1. Fix FIX-ROOT does not reconnect to router when disconnect reason is too many
2. Add API esp_mesh_print_scan_result
3. Modify not to reset mesh_xonseq of self and children when flush_upstream_packets
4. Fix not arm parent monitor when a parent candidate is cleared without sending an event to mesh layer
5. Fix the new voted root does not reconnect to router if rssi is weak
Exclude paths that were specified with slash as a path separator were compared
as strings. This would fail on Windows which uses backslash as a path separator.
Also add some explanation about configuring the project in the README
Previously with the default build-esp32.sh script, the build directory was
deleted and recreated so the project would always be built with default config.
1. Add STA checks during STA PMF operations
2. Fix WPA2-Ent issue with Open AP
3. Skip WPA-TKIP profile if PMF is required
4. Skip & clear Supplicant PMK Cache for mismatching AP config
Since we have provided separate functions for deleting node
information with node's unicast address, device uuid, etc.
So we update the behavior of this function, which will only
be used to delete device information which is not provisioned
or just under provisioning.
For getting secure_version field in anti rollback case, bootloader tries
to map whole firmware partition but fails for cases where partition size
is beyond available MMU free pages capacity.
Fix here ensures to map only required length upto application descriptor
size in firmware partition.
Closes https://github.com/espressif/esp-idf/issues/5911
Builds on previous commit.
Note: Getting the project version still pases --tags so still works with plain
tags, to keep compatibility for existing projects
1. Add g_av_with_rc to avrc_tg_init function
2. Remove g_av_open_fail
3. Add comment in API files
4. Add a2dp init & deinit state variable to inidcate a2dp statement
1. Add reaction for a2dp snk & src disconn req when no link up
2. Change state machine set before EVT upto APP layer in some cases
3. Add bool open_fail to choose whether post disconnection evt to app layer
1. Add logs for SPP not enabled successfully
2. Add information to Applicatioin layer that the A2DP is connecting to peer device
Regular BTC and BTU Task name & Add BTC Init protection
1. Regular the BTC Task Name to Close a Github issue
Closes https://github.com/espressif/esp-idf/issues/4966
2. Add BTC Init failure protection
3. Regular BTU Task Name with BTC
4. Correct the comment of
a. SDP_ServiceSearchRequest and SDP_ServiceSearchRequest2
b. l2cu_create_conn_after_switch
5. Correct comment of esp_bt_hf_vra();
6. Fix cpp compilier region
Provisioner should always uses the latest IV Index for provisioning.
For example, if the current IV Index is 0x00000001, but prov->iv_index
is still initialized with 0x00000000, and if Provisioner uses prov->
iv_index for provisioning, this will cause the Provisioner failing to
control the node.
So here bt_mesh.iv_index is used instead of prov->iv_index.
Fixup more for PSRAM issue;
Allow GDB work correctly with Privileged Registers;
Based on GCC 8.4.0;
Update a fix of C++ exception crashs from the GCC mainline;
Fixup strptime_l() and some locale in libc;
Closes https://github.com/espressif/esp-idf/issues/5765
H/W decryption of Mgmt frames was disabled for PMF and done through
S/W. If ESPNOW packets go through this path, it affects backward
compatibility since method of decrypting Mgmt packets is different in H/W.
To address PMF + ESPNOW Co-existance, CCMP decryption method is modified
for ESPNOW packets so that they can be decrypted correctly. Since Tx
of ESPNOW packets can still be done in H/W alongside PMF, no change
required in encryption method in S/W.
Co-Authored-By: Nachiket Kukade <nachiket.kukade@espressif.com>
Co-Authored-By: zhangyanjiao <zhangyanjiao@espressif.com>
Co-Authored-By: kapil.gupta <kapil.gupta@espressif.com>
If the olddelta argument is not a null pointer, the adjtime function returns information
about any previous time adjustment that has not yet completed.
Closes: https://github.com/espressif/esp-idf/issues/5194
Replace periph_module_enable/disable by periph_wifi_bt_common_module_enable which are in IRAM.
AddIRAM_ATTR periph_ll_wifi_bt_module_enable_clk_clear_rstandIRAM_ATTR periph_ll_wifi_bt_module_disable_clk_set_rstto fit O0 optimization level.
Delete duplicated spinlock and counter.
SD cards don't support CMD7 (select_card) in SPI mode. Highspeed probe
of sdspi will fail in this step and stop working in highspeed mode.
Remove the CMD7 in enable_hs_mode_and_check to fix this issue.
Please note that, on ESP32, you have to use the IOMUX pins to use sdspi
in 40MHz, otherwise the initialization process will report reading issue
and fail.
Currently only keep func pointer for the followings:
- Invalid parameter (mesh btc & mesh stack)
- Out of memory (mesh btc & mesh stack)
- Unknown act (mesh btc)
- Invalid model user data (mesh stack)
- BT_DBG("%s", __func__) (mesh btc & mesh stack)
- A few other specific situations (buf ref debug, send status check)
1. If Device is connected to AP in WPA3-PSK mode, AP switching
security to WPA2-PSK causes connection failures even after reset.
Fix is to not store WPA3's PMK in NVS for caching.
2. AP switching back to WPA3 causes even more connection failures.
This is due to device not clearing Supplicant level PMK Cache when
it is no longer valid. Fix is to clear the Cache when 4-way handshake
fails and to check Key Mgmt of Cache before using.
3. When AP switches from WPA3 to WPA2, device's PMF config in
Supplicant remains enabled. This may cause failures during
4-way handshake. So clear PMF config in when PMF is no longer used.
Different bluetooth host has different behaviors, so it's better
to maintain a scan check mechanism of BLE Mesh itself.
Fixes an issue when only PB-GATT is enabled for node, which will
output a scan error log when the device is provisioned.
mbedTLS stack does not keep TLS renegotiation enabled even if
relevant config option is turned on, it needs explicit API call
`mbedtls_ssl_conf_renegotiation` to do so.
This issue was observed in case of Azure IoT, where keys needs to
be refreshed periodically to keep TLS connection intact.
Update send_ttl mainly for server models. When a server model
receives a message, and the status is required to be replied
by the application, we need to set send_ttl to the msg context.
If send_ttl is not updated in btc, and the applcation does not
set the TTL either, then the status will be replied with TTL=0,
which may cause the client side (e.g. the phone App) failed to
receive the status.
Closes https://github.com/espressif/esp-idf/issues/5300
1. Fix TX DMA buffer issue
2. API esp_wifi_get_config add acquisition sta.listen_interval
3. Configure bandwidth and phy mode to store NVS
4. If AP's tsf has been restarted, STA will disconnect from AP.
5. Do not reset softAP's tsf except it restart
6. fix the wifi regdomain update bug
7. fix the bug for airkiss find hidden AP fail
8. fix addba and first ampdu send bar
The issue is caused by:
1. The disable_qio_mode inside read_id may have side effects.
2. read_id twice may have side effects.
Fix this issue by moving disable_qio_mode out of read_id and only do it
once before read_id. And retry read_id only when the first one is
failed.
Issue introduced in 3ecbb59c15.
Also remove some redundant CONFIG_BLE_MESH_PROXY checks, because
when the following options are satisfied, the CONFIG_BLE_MESH_PROXY
option will be selected mandatorily.
- Before updating the "beacon_sent" of a subnet, we need to check
if the subnet still exists, especially for a Provisioner.
- Fix a bug which will cause Provisioner failed to send Secure
Network Beacon if no device is provisioned
ctx values to cb_params for the mesh stack.
recv_rssi was not copied.
This means the rssi could not be read when receiving generic server messages using ble_mesh.
1.In the description of the ESP_API header file, it should be the event that the application layer needs to handle, not the BTA layer.
2.Fix name typo of BTA_GATTS_CREATE_SRVC_EVT event
Closes https://github.com/espressif/esp-idf/issues/5446
esp_flash: fix the regression of non-quad mode by default chip driver, bugs in add_device and deprecate cs_id (4.0)
See merge request espressif/esp-idf!8879
Prevent a remote device from doing a Bluetooth Impersonation Attack
(BIAS) by:
- Preventing remote device to downgrade secure connection
feature mask. Secure connection feature mask should remain same or
increase to enabled in link key generation and authentication.
- Doing a mutual authentication during Legacy Authentication.
Signed-off-by: Chinmay Chhajed <chinmay.chhajed@espressif.com>
- This issue was introduced in the commit: a788e7cd3d
which updated the bt_hex() function incorrectly.
- And in bt_hex(), we use 2 for the two-dimensional array, because currently
at most two bt_hex() will be used at the same time. Also this will save
some DRAM compared with using 4.
Move the TAG definition of each example to the corresponding
source file (previously it is defined in the common example
intialization header file), which can avoid the redefinition
of TAG in some situations.
This commit fixes a bug as well as changes a behaviour.
Bugfix: During softap/httpd based provisioning, if a session was closed
midway and a new one started, it would never proceed if the http server
assigns same socket number to the new session (which happens almost always).
Now, if a session is closed, using the http callbacks, the older session
data is cleared so that a new one can be created.
Behavioural change: If a client (mobile app particularly) does not use
persistent http session i.e. all provisioning communication on the same
socket, the provisioning may fail. Earlier, since the session context was
not getting cleared, even if the client closed a session and continued
on a new one, it would go through if the socket number assigned was same
(which happens almost always).
Ideally, from a security perspective, all communication related
to secure provisioning must happen on the same socket, and so, this
change is required.
Added following as part of this change
1. Removed auth_changed event
2. Updated doc regarding usage of WPA/WEP secuiry modes
3. Fixed WPA2 enterprise authmode threshold.
Some AP's keep NULL-padding at the end of some variable length WPS
Attributes. This is not as par the WPS2.0 specs, but to avoid interop
issues, ignore the padding by reducing the attribute length by 1.
When using DPort to read fifo, fifo_cnt is not credible, we need to calculate the real cnt based on the fifo read and write pointer. When using AHB to read FIFO, we can use fifo_cnt to indicate the data length in fifo.
Previously the model recv operation is a littl fuzzy.
With the changes, the model recv operation is splitted into:
- find op
- check app_idx
- check dst
- check length
- update info & handle message
The transport segmented TX nack and seg_pending fields must be at least
6 bits to avoid overflow for 32 segment messages. This change rearranges
the seg_tx fields to gather all state flag fields in one byte, while
making the counter fields whole bytes.
Ensures that friend messages are enqueued, even if the packet is
received with an appkey is unknown to the friend. Previously, sdu_recv
would return EINVAL if the appkey was unknown, which would prevent the
lower transport layer from adding the packet to the friend queue. This
is irrelevant for the logic in lower transport, and should not be
returned as an error.
esp_mqtt_abort_connection: Fixed an issue which could result in a race condition and subsequent crash
esp_mqtt: Change an error print to use ESP_LOGE instead of ESP_LOGI
Move Sending MQTT connect message log from Info to Debug level
docs: Makes clear that publish API could block
Change the message printed after MQTT connection failure
Since the scripts of CI will bypass common_components when
trying to get ALL_EXAMPLES, and these BLE Mesh components
will only be used by other mesh examples, i.e. no need to
be compiled as a single example, so we rename the folder
to common_componnets.
Previous implementation violated the RFC by having both the actual opcode and WS_FIN flag set for all fragments of a message.
Fixed by setting the opcode only for the first fragment and WS_FIN for the last one
Closes IDFGH-2938
Closes https://github.com/espressif/esp-idf/issues/4974
Device name will be reset when deinit mesh stack. If not
initializing device name during the next mesh stack init,
it will fail to set the device name when using bluedroid.
Since bt_mesh_friend_init() will only be invoked when
a device is provisioned or Provisioner is enabled, so
we add a flag to indicate if the friend functionality
is initialized in case deinit before initialization.
Compared with the latest idf, idf4.0 defines BIT(nr) in
the "esp_bit_defs.h" without the pre-condition "#ifndef BIT",
which will cause compile warning when using mesh. So we
include the header file "soc/soc.h" in "mesh_util.h",
which can bypass the definition of "BIT" in "mesh_util.h"
when compiling.
Old version of BLE Mesh has no device role storage, because
previously we only support storing mesh node info.
If the binary of the node is upgraded from old version to a
new version (support storing provisioner info), the mesh info
of the node will not be restored because mesh role does not
exist in the flash.
When using fast provisioning, the Provisioner functionality
will be enabled. Unicast addresses within the pre-allocated
range will be used for provisioning nodes. And during the
address assignment, the allocated unicast address will be
checked that if it's duplicated with other nodes addresses
and the Provisioner's own addresses.
So before starting using fast provisioning, we need to update
the Provisioner address.
Since the behavior of sending segmented messages has been
changed properly, the calculation of timeout value which
will be used when sending an acknowledged message by a
client model also needs to be updated.
Add mutex to protect some variables of the mesh segmented
messages.
Currently the timeout handler of mesh will be executed in
the btc task, while the received mesh messages will be
handled in the btu task. In case some variables are set to
NULL when handled in the btu task, meanwhile these variables
are also accessed in the timeout handler, so we add mutex to
protect these variables and related operations.
The publication context is checked for NULL in bt_mesh_model_publish()
however it was dereferenced before that. Move the assignment to
ctx.send_rel to the same place where other ctx members are set.
Split mesh_util.h into mesh_byteorder.h, mesh_compiler.h,
mesh_ffs.h and mesh_util.h based on the classification of
Zephyr, which will make further porting more clear.
The Friend queue uses the message SeqAuth to determine whether the
message is already in the queue. To facilitate this, the SeqAuth is
passed around as a pointer throughout the transport modules. In the
bt_mesh_ctl_send functions, this parameter is also exposed in the API,
but the internal usage is inconsistent and buggy. Also, no one actually
uses this parameter.
- Removes seq_auth param from bt_mesh_ctl_send, instead passing NULL
directly to the friend module, to enforce its addition to the queue.
- Makes the seq_auth pointer const throughout the friend module.
Changes the behavior of the message cache to optimize for cache
capacity. Previously, the message cache's primary function was to avoid
decrypting messages multiple times, although the cache's main function
in the spec is to avoid message rebroadcasting. Optimizing for minimal
decryption causes us to fill the network cache faster, which in turn
causes more cache misses, potentially outweighing the advantage.
Now stores src + seq in message cache instead of field hash value. This
cuts cache size in two, while including more of the sequence number than
before.
Adds messages to the cache only after the packet is successfully
decrypted. This reduces noise in the cache, and ensures that no
invalid deobfuscations are added.
Additionally, this fixes a bug where multiple calls to net_decrypt with
the same packet failed, as the message cache found its own entry from
the previous call.
Implements several changes to the transport layer segmented tx to
improve group message performance:
- Moves retransmit counter to tx context instead of per packet. As every
unacked packet is sent every retransmit, the retransmit counters would
be the same in each segment. This makes it easier to control progress.
- Delays the scheduling of the retransmit until the completion of the
last segment by adding a seg_pending counter. This is essentially the
same as the old behavior, except that the old behavior might retrigger
the sending before all segments are finished if the advertising is
slow.
- Allows the group transmits to stop as soon as all retransmits have
been exhausted, instead of timing out waiting for acks that won't
come. This allows group tx to finish without error.
- Fixes a bug where a failed TX would block IV update.
- Cancels any pending transmissions of acked segments.
- Reduces log level for several common group tx scenarios that aren't
erronous.
Backport of https://github.com/apache/mynewt-nimble/pull/724
Mesh spec 1.0.1 changes proxy disabling behavior to only affect the
relaying from proxy nodes. Previously, disabling proxy would shut down
all proxy and node activity.
Tweaks from the original commit:
- Removed redundant call to bt_mesh_adv_update() in gatt_proxy_set()
- Removed invalid ref to 4.2.11.1 in node_identity_set()
---
According to Mesh Profile Spec 1.0.1, Section 4.2.11:
"If the Proxy feature is disabled, a GATT client device can connect
over GATT to that node for configuration and control. Messages from
the GATT bearer are not relayed to the advertising bearer."
Moreover some notes have been removed from the spec compared to
version 1.0:
Mesh Profile Spec 1.0, Section 4.2.11:
"Upon transition from GATT Proxy state 0x01 to GATT Proxy state 0x00
the GATT Bearer Server shall disconnect all GATT Bearer Clients."
"The Configuration Client should turn off the Proxy state as the last
step in the configuration process."
Mesh Profile Spec 1.0, Section 4.2.11.1:
"When the GATT Proxy state is set to 0x00, the Node Identity state
for all subnets shall be set to 0x00 and shall not be changed."
Until now the choice of reliable sending (segmented messages with
acks) was implicitly dependent on the size of the payload. Add a new
member to the bt_mesh_model_pub to force using segment acks even when
the payload would fit a single unsegmented message.
When PB-GATT support has been enabled the provisioning code "borrows"
the buffer from the proxy code. However, the way that initialization
was happening the proxy buffers were initialized only after
provisioning initialization, resulting in a corrupted buffer with
buf->data pointing to NULL. Reorder the initialization calls so that
proxy is done first and provisioning only after it.
Allow models to skip a periodic publish interval by returning an error
from the publish update callback.
Previously, an error return from publish update would cancel periodic
publishing. This can't be recovered from, and as such, no valid model
implementation could return an error from this callback, and there was
no way to skip a periodic publish.
The function bt_mesh_ctl_send() used to support maximum length of
11 bytes. The segmentation complies with the BLE Mesh Standard.
The ack is disabled in case of non unicast address.
1. fix the bug for softAP update second channel wrong
2. change wifi_scan_time_t from union to struct
3. query country code wrong when policy change from manual to auto
4. fix the timeout issue of EAPOL four-way handshake
5. fix the crash issue when fetch peer after delete the cur_peer
This is not necessary for correct behaviour or to have valid sdkconfig files
(previous commit adds tests for this), but it's useful for consistency with
sdkconfig files generated by menuconfig.
As reported in https://github.com/espressif/vscode-esp-idf-extension/issues/83
And add tests for hex output formatting in all output formats.
Previously, Make & CMake outputs only formatted hex values with the 0x prefix
if they had the 0x prefix in the sdkconfig file. Now this prefix is always
applied.
Closes https://github.com/espressif/vscode-esp-idf-extension/issues/83
It was observed that after sending credentials, if the first
Wi-Fi conection attempt fails, the Wi-Fi provisioning managager
ignores subsequent Wi-Fi events. So, even if the device eventually
connects to the AP, the provisioning stays on indefinitely.
Changing a check so that subsequent events are also captured and
provisioning finishes successfully.
1. Buffers for SAE messages are not freed after the handshake.
This causes memory leak, free buffers after SAE handshake.
2. SAE global data is not freed until the next WPA3 connection
takes place, holding up heap space without reason. Free theis
data after SAE handshake is complete or event fails.
3. Update wifi lib which includes memory leak fix during BIP
encryption/decryption operations.
When fast provisioning is enabled, Provisioner shall not
ignore messages from the nodes whose addresses are not in
the provisioning database. Because other nodes which are
not provisioned by the Primary Provisioner will send node
address messages to the Primary Provisioner.
This MR imposes some determinism in the mapping rule order in the output
file. For each section, the archives are arranged alphabetically
(ascending), and the mapping rules in each archive are arranged by
increasing specificity then alphabetically (ascending). The default
rules remain the very first rule for each section.
Previously only mesh node info is supported to be stored
in flash. So when trying to reset the node, we only need
to judge if the BLE_MESH_VALID flag is set.
Currently we support storing both node & Provisioner info
in flash, when trying to erase the node info from flash,
the BLE_MESH_NODE flag will be checked. So we need to set
bt_mesh.flags to 0 when all the erase operations are done.
EAP reauth frames are dropped at various stages due to current
implementation of WPA2 ENT states and EAP SM init/deinit logic.
Route EAPOL frames based on EAP pkt type and maintain EAP SM
to facilitate EAP re-authentication process.
Added WPA3 Testcases support for -
1. Anti-Clogging Token Request support
2. Return correct status from SAE modules for invalid scenarios
3. Add PMK Caching support for WPA3
wifi lib includes fixes for below PMF Certification issues -
1. Check return status of decrypt operation. Fixes 5.3.3.1.
2. Allow PMF negotiation for WPA2-Enterprise. Fixes 5.3.3.2, 5.3.3.4.
3. Add NULL check on key before encrypting PMF, fixes crash.
1. Add changes in 4-way handshake path to allow SAE key mgmt.
2. Support for configuring WAP3 at init time, added Kconfig option.
3. Handle and propagate error conditions properly.
Under WPA3-Personal, SAE authentication is used to derive PMK
which is more secure and immune to offline dictionary attacks.
1. Add modules to generate SAE commit/confirm for the handshake
2. Add modules that build and parse SAE data in Auth frames
3. Add WPA3 association and key mgmt definitions
4. Invert y-bit while solving for ECC co-ordinate -
Once an X co-ordinate is obtained, solving for Y co-ordinate
using an elliptical curve equation results in 2 possible values,
Y and (P - Y), where p is the prime number. The co-ordinates are
used for deriving keys in SAE handshake. As par the 802.11 spec
if LSB of X is same as LSB of Y then Y is chosen, (P - Y) otherwise.
This is not what is implemented, so fix this behavior to obtain the
correct Y co-ordinate.
This change ports SAE(Simultaneous Authentication of Equals)
feature from wpa_supplicant and makes it work with mbedtls
crypto APIs. Currently only group 19 is supported. A sample
SAE handshake is included in the testcase. Other minor
changes for DH groups are also included.
1. Add CCMP, AES crypto modules for unicast protected Mgmt frames
2. Add support for computing SHA256 MIC on Bcast Mgmt frames
3. Add support for storing iGTK during 4-way handshake.
4. Provide APIs to MLME for utilizing the SW crypto modules
1. Add APIs for configuring PMF through set config.
2. Map Supplicant and Wifi Cipher types.
3. Add support for PMF negotiation while generating RSN IE.
1) Added PMK caching module from wpa_supplicant.
2) Modified wpa_sm to
a) Add entry to PMK cache when first time associated to an AP.
b) Maintain entry across the associations.
c) Clear current PMKSA when deauth happens.
d) Search for an entry when re-associating to the same AP and
set it as current PMKSA
e) Wait for msg 1/4 from AP instead of starting EAP authentication.
f) Check PMKID in msg 1 with current PMKSA/cache.
g) Use the cached PMK to complete 4-way handshake.
3) Remove config_bss callback as it was redundant and used to cause
problems for PMK caching flow.
Closes IDF-969
This was quite obviously a bad suggestion, and it seems that some
users have actually read the docs attentively enough to have followed
it. Replace be recommendation to create an alias.
Closes https://github.com/espressif/esp-idf/issues/3889
Fixes an issue with Python 3 in MSYS where it fails while trying to join
paths where one part is Unicode (default string on Python3) and the
second part are bytes (returned by the subprocess call).
Closes https://github.com/espressif/esp-idf/issues/5189
During BLE Mesh Provisioner initialization, the stack will restore
the nodes information if settings storage is enabled.
Previously when a failure happens (e.g. found the same uuid) during
the restore procedure, the information of the following nodes will
not be restored and error will be directly returned.
But this will introduce some problem with user experience, because
some newly provisioned nodes information will not be restored and
Provisioner will not be able to control those nodes.
So we change the operation here, when a failure happens during the
restore procedure, Provisioner will only ignore the information of
the current node and continue restoring other nodes information.
With this change, if a Provisioner has provisioned the maximum
number of nodes, it can still report the unprovisioned device
beacon from other nodes to the application layer. And this will
be more reasonable compared with the previous implementation.
Previously when the node array of Provisioner is full, no beacon
from unprovisioned devices will be reported, only some warning
logs will be given.
Previously only check the node address when it is assigned by the
application layer. Here we also check the address when the address
is allocated internally. And this will be useful when some mesh
internal tests are performed.
Previously the BLE_MESH_MAX_STORED_NODES option is added for
internal mesh test, which will be a little confusing for the
users to understand.
Here we remove this option, instead the BLE_MESH_MAX_PROV_NODES
will be used for all the cases. For mesh internal test, when
the test function is called to add some nodes info, the info
will be stored in the array of provisioned nodes directly.
The replay protection list of Provisioner should be at least equal
to the number of nodes with the precondition that each node contains
only one element.
The help information of replay protection list is updated, and the
maximum number of nodes for Provisioner is adjusted based on the
replay protection list size.
On Xtensa, backtrace can not recover the two most significant bits of
the address, as the window call size is encoded in these bits.
Because of this, __builtin_return_address modifies these MSBs to
match those of the callee, "fixing" the address. An unfortunate side
effect is that the zero return address, which usually terminates the
backtrace, gets converted to 0x40000000. While there is a valid
instruction at this address, its occurrence in the backtrace is
highly unlikely: this is the first instruction of WindowOverflow4
vector, and IDF apps switch VECBASE to an IRAM location very early at
startup.
Commit fc03161f updated esp_ota_get_app_elf_sha256 to store and return
X number of bytes of the hash, but the test case still expected 64 bytes.
Updated test case to use CONFIG value for expected length.
This is a regression from earlier commit related to TLSV12 which used
sha functions that are currently declared static.
Solution: Follow upstream code structure and resolve the errors.
1. add enable PSRAM 2T mode function
2. abort when himem and 2T mode are enabled meanwhile
3. set SPIRAM_2T_MODE as "n" by default, enable it when needed
1. Fix WiFi log print level
2. Optimize WiFi/BT coexist sleep
- If any data tx/rx in WiFi slice, fore wakeup next TBTT
- Increase active timeout time to coex TBTT interval to avoid sleep in WiFi slice
3. Fix esp_wifi_stop() crash
esp_wifi_stop() crashes if it's called when WiFi is not initialized.
4. Fix esp_wifi_stop() leads to memory leak
5. esp_wifi_sta_get_ap_info() returns actual values of group ciphers
It is not possible to write to Flash when the stack is located in
PSRAM, and it is not possible to write PSRAM buffers into Flash when
malloc can not be used.
and all ECDSA to be disabled if secure boot is not enabled
Previously if ECDSA disabled in config then secure_boot_signatures.c would
fail to build (whether or not secure boot was enabled).
To avoid breaking apps that might be using the signature scheme with custom OTA
without enabling secure boot signatures in config, this change just disables
this functionality if unavailable in mbedTLS config.
Possible fix for root cause of https://github.com/espressif/esp-idf/pull/3703
Closes https://github.com/espressif/esp-idf/issues/4758
Wifi_provisioning: Change service_uuid to non standard 128 bit UUID and add retries in test script (backport v4.0)
See merge request espressif/esp-idf!8163
Using the ble mesh white list test functions, a node can choose to
only receive mesh messages from a specific node and relay the
messages for it. Messages from other nodes will be ignored.
1,Reduce WiFi bin size
2,Add TX packets size check
3,Fix scan get rssi error
4,Add wifi stop check at WiFi deinit entry
5,coex adjust scheme when bt is connected status
6,Return fail when setting AP's channel is out of range
7,Fix the bug for setting channel when WiFi in NULL mode
When a handler instance is the last one in the list und unregisters
itself, the handler iterator will be invalidated by entering free'd
memory. Same applies for event base and id, if they become empty.
Merges https://github.com/espressif/esp-idf/pull/4139
Problem:
mbedtls_ctr_drbg_context was initialized in crypto_ec_point_mul. This
was okay in releases before 2.16.4 as entropy_len used to get set to
MBEDTLS_CTR_DRBG_ENTROPY_LEN in function mbedtls_ctr_drbg_seed. The
function is now changed to set the length to
MBEDTLS_CTR_DRBG_ENTROPY_LEN if previous length is 0 and hence the bug.
Solution:
Initialize mbedtls_ctr_drbg_context in crypto_ec_point_mul.
components/esp_https_ota: Changes in public requirement list for CMake and esp_ota_ops.h is included in esp_https_ota.c
bootloader_support is added in public requirements of CMake
bootloader_common.h is included in esp_https_ota.h
Closes https://github.com/espressif/esp-idf/issues/4198
Closes IDFGH-2066
Current implimentation of esp_http_ota does not perform any error-checking in the data writing phase calls `esp_ota_get_next_update_partition()` irrespetive of the received state of the image. A few additional error checking mechanism have now been added inside the esp_https_ota which returns the control in case an invalid header is received and a wrapper to the function `esp_http_client_is_complete_data_received()` of `esp_http_client` has been added.
While downloading OTA firmware, if their is a Origin Respnse Timeout or the binary is only partially downloaded, OTA failure is observed. Checking binary size can also be helpful for simple http client applications.
Closes https://github.com/espressif/esp-idf/issues/3004
The NimBLE host buffers that consume a significant amount of memory are
now allocated dynamically.
The advantage is that, the memory can be reclaimed in cases where BLE
is turned off and not required for the current boot cycle
- Add menuconfig option for NimBLE host flow control
- Include changes in `blecent` example from upstream PR!702
- add ble_hs_lock in ble_gap_unpair Upstream PR!584
- ble_hs_hci_rx_evt, upstream PR!738
Closes https://github.com/espressif/esp-idf/issues/4243
1. Bugfix for AG audio crash (change the return position)
2. Fix the error macro name and error return in hfp_hf demo
3. Fix the annotation error using UTF-8 ' (from Github)
4. Change or remove the log in SCO related code region.
5. Correct error of introduction of a function.
ledc.h includes two similar enums, ledc_clk_src_t & ledc_clk_cfg_t.
The two enums do different things but there are two similar names: LEDC_REF_TICK / LEDC_USE_REF_TICK
and LEDC_APB_CLK / LEDC_USE_APB_CLK.
Because C will accept any enum or integer value for an enum argument, there's no easy way to check
the correct enum is passed without using static analysis.
To avoid accidental errors, make the numeric values for the two similarly named enums the same.,
Noticed when looking into https://github.com/espressif/esp-idf/issues/4476
Allocation of the temporary internal buffer will now repeat until a small enough buffer can be
allocated, and only fail if less than a 256 byte block of internal RAM is free.
Adds unit test for the same, and generic test utility for creating memory pressure.
Previously would try allocate buffer of minimum size 16KB not maximum size 16KB, causing
out of memory errors for any large reads, or if less than 16KB contiguous free heap.
Also, if using legacy API and internal allocation failed then implementation would abort()
instead of returning the error to the caller.
Added test for using large buffers in PSRAM.
Closes https://github.com/espressif/esp-idf/issues/4769
Also reported on forum: https://esp32.com/viewtopic.php?f=13&t=14304&p=55972
As reported on forum:
https://esp32.com/viewtopic.php?f=2&t=13565&p=53476#p53453
split() with no arg will match any whitespace sequence not just
a single space, so takes care of case where two spaces are
inserted in the variable value.
enabled. The global interrupts will be disabled before taking current
time for scheduling further bluetooth events. Earlier disabling
interrupts was taking some time because of which WiFi events were
interrupting in current bluetooth scheduling.
Also fixes ble crash issue triggered by ble event irq miss(0x20000) (819979b9)
transport_ws can now be read multiple times in a row to read frames larger than the buffer.
Added reporting of total payload length and offset to the user in websocket_client.
Added local example test for long messages.
Closes IDF-1083
1. Add an API to set Provisioner static oob value
2. Add an API to deinit BLE Mesh stack
3. Add an API to set Provisioner unicast address
4. Add an API to provision devices with fixed address
5. Add an API to store node composition data
6. Add an API to get node with device uuid
7. Add an API to get node with unicast address
8. Add an API to delete node with device uuid
9. Add an API to delete node with unicast address
10. Add an API for Provisioner to update local AppKey
11. Add an API for Provisioner to update local NetKey
12. Support Provisioner persistent functionality
13. Fix Provisioner entering IV Update procedure
14. Fix an issue which may cause client failing to send msg
15. Use bt_mesh.flags to indicate device role
16. Remove several useless macros
17. Callback RSSI of received mesh provisioning packets
18. Modify the Provisioner disable function
19. Change some log level from debug to info
20. Add parameters to Provisioner bind AppKey completion event
21. Fix node ignoring relay messages issue
22. Support using a specific partition for BLE Mesh
23. Fix compile warning when proxy related macros are disabled
24. Clean up BLE Mesh stack included header files
25. NULL can be input if client message needs no parameters
26. Fix compile warning when BT log is disabled
27. Initilize BLE Mesh stack local variables
28. Support using PSRAM for BLE Mesh mutex, queue and task
29. Add a menuconfig option to enable using memory from PSRAM
30. Clean up sdkconfig.defaults of BLE Mesh examples
Fix an error of mesh deinit lock.
Fix mesh send returns TIMEOUT.
Increase root window close time to 120s.
Clear ie.child when the child connected or fully associated.
Update rc_rssi when roots direct/forward.
ix assoc in mesh ie wrong.
Mesh packets use AC_VI instead of AC_BE.
For SoftAP provisioning, if we start Wi-Fi connection as soon as the
apply config request is received, it has been observed that the clients
(Phone apps mostly) do not get the response for this as we switch the
Wi-Fi channel, causing a momentary disconnection.
Delaying the connection calls helps mitigate the issue.
Signed-off-by: Piyush Shah <piyush@espressif.com>
this issue is mainly exposed when using larger (4096) client key in TLS mutual auth,
since it uses multiplications > 2048 when mbedtls_mpi_mul_mpi is used in recursion,
which works only if both operands point to different location than result since
mpi_mult_mpi_overlong() called mbedtls_mpi_grow() to reallocate buffers used in previous
pointer arithmetics and thus corrupting it. Fixed by growing the mpi buffer before
calling mpi_mult_mpi_overlong()
This is not required for v4.0 (the previous zip will work for v4.0), but it will be
required if updating to a newer ESP-IDF and there's no downside to requiring it here.
1. The fault signal 3 for unit 2 is corrected to the right value
2. Now `mcpwm_fault_set_oneshot_mode` will clear the status before, no
need to reset the peripheral.
3. The capture feature relies on the prescaler, but it's only
initialized in the mcpwm_init funciton, which is used to initialize a
PWM channel. This means, the capture may not work correctly if no PWM
channel is enabled.
Now the prescaler is also updated when `mcpwm_capture_enable` is
called.
Fix following WiFi issues:
1. Fix WiFi buffer reload issue
2. Fix AMSDU decrypt issue
3. Fix some WiFi timer issues
4. Fix the crash caused by too big of association request RSN information
5. Fix the crash caused by block scan
6. Fix the bug for getting channel and bandwidth
7. Fix some Sniffer bugs
8. Fix some ESP-NOW issues
1> fix the bug when modifying the channel info of peer node
2> fix the crash when modifying peer node between unencrypted and encrypted
3> fix the bug for fetch peer
4> modify the esp_wifi_set_channel() function
5> fix the bug that the channel parameter doesn't work when adding peer node
Closes https://github.com/espressif/esp-idf/issues/2833
Closes https://github.com/espressif/esp-idf/issues/4311
When node is being reset, the init functions of each sig-defined
models will be invoked again, this will cause memory leak because
some model internal data will be allocated again.
Hence before trying to allocate memory for them, we add some check
to make sure no memory has been allocated previously.
And for client model, when the init functions are invoked again,
we will clear the list items.
Even if frame->exccause != PANIC_RSN_CACHEERR, it is possible that
the cache error interrupt status is set. For example, this may happen
due to an invalid cache access in the panic handler itself.
Check cache error interrupt status instead of frame->exccause to
decide whether to do CPU reset or digital reset.
Also remove unnecessary esp_dport_access_int_pause from
esp_cache_err_get_cpuid, since the panic handler already calls
esp_dport_access_int_abort on entry.
This allows the processes launched from idf_tools.py to use the value
of IDF_PATH. One such example is the installation of Python packages,
when requirements.txt uses IDF_PATH to refer to the esp-windows-curses
package.
Closes https://github.com/espressif/esp-idf/issues/4341
implemented workaround in idf_monitor for known usbser.sys bug which
causes changes in RTS signal to only be sent if DTR signal also changes.
Works by forcing a dummy change in DTR signal (re-assigning its current
value, as it has previously been done in the flashing tool)
Closes https://github.com/espressif/esp-idf/pull/4676
Previous version used exception handling for program control flow,
which makes for long and confusing backtrace messages if an error
occurs while parsing the CSV (as it also prints the phony error that
occurred while trying to parse as a binary, then prints
"During handling of the above exception, another exception occurred:",
then prints the real error).
Use the same heuristic that is applied in gen_esp32_part, instead.
Also, avoid printing the entire backtrace if the error is a gen_esp32_part
InputError, same as gen_esp32part does.
Found while looking into https://github.com/espressif/esp-idf/issues/4474
The upper transport layer is using big endian ordering. The
PreviousAddress field of a Friend Request message should therefore
be converted to native endianess using sys_cpu_to_be16().
If the duration to publish is roughly the same as the period, we might
end up with elapsed == period, which returns 0 and cancel the periodic
publication. Instead 1 should be returned, just like when the elapsed
time is greater than the period.
Previously the FastPeriodDivisor value was introduced to the model
publication struct. Based on the way it was grouped it seems the
intention was to fit it within the same octet as other bit fields,
but it actually makes the octet overflow by one bit. This ends up
creating another u8_t variable which in turn adds 24 bits of padding
after it.
To keep the size of the struct as compact as possible, group the flag
together with the key index, since that only requires 12 bits. Some
care is needed here, since the mesh stack does have special internal
key index values that require more than 12 bits such as
BLE_MESH_KEY_UNUSED and BLE_MESH_KEY_DEV. In this case restricting
ourselves to 12 bits is fine since the value in the model publication
struct follows 1:1 the value received in the Config Model Publication
Set message, and there the parameter is defined to be exactly 12 bits.
For Low Power node and Proxy Server, the two features
depend on BLE_MESH_NODE in Kconfig.in, here in the
stack there is no need to judge if CONFIG_BLE_MESH_NODE
is enabled.
According to Mesh Spec 3.4.5.3, a node shall implement a local
network interface. And here we limit the situation just based
on the spec, and Provisioner directly sending the msg without
passing through the local network interface.
The 16-bit format group addresses will be stored,
but we don't store (or restore) the virtual label UUIDs,
i.e. after a power cycle the 16-bit group addresses
would be meaningless.
Creates macros for determining model message lengths based on opcode,
payload length and MIC size. Also adds macro wrapping
NET_BUF_SIMPLE_DEFINE to serve the most common use case.
Re-encrypts single-segment application messages when the network seqnum
has changed, to avoid encrypting messages with different seqnums in
network and transport. This operation is only required for unsegmented
messages, as segmented messages don't need to use the same seqnum in
network.
Reinstates the special adv data for friend messages to store the app key
index.
Stores friend queue packets unencrypted, removing any out-of-order
issues caused by seqnum allocation. Also moves as much of the metadata
storage as possible into the packet, allowing us to free up some bytes
of net_buf user data for friend packets.
mdns library deinitialization destroys internal structures including action queue. if an event (e.g. network update) received
after some essential stucture is destoyed, an unexpected behavour might be introduced (e.g. crash of adding the event notification
to the action queue which was already destroyed
Closes WIFI-1485
user_context was missing from websocket event data, added. Also added the websocket client handle to the event data.
Removed unused event data struct.
Closes: IDF-1271
If an application wants to use webserver, instead of creating another
webserver instance, the wifi provisioning manager can re-use the same.
The webserver handle can be passed using this new API.
Signed-off-by: Piyush Shah <piyush@espressif.com>
Before handling status message, client models need check if the
corresponding timer has expired. And if timeout happens, the
status message will be treated as a publish message.
Latest versions of apt-get don't install as much packages by default
as older ones. This fix adds some missing packages discovered for an OS
derived from the latest stable Debian. These packages exist for older
Debian derivatives as well (tested with Ubuntu 16.04 LTS).
Closes https://github.com/espressif/esp-idf/issues/4480
* closes IDF-1135:
same namespace was used in different tests which
could lead to conflicts when re-running tests
* removes duplicated functions declarations
* correct argument order for test case macros
arg1 = MAX(int64_t arg1, uint64_t arg2) gave the wrong result, if arg1 < 0, it was presented as a larger value.
And ALARM_REG = (uin32_t)arg1. This leads to an infinite loop.
Fixed: both args are int64_t.
Closes: WIFI-1511
use CI dependencies could waste a lot bandwidth for target test jobs, as
example binary artifacts are very large. Now we will parse required
artifacts first, then use API to download required files in artifacts.
we should only load one module once.
if we load one module twice, python will regard the same object loaded in the first time and second time as different objects.
it will lead to strange errors like `isinstance(object, type_of_this_object)` return False
If we have multiple configs, we need to flash DUT with different binaries. But if we don't close DUT before apply new config, the old DUT will be reused, so new config name will not be applied.
Currently we use config and test function as filter when assign cases to one CI job. It's not necessary as the runner can run test with different configs / test functions. Now we will try to assign as many cases to a job as possible, to reduce the amount of jobs required.
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
Sometimes, libphy.a call phy_enter_critical() to protect accessing
critical sections, such like operating on I2C, but it may not effect
when both the CPU core call it. It may cause accessing I2C blocking
and cannot recover by esp_restart(), until do HW reboot.
1. Fix WiFi scan leads to poor performance of Bluetooth.
2. Improve WiFi connect success ratio when coexist with Bluetooth.
3. Check if WiFi is still connected when CSA or beacon timeout happen.
4. add coex pre init
Basically, in the portability layer, it is checked if the socket is
NON-block, and if not, then even the EAGAIN and EWOULDBLOCK errors are
diverted to a RECV error. This causes a problem for sockets with
receive timeouts set. When such a timeout is set, the condition for
NON_BLOCK isn't met and hence a hard error is returned.
Searching for EAGAIN and EWOULDBLOCK in lwip returns only 3 results
(accept, recvfrom, close) and all of them look to be genuine cases for
EWOULDBLOCK. So removing this check to make receive timeout with TLS
work.
... to reduce the number of simultaneously open files at link time.
When plugin support is enabled in the linker, BFD's (and the
corresponding file handles) are cached for the plugin to use. This
results in quite a large number of simultaneously open files, which
hits the default limit on macOS (256 files).
Since we aren't using LTO now, disable it explicitly when invoking the
linker.
Closes IDF-923
Closes IDFGH-1764
Closes https://github.com/espressif/esp-idf/issues/3989
Tests for external flash chips used to controlled by macros, one bin for
one chip. And tests are done manually. This commit refactored the test
so that all 3 chips can all run in single test.
There used to be dummy phase before out phase in common command
transactions. This corrupts the data.
The code before never actually operate (clear) the QE bit, once it finds
the QE bit is set. It's hard to check whether the QE set/disable
functions work well.
This commit:
1. Cancel the dummy phase
2. Set and clear the QE bit according to chip settings, allowing tests
for QE bits. However for some chips (Winbond for example), it's not
forced to clear the QE bit if not able to.
3. Also refactor to allow chip_generic and other chips to share the same
code to read and write qe bit; let common command and read command share
configure_host_io_mode.
4. Rename read mode to io mode since maybe we will write data with quad
mode one day.
Regression introduced in commit 79e74e5d5f
It is possible that some FreeRTOS APIs are invoked prior to
scheduler start condition (e.g. flash initialization in unicore mode).
In that condition these asserts should not trigger (scheduler state being yet to be started),
hence changes per this fix.
This commit fixes thread safety issues with configASSERT() calls
regarding the value of uxSchedulerSuspended. A false negative
occurs if a context switch to the opposite core occurs in between
the getting the core ID and the assesment.
Closes https://github.com/espressif/esp-idf/issues/4230
Previously set to wait until IP address got from tcpip_adapter, but since common example connect code blocks until both IP4 and IPv6 address received it could happen that test code might have started connection to the ASIO counter-part while ESP32 still waiting for IPv6 address
From issue:
I did some poking around in components/spiffs/project_include.cmake and
it looks like what is failing is it isn't getting an offset from
parttool.py. I think what the problem is that the parttool.py cannot
determine the offset of the storage partition as I can't run menuconfig
to set the custom partitions.
I have tested that this problem is solvable by manually setting the
custom partitions in sdkconfig and that is a workaround but there should
be at least a better error message in the configuration when it cannot
find the partition as this would also happen if the partition name was
misspelled.
Closes https://github.com/espressif/esp-idf/issues/4236
GCC prefixes underscore on symbol names on macOS, and so the set
undefined symbol `-u app_main` won't work. However, this is unecessary,
as the source file that defines it is specified directly to the
executable, and not through a static library.
Closes https://github.com/espressif/esp-idf/issues/4168
Use rev-parse to get the HEAD directory instead of manually looking for
it. This method works in the main repository, worktrees and submodules.
Closes https://github.com/espressif/esp-idf/issues/4136
This commit makes it so that BUILD_COMPONENT holds only the component,
and a new property BUILD_COMPONENT_ALIASES hold the full name of the
component.
This also removes erroneous check for duplicate components, as this can
never happen:
(1) if two components have the same name but different prefixes,
the internal names are still unique between them
(2)if two components happen to have the same name and same prefix, the
latter would override the former
Previous implementation only builds list of components included in the
build during component registration.
Since the build components is known as the requirements expansion is
ongoing, update the list here instead.
Use imported library, which does not create additional rules, but still
allows attaching arbitraty properties instead of custom targets. This
allows the targets to not appear in the target list of IDEs such as
CLion.
The workaround for PSRAM that will occupy an SPI bus is enabled only when:
1. used on 32MBit ver 0 PSRAM.
2. work at 80MHz.
The test used to only check 32MBit by the config option, but for PSRAM
on Wrover-B module seems to use a newer version of 32MBit PSRAM. So it
expects the workaround to be enabled, but actually not.
This commit split the unit test into two parts:
1. check all SPI buses are available, for all configs except psram_hspi
and psram_vspi, run on regular runners (including Wrover and Wrover-B).
a hidden option is enabled so that the compiler knows it's not building
psram_hspi or psram_vspi.
2. check the specified bus are acquired, for config psram_hspi and
psram_vspi. This only run on special runner (legacy Wrover module).
Fix following WiFi bugs:
1. Fix the bug for API_CHECK_ENABLED
2. Fix the bug that wpa2 enterprise sta can connect open AP
3. Fix TKIP/CCMP PN replay attack detection bug
4. Increase buffer number type from uint8_t to uint16_t to support more than 255 buffers
5. Avoid recalculating PMK if esp_wifi_set_config() is not called
6. Fix 5.5M signal test bug
Executing event loop `esp_event_loop_run()` with timeout causes delays in receiving events from user code. Fixed by removing the timeout to post synchronously.
closes https://github.com/espressif/esp-idf/issues/3957
tcpip_adapter_dhcpc_get_status() returns the actual internal value of dhcp client without any locking or TCP/IP stack context call, so when CONNECTED event fired with default settings it started DHCP client in TCP/IP stack context and at the same time mdns event handler checking actual DHCP state, which could still be INIT (not STARTED). Purpose of this check is to enable PCB if DHCP was stopped before setting network interface up (typically static IP settings), so the solutin is to check against TCPIP_ADAPTER_DHCP_STOPPED state
Ref. https://github.com/espressif/esp-idf/issues/1684
Also, for full RTTI support, libstdc++.a in the toolchain should be built
in both with RTTI and w/o RTTI options. Multilib with -fno-rtti
flag is used for that.
Note that this commit does not actually enable RTTI support.
The respective Kconfig option is hidden, and will be made visible when
the toolchain is updated.
This commit fixes multiple bugs caused by incorrect calls to
can_enter_reset_mode() and can_exit_reset_mode. Example test
scripts also updated to match new runners.
1. Since BLE full-scan feature for BLE mesh change the controller code cause this problem,
it cause coex semaphore take in "interrupt disable", then it may cause task schedule
and cause crash in freertos
2. Fix newlib lock ISR context and critical section check
3. Fix bt controller ISR context and critical section check
The MQTT broker URL used as default in the examples has ceased operation. All examples and documention have been updated to point to the new domain mqtt.eclipse.org.
This also required an update of the python example test scripts to use TLS 1.2
Ref. https://github.com/espressif/esp-idf/issues/1684
This change allows RTTI to be enabled in menuconfig. For full RTTI
support, libstdc++.a in the toolchain should be built without
-fno-rtti, as it is done now.
Generally if libstdc++.a is built with RTTI, applications which do not
use RTTI (and build with -fno-rtti) could still include typeinfo
structures referenced from STL classes’ vtables. This change works
around this, by moving all typeinfo structures from libstdc++.a into
a non-loadable section, placed into a non-existent memory region
starting at address 0. This can be done because when the application
is compiled with -fno-rtti, typeinfo structures are not used at run
time. This way, typeinfo structures do not contribute to the
application binary size.
If the application is build with RTTI support, typeinfo structures are
linked into the application .rodata section as usual.
Note that this commit does not actually enable RTTI support.
The respective Kconfig option is hidden, and will be made visible when
the toolchain is updated.
esp_partition_register_external did not call load_partitions, so if
it was called before any call to esp_partition_find, then the main
partition table would never be loaded. Introduce new function,
ensure_partitions_loaded, and call it both from esp_partition_find and
esp_partition_register_external.
Closes https://github.com/espressif/esp-idf/issues/4116
Replace the outdated build_installer.sh with the steps used in CI,
call build_installer.sh from CI. Move the signing part into the new
script, sign_installer.sh.
- Minor fix to remove while loop & delay while transmitting data from
host to controller.
- Add menuconfig option to enable predefined public-private keypair for
debugging (Vol. 3, Part H, 2.3.5.6.1)
- Update NimBLE submodule: Fix for #4028, #4012
Currently, the only way of exiting the idf_monitor program is to hit the CTRL+] button, if your keyboard doesn't have that key unless you hit another modifier key, it's not super trivial to exit.
This change adds the option to exit with CTRL+T (for menu) then hitting X (or CTRL+X) for exiting.
Closes https://github.com/espressif/esp-idf/pull/4167
Closes https://github.com/espressif/esp-idf/issues/4129
1. Fix high beacon and broadcast packets loss ratio of WiFi to
make MDNS test pass.
2. Improve stability of WiFi performance with a little sacrifice
of throughput.
3. Improve BLE advertising and connection performance with
dynamic priority. It sacrifices a little WiFi throughput, but
achieves balance between WiFi and Bluetooth.
- Backport of IDF MR!6122.
- Modifies `smp_utils.c` to add check on encryption key size received from
peer.
- Modifies `esp_ble_gap_set_security_param` API to add minimum encryption key
size requirement.
- NimBLE: Additional menuconfig option to enable mbedTLS instead of Tinycrypt from
NimBLE, changes `component.mk` & `CMakeLists.txt` for the same.
- Addition of NimBLE stack size configuration and misc changes.
- mbedTLS: Addition of `CMAC` and `ECP_RESTARTABLE` to mbedTLS menuconfig option and
`esp_config.h`.
- Example: Minor changes to `app_mesh.c` application.
Also put esp_flash functions into noflash region, when
ESP32_PANIC_HANDLER_IRAM and coredump are enabled. The option disables
the re-enabling of the CPU-cache when it's disabled during coredump.
This requires all the coredump functions including the flash API to be
in the D/IRAM.
During coredump, dangerous-area-checking should be disabled, and cache
disabling should be replaced by a safer version.
Dangerous-area-checking used to be in the HAL, but it seems to be more
fit to os functions. So it's moved to os functions. Interfaces are
provided to switch between os functions during coredump.
When legacy mode is used, the coredump still fails during linking
because "esp_flash_init_default_chip", "esp_flash_app_init" and
"esp_flash_default_chip " are not compiled and linked.
Instead of using ``if`` macros in callers, these functions are protected
by ``if`` macros in the header, and also not compiled in the sources.
"esp_flash_default_chip" variable is compiled with safe default value.
A lot of internally used crypto headers are publicly includeable
in user projects. This leads to bug reports when these headers
are incorrectly used or the API's are not used as intended.
Move all crypto headers into private crypto src folder, also move
crypto_ops into Supplicant to remove dependecy on crypto headers.
Closes IDF-476
If httpd_sess_trigger_close() gets called twice for the same socket,
the first httpd_sess_close() cb closes the correct socket, but the second
invocation closes the wrong socket which was just accepted and added to
the socket db. Checking for the lru counter will help identify this as the
counter is set only for requests actually served.
Commit 8cd04c80 has added a dependency of efuse component on
esp_clk_apb_freq, however there was no definition of this function in
the bootloader context.
Reported at https://esp32.com/viewtopic.php?f=13&t=12035
In testing environment, to avoid accidentally enabling flash
encryption on a device, CONFIG_SECURE_FLASH_REQUIRE_ALREADY_ENABLED
can be set. If set, the bootloader will refuse to boot if flash
encryption is not enabled, instead of enabling it.
- rely on OPENOCD_SCRIPTS variable in all cases, remove -s flags
- replace installation section with a reference to the Getting Started
guides
- add Windows-specific commands in a few cases
1. fix the bug that bb reset lock unhandled may cause assert in vPortCPUReleaseMutexIntsDisabledInternal
2. fix wifi mac reset may blocking then cause wdt timeout
3. fix bug wifi mac reset before coex init
This MR is intended to fix incorrect parsing of HTTP requests when empty header values are present.
The issue is was due to asymmetric behavior of `http_parser` library, which in case of:
non-empty header values : invokes callbacks with the pointer to the start of a value
empty header values : invokes callbacks with pointer to the start of next header or section
Since HTTP server relies on this pointer (along with length of the value) to locate the end of a value, and replace the line terminators (CRLFs) with null characters, the second case needed to be handled correctly.
Closes IDFGH-1539
Closes https://github.com/espressif/esp-idf/issues/3803
v4.0)
Some APs incorrectly advertize newer WPS2.0 config method bits
without setting bits for the corresponding older methods. This
results in failures during 8-way handshake. Add a workaround to
relax this check so that WPS handshake can proceed.
Fixes an issue where build fails if the build directory is a symlink.
The issue is caused by the rule target and the final executable dependency
not matching.
Closes https://github.com/espressif/esp-idf/issues/3626
Fixes an issue where if idf_build_process is called in a CMake
subdirectory, menuconfig looks for the mconf-idf binary in the wrong
place (in the subdirectory build dir instead of root binary dir).
1.Bugfix btdm sleep twice after wakeup request
2.Set the minimum encryption key size to be 7 octects for BR/EDR link for preventing KNOB attack
This patch is to address the CVE-2019-9506 vulnerability.
Add support for get write protection support, fixed the duplicated
set_write_protection link.
All the write_protection check in the top layer are removed. The lower
levels (chip) should ensure to disable write protection before the
operation start.
As per upgrade notes of lwIP v2.1.0:
socket API: according to the standard, SO_ERROR now only returns asynchronous errors.
All other/normal/synchronous errors are (and always were) available via 'errno'.
LWIP_SOCKET_SET_ERRNO has been removed - 'errno' is always set - and required!
Refer: https://www.nongnu.org/lwip/2_1_x/upgrading.html
Fixes https://github.com/espressif/esp-azure/issues/51
description:Before submitting a new issue, please follow the checklist and try to find the answer.
options:
- label:I have read the documentation [ESP-IDF Programming Guide](https://docs.espressif.com/projects/esp-idf/en/latest/) and the issue is not addressed there.
required:true
- label:I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
required:true
- label:I have searched the issue tracker for a similar issue and not found a similar issue.
required:true
- type:input
id:idf_version
attributes:
label:IDF version.
description:Onwhich IDF version does this issue occur on? Run `git describe --tags` to find it.
placeholder:ex. v3.2-dev-1148-g96cd3b75c
validations:
required:true
- type:dropdown
id:operating_system
attributes:
label:Operating System used.
multiple:false
options:
- Windows
- Linux
- macOS
validations:
required:true
- type:dropdown
id:build
attributes:
label:How did you build your project?
multiple:false
options:
- Command line with Make
- Command line with CMake
- Command line with idf.py
- Eclipse IDE
- CLion IDE
- VS Code IDE
- Other (please specify in More Information)
validations:
required:true
- type:dropdown
id:windows_comand_line
attributes:
label:If you are using Windows, please specify command line type.
multiple:false
options:
- PowerShell
- CMD
validations:
required:false
- type:textarea
id:expected
attributes:
label:What is the expected behavior?
description:Please provide a clear and concise description of the expected behavior.
placeholder:I expected it to...
validations:
required:true
- type:textarea
id:actual
attributes:
label:What is the actual behavior?
description:Please describe actual behavior.
placeholder:Instead it...
validations:
required:true
- type:textarea
id:steps
attributes:
label:Steps to reproduce.
description:'How do you trigger this bug? Please walk us through it step by step. If this is build bug, please attach sdkconfig file (from your project folder). Please attach your code here.'
value:|
1. Step
2. Step
3. Step
...
validations:
required:true
- type:textarea
id:debug_logs
attributes:
label:Build or installation Logs.
description:Build or installation log goes here, should contain the backtrace, as well as the reset source if it is a crash.
placeholder:Your log goes here.
render:plain
validations:
required:false
- type:textarea
id:more-info
attributes:
label:More Information.
description:Do you have any other information from investigating this?
placeholder:ex. I tried on my friend's Windows 10 PC and the command works there.
description:Before submitting a new issue, please follow the checklist and try to find the answer.
options:
- label:I have read the documentation [ESP-IDF Programming Guide](https://docs.espressif.com/projects/esp-idf/en/latest/) and the issue is not addressed there.
required:true
- label:I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
required:true
- label:I have searched the issue tracker for a similar issue and not found a similar issue.
required:true
- type:input
id:idf_version
attributes:
label:IDF version.
description:Onwhich IDF version does this issue occur on? Run `git describe --tags` to find it.
placeholder:ex. v3.2-dev-1148-g96cd3b75c
validations:
required:true
- type:input
id:chip_revision
attributes:
label:Espressif SoC revision.
description:Onwhich Espressif SoC revision does your application run on? Run `esptool chip_id` to find it.
placeholder:ex. ESP32-C3 (QFN32) (revision v0.3)
validations:
required:true
- type:dropdown
id:operating_system
attributes:
label:Operating System used.
multiple:false
options:
- Windows
- Linux
- macOS
validations:
required:true
- type:dropdown
id:build
attributes:
label:How did you build your project?
multiple:false
options:
- Command line with Make
- Command line with CMake
- Command line with idf.py
- Eclipse IDE
- CLion IDE
- VS Code IDE
- Other (please specify in More Information)
validations:
required:true
- type:dropdown
id:windows_comand_line
attributes:
label:If you are using Windows, please specify command line type.
multiple:false
options:
- PowerShell
- CMD
validations:
required:false
- type:input
id:devkit
attributes:
label:Development Kit.
description:Onwhich Development Kit does this issue occur on?
* We welcome any ideas or feature requests! It’s helpful if you can explain exactly why the feature would be useful.
* There are usually some outstanding feature requests in the [existing issues list](https://github.com/espressif/esp-idf/labels/Type%3A%20Feature%20Request), feel free to add comments to them.
* If you would like to contribute, please read the [contributions guide](https://docs.espressif.com/projects/esp-idf/en/stable/esp32/contribute/index.html).
- type:textarea
id:problem-related
attributes:
label:Is your feature request related to a problem?
description:Please provide a clear and concise description of what the problem is.
placeholder:ex. I'm always frustrated when ...
- type:textarea
id:solution
attributes:
label:Describe the solution you'd like.
description:Please provide a clear and concise description of what you want to happen.
placeholder:ex. When connecting to an Espressif chip ...
- type:textarea
id:alternatives
attributes:
label:Describe alternatives you've considered.
description:Please provide a clear and concise description of any alternative solutions or features you've considered.
placeholder:ex. Choosing other approach wouldn't work, because ...
- type:textarea
id:context
attributes:
label:Additional context.
description:Please add any other context or screenshots about the feature request here.
description:Before submitting a new issue, please follow the checklist and try to find the answer.
options:
- label:I have read the documentation [ESP-IDF Programming Guide](https://docs.espressif.com/projects/esp-idf/en/latest/) and the issue is not addressed there.
required:true
- label:I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
required:true
- label:I have searched the issue tracker for a similar issue and not found a similar issue.
The main development is done in Espressif Gitlab project.
Espressif [GitHub project espressif/esp-idf](https://github.com/espressif/esp-idf) is only a public mirror.
Therefore, all changes and updates to DangerJS files (`.github/dangerjs`) must be made via MR in the **Gitlab** repository by Espressif engineer.
When adding a new Danger rule or updating existing one, might be a good idea to test it on the developer's fork of GitHub project. This way, the new feature can be tested using a GitHub action without concern of damaging Espressif's GitHub repository.
Danger for Espressif GitHub is implemented in TypeScript. This makes the code more readable and robust than plain JavaScript.
Compilation to JavaScript code (using `tsc`) is not necessary; Danger handles TypeScript natively.
A good practice is to store each Danger rule in a separate module, and then import these modules into the main Danger file `.github/dangerjs/dangerfile.ts` (see how this is done for currently present modules when adding a new one).
If the Danger module (new check/rule) uses an external NPM module (e.g. `axios`), be sure to add this dependency to `.github/dangerjs/package.json` and also update `.github/dangerjs/package-lock.json`.
In the GitHub action, `danger` is not installed globally (nor are its dependencies) and the `npx` call is used to start the `danger` checks in CI.
## Adding new Danger rule
For local development you can use following strategy
#### Install dependencies
```sh
cd .github/dangerjs
npm install
```
(If the IDE still shows compiler/typing errors, reload the IDE window.)
#### Add new code as needed or make updates
#### Test locally
Danger rules can be tested locally (without running the GitHub action pipeline).
To do this, you have to first export the ENV variables used by Danger in the local terminal:
👋 **Hi ${authorLogin}**, thank you for your another contribution to \`espressif/esp-idf\` project!
If the change is approved and passes the tests in our internal git repository, it will appear in this public Github repository on the next sync.
***
`;
constmessageFirstContributor: string=`
***
👋 **Welcome ${authorLogin}**, thank you for your first contribution to \`espressif/esp-idf\` project!
📘 Please check [Contributions Guide](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/contribute/index.html#contributions-guide) for the contribution checklist, information regarding code and documentation style, testing and other topics.
🖊️ Please also make sure you have **read and signed** the [Contributor License Agreement for espressif/esp-idf project](https://cla-assistant.io/espressif/esp-idf).
#### Pull request review and merge process you can expect
Espressif develops the ESP-IDF project in an internal repository (Gitlab). We do welcome contributions in the form of bug reports, feature requests and pull requests via this public GitHub repository.
1. An internal issue has been created for the PR, we assign it to the relevant engineer
2. They review the PR and either approve it or ask you for changes or clarifications
3. Once the Github PR is approved, we synchronize it into our internal git repository
4. In the internal git repository we do the final review, collect approvals from core owners and make sure all the automated tests are passing
- At this point we may do some adjustments to the proposed change, or extend it by adding tests or documentation.
5. If the change is approved and passes the tests it is merged into the \`master\` branch
6. On next sync from the internal git repository merged change will appear in this public Github repository
***
`;
/**
* Check whether the author of the pull request is known or a first-time contributor, and add a message to the PR with information about the review and merge process.
- [Manifest File to Control the Build/Test apps](#manifest-file-to-control-the-buildtest-apps)
- [Grammar](#grammar)
- [Special Rules](#special-rules)
- [Upload/Download Artifacts to Internal Minio Server](#uploaddownload-artifacts-to-internal-minio-server)
- [Users Without Access to Minio](#users-without-access-to-minio)
- [Users With Access to Minio](#users-with-access-to-minio)
- [Env Vars for Minio](#env-vars-for-minio)
- [Artifacts Types and File Patterns](#artifacts-types-and-file-patterns)
- [Upload](#upload)
- [Download](#download)
## General Workflow
1. Push to a remote branch
2. Create an MR, choose related labels (not required)
3. A `detached` pipeline will be created.
4. if you push a new commit, a new pipeline will be created automatically.
## What if Expected Jobs ARE NOT Created?
1. check the file patterns
If you found a job that is not running as expected with some file changes, a git commit to improve the `pattern` will be appreciated.
2. please add MR labels to run additional tests, currently we have to do this only for `target-test` jobs, please use it as few as possible. Our final goal is to remove all the labels and let the file changes decide everything!
## MR labels for additional jobs
### Supported MR Labels
-`build`
-`build_docs`
-`component_ut[_esp32/esp32s2/...]`
-`custom_test[_esp32/esp32s2/...]`
-`docker`
-`docs`
-`docs_full`, triggers a full docs build, regardless of files changed
-`example_test[_esp32/esp32s2/...]`
-`fuzzer_test`
-`host_test`
-`integration_test`
-`iperf_stress_test`
-`macos`
-`macos_test`
-`nvs_coverage`
-`submodule`
-`windows`
There are two general labels (not recommended since these two labels will trigger a lot of jobs)
-`target_test`: includes all target for `example_test`, `custom_test`, `component_ut`, `integration_test`
-`all_test`: includes all test labels
### How to trigger a `detached` pipeline without pushing new commits?
Go to MR web page -> `Pipelines` tab -> click `Run pipeline` button.
In very rare case, this tab will not show up because no merge_request pipeline is created before. Please use web API then.
```shell
curl -X POST --header "PRIVATE-TOKEN: [YOUR PERSONAL ACCESS TOKEN]"[GITLAB_SERVER]/api/v4/projects/103/merge_requests/[MERGE_REQUEST_IID]/pipelines
```
## How to Develop With `rules.yml`?
### General Concepts
-`pattern`: Defined in an array. A GitLab job will be created if the changed files in this MR matched one of the patterns. For example:
```yaml
.patterns-python-files: &patterns-python-files
- "**/*.py"
```
- `label`: Defined in an if clause, similar as the previous bot command. A GitLab job will be created if the pipeline variables contains variables in `BOT_LABEL_xxx` format (DEPRECATED) or included in the MR labels. For example:
- `rule`: A combination of various patterns, and labels. It will be used by GitLab YAML `extends` keyword to tell GitLab in what conditions will this job be created. For example:
```yaml
.rules:build:docs:
rules:
- <<: *if-protected
- <<: *if-label-build_docs
- <<: *if-label-docs
- <<: *if-dev-push
changes: *patterns-docs
```
An example for GitLab job on how to use extends:
```yaml
check_docs_lang_sync:
extends:
- .pre_check_template
- .rules:build:docs
script:
- cd docs
- ./check_lang_folder_sync.sh
```
### How to Add a New `Job`?
check if there's a suitable `.rules:<rules-you-need>` template
1. if there is, put this in the job `extends`. All done, now you can close this window. (`extends` could be array or string)
2. if there isn't
1. check [How to Add a New `Rules` Template?](#how-to-add-a-new-rules-template), create a suitable one
2. follow step 1
### How to Add a New `Rules` Template?
check if this rule is related to `labels`, `patterns`
1. if it is, please refer to [dependencies/README.md](./dependencies/README.md) and add new rules by auto-generating
2. if it isn't, please continue reading
check if there's a suitable `.if-<if-anchor-you-need>` anchor
1. if there is, create a rule following [`rules` Template Naming Rules](#rules-template-naming-rules).For detail information, please refer to [GitLab Documentation `rules-if`](https://docs.gitlab.com/ee/ci/yaml/README.html#rulesif). Here's an example.
```yaml
.rules:patterns:python-files:
rules:
- <<: *if-protected
- <<: *if-dev-push
changes: *patterns-python-files
```
2. if there isn't
1. check [How to Add a New `if` Anchor?](#how-to-add-a-new-if-anchor), create a suitable one
2. follow step 1
### How to Add a New `if` Anchor?
Create an `if` anchor following [`if` Anchors Naming Rules](#if-anchors-naming-rules). For detailed information about how to write the condition clause, please refer to [GitLab Documentation `only/except (advanced)](https://docs.gitlab.com/ee/ci/yaml/README.html#onlyexcept-advanced). Here's an example.
```yaml
.if-schedule:&if-schedule:
if:'$CI_PIPELINE_SOURCE == "schedule"'
```
### Naming Rules
#### Common Naming Rules
if a phrase has multi words, use `_` to concatenate them.
> e.g. `regular_test`
if a name has multi phrases, use `-` to concatenate them.
> e.g. `regular_test-example_test`
#### `if` Anchors Naming Rules
- if it's a label: `.if-label-<label_name>`
- if it's a ref: `.if-ref-<ref_name>`
- if it's a branch: `.if-branch-<branch_name>`
- if it's a tag: `.if-tag-<tag_name>`
- if it's multi-type combination: `.if-ref-<release_name>-branch-<branch_name>`
a combination of `example_test`, `custom_test`, `component_ut`, `integration_test` and all targets
#### `rules` Template Naming Rules
- if it's tag related: `.rules:tag:<tag_1>-<tag_2>`
- if it's label related: `.rules:labels:<label_1>-<label_2>`
- if it's test related: `.rules:test:<test_type>`
- if it's build related: `.rules:build:<build_type>`
- if it's pattern related: `.rules:patterns:<patterns>`
## Reusable Shell Script `tools/ci/utils.sh`
It is used to put all the reusable shell scripts as small functions. If you want to set `before_script: []` for you job, now you can set `extends: .before_script_slim` instead. it will only run `source tools/ci/utils.sh`
If you're developing CI shell scripts, you can use these functions without `source` them. They're already included in all `before_script`
To run these commands in shell script locally, place `source tools/ci/utils.sh` at the very beginning.
### Functions
#### CI Job Related
-`add_gitlab_ssh_keys`
-`add_github_ssh_keys`
-`add_doc_server_ssh_keys`
-`fetch_submodules`
-`get_all_submodules`
#### Shell Script Related
-`error`: log in red color
-`warning`: log in orange color
-`info`: log in green color
-`run_cmd`: run the command with duration seconds info
-`retry_failed`: run the command with duration seconds info, retry when failed
## Manifest File to Control the Build/Test apps
`.build-test-rules.yml` file is a manifest file to control if the CI is running the build and test job or not. The Supported Targets table in `README.md` for apps would be auto-generated by `pre-commit` from the app's `.build-test-rules.yml`.
### Grammar
We're using the latest version of [idf-build-apps][idf-build-apps]. Please refer to their [documentation][manifest-doc]
- if [ -n "$LOCAL_GITLAB_HTTPS_HOST" ]; then export LOCAL_CI_REPOSITORY_URL="https://gitlab-ci-token:${CI_JOB_TOKEN}@${LOCAL_GITLAB_HTTPS_HOST}/${CI_PROJECT_PATH}"; fi
- if [ -n "$LOCAL_GIT_MIRROR" ]; then export LOCAL_CI_REPOSITORY_URL="${LOCAL_GIT_MIRROR}/${CI_PROJECT_PATH}"; fi
- echo "Using repository at $LOCAL_CI_REPOSITORY_URL"
1. Auto-generate some labels/rules we need and update them in `rules.yml`
2. Generate a dependency tree graph
## Schema
This file only used basic YAML grammar and has nothing to do with the GitLab version YAML file.
It has five custom keywords:
-`matrix`: An array of sub-arrays, used to replicate rules by formatting strings. You can use the format string everywhere, it will be formatted recursively
-`labels`: An array of `labels`.
-`patterns`: An array of `patterns`. Patterns that not included
-`included_in`: An array of other `rule` names. It indicates the `labels` and `patterns` will be included in all specified `rules` as well
-`deploy`: An array of strings, used to replicate rules by adding postfix `-<item in deploy array>`. It indicates the extra `label` used in `rules`, which will explain later.
## How to use this file to generate `rules.yml`
Let's take a complicated example to help understand the process
```yaml
"test-{0}-{1}":
matrix:
- [a, b]
- [c, d]
labels:
- "{0}-{1}"
patterns:
- "{0}"
- pattern-not-exist
included_in:
- build-{0}
```
1. expand the mapping dicts defined by `matrix`
After this step, it will turn into 4 dicts:
| key | labels | patterns | included_in |
| -------- | ------ | -------- | ----------- |
| test-a-c | a-c | a | build-a |
| test-a-d | a-d | a | build-a |
| test-b-c | b-c | b | build-b |
| test-b-d | b-d | b | build-b |
**Advanced Usage: You can overwrite a mapping by declaring it again later**, For example:
If we concatenate this part to the previous example,
```yaml
# ... The same as the previous example
test-a-c:
labels:
- overwrite
```
`rule` `test-a-c` will be turned into:
| key | labels |
| -------- | --------- |
| test-a-c | overwrite |
**Mappings with the keyword `deploy` will also replicate by adding a postfix `-<item in deploy array>` to the mapping key**
2. create rules by `included_in`
After this step, it will turn into 6 mapping dicts:
| key | labels | patterns |
| -------- | -------- | -------- |
| test-a-c | a-c | a |
| test-a-d | a-d | a |
| test-b-c | b-c | b |
| test-b-d | b-d | b |
| build-a | a-c, a-d | a |
| build-b | b-c, b-d | b |
3. replace the auto-generated region in `rules.yml` with `labels`, and `rules`. Each mapping will generate a `rule` and all the required labels. `patterns` are pre-defined in `rules.yml` and could not be generated automatically. If a mapping is using a `pattern` undefined, the `pattern` will be ignored.
- If a mapping key has postfix `-preview`, no `if-protected-xxx` clause will be added
- else if a mapping key has postfix `-production`, an `if-protected-no_label` clause will be added
- else, an `if-protected` clause will be added
## Graph
All `label` nodes are in green, `pattern` nodes are in cyan, `rule` nodes are in blue
### Requirements
There are a few extra dependencies while generating the dependency tree graph, please refer to [pygraphviz](https://github.com/pygraphviz/pygraphviz/blob/master/INSTALL.txt) documentation to install both `graphviz` and `pygraphviz`
- git diff --exit-code -- ${IDF_TARGET}/esp_efuse_table.c || { echo 'Differences found for ${IDF_TARGET} target. Please run idf.py efuse-common-table and commit the changes.'; exit 1; }
- cd ${IDF_PATH}/components/efuse/test_efuse_host
- ./efuse_tests.py
test_efuse_table_on_host_esp32:
extends:.test_efuse_table_on_host_template
test_efuse_table_on_host_esp32s2:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32s2
test_efuse_table_on_host_esp32s3:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32s3
test_efuse_table_on_host_esp32c3:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32c3
test_efuse_table_on_host_esp32h2:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32h2
test_efuse_table_on_host_esp32c6:
extends:.test_efuse_table_on_host_template
variables:
IDF_TARGET:esp32c6
test_logtrace_proc:
extends:.host_test_template
artifacts:
when:on_failure
paths:
- tools/esp_app_trace/test/logtrace/output
- tools/esp_app_trace/test/logtrace/.coverage
expire_in:1week
script:
- cd ${IDF_PATH}/tools/esp_app_trace/test/logtrace
- ./test.sh
test_sysviewtrace_proc:
extends:.host_test_template
artifacts:
when:on_failure
paths:
- tools/esp_app_trace/test/sysview/output
- tools/esp_app_trace/test/sysview/.coverage
expire_in:1week
script:
- cd ${IDF_PATH}/tools/esp_app_trace/test/sysview
- ./test.sh
test_mkdfu:
extends:.host_test_template
variables:
LC_ALL:C.UTF-8
script:
- cd ${IDF_PATH}/tools/test_mkdfu
- ./test_mkdfu.py
test_autocomplete:
extends:
- .host_test_template
artifacts:
when:on_failure
paths:
- ${IDF_PATH}/*.out
expire_in:1week
script:
- ${IDF_PATH}/tools/ci/test_autocomplete.py
test_detect_python:
extends:
- .host_test_template
script:
- cd ${IDF_PATH}
- shellcheck -s sh tools/detect_python.sh
- shellcheck -s bash tools/detect_python.sh
- shellcheck -s dash tools/detect_python.sh
- "bash -c '. tools/detect_python.sh && echo Our Python: ${ESP_PYTHON?Python is not set}'"
- "dash -c '. tools/detect_python.sh && echo Our Python: ${ESP_PYTHON?Python is not set}'"
- "zsh -c '. tools/detect_python.sh && echo Our Python: ${ESP_PYTHON?Python is not set}'"
return`You are a helpful assistant that creates suggestions for single git commit message, that user can use to describe all the changes in their merge request.
Use git diff: {mrDiff} and users current commit messages: {mrCommitMessages} to get the changes made in the commit.
Output should be git commit message following the conventional commit format.
Output only git commit message in desired format, without comments and other text.
Do not include the closing statements ("Closes https://....") in the output.
Here are the strict rules you must follow:
- Avoid mentioning any JIRA tickets (e.g., "Closes JIRA-123").
- Be specific. Don't use vague terms (e.g., "some checks", "add new ones", "few changes").
- The commit message structure should be: <type><(scope/component)>: <summary>
- Types allowed: ${allowedTypes.join(", ")}
- If 'scope/component' is used, it must start with a lowercase letter.
- The 'summary' must NOT end with a period.
- The 'summary' must be between ${minimumSummaryChars} and ${maximumSummaryChars} characters long.
If a 'body' of commit message is used:
- Each line must be no longer than ${maximumBodyLineChars} characters.
- It must be separated from the 'summary' by a blank line.
`- correct format of commit message should be: \`<type/action>(<scope/component>): <summary>\`, for example \`fix(esp32): Fixed startup timeout issue\``,
`- allowed types are: \`${allowedTypes}\``,
`- sufficiently descriptive message summary should be between ${minimumSummaryChars} to ${maximumSummaryChars} characters and start with upper case letter`,
`- avoid Jira references in commit messages (unavailable/irrelevant for our customers)`,
`- follow this [commit messages guide](${process.env.DANGER_GITLAB_HOST}/espressif/esp-idf/-/wikis/dev-proc/Commit-messages)`,
];
letdangerMessage=`\n**Some issues found for the commit messages in this MR:**\n${issuesAllCommitMessages.join(
"\n"
)}
\n***
\n**Please consider updating these commit messages** - here are some basic tips:\n${basicTips.join(
"\n"
)}
\n\`TIP:\` You can install commit-msg pre-commit hook (\`pre-commit install -t pre-commit -t commit-msg\`) to run this check when committing.
\n***
`;
if(generateAISuggestion){
// Create AI generated suggestion for git commit message based of gitDiff and current commit messages
`- closing ticket \`${ticket.record}\` seems to be in the wrong format (or inaccessible to Jira DangerBot).. The correct format is for example \`- Closes JIRA-123\`.`
);
}
// Get closing GitHub issue links from JIRA tickets
`- the GitHub issue [\`${ticket.closingGithubLink}\`](${ticket.closingGithubLink}) does not seem to exist on GitHub (referenced from JIRA ticket [\`${ticket.ticketName}\`](${ticket.jiraUIUrl}) )`
);
continue;// skip the following checks
}
// Search in commit message if there are all GitHub closing links (from Related section) for still open GH issues
:"";// if the Jira ticket has an unfilled Description, the ".description" property is missing in API response - in that case set "jiraDescription" to an empty string
constdescriptionChunk=mrDescription.match(/^([^#]*)/)[1].trim();// Extract all text before the first section header (i.e., the text before the "## Release notes")
constshortMrDescriptionThreshold=50;// Description is considered too short below this number of characters
returnwarn(`The \`Release Notes\` section seems to be missing. Please check if the section header in MR description is present and in the correct markdown format ("## Release Notes").\n\nSee [Release Notes Format Rules](${wiki_link}).`);
error_output.push('`No release notes` comment shows up when there is valid entry. Remove bullets before comments in release notes section.');
}
}else{
if(!valid_entries_found){
error_output.push('The `Release Notes` section seems to have no valid entries. Add bullets before valid entries, or add `No release notes` comment to suppress this error if you mean to have no release notes.');
}
}
if(error_output.length>0){
// Paragraphs joined by double `\n`s.
error_output=[...error_output,`See [Release Notes Format Guide](${wiki_link}).`].join('\n\n');
recordRuleExitStatus(ruleName,"Failed");
returnwarn(error_output);
}
// At this point, the rule has passed
recordRuleExitStatus(ruleName,'Passed');
};
functioncheck_entry(entry){
constentry_str=`- \`${entry}\``;
constindent=" ";
if(entry.match(/no\s+release\s+note/i)){
return[entry_str,`${indent}- \`No release notes\` comment shouldn't start with bullet.`].join('\n');
}
// Remove a leading escaping backslash of the special characters, https://www.markdownguide.org/basic-syntax/#characters-you-can-escape
return[entry_str,`${indent}- Please specify the [area] to which the change belongs (see guide). If this line is just a comment, remove the bullet.`].join('\n');
}
// area is in match[2]
constdescription=match[3].trim();
letviolations=[];
if(match[1]){
violations.push(`${indent}- Release note entry should start from the beginning of line. (Nested release note not allowed.)`);
}
if(!/^[A-Z0-9]/.test(description)){
violations.push(`${indent}- Release note statement should start with a capital letter or digit.`);
constdocsFilesMR=parseMrDocsFiles(allMrFiles);// Create single object of all doc files in MR with names, paths and groups
// Both versions (EN and CN) of document found changed in this MR
for(constfileofdocsFilesMR.bothFilesInMr){
file.contentEn=awaitgetContentFileInMR(file.fileEnPath);// Get content of English file
file.linesEn=file.contentEn.split("\n").length;// Get number of lines of English file
file.contentCn=awaitgetContentFileInMR(file.fileCnPath);// Get content of Chinese file
file.linesCn=file.contentCn.split("\n").length;// Get number of lines of English file
// Compare number of lines in both versions
if(file.linesEn!==file.linesCn){
// Check if CN file is only link to EN file
if(!regexIncludeLink.test(file.contentCn)){
// if not just a link ...
partMessages.push(
`- please synchronize the EN and CN version of \`${file.fileName}\`. [\`${file.fileEnPath}\`](${file.fileUrlRepoEN}) has ${file.linesEn} lines; [\`${file.fileCnPath}\`](${file.fileUrlRepoCN}) has ${file.linesCn} lines.`
);
}
}
}
// Only Chinese version of document found changed in this MR
for(constfileofdocsFilesMR.onlyCnFilesInMr){
partMessages.push(
`- file \`${file.fileEnPath}\` doesn't exist in this MR or in the GitLab repo. Please add \`${file.fileEnPath}\` into this MR.`
);
}
// Only English version of document found in this MR
for(constfileofdocsFilesMR.onlyEnFilesInMr){
consttargetBranch=danger.gitlab.mr.target_branch;
file.contentCn=awaitgetContentFileInGitlab(
file.fileCnPath,
targetBranch
);// Try to fetch CN file from target branch of Gitlab repository and store content
if(file.contentCn){
// File found on target branch in Gitlab repository
if(!regexIncludeLink.test(file.contentCn)){
// File on Gitlab master is NOT just an ..include:: link to ENG version
`- file \`${file.fileCnPath}\` was not updated in this MR, but found unchanged full document (not just link to EN) in target branch of Gitlab repository [\`${file.fileCnPath}\`](${file.fileUrlRepoMasterCN}). Please update \`${file.fileCnPath}\` into this MR.`
);
}
}else{
// File failed to fetch, probably does not exist in the target branch
partMessages.push(
`- file \`${file.fileCnPath}\` probably doesn't exist in this MR or in the GitLab repo. Please add \`${file.fileCnPath}\` into this MR.`
);
}
}
// Create a report with found issues with documents in MR
createReport();
// At this point, the rule has passed
recordRuleExitStatus(ruleName,'Passed');
/**
* Generates an object that represents the relationships between files in two different languages found in this MR.
*
* @param {string[]} docsFilesEN - An array of file paths for documents in English.
* @param {string[]} docsFilesCN - An array of file paths for documents in Chinese.
* @returns {Object} An object with the following properties:
* - bothFilesInMr: An array of objects that represent files that found in MR in both languages. Each object has the following properties:
* - fileName: The name of the file.
* - fileEnPath: The path to the file in English.
* - fileCnPath: The path to the file in Chinese.
* - fileUrlRepoEN: The URL link to MR branch path to the file in English.
* - fileUrlRepoCN: The URL link to MR branch path to the file in Chinese.
* - onlyCnFilesInMr: An array of objects that represent files that only found in MR in English. Each object has the following properties:
* - fileName: The name of the file.
* - fileEnPath: The path to the file in English.
* - fileCnPath: The FUTURE path to the file in Chinese.
* - fileUrlRepoEN: The URL link to MR branch path to the file in English.
* - fileUrlRepoCN: The URL link to MR branch path to the file in Chinese.
* - onlyEnFilesInMr: An array of objects that represent files that only found in MR in Chinese. Each object has the following properties:
* - fileName: The name of the file.
* - fileEnPath: The FUTURE path to the file in English.
* - fileCnPath: The path to the file in Chinese.
* - fileUrlRepoEN: The URL link to MR branch path to the file in English.
* - fileUrlRepoCN: The URL link to MR branch path to the file in Chinese.
constfileContentAfter=content.after.trim();// Trim trailing new lines
returnfileContentAfter;
}catch(error){
console.error(`Error while getting file content MR: ${error}`);
returnnull;
}
}
/**
* Creates a compiled report for found documentation issues in the current MR and alerts the Documentation team if there are any "needs translation" labels present.
*
* Report if documentation labels have been added by mistake.
*/
functioncreateReport(){
constmrLabels=danger.gitlab.mr.labels;// Get MR labels
`Please remove the \`needs translation: XX\` labels. For documents that need to translate from scratch, Doc team will translate them in the future. For the current stage, we only focus on updating exiting EN and CN translation to make them in sync.`
);
}
// Docs issues found in this MR
partMessages.sort();
letdangerMessage=`Some of the documentation files in this MR seem to have translations issues:\n${partMessages.join(
"\n"
)}\n`;
if(partMessages.length){
if(!translationLabelsPresent){
dangerMessage+=`
\nWhen synchronizing the EN and CN versions, please follow the [Documentation Code](https://docs.espressif.com/projects/esp-idf/zh_CN/latest/esp32/contribute/documenting-code.html#standardize-document-format). The total number of lines of EN and CN should be same.\n
\nIf you have difficulty in providing translation, you can contact Documentation team by adding <kbd>needs translation: CN</kbd> or <kbd>needs translation: EN</kbd> labels into this MR and retrying Danger CI job. The documentation team will be automatically notified and will help you with the translations before the merge.\n`;
recordRuleExitStatus(ruleName,"Failed");
returnwarn(dangerMessage);// no "needs translation: XX" labels in MR; report issues as warn
}else{
dangerMessage+=`\nTranslation labels <kbd>needs translation: CN</kbd> or <kbd>needs translation: EN</kbd> were added - this will automatically notify the Documentation team to help you with translation issues.`;
`The source branch name \`${sourceBranch}\` contains more than one slash. This can cause troubles with git sync. Please rename the branch.`
);
}
// Check if the source branch name contains any uppercase letters
if(sourceBranch!==sourceBranch.toLowerCase()){
recordRuleExitStatus(ruleName,"Failed");
returnwarn(
`The source branch name \`${sourceBranch}\` contains uppercase letters. This can cause troubles on case-insensitive file systems (macOS). Please use only lowercase letters.`
_All texts in italics are instructional and should be replaced by contents or removed._
## Checklist
_This entire section can be deleted if all items are checked._
* [ ] Enough information to help reviewers understand the issue, its root cause, impact, and the proposed solution
* [ ] Enough information to help reviewers understand the feature, its functional description, example, documentation, test cases, test results, feature TODO list
* [ ] The MR Title describes the change, including the component name, ie "lwip: Add support for IP over Pigeon"
* [ ] All related links, including JIRA, backport, submodule MR, are mentioned in the `Related` subsection.
* [ ] Any GitHub issues are linked inside the git commit message and corresponding release notes
* [ ] Add label for the area this MR is part of
* [ ] For documentation updates, check if label `Docs` and `needs translation:CN` or `needs translation:EN` have been added when the other language version still needs the update. Skip adding the label if the document is not yet translated.
* [ ] Check if documents requiring translation fall under get-started section. If yes, add the labels mentioned above. Then the documentation team will assign a translator for you. Please inform the translator to prepare translation once your MR is ready to merge. The translation should be included in your MR to get it merged. For more information, see documentation workflow in Wiki.
* [ ] Any necessary "needs backport" labels are added
* [ ] Check if this is a breaking change. If it is, add notes to the `Breaking change notes` subsection below
* [ ] Release note entry if this is a new public feature, or a fix for an issue introduced in the previous release.
* [ ] The commit log is clean and ready to merge.
* [ ] All relevant CI jobs have been run, i.e. jobs which cover the code changed by the MR.
---
_For issues, put enough information here to help reviewers understand the issue, its root cause, impact, and the proposed solution._
_If this issue is a regression, specify in the `Related` subsection below, in which commit or MR it was introduced. This helps reviewers to check if the backport labels are set correctly._
---
_For features, put enough information here to help reviewers understand the feature, its functional description, example, documentation, test cases, test results, feature TODO list._
---
_For other small/non-public changes, which are not expected to be in the release notes, can be mentioned here, include:_
* changes that don't apply to customers, e.g. some CI fixes, test only MR and etc
## Related
* Mention or related JIRA tasks (e.g. IDF-0000), to make sure they get updated.
* Mention submodule MR, if there is
* Mention backport(ed) MR, if there is
_Don't touch the subsection titles below, they will be parsed by scripts._
## Release notes (Mandatory)
_Changes made in this MR that should go into the **Release Notes** should be listed here. Please use **past tense** and *specify the area (see maintainers page of IDF internal wiki)*. If there is a subscope, include it and separate with slash (`/`). Minor changes can go to the descriptions above without a release notes entry._
_Write all the changes in a **list** (Start at the beginning of the line with `-` or `*`). If multiple changes are made, each of them should take a single line. If there is only one change to list, it should still be the only line of a list. If this MR does not need any release notes, write "No release notes" here without the `-` or `*`. e.g._
_If there are any breaking changes, please mention it here. Talking about (1) what is not accepted any more, (2) the alternative solution and (3) the benefits/reason. e.g._
_Please strictly follow the breaking change restriction, which means, if there is a breaking change but you are merging to non-major versions, you have to separate the breaking part out to another MR for a major version. The breaking change subsection is only accepted in MRs merging to major versions._
* [VFS/UART] Now vfs_uart_set_rts_cts accept one more instance argument, to support configuration to different ports.
# Compatibility Between ESP-IDF Releases and Revisions of Espressif SoCs
* [中文版](./COMPATIBILITY_CN.md)
Espressif keeps improving the performance of its SoCs by providing new chip revisions. However, some of the improvements require special software support. Some of the software supports are even mandatory for the chip revisions to run normally.
This document describes the compatibility between ESP-IDF releases and Espressif SoC revisions.
NOTE: This document on release branches may be out-of-date. Check the [Compatibility file on master](https://github.com/espressif/esp-idf/blob/master/COMPATIBILITY.md) for the most accurate information.
See [Compatibility Advisory for Chip Revision Numbering Scheme](https://www.espressif.com.cn/sites/default/files/advisory_downloads/AR2022-005%20Compatibility%20Advisory%20for%20Chip%20Revision%20Numbering%20%20Scheme.pdf) on the versioning of Espressif SoC revisions.
You can run `esptool chip_id` to detect the series and revision of an SoC. See [SoC Errata](https://www.espressif.com.cn/en/support/documents/technical-documents?keys=errata) for more on how to distinguish between chip revisions, and the improvements provided by chip revisions. And run `idf.py --version` to know the version of current ESP-IDF.
## ESP-IDF Support for Different Chip Revisions
The sections below show the requirements to ESP-IDF version of chip revisions. Each chip revision corresponds to specific `Recommended` and `Required` versions of ESP-IDF:
-`Recommended`: shows from which version of ESP-IDF you can make use of all the improvements of the chip revision. Running binary compiled with ESP-IDF below the `Recommended` version of a chip revision, software may not benefit from the bugfix/features provided by the chip revision. The chip will have almost the same behavior as its previous revision.
-`Required`: shows the minimum version required to run the chip revision normally. Running binary compiled below the `Required` version, the binary may have unpredictable behavior.
Though the software can make use of all the features of a chip revision, if its version is higher than the `Recommended` version of the chip, it is still recommended to use the latest bugfix version of the release branch. The latest bugfix version fixes a number of issues and helps improve product stability.
For example, if we have a chip, whose `Required`/`Recommended` version of `release/v5.1` branch is `v5.1.2`/`v5.1.4`, and the latest release on that branch is `v5.1.6`. Then the chip will not boot up with ESP-IDF `v5.1`-`v5.1.1` or will have unpredictable behavior, and application may not make use of all benefits of the chip, when running with ESP-IDF `v5.1.2` or `v5.1.3`. Though `v5.1.4` well supports the chip revision, it is still recommended to upgrade ESP-IDF to `v5.1.6`.
## What If the ESP-IDF Version Is Lower than the `Required` Version?
Latest ESP-IDF versions can prevent from downloading to, or even execute binaries on unsupported chips. ESP-IDF of versions v4.4.5+, v5.0.1+, v5.1 and above have both esptool download check and bootloader loading check against the chip revision. While ESP-IDF v4.3.5 has only esptool downloading check.
For earlier ESP-IDF versions without such checking, which is incompatible with the given chip revision, the chips running such versions will have unpredictable behavior.
Contributions to esp-idf - fixing bugs, adding features, adding documentation - are welcome. We accept contributions via `Github Pull Requests <https://help.github.com/articles/about-pull-requests/>`_.
Before Contributing
-------------------
Before sending us a Pull Request, please consider this list of points:
* Is the contribution entirely your own work, or already licensed under an Apache License 2.0 compatible Open Source License? If not then we unfortunately cannot accept it.
* Does any new code conform to the esp-idf :doc:`Style Guide <style-guide>`?
* Does the code documentation follow requirements in :doc:`documenting-code`?
* Is the code adequately commented for people to understand how it is structured?
* Is there documentation or examples that go with code contributions? There are additional suggestions for writing good examples in :idf:`examples` readme.
* Are comments and documentation written in clear English, with no spelling or grammar errors?
* Example contributions are also welcome. Please check the :doc:`creating-examples` guide for these.
* If the contribution contains multiple commits, are they grouped together into logical changes (one major change per pull request)? Are any commits with names like "fixed typo" `squashed into previous commits <http://eli.thegreenplace.net/2014/02/19/squashing-github-pull-requests-into-a-single-commit/>`_?
* If you're unsure about any of these points, please open the Pull Request anyhow and then ask us for feedback.
Pull Request Process
--------------------
After you open the Pull Request, there will probably be some discussion in the comments field of the request itself.
Once the Pull Request is ready to merge, it will first be merged into our internal git system for in-house automated testing.
If this process passes, it will be merged onto the public github repository.
Legal Part
----------
Before a contribution can be accepted, you will need to sign our :doc:`contributor-agreement`. You will be prompted for this automatically as part of the Pull Request process.
- Please read [the support policy](SUPPORT_POLICY.md) and [the documentation](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/versions.html) for more information about ESP-IDF versions.
- Please see the [End-of-Life Advisories](https://www.espressif.com/en/support/documents/advisories?keys=&field_type_of_advisory_tid%5B%5D=817) for information about ESP-IDF releases with discontinued support.
# ESP-IDF Release and SoC Compatibility
The following table shows ESP-IDF support of Espressif SoCs where ![alt text][preview] and ![alt text][supported] denote preview status and support, respectively. The preview support is usually limited in time and intended for beta versions of chips. Please use an ESP-IDF release where the desired SoC is already supported.
There are variants of revisions for a series of chips. See [Compatibility Between ESP-IDF Releases and Revisions of Espressif SoCs](https://github.com/espressif/esp-idf/blob/master/COMPATIBILITY.md) for the details of the compatibility between ESP-IDF and chip revisions.
Espressif SoCs released before 2016 (ESP8266 and ESP8285) are supported by [RTOS SDK](https://github.com/espressif/ESP8266_RTOS_SDK) instead.
ESP-IDF is the official development framework for the [ESP32](https://espressif.com/en/products/hardware/esp32/overview) chip provided for Windows, Linux and macOS.
# Developing With ESP-IDF
## Setting Up ESP-IDF
See https://idf.espressif.com/ for links to detailed instructions on how to set up the ESP-IDF depending on chip you use.
See setup guides for detailed instructions to set up the ESP-IDF:
**Note:** Each SoC series and each ESP-IDF release has its own documentation. Please see Section [Versions](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/versions.html) on how to find documentation and how to checkout specific release of ESP-IDF.
* [Getting Started Guide for the stable ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/stable/get-started/)
* [Getting Started Guide for the latest (master branch) ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/latest/get-started/)
### Non-GitHub forks
ESP-IDF uses relative locations as its submodules URLs ([.gitmodules](.gitmodules)). So they link to GitHub. If ESP-IDF is forked to a Git repository which is not on GitHub, you will need to run the script [tools/set-submodules-to-github.sh](tools/set-submodules-to-github.sh) after git clone.
The script sets absolute URLs for all submodules, allowing `git submodule update --init --recursive` to complete. If cloning ESP-IDF from GitHub, this step is not needed.
ESP-IDF uses relative locations as its submodules URLs ([.gitmodules](.gitmodules)). So they link to GitHub.
If ESP-IDF is forked to a Git repository which is not on GitHub, you will need to run the script
[tools/set-submodules-to-github.sh](tools/set-submodules-to-github.sh) after git clone.
The script sets absolute URLs for all submodules, allowing `git submodule update --init --recursive` to complete.
If cloning ESP-IDF from GitHub, this step is not needed.
## Finding a Project
@@ -64,13 +36,12 @@ See the Getting Started guide links above for a detailed setup guide. This is a
(See the Getting Started guide listed above for a full list of required steps with more details.)
* Install host build dependencies mentioned in the Getting Started guide.
* Run the install script to set up the build environment. The options include `install.bat` or `install.ps1` for Windows, and `install.sh` or `install.fish` for Unix shells.
* Run the install script to set up the build environment. The options include `install.bat` for Windows, and `install.sh` for Unix shells.
* Run the export script on Windows (`export.bat`) or source it on Unix (`source export.sh`) in every shell environment before using ESP-IDF.
## Configuring the Project
*`idf.py set-target <chip_name>` sets the target of the project to `<chip_name>`. Run `idf.py set-target` without any arguments to see a list of supported targets.
*`idf.py menuconfig` opens a text-based configuration menu where you can configure the project.
`idf.py menuconfig` opens a text-based configuration menu where you can configure the project.
## Compiling the Project
@@ -92,7 +63,7 @@ You don't need to run `idf.py build` before running `idf.py flash`, `idf.py flas
## Viewing Serial Output
The `idf.py monitor` target uses the [esp-idf-monitor tool](https://github.com/espressif/esp-idf-monitor) to display serial output from Espressif SoCs. esp-idf-monitor also has a range of features to decode crash output and interact with the device. [Check the documentation page for details](https://docs.espressif.com/projects/esp-idf/en/latest/get-started/idf-monitor.html).
The `idf.py monitor` target uses the [idf_monitor tool](https://docs.espressif.com/projects/esp-idf/en/latest/get-started/idf-monitor.html) to display serial output from the ESP32. idf_monitor also has a range of features to decode crash output and interact with the device. [Check the documentation page for details](https://docs.espressif.com/projects/esp-idf/en/latest/get-started/idf-monitor.html).
Exit the monitor by typing Ctrl-].
@@ -113,18 +84,18 @@ After the initial flash, you may just want to build and flash just your app, not
## Erasing Flash
The `idf.py flash` target does not erase the entire flash contents. However it is sometimes useful to set the device back to a totally erased state, particularly when making partition table changes or OTA app updates. To erase the entire flash, run `idf.py erase-flash`.
The `idf.py flash` target does not erase the entire flash contents. However it is sometimes useful to set the device back to a totally erased state, particularly when making partition table changes or OTA app updates. To erase the entire flash, run `idf.py erase_flash`.
This can be combined with other targets, ie `idf.py -p PORT erase-flash flash` will erase everything and then re-flash the new app, bootloader and partition table.
This can be combined with other targets, ie `idf.py -p PORT erase_flash flash` will erase everything and then re-flash the new app, bootloader and partition table.
# Resources
* Documentation for the latest version: https://docs.espressif.com/projects/esp-idf/. This documentation is built from the [docs directory](docs) of this repository.
* [Beginner's Guide to Key Concepts and Resources of ESP-IDF](https://youtu.be/J8zc8mMNKtc?feature=shared)
* The [esp32.com forum](https://esp32.com/) is a place to ask questions and find community resources.
* [Check the Issues section on github](https://github.com/espressif/esp-idf/issues) if you find a bug or have a feature request. Please check existing Issues before opening a new one.
* If you're interested in contributing to ESP-IDF, please check the [Contributions Guide](https://docs.espressif.com/projects/esp-idf/en/latest/contribute/index.html).
Please refer to https://docs.espressif.com/projects/esp-idf/en/latest/esp32/versions.html#support-periods for more details on ESP-IDF supported versions and support period policy.
## Reporting a Vulnerability
Please refer to [Espressif Security Incident Response Process](https://www.espressif.com/sites/default/files/Espressif%20Security%20Incident%20Response%20Process%20v1.0_EN.pdf) on the guidelines to report a security vulnerability. Please do **NOT** create a public GitHub issue.
@@ -3,56 +3,64 @@ The latest support policy for ESP-IDF can be found at [https://github.com/espres
Support Period Policy
=====================
* [中文版](./SUPPORT_POLICY_CN.md)
Each ESP-IDF major and minor release (V4.0, V4.1, etc) is supported for
18 months after the initial stable release date.
Each ESP-IDF major and minor release (V4.1, V4.2, etc) is supported for 30 months after the initial stable release date.
Supported means that the ESP-IDF team will continue to apply bug fixes,
security fixes, etc to the release branch on GitHub, and periodically
make new bugfix releases as needed.
Supported means that the ESP-IDF team will continue to apply bug fixes, security fixes, etc to the release branch on GitHub, and periodically make new bugfix releases as needed.
Users are encouraged to upgrade to a newer ESP-IDF release before the
support period finishes and the release becomes End of Life (EOL). It is
our policy to not continue fixing bugs in End of Life releases.
Support period is divided into "Service" and "Maintenance" period:
Pre-release versions (betas, previews, `-rc` and `-dev` versions, etc)
are not covered by any support period. Sometimes a particular feature is
marked as \"Preview\" in a release, which means it is also not covered
by the support period.
| Period | Duration | Recommended for new projects? |
The ESP-IDF Programming Guide has information about the
[different versions of ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/versions.html)
(major, minor, bugfix, etc).
During the Service period, bugfixes releases are more frequent. In some cases, support for new features may be added during the Service period (this is reserved for features which are needed to meet particular regulatory requirements or standards for new products, and which carry a very low risk of introducing regressions.)
Long Term Support releases
--------------------------
During the Maintenance period, the version is still supported but only bugfixes for high severity issues or security issues will be applied.
Some releases (starting with ESP-IDF V3.3) are designated Long Term
Support (LTS). LTS releases are supported for 30 months (2.5 years)
after the initial stable release date.
Using an “In Service” version is recommended when starting a new project.
Users are encouraged to upgrade all projects to a newer ESP-IDF release before the support period finishes and the release becomes End of Life (EOL). It is our policy to not continue fixing bugs in End of Life releases.
Pre-release versions (betas, previews, `-rc` and `-dev` versions, etc) are not covered by any support period. Sometimes a particular feature is marked as "Preview" in a release, which means it is also not covered by the support period.
ESP-IDF should be used in an up-to-date software environment. The operating system and other third-party tools should be supported by their maintainers. ESP-IDF cannot keep compatibility with unsupported third-party tools.
The ESP-IDF Programming Guide has information about the [different versions of ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/versions.html) (major, minor, bugfix, etc).
A new LTS release will be made at least every 18 months. This means
there will always be a period of at least 12 months to upgrade from the
previous LTS release to the following LTS release.
Example
-------
ESP-IDF V3.3 was released in September 2019. It was supported for 30 months until February 2022.
ESP-IDF V3.3 was released in September 2019 and is a Long Term Support
(LTS) release, meaning it will be supported for 30 months until February
2022.
- The first V3.3 release was `v3.3` in September 2019.
- The ESP-IDF team continues to backport bug fixes, security fixes, etc to the release branch `release/v3.3`.
- Periodically stable bugfix releases are created from the release branch. For example `v3.3.1`, `v3.3.2`, etc. Users are encouraged to always update to the latest bugfix release.
-V3.3 bugfix releases continue until February 2022, when all V3.3.x releases become End of Life.
- The ESP-IDF team continues to backport bug fixes, security fixes,
etc to the release branch `release/v3.3`.
-Periodically stable bugfix releases are created from the release
branch. For example `v3.3.1`, `v3.3.2`, etc. Users are encouraged to
always update to the latest bugfix release.
- V3.3 bugfix releases continue until February 2022, when all V3.3.x
releases become End of Life.
Existing Releases
-----------------
ESP-IDF release V4.1 and all newer releases will follow this support period policy. The support period for each release will be announced when the release is made.
ESP-IDF release V3.3 and all newer releases will follow this support
period policy. The support period for each release will be announced
when the release is made.
For releases made before the current support period policy was announced, the original support periods apply:
For releases made before the support period policy was announced,
the following support periods apply:
* ESP-IDF V4.0.x will be supported until October 2021
* ESP-IDF V3.3.x will be supported until February 2022
* ESP-IDF versions before V3.3 are already End of Life.
Policy History
--------------
* September 2019. This policy splits ESP-IDF releases into Standard and Long Term Support.
* July 2020. All releases from now will have the same support period, which is equal to the previous Long Term Support period. Added “In Service” period, during which versions will receive more updates.
- ESP-IDF V3.1.x and V3.2.x will both be supported until October 2020.
- ESP-IDF V3.0.9 (planned for October 2019) will be the last V3.0
bugfix release. ESP-IDF V3.0.x is End of Life from October 2019.
- ESP-IDF versions before V3.0 are already End of Life.
This document contains details about what the core components are, what they contain, and how they are organized.
## Organization
The core components are organized into two groups.
The first group (referred to as `G0`) includes `hal`, `arch` (where `arch` is either `riscv` or `xtensa` depending on the chip), `esp_rom`, `esp_common`, and `soc`. This group contains information about and provides low-level access to the underlying hardware. In the case of `esp_common`, it contains hardware-agnostic code and utilities. These components may have dependencies on each other within the group, but outside dependencies should be minimized. The reason for this approach is that these components are fundamental, and many other components may require them. Ideally, the dependency relationship only goes one way, making it easier for this group to be usable in other projects.
The second group (referred to as `G1`) operates at a higher level than the first group. `G1` includes the components `esp_hw_support`, `esp_system`, `newlib`, `spi_flash`, `freertos`, `log`, and `heap`. Like the first group, circular dependencies within this group are allowed, and these components can have dependencies on the first group. G1 components represent essential software mechanisms for building other components.
## Descriptions
The following is a short description of the components mentioned above.
### `G0` Components
#### `hal`
Contains the hardware abstraction layer and low-level operation implementations for the various peripherals. The low-level functions assign meaningful names to register-level manipulations; the hardware abstraction provide operations one level above this, grouping these low-level functions
into routines that achieve a meaningful action or state of the peripheral.
Example:
-`spi_flash_ll_set_address` is a low-level function part of the hardware abstraction `spi_flash_hal_read_block`
#### `arch`
Contains low-level architecture operations and definitions, including those for customizations (can be thought of on the same level as the low-level functions of `hal`).
This can also contain files provided by the architecture vendor.
Example:
-`xt_set_exception_handler`
-`rv_utils_intr_enable`
-`ERI_PERFMON_MAX`
#### `esp_common`
Contains hardware-agnostic definitions, constants, macros, utilities, 'pure' and/or algorithmic functions that is useable by all other components (that is, barring there being a more appropriate component to put them in).
Example:
-`BIT(nr)` and other bit manipulation utilities in the future
-`IDF_DEPRECATED(REASON)`
-`ESP_IDF_VERSION_MAJOR`
#### `soc`
Contains description of the underlying hardware: register structure, addresses, pins, capabilities, etc.
Example:
-`DR_REG_DPORT_BASE`
-`SOC_MCPWM_SUPPORTED`
-`uart_dev_s`
#### `esp_rom`
Contains headers, linker scripts, abstraction layer, patches, and other related files to ROM functions.
Example:
-`esp32.rom.eco3.ld`
-`rom/aes.h`
### `G1` Components
#### `spi_flash`
SPI flash device access implementation.
#### `freertos`
FreeRTOS port to targets supported by ESP-IDF.
#### `log`
Logging library.
#### `heap`
Heap implementation.
#### `newlib`
Some functions n the standard library are implemented here, especially those needing other `G1` components.
Example:
-`malloc` is implemented in terms of the component `heap`'s functions
-`gettimeofday` is implemented in terms of system time in `esp_system`
#### `esp_mm`
Memory management. Currently, this encompasses:
- Memory mapping for MMU supported memories
- Memory synchronisation via Cache
- Utils such as APIs to convert between virtual address and physical address
#### `esp_psram`
Contains implementation of PSRAM services
#### `esp_system`
Contains implementation of system services and controls system behavior. The implementations
here may take hardware resources and/or decide on a hardware state needed for support of a system service/feature/mechanism.
Currently, this encompasses the following, but not limited to:
- Startup and initialization
- Panic and debug
- Reset and reset reason
- Task and interrupt watchdogs
#### `esp_hw_support`
Contains implementations that provide hardware operations, arbitration, or resource sharing, especially those that
is used in the system. Unlike `esp_system`, implementations here do not decide on a hardware state or takes hardware resource, acting
merely as facilitator to hardware access. Currently, this encompasses the following, but not limited to:
if SYSVIEW_TS_SOURCE_TIMER_00 = y || SYSVIEW_TS_SOURCE_TIMER_01 = y
|| SYSVIEW_TS_SOURCE_TIMER_10 = y || SYSVIEW_TS_SOURCE_TIMER_11 = y:
timer (noflash)
else:
* (default)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
