Commit Graph

551 Commits

Author SHA1 Message Date
7e19495fb0 Add esp_tls option to skip server verification 2023-10-24 16:31:07 +02:00
dffabb067f Merge branch 'feature/esp_mqtt_as_managed_component' into 'master'
Moves mqtt build files to esp-mqtt and adds a component file

See merge request espressif/esp-mqtt!149
2022-12-17 01:46:11 +08:00
159b1638b2 Reorganize mqtt build structure
- Integrate build definitions from idf
- Changes CMakeLists to allow import
- Added host test from idf
- Added test code from idf
2022-12-16 08:52:54 +01:00
f27cb47c0f [MQTT] Simplifies MQTT CMakeLists file
- Uses mocks from tools instead of creating them.
- Move host based definition to test code.
2022-12-16 08:38:48 +01:00
ee9b52b7d2 build system: re-add -Wno-format as private flag for some components 2022-12-16 08:38:48 +01:00
01d0f34654 test: Add mqtt5 unit-test 2022-12-16 08:38:48 +01:00
0b98c63247 mqtt: Add mqtt5 Kconfig 2022-12-16 08:38:48 +01:00
94ecca2073 esp_netif/lwip: Fix deps cycles to "lwip -> esp_netif -> phy-drivers"
Fix dependency tree so that lwip doesn't depend on any specific network
interface component.
Network interface drivers shall depend on esp_netif.
esp_netif shall depend on lwip (but not on any specific interface
driver) -- it optionally depends on vfs and esp_eth (need ethernet
header for L2/bridge mode)
2022-12-16 08:38:48 +01:00
1e598b071b [MQTT] - Updates esp_mqtt configuration struct
- Layered config struct
- Fix examples.
2022-12-16 08:38:48 +01:00
d4ee31f627 docs: changes docs supported targets tables 2022-12-16 08:38:48 +01:00
411ea65ec7 [MQTT] - Adds esp-timer as dependency and change version.
- Current time is now from esp_timer.
2022-12-16 08:38:48 +01:00
bc30b9a8e3 mqtt: Update tests to start with valid transport 2022-12-16 08:38:48 +01:00
d372bbf09e mqtt: Fix client_enqueue(len=0), Improve transport memory
* Update submodule: git log --oneline 64f88b4412ea6649dbf207a07370c2617160d044..a21c387d6280260894981c22494017c893d505b9

Detailed description of the changes:
* mqtt_client: Added checks for cleanly-closed connection and timeout
  - See merge request espressif/esp-mqtt!118
  - Added checks for cleanly-closed connection and timeout (espressif/esp-mqtt@e05d873)
* mqtt_client: fix esp_mqtt_client_enqueue for len=0 (GitHub PR)
  - See merge request espressif/esp-mqtt!135
  - mqtt_client: fix esp_mqtt_client_enqueue for len=0 (espressif/esp-mqtt@69b6493)
* Fix implicit malloc/free inclusion
  - See merge request espressif/esp-mqtt!134
  - See commit https://github.com/espressif/esp-mqtt/commit/9299f54
* feat(mqtt): Optimize mqtt transport list and remove unused transport
  - See merge request espressif/esp-mqtt!131
  - See commit https://github.com/espressif/esp-mqtt/commit/647e0ef
* Fix WSS default port selection through menuconfig.
  - See merge request espressif/esp-mqtt!132
  - - Closes https://github.com/espressif/esp-mqtt/issues/223
  - See commit https://github.com/espressif/esp-mqtt/commit/f6caaff
2022-12-16 08:38:48 +01:00
610db7b934 mqtt: Fix and add mqtt host test to CI 2022-12-16 08:38:48 +01:00
62c5545f2d esp-netif: Make dependency on esp-eth optional
* esp-netif to optionally depend on esp-eth (only for l2tap config)
* esp_eth.h now includes the original ethernet header and the
ethernet-netif glue layer
* Updated examples and test to explicitely use esp-eth dependency if
needed
2022-12-16 08:38:48 +01:00
b80d4d468c tools: Increase the minimal supported CMake version to 3.16
This updates the minimal supported version of CMake to 3.16, which in turn enables us to use more CMake features and have a cleaner build system.
This is the version that provides most new features and also the one we use in our latest docker image for CI.
2022-12-16 08:38:48 +01:00
5539c83c0a mqtt: Fix incorrect reads on error; Update ping processing
* Update submodule: git log --oneline 4874bab35659bd2301e65fd849f6559d7380d4f1..64f88b4412ea6649dbf207a07370c2617160d044

Detailed description of the changes:
* Fix documentation of config struct
  - See merge request espressif/esp-mqtt!129
  - See commit https://github.com/espressif/esp-mqtt/commit/e31834c
* Changes the moment we update keepalive_tick.
  - See merge request espressif/esp-mqtt!127
  - See commit https://github.com/espressif/esp-mqtt/commit/2c2e6f3
* MQTT: Fix signature matching for some integer values
  - See merge request espressif/esp-mqtt!128
  - Closes https://github.com/espressif/esp-idf/issues/8482
  - MQTT: Fix signature matching for integer values (espressif/esp-mqtt@6b794e4)
* Make the mqtt submodule logging tags lower case
  - See merge request espressif/esp-mqtt!122
  - See commit https://github.com/espressif/esp-mqtt/commit/fb3184c

Closes https://github.com/espressif/esp-idf/issues/8482
Closes https://github.com/espressif/esp-idf/issues/8550
2022-12-16 08:38:48 +01:00
0a507f1205 build system: remove lwip from common requirements
lwip was added to common requirements list to provide "sys/socket.h"
header to all components without additional requirements specified.

However, lwip pulls in a lot of dependencies on other components.
This commit removes lwip from common requirements to reduce the number
of components in G1-only apps.

To compensate for this removal, the following changes are made:
- newlib (which is a common requirement) has a public dependency on
  lwip if lwip is present in the build. This ensures that sys/socket.h
  is available as long as lwip component is included into the build.
- lwip is now a public requirement of esp-tls since esp_tls.h includes
  sys/socket.h header.
- lwip is now a public requirement o esp_http_client because
  sys/socket.h is included from esp_http_client.h
- lwip is now a private requirement of esp_wifi for "smartconfig_ack"
- lwip is now a private requirement of mqtt for socket functions
- lwip is now a public requirement of tcp_transport because
  esp_transport_tcp.h includes sys/socket.h header.
- mbedtls checks if lwip component is present in the build. If yes,
  net_sockets.c is added to the build, along with the dependency on
  lwip. Previously lwip was a public requirement of mbedtls
  unconditionally.

system/g1_components test app is updated to reflect the changes

Default public dependencies of a component before and after this
change, except common requirements:

- esp_timer (public dependency of freertos)
- bootloader_support (public dependency of esp_hw_support)
- vfs (public dependency of lwip)
- esp_wifi (public dependency of lwip)
- esp_event (public dependency of esp_wifi)
- esp_netif (public dependency of esp_event)
- esp_eth (public dependency of esp_netif)
- esp_phy (public dependency of esp_wifi)

After:

- esp_timer (public dependency of freertos)
- bootloader_support (public dependency of esp_hw_support)

Altogether, the following components have been always added as
public requirements to all other components, and are not added now
([breaking-change]):

- lwip
- vfs
- esp_wifi
- esp_event
- esp_netif
- esp_eth
- esp_phy

Application components now need to explicitly declare dependencies on
these components.
2022-12-16 08:38:48 +01:00
77e41479d7 components: correct printf() placeholder for time_t
Using C99 %jd, https://en.cppreference.com/w/c/chrono/time_t
2022-12-16 08:38:48 +01:00
a88eaebad9 esp-mqtt: fix warnings 2022-12-16 08:38:48 +01:00
8dc8e5ae0b kconfig: Changed default values of bool configs
- Some bool configs were using default values true and false,
  instead of y and n.
2022-12-16 08:38:48 +01:00
933da9efed Remove legacy system event framework. 2022-12-16 08:38:48 +01:00
a840bcf274 esp_hw_support/esp_system: Re-evaluate header inclusions and include directories
This commit updates the visibility of various header files and cleans up
some unnecessary inclusions. Also, this commit removes certain header
include paths which were maintained for backward compatibility.
2022-12-16 08:38:48 +01:00
fcd548daab freertos: Remove legacy hooks
This commit refactors the legacy hooks as follows:

- Removed CONFIG_FREERTOS_LEGACY_HOOKS
- FreeRTOS hooks are now enabled via:
    - CONFIG_FREERTOS_USE_IDLE_HOOK
    - CONFIG_FREERTOS_USE_TICK_HOOK
- Update IDF hooks documentation
2022-12-16 08:38:48 +01:00
23e5c5a02c esp_eth: Update esp32's EMAC API to decouple driver and vendor config 2022-12-16 08:38:48 +01:00
71d9236275 mqtt: Fix sending log data; dup flag after queue
* Fix sending mqtt message longer than Tx buffer size
* Fix enqueue API to send data with correct dup flag
* Update submodule: git log --oneline b86d42c130ac64a916ce6cf299d99f9756692394..985078affa8a2d2b56b87c8e6455252850f895c6

Detailed description of the changes:
* Isolate IDF env for v4.4 and v5.0(master)
  - See merge request espressif/esp-mqtt!111
  - ci: Isolate IDF env for v4.4 and v5.0(master) (espressif/esp-mqtt@4c5a65c)
* Client: Remove usage of legacy FreeRTOS types
  - See merge request espressif/esp-mqtt!120
  - ci: Fix build issues with IDF-4.4 against master (espressif/esp-mqtt@c28a56d)
  - See commit https://github.com/espressif/esp-mqtt/commit/6ef98d6
* mqtt_client: Fix mqtt send long data error
  - See merge request espressif/esp-mqtt!117
  - Closes https://github.com/espressif/esp-mqtt/issues/214
  - See commit https://github.com/espressif/esp-mqtt/commit/372b323
* Client: Fix use esp_mqtt_client_enqueue API to send data, data dup flag will be set 1
  - See merge request espressif/esp-mqtt!116
  - See commit https://github.com/espressif/esp-mqtt/commit/df8dc92
2022-12-16 08:38:48 +01:00
f8cb9f7286 mqtt: Add docs on MQTT_CUSTOM_OUTBOX implentation
Closes https://github.com/espressif/esp-mqtt/issues/217
2022-12-16 08:38:48 +01:00
47ac29b1e2 mqtt: replace nghttp with http_parser references 2022-12-16 08:38:48 +01:00
174c6781de refactor (test_utils)!: separate file for memory check functions
Memory check (leaks and heap tracing) functions for unit tests
now have a separate file now and are renamed for more consistency.

BREAKING CHANGE: renamed memory check function names which may be used
                 in unit tests outside IDF.
2022-12-16 08:38:48 +01:00
1766fd4d5b MQTT: Fix disconnect/reconnect, Adds empty client id, ...
Updated MQTT submodule: git log --oneline 89894bd0c611b1392967fe90bb49682eba858383...b86d42c130ac64a916ce6cf299d99f9756692394
* Added support for client with empty id
* Fixed user requested disconnect to correctly send MQTT disconnection message
* Fixed reconnection request with disabled autoreconnect
* Added qos and dup flags to data events
* Added Support for suback massage payload in mqtt events

Detailed description of the changes (89894bd0c6...b86d42c130):
* Adds the possibility of client with empty id
    - See merge request esp-mqtt!114
    - esp_mqtt commit 09287a1156
    - esp_mqtt commit 1fd50dd2cb
    - Related IDF-4124
* Client: Disconnect/Reconnect improvements
    - See merge request esp-mqtt!113
    - esp_mqtt commit 3f05b1aedc
    - esp_mqtt commit 86e40f8615
    - Related https://github.com/espressif/esp-mqtt/issues/206
    - Related https://github.com/espressif/esp-mqtt/issues/208
* Events: Support qos/dup flags and suback payload in mqtt events (GitHub PR)
    - See merge request esp-mqtt!112
    - esp_mqtt commit de47f1c341
    - esp_mqtt commit e1d5a9402f
    - Related https://github.com/espressif/esp-mqtt/issues/200
    - Related https://github.com/espressif/esp-mqtt/pull/203
2022-12-16 08:38:48 +01:00
4afe71b132 MQTT: Add more tests 2022-12-16 08:38:48 +01:00
74882b18f0 MQTT: Add more unit tests with actual broker 2022-12-16 08:38:48 +01:00
ff8a93585f freertos: Add portTRY_ENTRY_CRITICAL() and deprecate legacy spinlock fucntions
Add TRY_ENTRY_CRITICAL() API to all for timeouts when entering critical sections.
The following port API were added:
- portTRY_ENTER_CRITICAL()
- portTRY_ENTER_CRITICAL_ISR()
- portTRY_ENTER_CRITICAL_SAFE()

Deprecated legacy spinlock API in favor of spinlock.h. The following API were deprecated:
- vPortCPUInitializeMutex()
- vPortCPUAcquireMutex()
- vPortCPUAcquireMutexTimeout()
- vPortCPUReleaseMutex()

Other Changes:
- Added portMUX_INITIALIZE() to replace vPortCPUInitializeMutex()
- The assembly of the critical section functions ends up being about 50 instructions longer,
  thus the spinlock test pass threshold had to be increased to account for the extra runtime.

Closes https://github.com/espressif/esp-idf/issues/5301
2022-12-16 08:38:48 +01:00
b4474ac16d Build & config: Remove leftover files from the unsupported "make" build system 2022-12-16 08:38:48 +01:00
3c764f0fc1 freertos: update freertos folder structure to match upstream
The following changes have been made:
1. All FreeRTOS kernel source files are now placed in the
   freertos/FreeRTOS-Kernel folder to match with the upstream folder structure.
2. All kernel include files are now placed in freertos/FreeRTOS-Kernel/include.
3. All port files are now placed in freertos/FreeRTOS-Kernel/portable.
4. All additions/customizations are placed in freertos/esp_additions.
5. All other miscellaneous files (README, License files etc.) are moved to
   freertos/FreeRTOS-Kernel folder to match with the upstream.
6. Updated esp-cryptoauthlib to latest commit to resolve FreeRTOS
   include dependencies.

Signed-off-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>
2022-12-16 08:38:48 +01:00
c03353333b MQTT: Fix build if CONFIG_WS_TRANSPORT is not set and client cleanups
Updated MQTT submodule: git log --oneline f10321a53b53a146ee299cfecc320b89c0cf6611...89894bd0c611b1392967fe90bb49682eba858383
* Fix build issue if cert bundle disabled
* Fix build issue if ws transport disabled
* Add config to set retransmission interval

Detailed description of the changes (f10321a53b...89894bd0c6):
* Added config option to configure custom retransmission interval
    - See merge request espressif/esp-mqtt!110
    - esp_mqtt commit 1b009c840b
    - Related https://github.com/espressif/esp-mqtt/pull/199
* Configuration conflicts were verified, logged but not reported to the user.
    - See merge request espressif/esp-mqtt!102
    - esp_mqtt commit 88f4b8ed50
* Fixed build issue if cert bundle disabled
    - See merge request espressif/esp-mqtt!109
    - esp_mqtt commit 4a89bff610
    - esp_mqtt commit 1b71980575
    - esp_mqtt commit 5b3c81ee48
    - Related https://github.com/espressif/esp-mqtt/pull/198
    - Related https://github.com/espressif/esp-idf/issues/7535
* Removes unnecessary outbox_cleanup
    - This function were used on old version to handle QoS 2 messages. It's no longer necessary in current implementation.
    - See merge request espressif/esp-mqtt!108
    - esp_mqtt commit ebef896b00
* Fixed return an error when fail to enqueue
    - The functions that enqueue messages didn't had a return for the handler, with this the error was only logged instead of returned whichmay cause the user to have an ID for a message that was not published.
    - See merge request espressif/esp-mqtt!103
    - esp_mqtt commit 7471177fe7
* CI: Use qemu image based on esp-env:v4.4-1
    - Replaced the temporary qemu image with the official qemu:v4.4-1-20210517 derived from the esp-env:v4.4-1 test environment
    - See merge request espressif/esp-mqtt!107
    - esp_mqtt commit 231b274962

Closes https://github.com/espressif/esp-idf/issues/7535
2022-12-16 08:38:48 +01:00
382caedbbb upgrade freertos version and history 2022-12-16 08:38:48 +01:00
93158d5d18 mqtt: Adds host tests for mqtt client 2022-12-16 08:38:48 +01:00
a8b0f474ec MQTT: Support for certificate bundle; Client clean-up
* Closes https://github.com/espressif/esp-idf/issues/7040
* Merges https://github.com/espressif/esp-idf/pull/7041
* Update submodule: git log --oneline 9fdf7b61385633075d5c3b84803f2dd0578d7869..f10321a53b53a146ee299cfecc320b89c0cf6611

Detailed description of the changes:
* Remove unnecessary parentheses
  - esp-mqtt commit: db13533904
  - esp-mqtt MR: espressif/esp-mqtt!101
* outbox: Cleanup all items when connection closes
  - esp-mqtt commit: 1a94efe8b9
  - esp-mqtt MR: espressif/esp-mqtt!104
* Outbox: Removes unnecessary calls to outbox_set_pending
  - esp-mqtt commit: 36a3990404
  - esp-mqtt MR: espressif/esp-mqtt!105
* MQTT: Makes abort connection function void.
  - esp-mqtt commit: 67553aba45
  - esp-mqtt MR: espressif/esp-mqtt!106
* Client: Removes unused defines
  - esp-mqtt commit: eec6f0e17d
  - esp-mqtt MR: espressif/esp-mqtt!100
  - Closes https://github.com/espressif/esp-mqtt/issues/194
* Config: Added support for certificate bundle
  - esp-mqtt commit: 06157be118
  - esp-mqtt MR: espressif/esp-mqtt!98
  - Closes https://github.com/espressif/esp-mqtt/issues/190
* Config: Adds missing field at config struct (path field)
  - esp-mqtt commit: 5b27d1896e
  - esp-mqtt MR: espressif/esp-mqtt!96
* Client: Add support for partial transport writes
  - esp-mqtt commit: d8c9c7a9e7
  - esp-mqtt MR: espressif/esp-mqtt!99
  - Partially addresses https://github.com/espressif/esp-idf/issues/6940
* Client: Add support for Retain flag in messages posted by events
  - esp-mqtt commit: a00a3134c6
  - esp-mqtt MR: espressif/esp-mqtt!99
  - Closes https://github.com/espressif/esp-mqtt/issues/193
* esp-mqtt: Added nullchecks for public APIs
  - esp-mqtt commit: 2f57985c0b
  - esp-mqtt MR: espressif/esp-mqtt!94
  - Closes https://github.com/espressif/esp-mqtt/issues/185
* esp-mqtt: Reduce the includes used in all files
  - esp-mqtt commit: 87fcce72c9
  - esp-mqtt MR: espressif/esp-mqtt!93
* mqtt_outbox: Use STAILQ_FOREACH for outbox_delete_single_expired
  - esp-mqtt commit: ff8e64839a
  - esp-mqtt MR: espressif/esp-mqtt!97
  - Merges https://github.com/espressif/esp-mqtt/pull/187
* Client: Add optimize for depend on ssl
  - esp-mqtt commit: 8f3cac8c36
  - esp-mqtt MR: espressif/esp-mqtt!95
2022-12-16 08:38:48 +01:00
0fda7d7062 transport: Add CONFI_WS_TRANSPORT for optimize the code size 2022-12-16 08:38:48 +01:00
128be3f402 mqtt: Moved weekend tests to test apps 2022-12-16 08:38:48 +01:00
f2f30baec0 esp-mqtt: Remove __FILE__ macro from error logs 2022-12-16 08:38:48 +01:00
f072c9f753 style: format python files with isort and double-quote-string-fixer 2022-12-16 08:38:48 +01:00
3883bde0b0 ci: Add MQTT publish test to standard test apps 2022-12-16 08:38:48 +01:00
40fdf32793 ci: Extend the MQTT weekend test to check mqtt-enqueue api 2022-12-16 08:38:48 +01:00
2c3363e23a MQTT: Add new config modes (outbox related, incremental id) 2022-12-16 08:38:48 +01:00
6756e62387 MQTT: Update submodule reference to support new config modes
* Queueing publish messages to outbox when the client is not connected (default=off -> messages are queued if disconnected)
* Use of incremental msg-id instead of random id (default=off -> msg-id uses platform_random())
* Posting a new event-id if a queued message gets deleted from the outbox (default=off -> events are not posted)

Detailed description of included `esp-mqtt` changes
(da850b0add1e71b3659bfac5d797cc834dc3e89b...9ea804e0ab5368d5ab53ae2301a5fec9d1f12f1a)
* mqtt: Remove unused mqtt_header_state_t
  - esp-mqtt commit: b7158a4aea
  - esp-mqtt MR: espressif/esp-mqtt!84
  - Merges https://github.com/espressif/esp-mqtt/pull/180
* Cleanup public include dirs
  - esp-mqtt commit: f65d5d05db
  - esp-mqtt MR: espressif/esp-mqtt!85
* Config: Add a new option to use incremental message id
  - esp-mqtt commit: 8bb4a26f46
  - esp-mqtt MR: espressif/esp-mqtt!85
  - Closes https://github.com/espressif/esp-mqtt/issues/176
* Publish: Add new API to enqueue qos>0 messages
  - esp-mqtt commit: dc7fd5c0b1
  - esp-mqtt MR: espressif/esp-mqtt!85
  - Closes https://github.com/espressif/esp-mqtt/issues/155
* Config: Add a new option to disable publishing when disconnected
  - esp-mqtt commit: f44dcb1c26
  - esp-mqtt MR: espressif/esp-mqtt!85
  - Related https://github.com/espressif/esp-mqtt/issues/177
* Events: Add new event to report deleted messages from outbox
  - esp-mqtt commit: 2e35d4d4d5
  - esp-mqtt MR: espressif/esp-mqtt!85
* Publish: Allow for qos=0 messages to be stored using esp_mqtt_client_enqueue()
  - esp-mqtt commit: e2de0f3e3e
  - esp-mqtt MR: espressif/esp-mqtt!85
2022-12-16 08:38:48 +01:00
10bba73d21 MQTT: Update submodule reference: config, error handle, minor fixes
Updates esp-mqtt reference to include fixes below related mainly to:
* configuration update (disable keepalive, network timeout)
* minor fixes (size_t for all sizes, unbalanced lock, api for
  outbox-size)
* extended error handle to include socket's errno

Closes https://github.com/espressif/esp-idf/issues/5906

Config: Added config value to disable keepalive mechanism
esp-mqtt commit: 8562437c8a
Related https://github.com/espressif/esp-mqtt/issues/179

Added esp_mqtt_client_get_outbox_size API
esp-mqtt commit: 0a1d9d0300
Related https://github.com/espressif/esp-mqtt/pull/178

mqtt_outbox: Removed unused retry_count field from outbox_item_t
esp-mqtt commit: 673086e13a

config: Fixed typo for configuring OUTBOX_EXPIRED_TIMEOUT_MS
esp-mqtt commit: 259baaec96

Fixed missing MQTT_API_UNLOCK in esp_mqtt_client_stop error path
esp-mqtt commit: 845c2a3a1e
Related https://github.com/espressif/esp-mqtt/issues/173
Related https://github.com/espressif/esp-mqtt/pull/174

Extended mqtt error handle to capture transport's socket errno (IDF
v4.3+)
esp-mqtt commit: 23c8e1ecf5

Config: Added configuration value to set network timeout
esp-mqtt commit: a03228ac46
Related https://github.com/espressif/esp-mqtt/pull/166

Used size_t for all lengths to allow for other architectures
esp-mqtt commit: b9db8d9020
2022-12-16 08:38:48 +01:00
b342444746 riscv: Add new arch-level component
Changes come from internal branch commit a6723fc
2022-12-16 08:38:48 +01:00
5facf78673 MQTT: Restore default MQTT_OUTBOX_EXPIRED_TIMEOUT_MS to 30 sec
The OUTBOX_EXPIRED_TIMEOUT_MS was 30*1000 in original esp-mqtt code.
Don't change the default OUTBOX_EXPIRED_TIMEOUT_MS without good reason,
which may has impact on memory usage for existing applications.

Fixes: e931168ed695 ("MQTT: add configurable msg expired timeout")
Signed-off-by: Axel Lin <axel.lin@gmail.com>
2022-12-16 08:38:48 +01:00