Merge pull request #405 from euripedesrocha/bump_mqtt_cxx

bump(mqtt_cxx): 0.1.0 -> 0.2.0

.github/workflows/modem__target-test.yml

@@ -16,6 +16,10 @@ jobs:
         idf_ver: ["latest"]
         idf_target: ["esp32c3"]
         test: [ { app: pppd, path: test/target }, { app: sim800_c3, path: examples/pppos_client }, { app: sim800_cmux, path: examples/simple_cmux_client } ]
+        include:
+          - idf_ver: "latest"
+            idf_target: "esp32s2"
+            test: { app: usb_a7670_s2, path: examples/pppos_client }
     runs-on: ubuntu-20.04
     container: espressif/idf:${{ matrix.idf_ver }}
     env:
@@ -55,6 +59,10 @@ jobs:
         idf_ver: ["latest"]
         idf_target: ["esp32c3"]
         test: [ { app: pppd, path: test/target }, { app: sim800_c3, path: examples/pppos_client }, { app: sim800_cmux, path: examples/simple_cmux_client } ]
+        include:
+          - idf_ver: "latest"
+            idf_target: "esp32s2"
+            test: { app: usb_a7670_s2, path: examples/pppos_client }
     needs: build_esp_modem_tests
     runs-on:
       - self-hosted

CONTRIBUTING.md

@@ -2,6 +2,11 @@
 
 Contributions in the form of pull requests, issue reports, and feature requests are welcome!
 
+## Common Terminology:
+* [Type]: Examples include feat (for new features), fix (for bug fixes), ci (for continuous integration), bump (for version updates), etc. You can find a comprehensive list of types in .pre-commit-config.yaml on line 65.
+* [Scope]: Refers to specific sections or areas within the project, such as mdns, modem, common, console, etc. You can discover additional scopes in .pre-commit-config.yaml on line 65.
+* [Component]: This is the name of the component, and it should match the directory name where the component code is located.
+
 ## Submitting a PR
 
 - [ ] Fork the [esp-protocols repository on GitHub](https://github.com/espressif/esp-protocols) to start making your changes.
@@ -14,6 +19,37 @@ For quick merging, the contribution should be short, and concentrated on a singl
 
 Please follow the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) rule when writing commit messages.
 
+A typical commit message title template:
+
+Template:
+`[type]([scope]): Message`
+
+e.g.
+`feat(console): Added fully operational ifconfig command`
+
+
+## Creating a new component
+
+Steps:
+1. Add a file named .cz.yaml to the root of the component.
+
+The template for .cz.yaml should look like this:
+```
+---
+commitizen:
+  bump_message: 'bump([scope]): $current_version -> $new_version'
+  pre_bump_hooks: python ../../ci/changelog.py [component]
+  tag_format: [component]-v$version
+  version: 0.0.0
+  version_files:
+  - idf_component.yml
+```
+2. Run the following command to bump the version of the component:
+
+`ci/bump [component] [version] --bump-message "bump([scope]): First version [version]"`
+
+Replace [component], [version] and [scope] with the specific component name, version and scope you are working with. This command will help you bump the version of the component with the provided details.
+
 ## Release process
 
 When releasing a new component version we have to:

ci/bump

@@ -14,7 +14,7 @@ if ! cz bump --dry-run; then
 fi
 
 cz_bump_out=`cz bump --files-only "$@"`
-commit_title=`echo "${cz_bump_out}" | head -1`
+commit_title=`echo "${cz_bump_out}" | grep "bump(" | head -1`
 commit_body=`cat ../../release_notes.txt`
 
 git add -u .

ci/changelog.py

@@ -31,6 +31,11 @@ def main():
         'breaking': 'Breaking changes',
         'major': 'Major changes'
     }
+
+    res = git('show-ref', '--tags', _tty_out=False)
+    if old_ref not in res:
+        old_ref = git('rev-list', '--max-parents=0', 'HEAD', _tty_out=False).strip()
+
     brief_log = git.log('--oneline', '{}..HEAD'.format(old_ref), '--', 'components/' + component, _tty_out=False)
     for oneline in brief_log.splitlines():
         [commit, brief_msg] = oneline.split(' ', 1)
@@ -80,6 +85,11 @@ def main():
                 changelog += '- {}\n'.format(it)
     changelog += '\n'
     filename = os.path.join(root_path, 'components', component, 'CHANGELOG.md')
+    # Check if the changelog file exists.
+    if not os.path.exists(filename):
+        # File does not exist, create it
+        with open(filename, 'w') as file:
+            file.write('# Changelog\n\n')
     # insert the actual changelog to the beginning of the file, just after the title (2nd line)
     with open(filename, 'r') as orig_changelog:
         changelog_title = orig_changelog.readline(

ci/detect_component_bump

@@ -12,6 +12,7 @@ if git log -1 -m --name-only --pretty="" | grep -q components/${comp}/idf_compon
     echo "${comp}: Component version file has changed"
     version=`grep version: components/${comp}/.cz.yaml`
     version=${version#*version: }
+    version="${version//\~/_}"
 
     tag_format=`grep tag_format: components/${comp}/.cz.yaml`
     tag_format=${tag_format#*tag_format: }

components/asio/.cz.yaml

@@ -0,0 +1,7 @@
+commitizen:
+  bump_message: 'bump(asio): $current_version -> $new_version'
+  pre_bump_hooks: python ../../ci/changelog.py asio
+  tag_format: asio-v$version
+  version: 1.28.0~0
+  version_files:
+  - idf_component.yml

components/asio/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## [1.28.0~0](https://github.com/espressif/esp-protocols/commits/asio-1.28.0~0)
+
+### Features
+
+- Updates asio to 1.28 ([b310abe](https://github.com/espressif/esp-protocols/commit/b310abe))
+
+### Bug Fixes
+
+- Makes the examples to override path ([842b2b2](https://github.com/espressif/esp-protocols/commit/842b2b2))
+- Removes esp_exception and make all examples to use exceptions ([c1c9350](https://github.com/espressif/esp-protocols/commit/c1c9350))
+- removed Wno-format flag and fixed formatting warnings ([c48e442](https://github.com/espressif/esp-protocols/commit/c48e442))
+- added idf_component.yml for examples ([d273e10](https://github.com/espressif/esp-protocols/commit/d273e10))
+- Reintroduce missing CHANGELOGs ([200cbb3](https://github.com/espressif/esp-protocols/commit/200cbb3), [#235](https://github.com/espressif/esp-protocols/issues/235))
+
+### Updated
+
+- docs(common): updated component and example links ([f48d9b2](https://github.com/espressif/esp-protocols/commit/f48d9b2))
+- docs(common): improving documentation ([ca3fce0](https://github.com/espressif/esp-protocols/commit/ca3fce0))
+- Add homepage URL and License to all components ([ef3f0ee](https://github.com/espressif/esp-protocols/commit/ef3f0ee))
+- Added badges with version of components to the respective README files ([e4c8a59](https://github.com/espressif/esp-protocols/commit/e4c8a59))
+- CI: Fix ASIO example test ([6e2bb51](https://github.com/espressif/esp-protocols/commit/6e2bb51))
+- Examples: using pytest.ini from top level directory ([aee016d](https://github.com/espressif/esp-protocols/commit/aee016d))
+- CI: fixing the files to be complient with pre-commit hooks ([945bd17](https://github.com/espressif/esp-protocols/commit/945bd17))
+
 ## [1.14.1~3](https://github.com/espressif/esp-protocols/commits/f148c98)
 
 ### Updated

components/asio/idf_component.yml

@@ -1,4 +1,4 @@
-version: "1.14.1~3"
+version: "1.28.0~0"
 description: ASIO
 url: https://github.com/espressif/esp-protocols/tree/master/components/asio
 dependencies:

components/esp_modem/examples/pppos_client/sdkconfig.ci.usb_a7670_s2

@@ -0,0 +1,14 @@
+CONFIG_IDF_TARGET="esp32s2"
+# Override some defaults to enable PPP
+CONFIG_LWIP_PPP_SUPPORT=y
+CONFIG_LWIP_PPP_NOTIFY_PHASE_SUPPORT=y
+CONFIG_LWIP_PPP_PAP_SUPPORT=y
+CONFIG_LWIP_TCPIP_TASK_STACK_SIZE=4096
+CONFIG_LWIP_PPP_ENABLE_IPV6=n
+CONFIG_EXAMPLE_SERIAL_CONFIG_USB=y
+CONFIG_EXAMPLE_MODEM_DEVICE_A7670=y
+CONFIG_EXAMPLE_MODEM_PPP_APN="lpwa.vodafone.com"
+CONFIG_EXAMPLE_MODEM_PPP_AUTH_NONE=y
+CONFIG_EXAMPLE_MQTT_TEST_TOPIC="/ci/esp-modem/pppos-client"
+CONFIG_ESP_SYSTEM_PANIC_PRINT_HALT=y
+CONFIG_ESP32_PANIC_PRINT_HALT=y

components/esp_modem/src/esp_modem_dte.cpp

@@ -33,7 +33,7 @@ DTE::DTE(std::unique_ptr<Terminal> terminal):
 DTE::DTE(const esp_modem_dte_config *config, std::unique_ptr<Terminal> t, std::unique_ptr<Terminal> s):
     buffer(config->dte_buffer_size),
     cmux_term(nullptr), primary_term(std::move(t)), secondary_term(std::move(s)),
-    mode(modem_mode::UNDEF)
+    mode(modem_mode::DUAL_MODE)
 {
     set_command_callbacks();
 }
@@ -41,7 +41,7 @@ DTE::DTE(const esp_modem_dte_config *config, std::unique_ptr<Terminal> t, std::u
 DTE::DTE(std::unique_ptr<Terminal> t, std::unique_ptr<Terminal> s):
     buffer(dte_default_buffer_size),
     cmux_term(nullptr), primary_term(std::move(t)), secondary_term(std::move(s)),
-    mode(modem_mode::UNDEF)
+    mode(modem_mode::DUAL_MODE)
 {
     set_command_callbacks();
 }
@@ -78,9 +78,9 @@ void DTE::set_command_callbacks()
             if (command_cb.process_line(data, 0, len)) {
                 return true;
             }
-            // cannot inflate and the processing hasn't finishes in the first iteration -> report a failure
-            command_cb.give_up();
-            return true;
+            // cannot inflate and the processing hasn't finishes in the first iteration, but continue
+            // (will post next fragments to the parser, since we might be just missing a last token or OK
+            return false;
 #endif
         }
         // data == nullptr: Terminals which request users to read current data

components/esp_mqtt_cxx/.cz.yaml

@@ -0,0 +1,8 @@
+---
+commitizen:
+  bump_message: 'bump(mqtt_cxx): $current_version -> $new_version'
+  pre_bump_hooks: python ../../ci/changelog.py esp_mqtt_cxx
+  tag_format: mqtt_cxx-v$version
+  version: 0.2.0
+  version_files:
+  - idf_component.yml

components/esp_mqtt_cxx/CHANGELOG.md

@@ -1,5 +1,26 @@
 # Changelog
 
+## [0.2.0](https://github.com/espressif/esp-protocols/commits/mqtt_cxx-v0.2.0)
+
+### Features
+
+- configure client authentication via certificate/key or secure element ([ee09ff4](https://github.com/espressif/esp-protocols/commit/ee09ff4))
+
+### Bug Fixes
+
+- removed Wno-format flag and fixed formatting warnings ([c48e442](https://github.com/espressif/esp-protocols/commit/c48e442))
+- Removes meaningless printf on subscribed handler (#358) ([bac742d](https://github.com/espressif/esp-protocols/commit/bac742d), [#356](https://github.com/espressif/esp-protocols/issues/356))
+- Removes unused type for configuration ([839c79d](https://github.com/espressif/esp-protocols/commit/839c79d))
+- added idf_component.yml for examples ([d273e10](https://github.com/espressif/esp-protocols/commit/d273e10))
+- Reintroduce missing CHANGELOGs ([200cbb3](https://github.com/espressif/esp-protocols/commit/200cbb3), [#235](https://github.com/espressif/esp-protocols/issues/235))
+
+### Updated
+
+- docs(common): updated component and example links ([f48d9b2](https://github.com/espressif/esp-protocols/commit/f48d9b2))
+- docs(esp_mqtt_cxx): updated documentation and deployment file ([a547ec8](https://github.com/espressif/esp-protocols/commit/a547ec8))
+- docs(common): improving documentation ([ca3fce0](https://github.com/espressif/esp-protocols/commit/ca3fce0))
+- Add homepage URL and License to all components ([ef3f0ee](https://github.com/espressif/esp-protocols/commit/ef3f0ee))
+
 ## [0.1.0](https://github.com/espressif/esp-protocols/commits/1407dfc)
 
 ### Updated

components/esp_mqtt_cxx/idf_component.yml

@@ -1,4 +1,4 @@
-version: "0.1.0"
+version: "0.2.0"
 description: esp mqtt cxx
 url: https://github.com/espressif/esp-protocols/tree/master/components/esp_mqtt_cxx
 dependencies:

components/esp_websocket_client/.gitignore

@@ -1,93 +0,0 @@
-.config
-*.o
-*.pyc
-
-# gtags
-GTAGS
-GRTAGS
-GPATH
-
-# emacs
-.dir-locals.el
-
-# emacs temp file suffixes
-*~
-.#*
-\#*#
-
-# eclipse setting
-.settings
-
-# MacOS directory files
-.DS_Store
-
-# Components Unit Test Apps files
-components/**/build
-components/**/sdkconfig
-components/**/sdkconfig.old
-
-# Example project files
-examples/**/sdkconfig
-examples/**/sdkconfig.old
-examples/**/build
-
-# Doc build artifacts
-docs/_build/
-docs/doxygen_sqlite3.db
-
-# Downloaded font files
-docs/_static/DejaVuSans.ttf
-docs/_static/NotoSansSC-Regular.otf
-
-# Unit test app files
-tools/unit-test-app/sdkconfig
-tools/unit-test-app/sdkconfig.old
-tools/unit-test-app/build
-tools/unit-test-app/builds
-tools/unit-test-app/output
-tools/unit-test-app/test_configs
-
-# Unit Test CMake compile log folder
-log_ut_cmake
-
-# test application build files
-test/**/build
-test/**/sdkconfig
-test/**/sdkconfig.old
-
-# IDF monitor test
-tools/test_idf_monitor/outputs
-
-TEST_LOGS
-
-# gcov coverage reports
-*.gcda
-*.gcno
-coverage.info
-coverage_report/
-
-test_multi_heap_host
-
-# VS Code Settings
-.vscode/
-
-# VIM files
-*.swp
-*.swo
-
-# Clion IDE CMake build & config
-.idea/
-cmake-build-*/
-
-# Results for the checking of the Python coding style and static analysis
-.mypy_cache
-flake8_output.txt
-
-# ESP-IDF default build directory name
-build
-
-# lock files for examples and components
-dependencies.lock
-
-# ignore generated docs
-docs/html

components/esp_websocket_client/esp_websocket_client.c

@@ -792,6 +792,51 @@ esp_err_t esp_websocket_client_set_headers(esp_websocket_client_handle_t client,
     return ret;
 }
 
+esp_err_t esp_websocket_client_append_header(esp_websocket_client_handle_t client, const char *key, const char *value)
+{
+    // Validate the input parameters
+    if (client == NULL || key == NULL || value == NULL) {
+        return ESP_ERR_INVALID_ARG;
+    }
+
+    websocket_config_storage_t *cfg = client->config;
+
+    // Calculate the length for "key: value\r\n"
+    size_t len = strlen(key) + strlen(value) + 5; // 5 accounts for ": \r\n" and null-terminator
+
+    // If no previous headers exist
+    if (cfg->headers == NULL) {
+        cfg->headers = (char *)malloc(len);
+        if (cfg->headers == NULL) {
+            ESP_LOGE(TAG, "Failed to allocate...");
+            return ESP_ERR_NO_MEM;
+        }
+        snprintf(cfg->headers, len, "%s: %s\r\n", key, value);
+        return ESP_OK;
+    }
+
+    // Extend the current headers to accommodate the new key-value pair
+    size_t current_len = strlen(cfg->headers);
+    size_t new_len = current_len + len;
+
+    // Allocate memory for new headers
+    char *new_headers = (char *)malloc(new_len);
+    if (new_headers == NULL) {
+        ESP_LOGE(TAG, "Failed to allocate...");
+        return ESP_ERR_NO_MEM;
+    }
+
+    // Copy old headers and append the new header
+    strcpy(new_headers, cfg->headers);
+    snprintf(new_headers + current_len, len, "%s: %s\r\n", key, value);
+
+    // Free old headers and assign the new header pointer to cfg->headers
+    free(cfg->headers);
+    cfg->headers = new_headers;
+
+    return ESP_OK;
+}
+
 static esp_err_t esp_websocket_client_recv(esp_websocket_client_handle_t client)
 {
     int rlen;
@@ -1155,6 +1200,7 @@ int esp_websocket_client_send_fin(esp_websocket_client_handle_t client, TickType
 
 int esp_websocket_client_send_with_opcode(esp_websocket_client_handle_t client, ws_transport_opcodes_t opcode, const uint8_t *data, int len, TickType_t timeout)
 {
+    int ret = ESP_OK;
     if (client == NULL || len < 0 || (data == NULL && len > 0)) {
         ESP_LOGE(TAG, "Invalid arguments");
         return ESP_FAIL;
@@ -1167,24 +1213,28 @@ int esp_websocket_client_send_with_opcode(esp_websocket_client_handle_t client,
 
     if (!esp_websocket_client_is_connected(client)) {
         ESP_LOGE(TAG, "Websocket client is not connected");
+        ret = ESP_FAIL;
         goto unlock_and_return;
     }
 
     if (client->transport == NULL) {
         ESP_LOGE(TAG, "Invalid transport");
+        ret = ESP_FAIL;
         goto unlock_and_return;
     }
     if (esp_websocket_new_buf(client, true) != ESP_OK) {
         ESP_LOGE(TAG, "Failed to setup tx buffer");
+        ret = ESP_FAIL;
         goto unlock_and_return;
     }
     if (esp_websocket_client_send_with_exact_opcode(client, opcode | WS_TRANSPORT_OPCODES_FIN, data, len, timeout) != true) {
         ESP_LOGE(TAG, "Failed to send the buffer");
+        ret = ESP_FAIL;
         goto unlock_and_return;
     }
 unlock_and_return:
     xSemaphoreGiveRecursive(client->lock);
-    return ESP_FAIL;
+    return ret;
 }
 
 bool esp_websocket_client_is_connected(esp_websocket_client_handle_t client)

components/esp_websocket_client/examples/linux/main/main.c

@@ -79,6 +79,9 @@ static void websocket_app_start(void)
     ESP_LOGI(TAG, "Connecting to %s...", websocket_cfg.uri);
 
     esp_websocket_client_handle_t client = esp_websocket_client_init(&websocket_cfg);
+    // This call demonstrates adding another header; it's called to increase code coverage
+    esp_websocket_client_append_header(client, "HeaderNewKey", "value");
+
     esp_websocket_register_events(client, WEBSOCKET_EVENT_ANY, websocket_event_handler, (void *)client);
 
     esp_websocket_client_start(client);
@@ -93,6 +96,14 @@ static void websocket_app_start(void)
         vTaskDelay(1000 / portTICK_PERIOD_MS);
     }
 
+    ESP_LOGI(TAG, "Sending fragmented message");
+    vTaskDelay(1000 / portTICK_PERIOD_MS);
+    memset(data, 'a', sizeof(data));
+    esp_websocket_client_send_text_partial(client, data, sizeof(data), portMAX_DELAY);
+    memset(data, 'b', sizeof(data));
+    esp_websocket_client_send_cont_msg(client, data, sizeof(data), portMAX_DELAY);
+    esp_websocket_client_send_fin(client, portMAX_DELAY);
+
     esp_websocket_client_destroy(client);
 }
 

components/esp_websocket_client/include/esp_websocket_client.h

@@ -158,13 +158,31 @@ esp_err_t esp_websocket_client_set_uri(esp_websocket_client_handle_t client, con
  * @brief      Set additional websocket headers for the client, when performing this behavior, the headers will replace the old ones
  * @pre        Must stop the WebSocket client before set headers if the client has been connected
  *
- * @param[in]  client  The client
- * @param headers  additional header strings each terminated with \r\n
+ *    - This API should be used after the WebSocket client connection has succeeded (i.e., once the transport layer is initialized).
+ *    - If you wish to set or append headers before the WebSocket client connection is established(before handshake), consider the following options:
+ *       1. Input headers directly into the config options, terminating each item with [CR][LF]. This approach will replace any previous headers.
+ *          Example: websocket_cfg.headers = "Sec-WebSocket-Key: my_key\r\nPassword: my_pass\r\n";
+ *       2. Use the `esp_websocket_client_append_header` API to append a single header to the current set.
+ *
+ * @param[in]  client  The WebSocket client handle
+ * @param[in]  headers Additional header strings each terminated with [CR][LF]
  *
  * @return     esp_err_t
  */
 esp_err_t esp_websocket_client_set_headers(esp_websocket_client_handle_t client, const char *headers);
 
+/**
+ * @brief      Appends a new key-value pair to the headers of a WebSocket client.
+ * @pre        Ensure that this function is called before starting the WebSocket client.
+ *
+ * @param[in]  client  The WebSocket client handle
+ * @param[in]  key     The header key to append
+ * @param[in]  value   The associated value for the given key
+ *
+ * @return     esp_err_t
+ */
+esp_err_t esp_websocket_client_append_header(esp_websocket_client_handle_t client, const char *key, const char *value);
+
 /**
  * @brief      Open the WebSocket connection
  *

pytest.ini

@@ -23,3 +23,6 @@ log_file = test.log
 log_file_level = INFO
 log_file_format = %(asctime)s %(levelname)s %(message)s
 log_file_date_format = %Y-%m-%d %H:%M:%S
+
+# Directory patterns to avoid for recursion
+norecursedirs = "managed_components"