bump(modem): 0.1.28 -> 1.0.0

Also added .cz config for modem components

.github/ISSUE_TEMPLATE/bug-report.yml

@@ -0,0 +1,51 @@
+name: Installation or build bug report
+description: Report installation or build bugs
+labels: ['Type: Bug']
+body:
+  - type: checkboxes
+    id: checklist
+    attributes:
+      label: Answers checklist.
+      description: Before submitting a new issue, please follow the checklist and try to find the answer.
+      options:
+        - label: I have read the documentation for [esp-protocols components](https://espressif.github.io/esp-protocols/) and the issue is not addressed there.
+          required: true
+        - label: I have updated my esp-protocols branch (master or release) to the latest version and checked that the issue is present there.
+          required: true
+        - label: I have searched the issue tracker for a similar issue and not found a similar issue.
+          required: true
+  - type: dropdown
+    id: component
+    attributes:
+      label: What component are you using? If you choose Other, provide details in More Information.
+      multiple: false
+      options:
+        - ASIO port
+        - esp_modem
+        - esp_websocket_client
+        - mDNS
+        - Other
+    validations:
+      required: true
+  - type: input
+    id: component_id
+    attributes:
+      label: component version
+      description: Which component version does this issue occur on? (see your `idf_component.yml`)
+      placeholder: ex. 1.0.0
+  - type: input
+    id: idf_version
+    attributes:
+      label: IDF version.
+      description: On which IDF version does this issue occur on? Run `git describe --tags` to find it.
+      placeholder: ex. v3.2-dev-1148-g96cd3b75c
+    validations:
+      required: true
+  - type: textarea
+    id: more-info
+    attributes:
+      label: More Information.
+      description: Do you have any other information from investigating this?
+      placeholder: ex. I tried on my friend's Windows 10 PC and the command works there.
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,14 @@
+blank_issues_enabled: false
+contact_links:
+  - name: esp-protocols documentation
+    url: https://espressif.github.io/esp-protocols/
+    about: Documenation for esp-protocols components.
+  - name: ESP-IDF Programming Guide
+    url: https://docs.espressif.com/projects/esp-idf/en/latest/
+    about: Documentation for configuring and using ESP-IDF.
+  - name: Espressif documentation page
+    url: https://www.espressif.com/en/support/download/documents
+    about: Hardware documentation (datasheets, Technical Reference Manual, etc).
+  - name: Forum
+    url: https://esp32.com
+    about: For questions about using ESP-IDF and/or ESP32 series chips. Please submit all questions starting "How do I..." here.

.github/ISSUE_TEMPLATE/feature-request.yml

@@ -0,0 +1,33 @@
+name: Feature request
+description: Suggest an idea or new component for this project.
+labels: ['Type: Feature Request']
+body:
+  - type: markdown
+    attributes:
+      value: |
+        * We welcome any ideas or feature requests! It’s helpful if you can explain exactly why the feature would be useful.
+        * There are usually some outstanding feature requests in the [existing issues list](https://github.com/espressif/esp-protocols/labels/Type%3A%20Feature%20Request), feel free to add comments to them.
+  - type: textarea
+    id: problem-related
+    attributes:
+      label: Is your feature request related to a problem?
+      description: Please provide a clear and concise description of what the problem is.
+      placeholder: ex. I'm always frustrated when ...
+  - type: textarea
+    id: solution
+    attributes:
+      label: Describe the solution you'd like.
+      description: Please provide a clear and concise description of what you want to happen.
+      placeholder: ex. When building my application ...
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Describe alternatives you've considered.
+      description: Please provide a clear and concise description of any alternative solutions or features you've considered.
+      placeholder: ex. Choosing other approach wouldn't work, because ...
+  - type: textarea
+    id: context
+    attributes:
+      label: Additional context.
+      description: Please add any other context or screenshots about the feature request here.
+      placeholder: ex. This would work only when ...

.github/ISSUE_TEMPLATE/other-issue.yml

@@ -0,0 +1,23 @@
+name: General issue report
+description: File an issue report
+body:
+  - type: checkboxes
+    id: checklist
+    attributes:
+      label: Answers checklist.
+      description: Before submitting a new issue, please follow the checklist and try to find the answer.
+      options:
+        - label: I have read the documentation for [esp-protocols components](https://espressif.github.io/esp-protocols/) and the issue is not addressed there.
+          required: true
+        - label: I have updated my esp-protocols branch (master or release) to the latest version and checked that the issue is present there.
+          required: true
+        - label: I have searched the issue tracker for a similar issue and not found a similar issue.
+          required: true
+  - type: textarea
+    id: issue
+    attributes:
+      label: General issue report
+      description: Your issue report goes here.
+      placeholder: ex. How do I...
+    validations:
+      required: true

.github/workflows/asio__build-target-test.yml

@@ -0,0 +1,100 @@
+name: "asio: build/target-tests"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    types: [opened, synchronize, reopened, labeled]
+
+jobs:
+  build_asio:
+    if: contains(github.event.pull_request.labels.*.name, 'asio') || github.event_name == 'push'
+    name: Build
+    strategy:
+      matrix:
+        idf_ver: ["latest", "release-v5.0"]
+        idf_target: ["esp32", "esp32s2"]
+        example: ["asio_chat", "async_request", "socks4", "ssl_client_server", "tcp_echo_server", "udp_echo_server"]
+    runs-on: ubuntu-20.04
+    container: espressif/idf:${{ matrix.idf_ver }}
+    env:
+      TEST_DIR: components/asio/examples
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
+        working-directory: ${{ env.TEST_DIR }}/${{ matrix.example }}
+        env:
+          IDF_TARGET: ${{ matrix.idf_target }}
+        shell: bash
+        run: |
+          . ${IDF_PATH}/export.sh
+          test -f sdkconfig.ci && cat sdkconfig.ci >> sdkconfig.defaults || echo "No sdkconfig.ci"
+          idf.py set-target ${{ matrix.idf_target }}
+          idf.py build
+      - name: Merge binaries with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }} for ${{ matrix.example }}
+        working-directory: ${{ env.TEST_DIR }}/${{ matrix.example }}/build
+        env:
+          IDF_TARGET: ${{ matrix.idf_target }}
+        shell: bash
+        run: |
+          . ${IDF_PATH}/export.sh
+          esptool.py --chip ${{ matrix.idf_target }} merge_bin --fill-flash-size 4MB -o flash_image.bin @flash_args
+      - uses: actions/upload-artifact@v3
+        with:
+          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.example }}
+          path: |
+            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/bootloader/bootloader.bin
+            ${{ env.TEST_DIR }}/${{ matrix.example }}/build//partition_table/partition-table.bin
+            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/*.bin
+            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/*.elf
+            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/flasher_args.json
+            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/config/sdkconfig.h
+            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/config/sdkconfig.json
+          if-no-files-found: error
+
+  target_tests_asio:
+    # Skip running on forks since it won't have access to secrets
+    if: |
+      github.repository == 'espressif/esp-protocols' &&
+      ( contains(github.event.pull_request.labels.*.name, 'asio') || github.event_name == 'push' )
+    name: Target tests
+    strategy:
+      matrix:
+        idf_ver: ["latest", "release-v5.0"]
+        idf_target: ["esp32"]
+        example: ["asio_chat", "tcp_echo_server", "udp_echo_server", "ssl_client_server"]
+    needs: build_asio
+    runs-on:
+      - self-hosted
+      - ESP32-ETHERNET-KIT
+    env:
+      TEST_DIR: components/asio/examples
+    steps:
+      - name: Clear repository
+        run: sudo rm -fr $GITHUB_WORKSPACE && mkdir $GITHUB_WORKSPACE
+      - uses: actions/checkout@v3
+      - uses: actions/download-artifact@v3
+        with:
+          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.example }}
+          path: ${{ env.TEST_DIR }}/${{ matrix.example }}/build
+      - name: Install Python packages
+        env:
+          PIP_EXTRA_INDEX_URL: "https://www.piwheels.org/simple"
+        run: |
+            sudo apt-get install -y dnsutils
+      - name: Download Example Test to target ${{ matrix.config }}
+        run: |
+            python -m esptool --chip ${{ matrix.idf_target }} write_flash 0x0 ${{ env.TEST_DIR }}/${{ matrix.example }}/build/flash_image.bin
+      - name: Run Example Test ${{ matrix.example }} on target
+        working-directory: ${{ env.TEST_DIR }}/${{ matrix.example }}
+        run: |
+            python -m pytest --log-cli-level DEBUG --junit-xml=./examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.config }}.xml --target=${{ matrix.idf_target }}
+      - uses: actions/upload-artifact@v3
+        if: always()
+        with:
+          name: examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.example }}
+          path: ${{ env.TEST_DIR }}/${{ matrix.example }}/*.xml

.github/workflows/gcov_analyzer.yml

@@ -1,92 +0,0 @@
-name: Code Coverage Analyzer
-
-on: [push, pull_request]
-
-jobs:
-  gcovr_analyzer_esp_modem:
-    name: Run gcovr on esp modem host test
-    runs-on: ubuntu-22.04
-    container: espressif/idf:release-v4.3
-    env:
-      lwip: lwip-2.1.2
-      lwip_contrib: contrib-2.1.0
-      lwip_uri: http://download.savannah.nongnu.org/releases/lwip
-      COMP_DIR: esp-protocols/components/esp_modem
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@v3
-        with:
-          path: esp-protocols
-          persist-credentials: false
-      - name: Build and Test
-        shell: bash
-        run: |
-          apt-get update
-          apt-get update && apt-get install -y gcc-8 g++-8 python3-pip
-          apt-get install -y rsync
-          update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 800 --slave /usr/bin/g++ g++ /usr/bin/g++-8
-          export DEBIAN_FRONTEND=noninteractive
-          export LWIP_PATH=`pwd`/${{ env.lwip }}
-          export LWIP_CONTRIB_PATH=`pwd`/${{ env.lwip_contrib }}
-          . ${IDF_PATH}/export.sh
-          ${{ env.COMP_DIR }}/test/host_test/env.sh $lwip $lwip_uri $lwip_contrib
-          cd $GITHUB_WORKSPACE/${{ env.COMP_DIR }}/test/host_test
-          cat sdkconfig.ci.coverage >> sdkconfig.defaults
-          idf.py build
-          ./build/host_modem_test.elf
-      - name: Run gcovr
-        shell: bash
-        run: |
-          python -m pip install gcovr
-          cd $GITHUB_WORKSPACE/${{ env.COMP_DIR }}
-          gcov-8 `find . -name "esp_modem*gcda" -printf '%h\n' | head -n 1`/*
-          gcovr --gcov-ignore-parse-errors -g -k -r . --html index.html  -x esp_modem_coverage.xml
-          mkdir docs_gcovr
-          cp $GITHUB_WORKSPACE/${{ env.COMP_DIR }}/index.html docs_gcovr
-          touch docs_gcovr/.nojekyll
-
-      - name: Code Coverage Summary Report
-        uses: irongut/CodeCoverageSummary@v1.3.0
-        with:
-          filename: esp-protocols/**/esp_modem_coverage.xml
-          badge: true
-          fail_below_min: false
-          format: markdown
-          hide_branch_rate: false
-          hide_complexity: false
-          indicators: true
-          output: both
-          thresholds: '60 80'
-
-      - name: Write to Job Summary
-        run: cat code-coverage-results.md >> $GITHUB_STEP_SUMMARY
-
-      - name: Upload artifacts
-        uses: actions/upload-artifact@v3
-        if: always()
-        with:
-          name: docs_gcovr
-          path: |
-            ${{ env.COMP_DIR }}/docs_gcovr
-          if-no-files-found: error
-
-#  show_report_data:
-#    name: Publish-Results
-#    if: github.ref == 'refs/heads/master' || github.repository != 'espressif/esp-protocols'
-#    runs-on: ubuntu-22.04
-#    needs: gcovr_analyzer_esp_modem
-#    steps:
-#      - name: Checkout 🛎️
-#        uses: actions/checkout@v3
-#        with:
-#          persist-credentials: false
-#      - name: Download Artifacts
-#        uses: actions/download-artifact@v1
-#        with:
-#          name: docs_gcovr
-#
-#      - name: Deploy generated docs
-#        uses: JamesIves/github-pages-deploy-action@v4
-#        with:
-#          branch: gh-pages
-#          folder: 'docs_gcovr'

.github/workflows/host-test.yml

@@ -1,40 +0,0 @@
-name: Host test
-
-on: [push, pull_request]
-
-jobs:
-  host_test:
-    name: Build and Test on Host
-    runs-on: ubuntu-20.04
-    container: espressif/idf:release-v4.3
-    env:
-      lwip: lwip-2.1.2
-      lwip_contrib: contrib-2.1.0
-      lwip_uri: http://download.savannah.nongnu.org/releases/lwip
-      COMP_DIR: esp-protocols/components/esp_modem
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@master
-        with:
-          path: esp-protocols
-
-      - name: Build and Test
-        shell: bash
-        run: |
-          apt-get update && apt-get install -y gcc-8 g++-8
-          update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 800 --slave /usr/bin/g++ g++ /usr/bin/g++-8
-          export LWIP_PATH=`pwd`/${{ env.lwip }}
-          export LWIP_CONTRIB_PATH=`pwd`/${{ env.lwip_contrib }}
-          . ${IDF_PATH}/export.sh
-          $GITHUB_WORKSPACE/${{ env.COMP_DIR }}/test/host_test/env.sh $lwip $lwip_uri $lwip_contrib
-          cd $GITHUB_WORKSPACE/esp-protocols/components/esp_modem/examples/linux_modem
-          idf.py build
-          cd $GITHUB_WORKSPACE/esp-protocols/components/esp_modem/test/host_test
-          idf.py build
-          ./build/host_modem_test.elf -r junit -o junit.xml
-
-      - name: Publish Results
-        uses: EnricoMi/publish-unit-test-result-action@v1
-        if: always()
-        with:
-          files: esp-protocols/components/esp_modem/test/host_test/junit.xml

.github/workflows/mdns__build-target-test.yml

@@ -0,0 +1,173 @@
+name: "mdns: build/target-tests"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    types: [opened, synchronize, reopened, labeled]
+
+jobs:
+  build_mdns:
+    if: contains(github.event.pull_request.labels.*.name, 'mdns') || github.event_name == 'push'
+    name: Build
+    strategy:
+      matrix:
+        idf_ver: ["latest"]
+        idf_target: ["esp32", "esp32s2", "esp32c3"]
+        test: [ { app: example, path: "components/mdns/examples" }, { app: unit_test, path: "components/mdns/tests/unit_test" } ]
+    runs-on: ubuntu-20.04
+    container: espressif/idf:${{ matrix.idf_ver }}
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+      - name: Build ${{ matrix.test.app }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
+        shell: bash
+        working-directory: ${{ matrix.test.path }}
+        run: |
+          ${IDF_PATH}/install.sh --enable-pytest
+          . ${IDF_PATH}/export.sh
+          python $IDF_PATH/tools/ci/ci_build_apps.py . --target ${{ matrix.idf_target }} -vv --preserve-all --pytest-app
+          for dir in `ls -d build_*`; do
+          $GITHUB_WORKSPACE/ci/clean_build_artifacts.sh `pwd`/$dir
+          zip -qur artifacts.zip $dir
+          done
+      - uses: actions/upload-artifact@v3
+        if: ${{ matrix.idf_target }} == "esp32"
+        with:
+          name: mdns_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.test.app }}
+          path: ${{ matrix.test.path }}/artifacts.zip
+          if-no-files-found: error
+
+  build_mdns_app:
+    if: contains(github.event.pull_request.labels.*.name, 'mdns') || github.event_name == 'push'
+    name: Build Test Apps
+    strategy:
+      matrix:
+        idf_ver: ["latest"]
+        idf_target: ["esp32", "esp32s2", "esp32c3"]
+    runs-on: ubuntu-20.04
+    container: espressif/idf:${{ matrix.idf_ver }}
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+      - name: Setup Build Environment
+        working-directory: components/mdns/tests/test_apps
+        run: |
+          . ${IDF_PATH}/export.sh
+          pip install -r $GITHUB_WORKSPACE/ci/requirements.txt
+      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
+        env:
+          IDF_TARGET: ${{ matrix.idf_target }}
+        shell: bash
+        working-directory: components/mdns/tests/test_apps
+        run: |
+          . ${IDF_PATH}/export.sh
+          python $IDF_PATH/tools/ci/ci_build_apps.py . --target ${{ matrix.idf_target }} -vv --pytest-apps
+      - name: Merge binaries with IDF-${{ matrix.idf_ver }}
+        working-directory: components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default
+        env:
+          IDF_TARGET: ${{ matrix.idf_target }}
+        shell: bash
+        run: |
+          . ${IDF_PATH}/export.sh
+          esptool.py --chip ${{ matrix.idf_target }} merge_bin --fill-flash-size 4MB -o flash_image.bin @flash_args
+      - uses: actions/upload-artifact@v2
+        with:
+          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
+          path: |
+            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/bootloader/bootloader.bin
+            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/partition_table/partition-table.bin
+            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/*.bin
+            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/*.elf
+            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/flasher_args.json
+            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/config/sdkconfig.h
+            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/config/sdkconfig.json
+          if-no-files-found: error
+
+  target_test_apps_mdns:
+    # Skip running on forks since it won't have access to secrets
+    if: |
+      github.repository == 'espressif/esp-protocols' &&
+      ( contains(github.event.pull_request.labels.*.name, 'mdns') || github.event_name == 'push' )
+    name: Target Test Apps
+    strategy:
+      matrix:
+        idf_ver: ["latest"]
+        idf_target: ["esp32"]
+    needs: build_mdns_app
+    runs-on:
+      - self-hosted
+      - ESP32-ETHERNET-KIT
+    steps:
+      - name: Clear repository
+        run: sudo rm -fr $GITHUB_WORKSPACE && mkdir $GITHUB_WORKSPACE
+      - uses: actions/checkout@v3
+      - uses: actions/download-artifact@v2
+        with:
+          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
+          path: components/mdns/tests/test_apps/build
+      - name: Install Python packages
+        env:
+          PIP_EXTRA_INDEX_URL: "https://www.piwheels.org/simple"
+        run: |
+            sudo apt-get install -y dnsutils
+      - name: Download Test apps to target
+        run: |
+            python -m esptool --chip ${{ matrix.idf_target }} write_flash 0x0 components/mdns/tests/test_apps/build/flash_image.bin
+      - name: Run Example Test on target
+        working-directory: components/mdns/tests/test_apps
+        run: |
+            python -m pytest --log-cli-level DEBUG --junit-xml=./examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}.xml --target=${{matrix.idf_target}}
+      - uses: actions/upload-artifact@v2
+        if: always()
+        with:
+          name: test_apps_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
+          path: components/mdns/tests/test_apps/*.xml
+
+  target_tests_mdns:
+    # Skip running on forks since it won't have access to secrets
+    if: |
+      github.repository == 'espressif/esp-protocols' &&
+      ( contains(github.event.pull_request.labels.*.name, 'mdns') || github.event_name == 'push' )
+    name: Target Example and Unit tests
+    strategy:
+      matrix:
+        idf_ver: ["latest"]
+        idf_target: ["esp32"]
+        test: [ { app: example, path: "components/mdns/examples" }, { app: unit_test, path: "components/mdns/tests/unit_test" } ]
+    needs: build_mdns
+    runs-on:
+      - self-hosted
+      - ESP32-ETHERNET-KIT
+    steps:
+      - name: Clear repository
+        run: sudo rm -fr $GITHUB_WORKSPACE && mkdir $GITHUB_WORKSPACE
+      - uses: actions/checkout@v3
+      - uses: actions/download-artifact@v3
+        with:
+          name: mdns_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.test.app }}
+          path: ${{ matrix.test.path }}/ci/
+      - name: Install Python packages
+        env:
+          PIP_EXTRA_INDEX_URL: "https://www.piwheels.org/simple"
+        run: |
+            sudo apt-get install -y dnsutils
+      - name: Run ${{ matrix.test.app }} application on ${{ matrix.idf_target }}
+        working-directory: ${{ matrix.test.path }}
+        run: |
+            unzip ci/artifacts.zip -d ci
+            for dir in `ls -d ci/build_*`; do
+            rm -rf build sdkconfig.defaults
+            mv $dir build
+            python -m pytest --log-cli-level DEBUG --junit-xml=./results_${{ matrix.test.app }}_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${dir#"ci/build_"}.xml --target=${{ matrix.idf_target }}
+            done
+      - uses: actions/upload-artifact@v3
+        if: always()
+        with:
+          name: results_${{ matrix.test.app }}_${{ matrix.idf_target }}_${{ matrix.idf_ver }}.xml
+          path: ${{ matrix.test.path }}/*.xml

.github/workflows/mdns__host-tests.yml

@@ -0,0 +1,61 @@
+name: "mdns: host-tests"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    types: [opened, synchronize, reopened, labeled]
+
+jobs:
+  host_test_mdns:
+    if: contains(github.event.pull_request.labels.*.name, 'mdns') || github.event_name == 'push'
+    name: Host test
+    runs-on: ubuntu-20.04
+    container: espressif/idf:latest
+
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@master
+        with:
+          path: esp-protocols
+
+      - name: Build and Test
+        shell: bash
+        run: |
+          apt-get update && apt-get install -y dnsutils gcc g++
+          . ${IDF_PATH}/export.sh
+          cd $GITHUB_WORKSPACE/esp-protocols/components/mdns/tests/host_test
+          idf.py build
+          ./build/mdns_host.elf &
+          dig +short -p 5353 @224.0.0.251 myesp.local > ip.txt
+          cat ip.txt | xargs dig +short -p 5353 @224.0.0.251 -x
+          cat ip.txt
+
+  build_afl_host_test_mdns:
+    if: contains(github.event.pull_request.labels.*.name, 'mdns') || github.event_name == 'push'
+    name: Build AFL host test
+    strategy:
+      matrix:
+        idf_ver: ["latest"]
+        idf_target: ["esp32"]
+
+    runs-on: ubuntu-20.04
+    container: espressif/idf:${{ matrix.idf_ver }}
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@master
+        with:
+          path: esp-protocols
+      - name: Install Necessary Libs
+        run: |
+          apt-get update -y
+          apt-get install -y libbsd-dev
+      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
+        env:
+          IDF_TARGET: ${{ matrix.idf_target }}
+        shell: bash
+        run: |
+          . ${IDF_PATH}/export.sh
+          cd $GITHUB_WORKSPACE/esp-protocols/components/mdns/tests/test_afl_fuzz_host/
+          make INSTR=off

.github/workflows/modem__build-host-tests.yml

@@ -0,0 +1,177 @@
+name: "esp-modem: build/host-tests"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    types: [opened, synchronize, reopened, labeled]
+
+jobs:
+  build_esp_modem:
+    if: contains(github.event.pull_request.labels.*.name, 'modem') || github.event_name == 'push'
+    name: Build
+    strategy:
+      matrix:
+        idf_ver: ["latest", "release-v4.2", "release-v4.3", "release-v4.4", "release-v5.0"]
+        example: ["pppos_client", "modem_console", "modem_tcp_client", "ap_to_pppos", "simple_cmux_client"]
+        exclude:
+          - idf_ver: "release-v4.2"
+            example: simple_cmux_client
+          - idf_ver: "release-v4.2"
+            example: modem_tcp_client
+          - idf_ver: "release-v4.3"
+            example: modem_tcp_client
+        include:
+          - idf_ver: "release-v4.2"
+            skip_config: usb
+          - idf_ver: "release-v4.3"
+            skip_config: usb
+          - idf_ver: "release-v5.0"
+            example: "simple_cmux_client"
+            warning: "Warning: The smallest app partition is nearly full"
+
+    runs-on: ubuntu-20.04
+    container: espressif/idf:${{ matrix.idf_ver }}
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@v3
+        with:
+          path: protocols
+      - if: ${{ matrix.skip_config }}
+        run: rm -f $GITHUB_WORKSPACE/protocols/components/esp_modem/examples/${{ matrix.example }}/sdkconfig.ci.${{ matrix.skip_config }}*
+      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }}
+        env:
+          EXPECTED_WARNING: ${{ matrix.warning }}
+        shell: bash
+        run: |
+          . ${IDF_PATH}/export.sh
+          python -m pip install idf-build-apps
+          cd $GITHUB_WORKSPACE/protocols
+          python ./ci/build_apps.py components/esp_modem/examples/${{ matrix.example }}
+
+  host_test_esp_modem:
+    if: contains(github.event.pull_request.labels.*.name, 'modem') || github.event_name == 'push'
+    name: Host Tests
+    runs-on: ubuntu-20.04
+    container: espressif/idf:release-v4.3
+    env:
+      lwip: lwip-2.1.2
+      lwip_contrib: contrib-2.1.0
+      lwip_uri: http://download.savannah.nongnu.org/releases/lwip
+      COMP_DIR: esp-protocols/components/esp_modem
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@master
+        with:
+          path: esp-protocols
+
+      - name: Build and Test
+        shell: bash
+        run: |
+          apt-get update && apt-get install -y gcc-8 g++-8
+          update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 800 --slave /usr/bin/g++ g++ /usr/bin/g++-8
+          export LWIP_PATH=`pwd`/${{ env.lwip }}
+          export LWIP_CONTRIB_PATH=`pwd`/${{ env.lwip_contrib }}
+          . ${IDF_PATH}/export.sh
+          $GITHUB_WORKSPACE/${{ env.COMP_DIR }}/test/host_test/env.sh $lwip $lwip_uri $lwip_contrib
+          cd $GITHUB_WORKSPACE/esp-protocols/components/esp_modem/examples/linux_modem
+          idf.py build
+          cd $GITHUB_WORKSPACE/esp-protocols/components/esp_modem/test/host_test
+          idf.py build
+          ./build/host_modem_test.elf -r junit -o junit.xml
+
+      - name: Publish Results
+        uses: EnricoMi/publish-unit-test-result-action@v1
+        if: always()
+        with:
+          files: esp-protocols/components/esp_modem/test/host_test/junit.xml
+
+  host_test_gcov_esp_modem:
+    if: contains(github.event.pull_request.labels.*.name, 'modem') || github.event_name == 'push'
+    name: Run gcovr on esp modem host test
+    runs-on: ubuntu-22.04
+    container: espressif/idf:release-v4.3
+    env:
+      lwip: lwip-2.1.2
+      lwip_contrib: contrib-2.1.0
+      lwip_uri: http://download.savannah.nongnu.org/releases/lwip
+      COMP_DIR: esp-protocols/components/esp_modem
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@v3
+        with:
+          path: esp-protocols
+          persist-credentials: false
+      - name: Build and Test
+        shell: bash
+        run: |
+          apt-get update
+          apt-get update && apt-get install -y gcc-8 g++-8 python3-pip
+          apt-get install -y rsync
+          update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 800 --slave /usr/bin/g++ g++ /usr/bin/g++-8
+          export DEBIAN_FRONTEND=noninteractive
+          export LWIP_PATH=`pwd`/${{ env.lwip }}
+          export LWIP_CONTRIB_PATH=`pwd`/${{ env.lwip_contrib }}
+          . ${IDF_PATH}/export.sh
+          ${{ env.COMP_DIR }}/test/host_test/env.sh $lwip $lwip_uri $lwip_contrib
+          cd $GITHUB_WORKSPACE/${{ env.COMP_DIR }}/test/host_test
+          cat sdkconfig.ci.coverage >> sdkconfig.defaults
+          idf.py build
+          ./build/host_modem_test.elf
+      - name: Run gcovr
+        shell: bash
+        run: |
+          python -m pip install gcovr
+          cd $GITHUB_WORKSPACE/${{ env.COMP_DIR }}
+          gcov-8 `find . -name "esp_modem*gcda" -printf '%h\n' | head -n 1`/*
+          gcovr --gcov-ignore-parse-errors -g -k -r . --html index.html  -x esp_modem_coverage.xml
+          mkdir docs_gcovr
+          cp $GITHUB_WORKSPACE/${{ env.COMP_DIR }}/index.html docs_gcovr
+          touch docs_gcovr/.nojekyll
+
+      - name: Code Coverage Summary Report
+        uses: irongut/CodeCoverageSummary@v1.3.0
+        with:
+          filename: esp-protocols/**/esp_modem_coverage.xml
+          badge: true
+          fail_below_min: false
+          format: markdown
+          hide_branch_rate: false
+          hide_complexity: false
+          indicators: true
+          output: both
+          thresholds: '60 80'
+
+      - name: Write to Job Summary
+        run: cat code-coverage-results.md >> $GITHUB_STEP_SUMMARY
+
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v3
+        if: always()
+        with:
+          name: docs_gcovr
+          path: |
+            ${{ env.COMP_DIR }}/docs_gcovr
+          if-no-files-found: error
+
+#  show_report_data:
+#    name: Publish-Results
+#    if: github.ref == 'refs/heads/master' || github.repository != 'espressif/esp-protocols'
+#    runs-on: ubuntu-22.04
+#    needs: gcovr_analyzer_esp_modem
+#    steps:
+#      - name: Checkout 🛎️
+#        uses: actions/checkout@v3
+#        with:
+#          persist-credentials: false
+#      - name: Download Artifacts
+#        uses: actions/download-artifact@v1
+#        with:
+#          name: docs_gcovr
+#
+#      - name: Deploy generated docs
+#        uses: JamesIves/github-pages-deploy-action@v4
+#        with:
+#          branch: gh-pages
+#          folder: 'docs_gcovr'

.github/workflows/modem__target-test.yml

@@ -0,0 +1,75 @@
+name: "esp-modem: target-tests"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    types: [opened, synchronize, reopened, labeled]
+
+jobs:
+  build_esp_modem_tests:
+    if: contains(github.event.pull_request.labels.*.name, 'modem') || github.event_name == 'push'
+    name: Build Target tests
+    strategy:
+      matrix:
+        idf_ver: ["latest"]
+        idf_target: ["esp32c3"]
+        test: [ { app: pppd, path: test/target }, { app: sim800_c3, path: examples/pppos_client } ]
+    runs-on: ubuntu-20.04
+    container: espressif/idf:${{ matrix.idf_ver }}
+    env:
+      TEST_DIR: components/esp_modem/${{ matrix.test.path }}
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+      - name: Build esp-modem target tests with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
+        env:
+          IDF_TARGET: ${{ matrix.idf_target }}
+          SDKCONFIG: sdkconfig.ci.${{ matrix.test.app }}
+        shell: bash
+        working-directory: ${{ env.TEST_DIR }}
+        run: |
+          . ${IDF_PATH}/export.sh
+          rm -rf sdkconfig build
+          [ -f ${SDKCONFIG} ] && cp ${SDKCONFIG} sdkconfig.defaults
+          idf.py set-target ${{ matrix.idf_target }}
+          idf.py build
+          $GITHUB_WORKSPACE/ci/clean_build_artifacts.sh ${GITHUB_WORKSPACE}/${TEST_DIR}/build
+      - uses: actions/upload-artifact@v3
+        with:
+          name: modem_target_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.test.app }}
+          path: ${{ env.TEST_DIR }}/build
+          if-no-files-found: error
+
+  target_tests_esp_modem:
+    # Skip running on forks since it won't have access to secrets
+    if: |
+      github.repository == 'espressif/esp-protocols' &&
+      ( contains(github.event.pull_request.labels.*.name, 'modem') || github.event_name == 'push' )
+    name: Run Target tests
+    strategy:
+      matrix:
+        idf_ver: ["latest"]
+        idf_target: ["esp32c3"]
+        test: [ { app: pppd, path: test/target }, { app: sim800_c3, path: examples/pppos_client } ]
+    needs: build_esp_modem_tests
+    runs-on:
+      - self-hosted
+      - BrnoRPI-GH006
+    env:
+      TEST_DIR: components/esp_modem/${{ matrix.test.path }}
+    steps:
+      - name: Clear repository
+        run: sudo rm -fr $GITHUB_WORKSPACE && mkdir $GITHUB_WORKSPACE
+      - uses: actions/checkout@v3
+      - uses: actions/download-artifact@v3
+        with:
+          name: modem_target_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.test.app }}
+          path: ${{ env.TEST_DIR }}/build
+      - name: Run Example Test on target
+        working-directory: ${{ env.TEST_DIR }}
+        run: |
+            python -m pytest --log-cli-level DEBUG --target=${{ matrix.idf_target }}

.github/workflows/mqtt_cxx__build.yml

@@ -0,0 +1,32 @@
+name: "mqtt-cxx: build-tests"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    types: [opened, synchronize, reopened, labeled]
+
+jobs:
+  build_esp_mqtt_cxx:
+    if: contains(github.event.pull_request.labels.*.name, 'mqtt') || github.event_name == 'push'
+    name: Build
+    strategy:
+      matrix:
+        idf_ver: ["latest", "release-v5.0"]
+        idf_target: ["esp32"]
+        test: [ { app: example, path: "components/esp_mqtt_cxx/examples" }]
+    runs-on: ubuntu-20.04
+    container: espressif/idf:${{ matrix.idf_ver }}
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+      - name: Build ${{ matrix.test.app }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
+        shell: bash
+        working-directory: ${{matrix.test.path}}
+        run: |
+          ${IDF_PATH}/install.sh --enable-pytest
+          . ${IDF_PATH}/export.sh
+          python $IDF_PATH/tools/ci/ci_build_apps.py . --target ${{ matrix.idf_target }} -vv --preserve-all --pytest-app

.github/workflows/pre_commit_check.yml

@@ -26,6 +26,7 @@ jobs:
         run: |
           pip install pre-commit
           pre-commit install-hooks
+          pre-commit install --hook-type commit-msg --hook-type pre-push
       - name: Run pre-commit and check for any changes
         run: |
           echo "Commits being checked:"

.github/workflows/target-test.yml

@@ -1,384 +0,0 @@
-name: Build and Run Target tests
-
-on: [push, pull_request]
-
-jobs:
-  build_esp_modem:
-    strategy:
-      matrix:
-        idf_ver: ["latest", "release-v4.1", "release-v4.2", "release-v4.3", "release-v4.4", "release-v5.0"]
-        example: ["pppos_client", "modem_console", "ap_to_pppos", "simple_cmux_client"]
-        idf_target: ["esp32"]
-        exclude:
-          - idf_ver: "release-v4.1"
-            example: modem_console
-          - idf_ver: "release-v4.1"
-            example: ap_to_pppos
-          - idf_ver: "release-v4.1"
-            example: simple_cmux_client
-          - idf_ver: "release-v4.2"
-            example: simple_cmux_client
-
-    runs-on: ubuntu-20.04
-    container: espressif/idf:${{ matrix.idf_ver }}
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@master
-        with:
-          path: esp-protocols
-      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        run: |
-          . ${IDF_PATH}/export.sh
-          cd $GITHUB_WORKSPACE/esp-protocols/components/esp_modem/examples/${{ matrix.example }}
-          idf.py build
-
-  build_esp_modem_usb:
-    strategy:
-      matrix:
-        idf_ver: ["latest", "release-v4.4", "release-v5.0"]
-        example: ["modem_console", "pppos_client"]
-        idf_target: ["esp32s2", "esp32s3"]
-
-    runs-on: ubuntu-20.04
-    container: espressif/idf:${{ matrix.idf_ver }}
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@v3
-      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
-        working-directory: components/esp_modem/examples/${{ matrix.example }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        run: |
-          . ${IDF_PATH}/export.sh
-          cat sdkconfig.ci.usb >> sdkconfig.defaults
-          idf.py build
-
-  build_mdns:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32", "esp32s2", "esp32c3"]
-        config: ["eth_custom_netif", "eth_def", "eth_no_ipv6", "eth_socket"]
-    runs-on: ubuntu-20.04
-    container: espressif/idf:${{ matrix.idf_ver }}
-    env:
-      TEST_DIR: components/mdns/examples
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@v3
-        with:
-          submodules: recursive
-      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }} for ${{ matrix.config }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        working-directory: ${{ env.TEST_DIR }}
-        run: |
-          . ${IDF_PATH}/export.sh
-          rm -rf sdkconfig sdkconfig.defaults build build_${{ matrix.config }}
-          cat sdkconfig.ci.${{ matrix.config }} >> sdkconfig.defaults
-          idf.py set-target ${{ matrix.idf_target }}
-          idf.py build
-          mv build build_${{ matrix.config }}
-      - name: Merge binaries with IDF-${{ matrix.idf_ver }} for ${{ matrix.config }}
-        working-directory: ${{ env.TEST_DIR }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        run: |
-          . ${IDF_PATH}/export.sh
-          cd build_${{ matrix.config }}
-          esptool.py --chip ${{ matrix.idf_target }} merge_bin --fill-flash-size 4MB -o flash_image.bin @flash_args
-      - uses: actions/upload-artifact@v3
-        with:
-          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.config }}
-          path: |
-            ${{ env.TEST_DIR }}/build_${{ matrix.config }}/bootloader/bootloader.bin
-            ${{ env.TEST_DIR }}/build_${{ matrix.config }}/partition_table/partition-table.bin
-            ${{ env.TEST_DIR }}/build_${{ matrix.config }}/*.bin
-            ${{ env.TEST_DIR }}/build_${{ matrix.config }}/*.elf
-            ${{ env.TEST_DIR }}/build_${{ matrix.config }}/flasher_args.json
-            ${{ env.TEST_DIR }}/build_${{ matrix.config }}/config/sdkconfig.h
-            ${{ env.TEST_DIR }}/build_${{ matrix.config }}/config/sdkconfig.json
-          if-no-files-found: error
-
-  build_asio:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32", "esp32s2"]
-        example: ["asio_chat", "async_request", "socks4", "ssl_client_server", "tcp_echo_server", "udp_echo_server"]
-    runs-on: ubuntu-20.04
-    container: espressif/idf:${{ matrix.idf_ver }}
-    env:
-      TEST_DIR: components/asio/examples
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@v3
-        with:
-          submodules: recursive
-      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
-        working-directory: ${{ env.TEST_DIR }}/${{ matrix.example }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        run: |
-          . ${IDF_PATH}/export.sh
-          test -f sdkconfig.ci && cat sdkconfig.ci >> sdkconfig.defaults || echo "No sdkconfig.ci"
-          idf.py set-target ${{ matrix.idf_target }}
-          idf.py build
-      - name: Merge binaries with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }} for ${{ matrix.example }}
-        working-directory: ${{ env.TEST_DIR }}/${{ matrix.example }}/build
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        run: |
-          . ${IDF_PATH}/export.sh
-          esptool.py --chip ${{ matrix.idf_target }} merge_bin --fill-flash-size 4MB -o flash_image.bin @flash_args
-      - uses: actions/upload-artifact@v3
-        with:
-          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.example }}
-          path: |
-            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/bootloader/bootloader.bin
-            ${{ env.TEST_DIR }}/${{ matrix.example }}/build//partition_table/partition-table.bin
-            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/*.bin
-            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/*.elf
-            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/flasher_args.json
-            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/config/sdkconfig.h
-            ${{ env.TEST_DIR }}/${{ matrix.example }}/build/config/sdkconfig.json
-          if-no-files-found: error
-
-
-  build_websocket:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32"]
-    runs-on: ubuntu-20.04
-    container: espressif/idf:${{ matrix.idf_ver }}
-    env:
-      TEST_DIR: components/esp_websocket_client/examples
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@v3
-        with:
-          submodules: recursive
-      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        working-directory: ${{ env.TEST_DIR }}
-        run: |
-          . ${IDF_PATH}/export.sh
-          cat sdkconfig.ci >> sdkconfig.defaults
-          idf.py build
-      - name: Merge binaries
-        working-directory: ${{ env.TEST_DIR }}/build
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        run: |
-          . ${IDF_PATH}/export.sh
-          esptool.py --chip ${{ matrix.idf_target }} merge_bin --fill-flash-size 4MB -o flash_image.bin @flash_args
-      - uses: actions/upload-artifact@v3
-        with:
-          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
-          path: |
-            ${{ env.TEST_DIR }}/build/bootloader/bootloader.bin
-            ${{ env.TEST_DIR }}/build/partition_table/partition-table.bin
-            ${{ env.TEST_DIR }}/build/*.bin
-            ${{ env.TEST_DIR }}/build/*.elf
-            ${{ env.TEST_DIR }}/build/flasher_args.json
-            ${{ env.TEST_DIR }}/build/config/sdkconfig.h
-            ${{ env.TEST_DIR }}/build/config/sdkconfig.json
-          if-no-files-found: error
-
-  run-target-websocket:
-    name: Run Websocket Example Test on target
-    needs: build_websocket
-    strategy:
-      fail-fast: false
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32"]
-    runs-on:
-      - self-hosted
-      - ESP32-ETHERNET-KIT
-    env:
-      TEST_DIR: components/esp_websocket_client/examples
-    # Skip running on forks since it won't have access to secrets
-    if: github.repository == 'espressif/esp-protocols'
-    steps:
-      - uses: actions/checkout@v3
-      - uses: actions/download-artifact@v3
-        with:
-          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
-          path: ${{ env.TEST_DIR }}/build/
-      - name: Install Python packages
-        env:
-          PIP_EXTRA_INDEX_URL: "https://www.piwheels.org/simple"
-        run: |
-          pip install --only-binary cryptography --extra-index-url https://dl.espressif.com/pypi/ -r $GITHUB_WORKSPACE/ci/requirements.txt
-      - name: Download Example Test to target
-        run: python -m esptool --chip ${{ matrix.idf_target }} write_flash 0x0 components/esp_websocket_client/examples/build/flash_image.bin
-      - name: Run Example Test on target
-        working-directory: ${{ env.TEST_DIR }}
-        run: |
-          python -m pytest --log-cli-level DEBUG --junit-xml=./examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}.xml --target=${{ matrix.idf_target }}
-      - uses: actions/upload-artifact@v3
-        if: always()
-        with:
-          name: examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
-          path: ${{ env.TEST_DIR }}/*.xml
-
-  run-target-mdns:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32"]
-        config: ["eth_custom_netif", "eth_def", "eth_no_ipv6", "eth_socket"]
-    name: Run mDNS Example Test on target
-    needs: build_mdns
-    runs-on:
-      - self-hosted
-      - ESP32-ETHERNET-KIT
-    env:
-      TEST_DIR: components/mdns/examples
-    # Skip running on forks since it won't have access to secrets
-    if: github.repository == 'espressif/esp-protocols'
-    steps:
-      - name: Clear repository
-        run: sudo rm -fr $GITHUB_WORKSPACE && mkdir $GITHUB_WORKSPACE
-      - uses: actions/checkout@v3
-      - uses: actions/download-artifact@v3
-        with:
-          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.config }}
-          path: ${{ env.TEST_DIR }}/build_${{ matrix.config }}
-      - name: Install Python packages
-        env:
-          PIP_EXTRA_INDEX_URL: "https://www.piwheels.org/simple"
-        run: |
-            sudo apt-get install -y dnsutils
-      - name: Download Example Test to target ${{ matrix.config }}
-        run: |
-            python -m esptool --chip ${{ matrix.idf_target }} write_flash 0x0 components/mdns/examples/build_${{ matrix.config }}/flash_image.bin
-      - name: Run Example Test on target ${{ matrix.config }}
-        working-directory: components/mdns/examples
-        run: |
-            rm -rf build sdkconfig.defaults
-            mv build_${{ matrix.config }} build
-            cat sdkconfig.ci.${{ matrix.config }} >> sdkconfig.defaults
-            python -m pytest --log-cli-level DEBUG --junit-xml=./examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.config }}.xml --target=${{ matrix.idf_target }}
-      - uses: actions/upload-artifact@v3
-        if: always()
-        with:
-          name: examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.config }}
-          path: ${{ env.TEST_DIR }}/*.xml
-
-  run-target-asio:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32"]
-        example: ["asio_chat", "tcp_echo_server", "udp_echo_server", "ssl_client_server"]
-    name: Run ASIO Example Test on target
-    needs: build_asio
-    runs-on:
-      - self-hosted
-      - ESP32-ETHERNET-KIT
-    env:
-      TEST_DIR: components/asio/examples
-    # Skip running on forks since it won't have access to secrets
-    if: github.repository == 'espressif/esp-protocols'
-    steps:
-      - name: Clear repository
-        run: sudo rm -fr $GITHUB_WORKSPACE && mkdir $GITHUB_WORKSPACE
-      - uses: actions/checkout@v3
-      - uses: actions/download-artifact@v3
-        with:
-          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.example }}
-          path: ${{ env.TEST_DIR }}/${{ matrix.example }}/build
-      - name: Install Python packages
-        env:
-          PIP_EXTRA_INDEX_URL: "https://www.piwheels.org/simple"
-        run: |
-            sudo apt-get install -y dnsutils
-      - name: Download Example Test to target ${{ matrix.config }}
-        run: |
-            python -m esptool --chip ${{ matrix.idf_target }} write_flash 0x0 ${{ env.TEST_DIR }}/${{ matrix.example }}/build/flash_image.bin
-      - name: Run Example Test ${{ matrix.example }} on target
-        working-directory: ${{ env.TEST_DIR }}/${{ matrix.example }}
-        run: |
-            python -m pytest --log-cli-level DEBUG --junit-xml=./examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.config }}.xml --target=${{ matrix.idf_target }}
-      - uses: actions/upload-artifact@v3
-        if: always()
-        with:
-          name: examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.example }}
-          path: ${{ env.TEST_DIR }}/${{ matrix.example }}/*.xml
-
-  build_esp_modem_tests:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32c3"]
-        test: [ { app: pppd, path: test/target }, { app: sim800, path: examples/pppos_client } ]
-    runs-on: ubuntu-20.04
-    container: espressif/idf:${{ matrix.idf_ver }}
-    env:
-      TEST_DIR: components/esp_modem/${{ matrix.test.path }}
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@v3
-        with:
-          submodules: recursive
-      - name: Build esp-modem target tests with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-          SDKCONFIG: sdkconfig.ci.${{ matrix.test.app }}
-        shell: bash
-        working-directory: ${{ env.TEST_DIR }}
-        run: |
-          . ${IDF_PATH}/export.sh
-          rm -rf sdkconfig build
-          [ -f ${SDKCONFIG} ] && cp ${SDKCONFIG} sdkconfig.defaults
-          idf.py set-target ${{ matrix.idf_target }}
-          idf.py build
-          $GITHUB_WORKSPACE/ci/clean_build_artifacts.sh ${GITHUB_WORKSPACE}/${TEST_DIR}/build
-          ls build
-      - uses: actions/upload-artifact@v3
-        with:
-          name: modem_target_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.test.app }}
-          path: ${{ env.TEST_DIR }}/build
-          if-no-files-found: error
-
-  run_esp_modem_tests:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32c3"]
-        test: [ { app: pppd, path: test/target }, { app: sim800, path: examples/pppos_client } ]
-    name: Run esp_modem Test on target
-    needs: build_esp_modem_tests
-    runs-on:
-      - self-hosted
-      - BrnoRPI-GH006
-    env:
-      TEST_DIR: components/esp_modem/${{ matrix.test.path }}
-    # Skip running on forks since it won't have access to secrets
-    if: github.repository == 'espressif/esp-protocols'
-    steps:
-      - name: Clear repository
-        run: sudo rm -fr $GITHUB_WORKSPACE && mkdir $GITHUB_WORKSPACE
-      - uses: actions/checkout@v3
-      - uses: actions/download-artifact@v3
-        with:
-          name: modem_target_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}_${{ matrix.test.app }}
-          path: ${{ env.TEST_DIR }}/build
-      - name: Run Example Test on target
-        working-directory: ${{ env.TEST_DIR }}
-        run: |
-            python -m pytest --log-cli-level DEBUG --target=${{ matrix.idf_target }}

.github/workflows/test_afl_fuzzer.yml

@@ -1,30 +0,0 @@
-name: AFL fuzzer compilation test
-
-on: [push, pull_request]
-
-jobs:
-  build:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32"]
-
-    runs-on: ubuntu-20.04
-    container: espressif/idf:${{ matrix.idf_ver }}
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@master
-        with:
-          path: esp-protocols
-      - name: Install Necessary Libs
-        run: |
-          apt-get update -y
-          apt-get install -y libbsd-dev
-      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        run: |
-          . ${IDF_PATH}/export.sh
-          cd $GITHUB_WORKSPACE/esp-protocols/components/mdns/tests/test_afl_fuzz_host/
-          make INSTR=off

.github/workflows/test_apps.yml

@@ -1,94 +0,0 @@
-name: Build and Run Test apps
-
-on:
-   push:
-     paths:
-       - 'components/mdns/**'
-   pull_request:
-     paths:
-       - 'components/mdns/**'
-
-jobs:
-  build_mdns_app:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32", "esp32s2", "esp32c3"]
-    runs-on: ubuntu-20.04
-    container: espressif/idf:${{ matrix.idf_ver }}
-    steps:
-      - name: Checkout esp-protocols
-        uses: actions/checkout@v3
-        with:
-          submodules: recursive
-      - name: Setup Build Environment
-        working-directory: components/mdns/tests/test_apps
-        run: |
-          . ${IDF_PATH}/export.sh
-          pip install -r $GITHUB_WORKSPACE/ci/requirements.txt
-      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        working-directory: components/mdns/tests/test_apps
-        run: |
-          . ${IDF_PATH}/export.sh
-          python $IDF_PATH/tools/ci/ci_build_apps.py . --target ${{ matrix.idf_target }} -vv --pytest-apps
-      - name: Merge binaries with IDF-${{ matrix.idf_ver }}
-        working-directory: components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default
-        env:
-          IDF_TARGET: ${{ matrix.idf_target }}
-        shell: bash
-        run: |
-          . ${IDF_PATH}/export.sh
-          esptool.py --chip ${{ matrix.idf_target }} merge_bin --fill-flash-size 4MB -o flash_image.bin @flash_args
-      - uses: actions/upload-artifact@v2
-        with:
-          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
-          path: |
-            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/bootloader/bootloader.bin
-            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/partition_table/partition-table.bin
-            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/*.bin
-            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/*.elf
-            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/flasher_args.json
-            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/config/sdkconfig.h
-            components/mdns/tests/test_apps/build_${{ matrix.idf_target }}_default/config/sdkconfig.json
-          if-no-files-found: error
-
-  run-target-mdns-app:
-    strategy:
-      matrix:
-        idf_ver: ["latest"]
-        idf_target: ["esp32"]
-    name: Run mDNS Test apps on target
-    needs: build_mdns_app
-    runs-on:
-      - self-hosted
-      - ESP32-ETHERNET-KIT
-    # Skip running on forks since it won't have access to secrets
-    if: github.repository == 'espressif/esp-protocols'
-    steps:
-      - name: Clear repository
-        run: sudo rm -fr $GITHUB_WORKSPACE && mkdir $GITHUB_WORKSPACE
-      - uses: actions/checkout@v3
-      - uses: actions/download-artifact@v2
-        with:
-          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
-          path: components/mdns/tests/test_apps/build
-      - name: Install Python packages
-        env:
-          PIP_EXTRA_INDEX_URL: "https://www.piwheels.org/simple"
-        run: |
-            sudo apt-get install -y dnsutils
-      - name: Download Test apps to target
-        run: |
-            python -m esptool --chip ${{ matrix.idf_target }} write_flash 0x0 components/mdns/tests/test_apps/build/flash_image.bin
-      - name: Run Example Test on target
-        working-directory: components/mdns/tests/test_apps
-        run: |
-            python -m pytest --log-cli-level DEBUG --junit-xml=./examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}.xml --target=${{matrix.idf_target}}
-      - uses: actions/upload-artifact@v2
-        if: always()
-        with:
-          name: test_apps_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
-          path: components/mdns/tests/test_apps/*.xml

.github/workflows/websocket__build-target-test.yml

@@ -0,0 +1,95 @@
+name: "websocket: build/target-tests"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    types: [opened, synchronize, reopened, labeled]
+
+jobs:
+  build_websocket:
+    if: contains(github.event.pull_request.labels.*.name, 'websocket') || github.event_name == 'push'
+    name: Build
+    strategy:
+      matrix:
+        idf_ver: ["release-v5.0", "latest"]
+        idf_target: ["esp32"]
+    runs-on: ubuntu-20.04
+    container: espressif/idf:${{ matrix.idf_ver }}
+    env:
+      TEST_DIR: components/esp_websocket_client/examples
+    steps:
+      - name: Checkout esp-protocols
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+      - name: Build ${{ matrix.example }} with IDF-${{ matrix.idf_ver }} for ${{ matrix.idf_target }}
+        env:
+          IDF_TARGET: ${{ matrix.idf_target }}
+        shell: bash
+        working-directory: ${{ env.TEST_DIR }}
+        run: |
+          . ${IDF_PATH}/export.sh
+          cat sdkconfig.ci >> sdkconfig.defaults
+          idf.py build
+      - name: Merge binaries
+        working-directory: ${{ env.TEST_DIR }}/build
+        env:
+          IDF_TARGET: ${{ matrix.idf_target }}
+        shell: bash
+        run: |
+          . ${IDF_PATH}/export.sh
+          esptool.py --chip ${{ matrix.idf_target }} merge_bin --fill-flash-size 4MB -o flash_image.bin @flash_args
+      - uses: actions/upload-artifact@v3
+        with:
+          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
+          path: |
+            ${{ env.TEST_DIR }}/build/bootloader/bootloader.bin
+            ${{ env.TEST_DIR }}/build/partition_table/partition-table.bin
+            ${{ env.TEST_DIR }}/build/*.bin
+            ${{ env.TEST_DIR }}/build/*.elf
+            ${{ env.TEST_DIR }}/build/flasher_args.json
+            ${{ env.TEST_DIR }}/build/config/sdkconfig.h
+            ${{ env.TEST_DIR }}/build/config/sdkconfig.json
+          if-no-files-found: error
+
+  run-target-websocket:
+    # Skip running on forks since it won't have access to secrets
+    if: |
+      github.repository == 'espressif/esp-protocols' &&
+      ( contains(github.event.pull_request.labels.*.name, 'websocket') || github.event_name == 'push' )
+    name: Target test
+    needs: build_websocket
+    strategy:
+      fail-fast: false
+      matrix:
+        idf_ver: ["release-v5.0", "latest"]
+        idf_target: ["esp32"]
+    runs-on:
+      - self-hosted
+      - ESP32-ETHERNET-KIT
+    env:
+      TEST_DIR: components/esp_websocket_client/examples
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/download-artifact@v3
+        with:
+          name: examples_app_bin_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
+          path: ${{ env.TEST_DIR }}/build/
+      - name: Install Python packages
+        env:
+          PIP_EXTRA_INDEX_URL: "https://www.piwheels.org/simple"
+        run: |
+          pip install --only-binary cryptography --extra-index-url https://dl.espressif.com/pypi/ -r $GITHUB_WORKSPACE/ci/requirements.txt
+      - name: Download Example Test to target
+        run: python -m esptool --chip ${{ matrix.idf_target }} write_flash 0x0 components/esp_websocket_client/examples/build/flash_image.bin
+      - name: Run Example Test on target
+        working-directory: ${{ env.TEST_DIR }}
+        run: |
+          python -m pytest --log-cli-level DEBUG --junit-xml=./examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}.xml --target=${{ matrix.idf_target }}
+      - uses: actions/upload-artifact@v3
+        if: always()
+        with:
+          name: examples_results_${{ matrix.idf_target }}_${{ matrix.idf_ver }}
+          path: ${{ env.TEST_DIR }}/*.xml

.pre-commit-config.yaml

@@ -16,6 +16,7 @@ repos:
     rev: 5.0.4
     hooks:
       - id: flake8
+        args: ['--config=.flake8', '--tee', '--benchmark']
   - repo: https://github.com/pre-commit/mirrors-mypy
     rev: v0.981
     hooks:
@@ -32,6 +33,7 @@ repos:
     rev: "v0.32.0"
     hooks:
       - id: yapf
+        args: ['style={based_on_style: google, column_limit: 160, indent_width: 4}']
   - repo: https://github.com/pre-commit/mirrors-isort
     rev: v5.10.1
     hooks:
@@ -50,3 +52,17 @@ repos:
     hooks:
     -   id: astyle_py
         args: ['--style=otbs', '--attach-namespaces', '--attach-classes', '--indent=spaces=4', '--convert-tabs', '--align-pointer=name', '--align-reference=name', '--keep-one-line-statements', '--pad-header', '--pad-oper']
+  - repo: https://github.com/commitizen-tools/commitizen
+    rev: v2.42.1
+    hooks:
+      - id: commitizen
+      - id: commitizen-branch
+        stages: [push]
+  - repo: local
+    hooks:
+      - id: commit message scopes
+        name: "commit message must be scoped with: mdns, modem, websocket, asio, mqtt_cxx, common"
+        entry: '\A(?!(feat|fix|ci|bump|test|docs)\((mdns|modem|common|websocket|asio|mqtt_cxx)\)\:)'
+        language: pygrep
+        args: [--multiline]
+        stages: [commit-msg]

CONTRIBUTING.md

@@ -6,8 +6,26 @@ Contributions in the form of pull requests, issue reports, and feature requests
 
 - [ ] Fork the [esp-protocols repository on GitHub](https://github.com/espressif/esp-protocols) to start making your changes.
 
-- [ ] Install pre-commit hooks: `pip install pre-commit && pre-commit install`
+- [ ] Install pre-commit hooks: `pip install pre-commit && pre-commit install-hooks && pre-commit install --hook-type commit-msg --hook-type pre-push`
 
 - [ ] Send a pull request (PR) and work with us until it gets merged and published. Contributions may need some modifications, so a few rounds of review and fixing may be necessary.
 
 For quick merging, the contribution should be short, and concentrated on a single feature or topic. The larger the contribution is, the longer it would take to review it and merge it.
+
+Please follow the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) rule when writing commit messages.
+
+## Release process
+
+When releasing a new component version we have to:
+
+* Update the version number
+* Update the changelog
+* Create the version tag in this repository
+* Deploy the component to component registry
+* Update the documentation
+
+This process is not fully automated, the first three steps need to be performed manually by project maintainers running the `bump` command (from within this repository, rather than forks, to publish the release `tag`). Release procedure is as follows:
+* Create a branch in this repository (not from fork)
+* Run `cz bump [version]` (version number is optional, `cz` would automatically increment it if not present)
+* Check the updated `CHANGELOG.md`
+* Create and merge the branch to master

ci/build_apps.py

@@ -0,0 +1,60 @@
+# SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
+# SPDX-License-Identifier: Apache-2.0
+"""
+This file is used in CI for esp-protocols build tests
+"""
+
+import argparse
+import os
+import sys
+from pathlib import Path
+
+from idf_build_apps import build_apps, find_apps, setup_logging
+from idf_build_apps.constants import SUPPORTED_TARGETS
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description='Build all projects',
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    parser.add_argument('paths', nargs='+', help='Paths to the apps to build.')
+    parser.add_argument(
+        '-t',
+        '--target',
+        default='all',
+        help='Build apps for given target',
+    )
+    args = parser.parse_args()
+
+    IDF_PATH = os.environ['IDF_PATH']
+
+    print(args.paths)
+    setup_logging(2)
+    apps = find_apps(
+        args.paths,
+        recursive=True,
+        target=args.target,
+        build_dir='build_@t_@w',
+        config_rules_str=[
+            'sdkconfig.ci=default', 'sdkconfig.ci.*=', '=default'
+        ],
+        build_log_path='build_log.txt',
+        size_json_path='size.json',
+        check_warnings=True,
+        preserve=True,
+        manifest_files=[
+            str(p) for p in Path('.').glob('**/.build-test-rules.yml')
+        ],
+        default_build_targets=SUPPORTED_TARGETS,
+        manifest_rootpath='.',
+    )
+
+    for app in apps:
+        print(app)
+
+    sys.exit(
+        build_apps(apps,
+                   dry_run=False,
+                   keep_going=False,
+                   ignore_warning_strs=os.environ['EXPECTED_WARNING']
+                   if 'EXPECTED_WARNING' in os.environ else None))

ci/changelog.py

@@ -0,0 +1,98 @@
+# SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
+# SPDX-License-Identifier: Apache-2.0
+import argparse
+import os
+import re
+
+import sh
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='Generates a change log from cz commits')
+    parser.add_argument('component')
+    args = parser.parse_args()
+
+    old_ref = os.environ.get('CZ_PRE_CURRENT_TAG_VERSION')
+    new_tag = os.environ.get('CZ_PRE_NEW_TAG_VERSION')
+    new_ref = os.environ.get('CZ_PRE_NEW_VERSION')
+    component = args.component
+
+    root_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
+
+    git = sh.git.bake(_cwd=root_path)
+
+    keys = ['breaking', 'major', 'feat', 'fix', 'update']
+    items = {key: [] for key in keys}
+    sections = {
+        'feat': 'Features',
+        'fix': 'Bug Fixes',
+        'update': 'Updated',
+        'breaking': 'Breaking changes',
+        'major': 'Major changes'
+    }
+    brief_log = git.log('--oneline', '{}..HEAD'.format(old_ref), '--', 'components/' + component, _tty_out=False)
+    for oneline in brief_log.splitlines():
+        [commit, brief_msg] = oneline.split(' ', 1)
+        if 'Merge' in str(brief_msg) or brief_msg.startswith('bump('):  # skip Merge commits and bumps
+            continue
+        find_title = re.findall(r'^(fix|feat|esp_modem|esp-modem|mdns)(\([^\)]+\))?:\s*(.+)$', str(brief_msg))
+        item_type = ''             # default prefix
+        item_message = brief_msg   # default body
+        if find_title is not None and len(find_title) > 0:  # Use scopes & types
+            item_type = find_title[0][0]      # prefix (fix/feat/ci)
+            item_message = find_title[0][-1]  # title body
+
+        # Add details in parentheses (commit-id and references from the FOOTER section)
+        details = '[{}](https://github.com/espressif/esp-protocols/commit/{})'.format(commit, commit)
+        msg_details = git.show('-s', commit, _tty_out=False)
+        # check references
+        if any(s in str(msg_details) for s in ['esp-protocols/issues', 'BREAKING CHANGE', 'MAJOR CHANGE']):
+            # Closes <issue>
+            closes = re.findall(r'Closes ([^\d]+/(\d+))', str(msg_details), re.MULTILINE)
+            if closes and closes[0] is not None:
+                details += ', [#{}]({})'.format(closes[0][1], closes[0][0])
+            # Breaking changes
+            find_breaking = re.findall(r'BREAKING CHANGE:\s*([^\n]*)', str(msg_details), re.MULTILINE)
+            if find_breaking is not None and len(find_breaking) > 0:
+                if find_breaking[0] != '':
+                    items['breaking'].append('{} ([{}](https://github.com/espressif/esp-protocols/commit/{}))'.format(find_breaking[0], item_message, commit))
+                else:
+                    items['breaking'].append('{} ({})'.format(item_message, details))
+                details += ', !BREAKING'
+            # Major changes
+            find_major = re.findall(r'MAJOR CHANGE:\s*([^\n]*)',
+                                    str(msg_details), re.MULTILINE)
+            if find_major is not None and len(
+                    find_major) > 0 and find_major[0] != '':
+                items['major'].append('{} ([{}](https://github.com/espressif/esp-protocols/commit/{}))'.format(find_major[0], item_message, commit))
+        if item_type in ['fix', 'feat']:
+            items[item_type].append('{} ({})'.format(item_message, details))
+        else:
+            items['update'].append('{} ({})'.format(item_message, details))
+    # Generate changelog
+    changelog = '## [{}](https://github.com/espressif/esp-protocols/commits/{})\n'.format(
+        new_ref, new_tag)
+    for section, item in items.items():
+        if len(item) > 0:
+            changelog += '\n### {}\n\n'.format(sections[section])
+            for it in item:
+                changelog += '- {}\n'.format(it)
+    changelog += '\n'
+    filename = os.path.join(root_path, 'components', component, 'CHANGELOG.md')
+    # insert the actual changelog to the beginning of the file, just after the title (2nd line)
+    with open(filename, 'r') as orig_changelog:
+        changelog_title = orig_changelog.readline(
+        )  # expect # Changelog title on the first line
+        changelog_nl = orig_changelog.readline()
+        orig_items = orig_changelog.read()
+    with open(filename, 'w') as updated_changelog:
+        updated_changelog.write(changelog_title)
+        updated_changelog.write(changelog_nl)
+        updated_changelog.write(changelog)
+        updated_changelog.write(orig_items)
+    git.add(filename)
+
+
+if __name__ == '__main__':
+    main()

ci/clean_build_artifacts.sh

@@ -4,4 +4,4 @@
 # - flasher args
 # - sdkconfigs (header and json)
 # (Ignoring the command failure as it refuses to delete nonempty dirs)
-find $1 ! -regex ".*/build/[^/]+.\(bin\|elf\)" -a ! -regex ".*\(bootloader\|partition-table\).bin" -a ! -name "flasher_args.json" -a ! -regex ".*/build/config/sdkconfig.\(h\|json\)" -delete || true
+find $1 ! -regex ".*/build[^/]*/[^/]+.\(bin\|elf\)" -a ! -regex ".*\(bootloader\|partition-table\).bin" -a ! -name "flasher_args.json" -a ! -regex ".*/build[^/]*/config/sdkconfig.\(h\|json\)" -delete || true

components/asio/CHANGELOG.md

@@ -0,0 +1,43 @@
+# Changelog
+
+## [1.14.1~3](https://github.com/espressif/esp-protocols/commits/f148c98)
+
+### Updated
+
+- ASIO: Updated package version to "1.14.1~3" ([f148c98](https://github.com/espressif/esp-protocols/commit/f148c98))
+- ASIO: Example tests integration ([5193ebc](https://github.com/espressif/esp-protocols/commit/5193ebc))
+
+
+## [1.0.2](https://github.com/espressif/esp-protocols/commits/57afa38)
+
+### Updated
+
+- Bump asio/mdns/esp_websocket_client versions ([57afa38](https://github.com/espressif/esp-protocols/commit/57afa38))
+- ignore format warnings ([d66f9dc](https://github.com/espressif/esp-protocols/commit/d66f9dc))
+- asio: Fix exampels build ([3662c14](https://github.com/espressif/esp-protocols/commit/3662c14))
+
+
+## [1.0.1](https://github.com/espressif/esp-protocols/commits/055f051)
+
+### Updated
+
+- ASIO: Initial version based on IDF 5.0 with history ([055f051](https://github.com/espressif/esp-protocols/commit/055f051))
+- asio: Disable concepts support       Fix example for compatibility with C++20 ([ac7bf46](https://github.com/espressif/esp-protocols/commit/ac7bf46), [IDF@9dba047](https://github.com/espressif/esp-idf/commit/9dba0476a01cd80d76e21706ad350009606b877e))
+- lwip: Update socket API to include port-version of sockets/netdb ([057a5d2](https://github.com/espressif/esp-protocols/commit/057a5d2), [IDF@53c009e](https://github.com/espressif/esp-idf/commit/53c009e62631bae569fa849c6b6c9e70a10b3afe))
+- esp_netif/lwip: Implement basic support for vanilla-lwip (2.1.3-REL) ([63bff63](https://github.com/espressif/esp-protocols/commit/63bff63), [IDF@5b471a1](https://github.com/espressif/esp-idf/commit/5b471a18489b056f65fe8dcbb2c992d27909ebc9))
+- mbedtls: Remove certs.c and certs.h from port directory ([e3c4391](https://github.com/espressif/esp-protocols/commit/e3c4391), [IDF@f31d8dd](https://github.com/espressif/esp-idf/commit/f31d8dd2955be0fe949340dcf3b043ec6daf4378))
+- mbedtls-3 update: 1) Fix build issue in mbedtls 2) skip the public headers check in IDF 3)Update Kconfig Macros 4)Remove deprecated config options 5) Update the sha API according to new nomenclature 6) Update mbedtls_rsa_init usage 7) Include mbedtls/build_info.h instead of mbedtls/config.h 8) Dont include check_config.h 9) Add additional error message in esp_blufi_api.h ([9813818](https://github.com/espressif/esp-protocols/commit/9813818), [IDF@4512253](https://github.com/espressif/esp-idf/commit/45122533e0bca5d538585e22308f14b74c33e474))
+- asio: Use internal ssl context and engine impl ([f605fdd](https://github.com/espressif/esp-protocols/commit/f605fdd), [IDF@d823106](https://github.com/espressif/esp-idf/commit/d823106aa6b24b8bdcc30373513c8688c61438c4))
+- Build & config: Remove leftover files from the unsupported "make" build system ([abbc8d9](https://github.com/espressif/esp-protocols/commit/abbc8d9), [IDF@766aa57](https://github.com/espressif/esp-idf/commit/766aa5708443099f3f033b739cda0e1de101cca6))
+- openssl: Add deprecation warning to ssl.h ([a029774](https://github.com/espressif/esp-protocols/commit/a029774), [IDF@cfc0018](https://github.com/espressif/esp-idf/commit/cfc001870c5e0afed7b42b6bf8c326eae053fe96))
+- asio coap: If LWIP IPV6 is disabled, automatically don't build asio & coap ([cc0f2b3](https://github.com/espressif/esp-protocols/commit/cc0f2b3), [IDF@e305f29](https://github.com/espressif/esp-idf/commit/e305f2938278c2a39e75c21a3ed59d8f4d4e62fa))
+- asio: update copyright notice ([47d57a5](https://github.com/espressif/esp-protocols/commit/47d57a5), [IDF@2d0895e](https://github.com/espressif/esp-idf/commit/2d0895e9a98bc7846d0ac7321f2b86b47346bf21))
+- Whitespace: Automated whitespace fixes (large commit) ([622a360](https://github.com/espressif/esp-protocols/commit/622a360), [IDF@66fb5a2](https://github.com/espressif/esp-idf/commit/66fb5a29bbdc2482d67c52e6f66b303378c9b789))
+- cmake: Apply cmakelint fixes ([4358c3c](https://github.com/espressif/esp-protocols/commit/4358c3c), [IDF@e82eac4](https://github.com/espressif/esp-idf/commit/e82eac4354b8b4111697656f3acce7450eeff366))
+- asio: option to use wolfSSL as TLS stack for ASIO ([c05558b](https://github.com/espressif/esp-protocols/commit/c05558b), [IDF@1c8171c](https://github.com/espressif/esp-idf/commit/1c8171c3e8d5a67e47dc8d6abac27ad2989470c3))
+- asio: Basic SSL/TLS support in asio port for ESP platform ([dab1230](https://github.com/espressif/esp-protocols/commit/dab1230), [IDF@9459c0d](https://github.com/espressif/esp-idf/commit/9459c0dd432fdd0fccb49ea65bb5c72d1849e1ba))
+- asio: updated ASIO port to use latest asio and esp-idf features ([9190917](https://github.com/espressif/esp-protocols/commit/9190917), [IDF@13d603e](https://github.com/espressif/esp-idf/commit/13d603e486624380d49f2e89614b10425c208d14))
+- components: use new component registration api ([9e83b1e](https://github.com/espressif/esp-protocols/commit/9e83b1e), [IDF@9eccd7c](https://github.com/espressif/esp-idf/commit/9eccd7c0826d6cc2e9de59304d1e5f76c0063ccf))
+- Rename Kconfig options (root) ([3b49d1f](https://github.com/espressif/esp-protocols/commit/3b49d1f), [IDF@c5000c8](https://github.com/espressif/esp-idf/commit/c5000c83d250896fffbddd7a3991384ea0fc286d))
+- cmake: make main a component again ([57672d5](https://github.com/espressif/esp-protocols/commit/57672d5), [IDF@d9939ce](https://github.com/espressif/esp-idf/commit/d9939cedd9b44d63dc148354c3a0a139b9c7113d))
+- asio: initial idf port of asio library without ssl ([5472d5c](https://github.com/espressif/esp-protocols/commit/5472d5c), [IDF@1ef13c5](https://github.com/espressif/esp-idf/commit/1ef13c524c484e9fb62e6c0b11482c30c5383728))

components/asio/LICENSE

@@ -0,0 +1,23 @@
+Boost Software License - Version 1.0 - August 17th, 2003
+
+Permission is hereby granted, free of charge, to any person or organization
+obtaining a copy of the software and accompanying documentation covered by
+this license (the "Software") to use, reproduce, display, distribute,
+execute, and transmit the Software, and to prepare derivative works of the
+Software, and to permit third-parties to whom the Software is furnished to
+do so, all subject to the following:
+
+The copyright notices in the Software and this entire statement, including
+the above license grant, this restriction and the following disclaimer,
+must be included in all copies of the Software, in whole or in part, and
+all derivative works of the Software, unless such copies or derivative
+works are solely in the form of machine-executable object code generated by
+a source language processor.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
+SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
+FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+DEALINGS IN THE SOFTWARE.

components/asio/idf_component.yml

@@ -1,5 +1,6 @@
 version: "1.14.1~3"
 description: ASIO
+url: https://github.com/espressif/esp-protocols/tree/master/components/asio
 dependencies:
   idf:
     version: ">=5.0"

components/esp_modem/.cz.yaml

@@ -0,0 +1,8 @@
+---
+commitizen:
+  bump_message: 'bump(modem): $current_version -> $new_version'
+  pre_bump_hooks: python ../../ci/changelog.py esp_modem
+  tag_format: modem-v$version
+  version: 1.0.0
+  version_files:
+  - idf_component.yml

components/esp_modem/CHANGELOG.md

@@ -0,0 +1,337 @@
+# Changelog
+
+## [1.0.0](https://github.com/espressif/esp-protocols/commits/modem-v1.0.0)
+
+### Major changes
+
+- Enable DTE to redefine on_read() and write(cmd) directly ([DTE to support sending and receiving raw data](https://github.com/espressif/esp-protocols/commit/bf114d3))
+
+### Features
+
+- Add mqtt example in AT-only mode ([7547267](https://github.com/espressif/esp-protocols/commit/7547267))
+- DTE to support sending and receiving raw data ([bf114d3](https://github.com/espressif/esp-protocols/commit/bf114d3))
+
+### Bug Fixes
+
+- Changelog to correctly pick references ([c59e330](https://github.com/espressif/esp-protocols/commit/c59e330))
+- AT-only example: support MQTT over TLS on BG96 ([31d4323](https://github.com/espressif/esp-protocols/commit/31d4323))
+- AT-only example: initial adaptation to BG96 ([ceedcfc](https://github.com/espressif/esp-protocols/commit/ceedcfc))
+- warning in ap-2-pppos example when using lwip/napt ([bf32e45](https://github.com/espressif/esp-protocols/commit/bf32e45))
+- trigger PR jobs based on labels ([13eca12](https://github.com/espressif/esp-protocols/commit/13eca12))
+- Reintroduce missing CHANGELOGs ([200cbb3](https://github.com/espressif/esp-protocols/commit/200cbb3), [#235](https://github.com/espressif/esp-protocols/issues/235))
+- Use default UART CLK source ([d0814bf](https://github.com/espressif/esp-protocols/commit/d0814bf), [#241](https://github.com/espressif/esp-protocols/issues/241))
+- run CI build job for all targets ([a089e0d](https://github.com/espressif/esp-protocols/commit/a089e0d))
+
+### Updated
+
+- Update examples for USB v1.1 ([610372f](https://github.com/espressif/esp-protocols/commit/610372f))
+
+
+## [0.1.28](https://github.com/espressif/esp-protocols/commits/01c26c8)
+
+### Updated
+
+- Dual DTE support ([01c26c8](https://github.com/espressif/esp-protocols/commit/01c26c8))
+
+
+## [0.1.27](https://github.com/espressif/esp-protocols/commits/44bae24)
+
+### Updated
+
+- Return true from on_data callback in data mode ([44bae24](https://github.com/espressif/esp-protocols/commit/44bae24))
+
+
+## [0.1.26](https://github.com/espressif/esp-protocols/commits/05fff94)
+
+### Features
+
+- Added target test ([4314c78](https://github.com/espressif/esp-protocols/commit/4314c78))
+- Add support for manual CMUX operations ([ac5d438](https://github.com/espressif/esp-protocols/commit/ac5d438), [#168](https://github.com/espressif/esp-protocols/issues/168))
+
+### Bug Fixes
+
+- Example to use variable mqtt topic/data ([8958d5e](https://github.com/espressif/esp-protocols/commit/8958d5e))
+- Fix cmux client compilation issue ([f71192b](https://github.com/espressif/esp-protocols/commit/f71192b))
+- Exit data mode only after state change ([9a7bd90](https://github.com/espressif/esp-protocols/commit/9a7bd90))
+- Support AT with callback in C-API ([2180ab1](https://github.com/espressif/esp-protocols/commit/2180ab1), [#143](https://github.com/espressif/esp-protocols/issues/143))
+
+### Updated
+
+- update(esp_modem): Bump component version to 0.1.26 ([05fff94](https://github.com/espressif/esp-protocols/commit/05fff94), [#213](https://github.com/espressif/esp-protocols/issues/213))
+- Added the missing definition for the esp_modem_set_baud function (IDFGH-9181) (#209) ([0e215b1](https://github.com/espressif/esp-protocols/commit/0e215b1))
+- CI: Added coverage analyzer for esp_modem host tests ([6b684ce](https://github.com/espressif/esp-protocols/commit/6b684ce))
+- Update esp_modem_command_library.cpp ([5304a6e](https://github.com/espressif/esp-protocols/commit/5304a6e))
+- Added badges with version of components to the respective README files ([e4c8a59](https://github.com/espressif/esp-protocols/commit/e4c8a59))
+- esp_modem/examples: Add an example to synchronise PSM sleep in Sim70XX modem and esp32. ([1ca1391](https://github.com/espressif/esp-protocols/commit/1ca1391))
+
+
+## [0.1.25](https://github.com/espressif/esp-protocols/commits/f1ae14f)
+
+### Bug Fixes
+
+- Cleanup custom lib-commands and factory ([65c0e0e](https://github.com/espressif/esp-protocols/commit/65c0e0e))
+- Examples to configure MQTT Broker ([71a2388](https://github.com/espressif/esp-protocols/commit/71a2388))
+- Extend pppos example project config ([ebc36a3](https://github.com/espressif/esp-protocols/commit/ebc36a3))
+- Uart Terminal read_cb race ([a02bf05](https://github.com/espressif/esp-protocols/commit/a02bf05))
+
+### Updated
+
+- update(esp_modem): Bump component version ([f1ae14f](https://github.com/espressif/esp-protocols/commit/f1ae14f))
+- fix(esp_modem) Add example that reads GNSS info ([652314e](https://github.com/espressif/esp-protocols/commit/652314e))
+- CI: fixing the files to be complient with pre-commit hooks ([945bd17](https://github.com/espressif/esp-protocols/commit/945bd17))
+- Quick fix esp_modem: Added modem_console commands for deep-sleep and modem PSM. ([cf504ec](https://github.com/espressif/esp-protocols/commit/cf504ec))
+- fix(esp-modem) Make simple_cmux_client example configurable ([511ed54](https://github.com/espressif/esp-protocols/commit/511ed54))
+- Added modem_console commands for deep-sleep and modem PSM. ([85a2e25](https://github.com/espressif/esp-protocols/commit/85a2e25))
+- ci: Compile esp_modem examples with USB support ([988b3f9](https://github.com/espressif/esp-protocols/commit/988b3f9))
+- Expand PPPoS example with USB DTE ([53b5933](https://github.com/espressif/esp-protocols/commit/53b5933))
+- Allow USB DTE reconnection ([bf84ae9](https://github.com/espressif/esp-protocols/commit/bf84ae9))
+- Add ConsoleCommand destructor ([a89a0ab](https://github.com/espressif/esp-protocols/commit/a89a0ab))
+- esp_modem_get_gnss_power_mode (#136) ([fe536e4](https://github.com/espressif/esp-protocols/commit/fe536e4))
+- remove unused Config Parameters ! ([55b4775](https://github.com/espressif/esp-protocols/commit/55b4775))
+
+
+## [0.1.24](https://github.com/espressif/esp-protocols/commits/9b48b0a)
+
+### Features
+
+- Make some CMUX params compile-time configurable ([25ac2d9](https://github.com/espressif/esp-protocols/commit/25ac2d9))
+
+### Bug Fixes
+
+- CMUX to ignore MSC frames ([ce175df](https://github.com/espressif/esp-protocols/commit/ce175df), [#140](https://github.com/espressif/esp-protocols/issues/140))
+- Fix CRLF endlines in examples ([07a347f](https://github.com/espressif/esp-protocols/commit/07a347f))
+- Console example to use configurable flow-ctrl ([d9c9681](https://github.com/espressif/esp-protocols/commit/d9c9681))
+
+### Updated
+
+- update(esp_modem): Bump component version ([9b48b0a](https://github.com/espressif/esp-protocols/commit/9b48b0a))
+- Allow error handler register in C ([62cb235](https://github.com/espressif/esp-protocols/commit/62cb235))
+
+
+## [0.1.23](https://github.com/espressif/esp-protocols/commits/04c711f)
+
+### Features
+
+- Add exit PPP in example ([290197c](https://github.com/espressif/esp-protocols/commit/290197c))
+
+### Updated
+
+- Move common C definitions in to separate header ([04c711f](https://github.com/espressif/esp-protocols/commit/04c711f))
+- Minor formatting fix and spelling ([accf924](https://github.com/espressif/esp-protocols/commit/accf924))
+- extended the modem_console example.  (#120) ([ef0e48a](https://github.com/espressif/esp-protocols/commit/ef0e48a))
+
+
+## [0.1.22](https://github.com/espressif/esp-protocols/commits/187ef76)
+
+### Bug Fixes
+
+- DTE command race of timeout vs reply's signal ([a871473](https://github.com/espressif/esp-protocols/commit/a871473), [#110](https://github.com/espressif/esp-protocols/issues/110))
+- Correct timeouts for certain commands ([1029078](https://github.com/espressif/esp-protocols/commit/1029078), [#129](https://github.com/espressif/esp-protocols/issues/129))
+- Make get_operator_name() return also ACT value ([0015e54](https://github.com/espressif/esp-protocols/commit/0015e54), [#128](https://github.com/espressif/esp-protocols/issues/128))
+
+### Updated
+
+- update(esp_modem): Bump component version ([187ef76](https://github.com/espressif/esp-protocols/commit/187ef76))
+
+
+## [0.1.21](https://github.com/espressif/esp-protocols/commits/d07237b)
+
+### Updated
+
+- Fix IDF version resolution ([d07237b](https://github.com/espressif/esp-protocols/commit/d07237b))
+- Fix format warnings ([98bf3ef](https://github.com/espressif/esp-protocols/commit/98bf3ef), [#79](https://github.com/espressif/esp-protocols/issues/79))
+
+
+## [0.1.20](https://github.com/espressif/esp-protocols/commits/ae8479c)
+
+### Bug Fixes
+
+- Correct exit of CMUX mode ([2099434](https://github.com/espressif/esp-protocols/commit/2099434), [#103](https://github.com/espressif/esp-protocols/issues/103))
+- Add filename/line info to exception message ([89e1bd2](https://github.com/espressif/esp-protocols/commit/89e1bd2))
+
+### Updated
+
+- Expose set_error_cb method ([ae8479c](https://github.com/espressif/esp-protocols/commit/ae8479c))
+
+
+## [0.1.19](https://github.com/espressif/esp-protocols/commits/469f953)
+
+### Features
+
+- Add optional ACT to operator-name ([a286634](https://github.com/espressif/esp-protocols/commit/a286634), [#80](https://github.com/espressif/esp-protocols/issues/80))
+
+### Bug Fixes
+
+- Add missing set_pdp_context() to C-API ([4980ac8](https://github.com/espressif/esp-protocols/commit/4980ac8))
+
+### Updated
+
+- updated package version to 0.1.19 ([469f953](https://github.com/espressif/esp-protocols/commit/469f953))
+- CI: Fix build issues ([6e4e4fa](https://github.com/espressif/esp-protocols/commit/6e4e4fa))
+- esp_err_to_name (IDFGH-7793) (#78) ([d1129f3](https://github.com/espressif/esp-protocols/commit/d1129f3))
+
+
+## [0.1.18](https://github.com/espressif/esp-protocols/commits/e8145fc)
+
+### Features
+
+- Add CMUX mode to C-API ([3fd4391](https://github.com/espressif/esp-protocols/commit/3fd4391), [#41](https://github.com/espressif/esp-protocols/issues/41))
+- Add support to CMUX exit ([a16aab6](https://github.com/espressif/esp-protocols/commit/a16aab6), [#37](https://github.com/espressif/esp-protocols/issues/37))
+
+### Bug Fixes
+
+- Update ap2ppp example to recover network on disconnection ([a243d7e](https://github.com/espressif/esp-protocols/commit/a243d7e), [#44](https://github.com/espressif/esp-protocols/issues/44))
+- Implement movable unique_buffer to bundle data, size, ptr ([66e6d4c](https://github.com/espressif/esp-protocols/commit/66e6d4c))
+- DTE should own both command and data terminal ([f3ff98b](https://github.com/espressif/esp-protocols/commit/f3ff98b))
+- Update CMUX example to use CMUX mode automatically ([58a0b57](https://github.com/espressif/esp-protocols/commit/58a0b57))
+- Improve PPP exit sequence ([1e0aefd](https://github.com/espressif/esp-protocols/commit/1e0aefd), [#47](https://github.com/espressif/esp-protocols/issues/47))
+
+### Updated
+
+- Ignore format warnings ([e8145fc](https://github.com/espressif/esp-protocols/commit/e8145fc))
+- Allow to customize uart source_clk ([d723fb7](https://github.com/espressif/esp-protocols/commit/d723fb7))
+- Add "at" api for custom commands ([0cf08fb](https://github.com/espressif/esp-protocols/commit/0cf08fb))
+
+
+## [0.1.17](https://github.com/espressif/esp-protocols/commits/5addf9e)
+
+### Features
+
+- Bumped version number to 0.1.17 ([5addf9e](https://github.com/espressif/esp-protocols/commit/5addf9e))
+
+### Bug Fixes
+
+- Support 2 byte size packets ([128c0a2](https://github.com/espressif/esp-protocols/commit/128c0a2), [#46](https://github.com/espressif/esp-protocols/issues/46))
+- Correction of switching to CMUX ([503218b](https://github.com/espressif/esp-protocols/commit/503218b), [#33](https://github.com/espressif/esp-protocols/issues/33))
+
+### Updated
+
+- Fix param description for get_operator_name() ([a661e51](https://github.com/espressif/esp-protocols/commit/a661e51))
+- Update: Removed stringstream to decrease static sizes ([3ced2d9](https://github.com/espressif/esp-protocols/commit/3ced2d9))
+- Update: Renamed list_in to be type-specific Update: Removed modem-specific comment ([ba88d7f](https://github.com/espressif/esp-protocols/commit/ba88d7f))
+- Update: Changed tests to coincide with new initial esp_modem mode ([4f3c429](https://github.com/espressif/esp-protocols/commit/4f3c429))
+- typo ([52de8f1](https://github.com/espressif/esp-protocols/commit/52de8f1))
+- Update: Removed duplicate power down function Add: gnss power mode function ([e98cf16](https://github.com/espressif/esp-protocols/commit/e98cf16))
+- Update: Formatting ([be3d2ec](https://github.com/espressif/esp-protocols/commit/be3d2ec))
+- Add: Sim7600 extended support ([15ed885](https://github.com/espressif/esp-protocols/commit/15ed885))
+- Add: Support for SIM7000 modules ([0733ea8](https://github.com/espressif/esp-protocols/commit/0733ea8))
+- Update: modem_mode::UNDEF initially ([15cbc9b](https://github.com/espressif/esp-protocols/commit/15cbc9b))
+- Add: sim7070 support ([bb7f198](https://github.com/espressif/esp-protocols/commit/bb7f198))
+- Add: Expanded modem_command_library with more "standard" commands ([d3f7ea6](https://github.com/espressif/esp-protocols/commit/d3f7ea6))
+- Fix DCE selection in ‘pppos_client’ example ([d191a72](https://github.com/espressif/esp-protocols/commit/d191a72))
+- Fix AT command in ‘set_data_mode_sim8xx()’ for SIM800 modem device ([cc0d527](https://github.com/espressif/esp-protocols/commit/cc0d527))
+
+
+## [0.1.16](https://github.com/espressif/esp-protocols/commits/74040cf)
+
+### Updated
+
+- Extend modem_console example with USB DTE ([74040cf](https://github.com/espressif/esp-protocols/commit/74040cf))
+
+
+## [0.1.15](https://github.com/espressif/esp-protocols/commits/1f5eb39)
+
+### Updated
+
+- Fix IDFv5.0 include paths ([1f5eb39](https://github.com/espressif/esp-protocols/commit/1f5eb39))
+
+
+## [0.1.14](https://github.com/espressif/esp-protocols/commits/745201b)
+
+### Bug Fixes
+
+- missing default statement in a switch in the modem_console example ([325a193](https://github.com/espressif/esp-protocols/commit/325a193))
+- add virtual destructors to ModuleIf and CommandableIf ([face03e](https://github.com/espressif/esp-protocols/commit/face03e))
+
+### Updated
+
+- build: fix issue with passing cxx_std option, a common workaround ([745201b](https://github.com/espressif/esp-protocols/commit/745201b))
+- add(esp_modem): Add unit test to check polymorphic delete ([e54b240](https://github.com/espressif/esp-protocols/commit/e54b240))
+- Implement esp_modem_sync for the C API ([8b3d420](https://github.com/espressif/esp-protocols/commit/8b3d420))
+
+
+## [0.1.13](https://github.com/espressif/esp-protocols/commits/2cb74cf)
+
+### Bug Fixes
+
+- Update the test to exersise CBC parser ([d879e82](https://github.com/espressif/esp-protocols/commit/d879e82))
+- Fix battery status parse ([4f1d31f](https://github.com/espressif/esp-protocols/commit/4f1d31f))
+- Read module name with AT commands ([8417e23](https://github.com/espressif/esp-protocols/commit/8417e23))
+- linux port to work with lwip-2.1.2 ([8465b14](https://github.com/espressif/esp-protocols/commit/8465b14))
+
+### Updated
+
+- Update(esp_modem): Bump component version ([2cb74cf](https://github.com/espressif/esp-protocols/commit/2cb74cf))
+
+
+## [0.1.12](https://github.com/espressif/esp-protocols/commits/5d9ad9c)
+
+### Updated
+
+- Update FreeRTOS EventQueueHandle_t forward declaration ([5d9ad9c](https://github.com/espressif/esp-protocols/commit/5d9ad9c))
+
+
+## [0.1.11](https://github.com/espressif/esp-protocols/commits/9649876)
+
+### Updated
+
+- New version to publish to component registry ([9649876](https://github.com/espressif/esp-protocols/commit/9649876))
+- Docs: Add links to GitHub pages ([cf990d1](https://github.com/espressif/esp-protocols/commit/cf990d1))
+- Add missing AT commands to plain C-API ([5299b42](https://github.com/espressif/esp-protocols/commit/5299b42))
+- CI/Docs: Generate docs locally before uploading to registry ([2c21aa1](https://github.com/espressif/esp-protocols/commit/2c21aa1))
+- CI/Docs: Add jobs to deploy component docs ([b9ff1e4](https://github.com/espressif/esp-protocols/commit/b9ff1e4))
+- Update Task handle and Event Group handle to match freertos v10.4.3 ([5888717](https://github.com/espressif/esp-protocols/commit/5888717))
+- Examples: Fix print format for http_client() API ([01e2a9c](https://github.com/espressif/esp-protocols/commit/01e2a9c))
+- Examples: Fix build error when SIM needs no PIN ([c8c24ed](https://github.com/espressif/esp-protocols/commit/c8c24ed))
+- Examples: Use bigger 4M flash for OTA example ([b380ded](https://github.com/espressif/esp-protocols/commit/b380ded))
+
+
+## [0.1.9](https://github.com/espressif/esp-protocols/commits/2a2d270)
+
+### Updated
+
+- Fix CMake3.5 build with no cxx_std_17 feature ([2a2d270](https://github.com/espressif/esp-protocols/commit/2a2d270))
+
+
+## [0.1.8](https://github.com/espressif/esp-protocols/commits/9dd1bd5)
+
+### Updated
+
+- 0.1.8: Update comonent version ([9dd1bd5](https://github.com/espressif/esp-protocols/commit/9dd1bd5))
+- Add pre upload script to deploy the docs before component upload ([918db0d](https://github.com/espressif/esp-protocols/commit/918db0d))
+- Minor correction of the docs ([c1b1330](https://github.com/espressif/esp-protocols/commit/c1b1330))
+- Examples/console: Remove unnecessary event-base check on IP handler ([fb7295e](https://github.com/espressif/esp-protocols/commit/fb7295e))
+- Examples: Extend CMux client default stack size for the main task ([8236b3d](https://github.com/espressif/esp-protocols/commit/8236b3d))
+- CMUX: Refactor the protocol decoder to multiple methods per cmux state ([fb6029b](https://github.com/espressif/esp-protocols/commit/fb6029b))
+- Factory: Rename Builder class to Creator, since it's not 100% builder pattern ([6e34954](https://github.com/espressif/esp-protocols/commit/6e34954))
+- Formal  updates per code review ([148a930](https://github.com/espressif/esp-protocols/commit/148a930))
+- Docs: Update CMux collaboration diagram to resemble composite ([1fb9150](https://github.com/espressif/esp-protocols/commit/1fb9150))
+- Applied astyle code formatting ([a61e9e2](https://github.com/espressif/esp-protocols/commit/a61e9e2))
+- Minor corrections per code review ([dc64f86](https://github.com/espressif/esp-protocols/commit/dc64f86))
+- Bulk replace header guards to use ([3332c27](https://github.com/espressif/esp-protocols/commit/3332c27))
+- Host test fix: renamed Loopback inherited member ([84b0dcf](https://github.com/espressif/esp-protocols/commit/84b0dcf))
+- Update documentation and minor fixes ([e0e6585](https://github.com/espressif/esp-protocols/commit/e0e6585))
+
+
+## [0.1.7](https://github.com/espressif/esp-protocols/commits/28433de)
+
+### Updated
+
+- Update version to 0.1.7 -- support socket VFS ([28433de](https://github.com/espressif/esp-protocols/commit/28433de))
+- Reworked decouple resources from the file system ([8f17a90](https://github.com/espressif/esp-protocols/commit/8f17a90))
+- Make the component compatible with IDFv4.1, v4.2, v4.3 ([69ad3ea](https://github.com/espressif/esp-protocols/commit/69ad3ea))
+
+
+## [0.1.6](https://github.com/espressif/esp-protocols/commits/a67d749)
+
+### Updated
+
+- Bump the version and upload the component ([a67d749](https://github.com/espressif/esp-protocols/commit/a67d749))
+- Updated per review comments ([ab93c13](https://github.com/espressif/esp-protocols/commit/ab93c13))
+
+
+## [0.1.5](https://github.com/espressif/esp-protocols/commits/90641c8)
+
+### Updated
+
+- Moved to component folder ([90641c8](https://github.com/espressif/esp-protocols/commit/90641c8))

components/esp_modem/examples/.build-test-rules.yml

@@ -0,0 +1,7 @@
+components/esp_modem/examples/ap_to_pppos:
+  disable:
+    - if: IDF_TARGET in ["esp32h2"]
+
+components/esp_modem/examples/modem_console:
+  disable:
+    - if: IDF_TARGET in ["esp32h2"]

components/esp_modem/examples/ap_to_pppos/main/CMakeLists.txt

@@ -7,4 +7,6 @@ endif()
 idf_component_register(SRCS "ap_to_pppos.c"
                             ${NETWORK_DCE}
                     INCLUDE_DIRS ".")
-target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")
+
+# Ignore strict prototypes, as the network_dce.h can used in both C and C++ compilation
+target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-strict-prototypes")

components/esp_modem/examples/ap_to_pppos/main/ap_to_pppos.c

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD
+ * SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
  *
  * SPDX-License-Identifier: Unlicense OR CC0-1.0
  */
@@ -14,6 +14,7 @@
 #include "esp_log.h"
 #include "esp_idf_version.h"
 #include "nvs_flash.h"
+#include "lwip/netif.h"
 #include "lwip/lwip_napt.h"
 #include "freertos/FreeRTOS.h"
 #include "freertos/event_groups.h"
@@ -38,7 +39,7 @@ static const int DISCONNECT_BIT = BIT1;
 static void on_ip_event(void *arg, esp_event_base_t event_base,
                         int32_t event_id, void *event_data)
 {
-    ESP_LOGD(TAG, "IP event! %d", event_id);
+    ESP_LOGD(TAG, "IP event! %" PRId32, event_id);
     if (event_id == IP_EVENT_PPP_GOT_IP) {
         esp_netif_dns_info_t dns_info;
         ip_event_got_ip_t *event = (ip_event_got_ip_t *)event_data;

components/esp_modem/examples/modem_console/CMakeLists.txt

@@ -1,6 +1,7 @@
 # The following lines of boilerplate have to be in your project's CMakeLists
 # in this exact order for cmake to work correctly
-cmake_minimum_required(VERSION 3.5)
+cmake_minimum_required(VERSION 3.8)
+set(CMAKE_CXX_STANDARD 17)
 
 include($ENV{IDF_PATH}/tools/cmake/project.cmake)
 project(modem-console)

components/esp_modem/examples/modem_console/main/CMakeLists.txt

@@ -1,5 +1,6 @@
 idf_component_register(SRCS "modem_console_main.cpp"
                             "console_helper.cpp"
+                            "my_module_dce.cpp"
                             "httpget_handle.c"
                             "ping_handle.c"
                        REQUIRES console esp_http_client nvs_flash

components/esp_modem/examples/modem_console/main/Kconfig.projbuild

@@ -16,17 +16,18 @@ menu "Example Configuration"
     endchoice
 
     choice EXAMPLE_MODEM_DEVICE
-        depends on EXAMPLE_SERIAL_CONFIG_UART
         prompt "Choose supported modem device (DCE)"
-        default EXAMPLE_MODEM_DEVICE_BG96
+        default EXAMPLE_MODEM_DEVICE_SHINY
         help
             Select modem device connected to the ESP DTE.
         config EXAMPLE_MODEM_DEVICE_SHINY
+            depends on EXAMPLE_SERIAL_CONFIG_UART
             bool "SHINY"
             help
                 SHINY is a GSM/GPRS module.
                 It supports SHINY.
         config EXAMPLE_MODEM_DEVICE_SIM800
+            depends on EXAMPLE_SERIAL_CONFIG_UART
             bool "SIM800"
             help
                 SIMCom SIM800L is a GSM/GPRS module.
@@ -36,10 +37,12 @@ menu "Example Configuration"
             help
                 Quectel BG96 is a series of LTE Cat M1/Cat NB1/EGPRS module.
         config EXAMPLE_MODEM_DEVICE_SIM7000
+            depends on EXAMPLE_SERIAL_CONFIG_UART
             bool "SIM7000"
             help
                 SIM7000 is a Multi-Band LTE-FDD and GSM/GPRS/EDGE module.
         config EXAMPLE_MODEM_DEVICE_SIM7070
+            depends on EXAMPLE_SERIAL_CONFIG_UART
             bool "SIM7070"
             help
                 SIM7070 is Multi-Band CAT M and NB IoT module.
@@ -47,6 +50,11 @@ menu "Example Configuration"
             bool "SIM7600"
             help
                 SIM7600 is a Multi-Band LTE-TDD/LTE-FDD/HSPA+ and GSM/GPRS/EDGE module.
+        config EXAMPLE_MODEM_DEVICE_A7670
+            depends on EXAMPLE_SERIAL_CONFIG_USB
+            bool "A7670"
+            help
+                A7670X is Multi-Band LTE-FDD/LTE-TDD/GSM/GPRS/EDGE module.
     endchoice
 
     config EXAMPLE_MODEM_PPP_APN

components/esp_modem/examples/modem_console/main/idf_component.yml

@@ -6,7 +6,7 @@ dependencies:
     version: "^0.1.20"
     override_path: "../../../"
   espressif/esp_modem_usb_dte:
-    version: "^1.0.0"
+    version: "^1.1.0"
     rules:
       - if: "idf_version >=4.4"
       - if: "target in [esp32s2, esp32s3]"

components/esp_modem/examples/modem_console/main/modem_console_main.cpp

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD
+ * SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
  *
  * SPDX-License-Identifier: Unlicense OR CC0-1.0
  */
@@ -48,9 +48,9 @@
         } } while (0)
 
 /**
- * Please update the default APN name here (this could be updated runtime)
+ * Default APN name is taken from Kconfig (this could be updated runtime)
  */
-#define DEFAULT_APN "my_apn"
+#define DEFAULT_APN CONFIG_EXAMPLE_MODEM_PPP_APN
 
 #define GPIO_OUTPUT_PWRKEY    (gpio_num_t)CONFIG_EXAMPLE_MODEM_PWRKEY_PIN
 #define GPIO_OUTPUT_PIN_SEL  (1ULL<<GPIO_OUTPUT_PWRKEY)
@@ -89,6 +89,14 @@ void wakeup_modem(void)
     vTaskDelay(pdMS_TO_TICKS(2000));
 }
 
+#ifdef CONFIG_EXAMPLE_MODEM_DEVICE_SHINY
+command_result handle_urc(uint8_t *data, size_t len)
+{
+    ESP_LOG_BUFFER_HEXDUMP("on_read", data, len, ESP_LOG_INFO);
+    return command_result::TIMEOUT;
+}
+#endif
+
 extern "C" void app_main(void)
 {
     static RTC_RODATA_ATTR char apn_rtc[20] = DEFAULT_APN;
@@ -122,19 +130,19 @@ extern "C" void app_main(void)
     dte_config.dte_buffer_size = CONFIG_EXAMPLE_MODEM_UART_RX_BUFFER_SIZE / 2;
     auto uart_dte = create_uart_dte(&dte_config);
 
-#if CONFIG_EXAMPLE_MODEM_DEVICE_SHINY == 1
+#if defined(CONFIG_EXAMPLE_MODEM_DEVICE_SHINY)
     ESP_LOGI(TAG, "Initializing esp_modem for the SHINY module...");
     auto dce = create_shiny_dce(&dce_config, uart_dte, esp_netif);
-#elif CONFIG_EXAMPLE_MODEM_DEVICE_BG96 == 1
+#elif defined(CONFIG_EXAMPLE_MODEM_DEVICE_BG96)
     ESP_LOGI(TAG, "Initializing esp_modem for the BG96 module...");
     auto dce = create_BG96_dce(&dce_config, uart_dte, esp_netif);
-#elif CONFIG_EXAMPLE_MODEM_DEVICE_SIM800 == 1
+#elif defined(CONFIG_EXAMPLE_MODEM_DEVICE_SIM800)
     ESP_LOGI(TAG, "Initializing esp_modem for the SIM800 module...");
     auto dce = create_SIM800_dce(&dce_config, uart_dte, esp_netif);
-#elif CONFIG_EXAMPLE_MODEM_DEVICE_SIM7000 == 1
+#elif defined(CONFIG_EXAMPLE_MODEM_DEVICE_SIM7000)
     ESP_LOGI(TAG, "Initializing esp_modem for the SIM7000 module...");
     auto dce = create_SIM7000_dce(&dce_config, uart_dte, esp_netif);
-#elif CONFIG_EXAMPLE_MODEM_DEVICE_SIM7070 == 1
+#elif defined(CONFIG_EXAMPLE_MODEM_DEVICE_SIM7070)
     ESP_LOGI(TAG, "Initializing esp_modem for the SIM7070 module...");
     auto dce = create_SIM7070_dce(&dce_config, uart_dte, esp_netif);
 #elif CONFIG_EXAMPLE_MODEM_DEVICE_SIM7600 == 1
@@ -149,7 +157,18 @@ extern "C" void app_main(void)
 #elif defined(CONFIG_EXAMPLE_SERIAL_CONFIG_USB)
     while (1) {
         exit_signal.clear(1);
-        struct esp_modem_usb_term_config usb_config = ESP_MODEM_DEFAULT_USB_CONFIG(0x2C7C, 0x0296, 2); // VID, PID and interface num of BG96 modem
+#if CONFIG_EXAMPLE_MODEM_DEVICE_BG96 == 1
+        ESP_LOGI(TAG, "Initializing esp_modem for the BG96 module...");
+        struct esp_modem_usb_term_config usb_config = ESP_MODEM_BG96_USB_CONFIG();
+#elif CONFIG_EXAMPLE_MODEM_DEVICE_SIM7600 == 1
+        ESP_LOGI(TAG, "Initializing esp_modem for the SIM7600 module...");
+        struct esp_modem_usb_term_config usb_config = ESP_MODEM_SIM7600_USB_CONFIG();
+#elif CONFIG_EXAMPLE_MODEM_DEVICE_A7670 == 1
+        ESP_LOGI(TAG, "Initializing esp_modem for the A7670 module...");
+        struct esp_modem_usb_term_config usb_config = ESP_MODEM_A7670_USB_CONFIG();
+#else
+#error USB modem not selected
+#endif
         const esp_modem_dte_config_t dte_config = ESP_MODEM_DTE_DEFAULT_USB_CONFIG(usb_config);
         ESP_LOGI(TAG, "Waiting for USB device connection...");
         auto dte = create_usb_dte(&dte_config);
@@ -159,7 +178,13 @@ extern "C" void app_main(void)
                 exit_signal.set(1);
             }
         });
+#if CONFIG_EXAMPLE_MODEM_DEVICE_BG96 == 1
         std::unique_ptr<DCE> dce = create_BG96_dce(&dce_config, dte, esp_netif);
+#elif CONFIG_EXAMPLE_MODEM_DEVICE_SIM7600 == 1 || CONFIG_EXAMPLE_MODEM_DEVICE_A7670 == 1
+        std::unique_ptr<DCE> dce = create_SIM7600_dce(&dce_config, dte, esp_netif);
+#else
+#error USB modem not selected
+#endif
 
 #else
 #error Invalid serial connection to modem.
@@ -286,8 +311,9 @@ extern "C" void app_main(void)
 
     const ConsoleCommand GetOperatorName("get_operator_name", "reads the operator name", no_args, [&](ConsoleCommand * c) {
         std::string operator_name;
+        int act;
         ESP_LOGI(TAG, "Reading operator name...");
-        CHECK_ERR(dce->get_operator_name(operator_name), ESP_LOGI(TAG, "OK. Operator name: %s", operator_name.c_str()));
+        CHECK_ERR(dce->get_operator_name(operator_name, act), ESP_LOGI(TAG, "OK. Operator name: %s", operator_name.c_str()));
     });
 
     const struct GenericCommandArgs {
@@ -339,6 +365,20 @@ extern "C" void app_main(void)
         ESP_LOGI(TAG, "Resetting the module...");
         CHECK_ERR(dce->reset(), ESP_LOGI(TAG, "OK"));
     });
+#ifdef CONFIG_EXAMPLE_MODEM_DEVICE_SHINY
+    const ConsoleCommand HandleURC("urc", "toggle urc handling", no_args, [&](ConsoleCommand * c) {
+        static int cnt = 0;
+        if (++cnt % 2) {
+            ESP_LOGI(TAG, "Adding URC handler");
+            dce->set_on_read(handle_urc);
+        } else {
+            ESP_LOGI(TAG, "URC removed");
+            dce->set_on_read(nullptr);
+        }
+        return 0;
+    });
+#endif
+
     const struct SetApn {
         SetApn(): apn(STR1, nullptr, nullptr, "<apn>", "APN (Access Point Name)") {}
         CommandArgs apn;

components/esp_modem/examples/modem_console/main/my_module_dce.cpp

@@ -0,0 +1,114 @@
+/*
+ * SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Unlicense OR CC0-1.0
+ */
+/* Modem console example: Custom DCE
+
+   This example code is in the Public Domain (or CC0 licensed, at your option.)
+
+   Unless required by applicable law or agreed to in writing, this
+   software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+   CONDITIONS OF ANY KIND, either express or implied.
+*/
+
+#include <cstring>
+#include "cxx_include/esp_modem_api.hpp"
+#include "cxx_include/esp_modem_dce_module.hpp"
+#include "generate/esp_modem_command_declare.inc"
+#include "my_module_dce.hpp"
+
+using namespace esp_modem;
+
+//
+// Define preprocessor's forwarding to dce_commands definitions
+//
+
+// Helper macros to handle multiple arguments of declared API
+#define ARGS0
+#define ARGS1 , p1
+#define ARGS2 , p1 , p2
+#define ARGS3 , p1 , p2 , p3
+#define ARGS4 , p1 , p2 , p3, p4
+#define ARGS5 , p1 , p2 , p3, p4, p5
+#define ARGS6 , p1 , p2 , p3, p4, p5, p6
+
+#define _ARGS(x)  ARGS ## x
+#define ARGS(x)  _ARGS(x)
+
+#define CMD_OK    (1)
+#define CMD_FAIL  (2)
+
+//
+// Repeat all declarations and forward to the AT commands defined in esp_modem::dce_commands:: namespace
+//
+#define ESP_MODEM_DECLARE_DCE_COMMAND(name, return_type, arg_nr, ...) \
+     return_type Shiny::DCE::name(__VA_ARGS__) { return esp_modem::dce_commands::name(this ARGS(arg_nr) ); }
+
+DECLARE_ALL_COMMAND_APIS(return_type name(...) )
+
+#undef ESP_MODEM_DECLARE_DCE_COMMAND
+
+std::unique_ptr<Shiny::DCE> create_shiny_dce(const esp_modem::dce_config *config,
+        std::shared_ptr<esp_modem::DTE> dte,
+        esp_netif_t *netif)
+{
+    return Shiny::Factory::create(config, std::move(dte), netif);
+}
+
+/**
+ * @brief Definition of the command API, which makes the Shiny::DCE "command-able class"
+ * @param cmd Command to send
+ * @param got_line Recv line callback
+ * @param time_ms timeout in ms
+ * @param separator line break separator
+ * @return OK, FAIL or TIMEOUT
+ */
+command_result Shiny::DCE::command(const std::string &cmd, got_line_cb got_line, uint32_t time_ms, const char separator)
+{
+    if (!handling_urc) {
+        return dte->command(cmd, got_line, time_ms, separator);
+    }
+    handle_cmd = got_line;
+    signal.clear(CMD_OK | CMD_FAIL);
+    esp_modem::DTE_Command command{cmd};
+    dte->write(command);
+    signal.wait_any(CMD_OK | CMD_FAIL, time_ms);
+    handle_cmd = nullptr;
+    if (signal.is_any(CMD_OK)) {
+        return esp_modem::command_result::OK;
+    }
+    if (signal.is_any(CMD_FAIL)) {
+        return esp_modem::command_result::FAIL;
+    }
+    return esp_modem::command_result::TIMEOUT;
+}
+
+/**
+ * @brief Handle received data
+ *
+ * @param data Data received from the device
+ * @param len Length of the data
+ * @return standard command return code (OK|FAIL|TIMEOUT)
+ */
+command_result Shiny::DCE::handle_data(uint8_t *data, size_t len)
+{
+    if (std::memchr(data, '\n', len)) {
+        if (handle_urc) {
+            handle_urc(data, len);
+        }
+        if (handle_cmd) {
+            auto ret = handle_cmd(data, len);
+            if (ret == esp_modem::command_result::TIMEOUT) {
+                return command_result::TIMEOUT;
+            }
+            if (ret == esp_modem::command_result::OK) {
+                signal.set(CMD_OK);
+            }
+            if (ret == esp_modem::command_result::FAIL) {
+                signal.set(CMD_FAIL);
+            }
+        }
+    }
+    return command_result::TIMEOUT;
+}

components/esp_modem/examples/modem_console/main/my_module_dce.hpp

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD
+ * SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
  *
  * SPDX-License-Identifier: Unlicense OR CC0-1.0
  */
@@ -11,6 +11,8 @@
 #pragma once
 
 
+#include <utility>
+
 #include "cxx_include/esp_modem_dce_factory.hpp"
 #include "cxx_include/esp_modem_dce_module.hpp"
 
@@ -28,13 +30,84 @@ public:
     }
 };
 
+namespace Shiny {
+
+using namespace esp_modem;
+
+class DCE : public esp_modem::DCE_T<MyShinyModem>, public CommandableIf {
+public:
+    using DCE_T<MyShinyModem>::DCE_T;
+
+    command_result
+    command(const std::string &cmd, got_line_cb got_line, uint32_t time_ms) override
+    {
+        return command(cmd, got_line, time_ms, '\n');
+    }
+
+    command_result
+    command(const std::string &cmd, got_line_cb got_line, uint32_t time_ms, const char separator) override;
+
+    int write(uint8_t *data, size_t len) override
+    {
+        return dte->write(data, len);
+    }
+
+    void on_read(got_line_cb on_data) override
+    {
+        return dte->on_read(on_data);
+    }
+
+#define ESP_MODEM_DECLARE_DCE_COMMAND(name, return_type, num, ...) \
+    esp_modem::return_type name(__VA_ARGS__);
+
+    DECLARE_ALL_COMMAND_APIS(forwards name(...))
+
+#undef ESP_MODEM_DECLARE_DCE_COMMAND
+
+    void set_on_read(esp_modem::got_line_cb on_read_cb)
+    {
+        if (on_read_cb == nullptr) {
+            handling_urc = false;
+            handle_urc = nullptr;
+            dte->on_read(nullptr);
+            return;
+        }
+        handle_urc = std::move(on_read_cb);
+        dte->on_read([this](uint8_t *data, size_t len) {
+            this->handle_data(data, len);
+            return command_result::TIMEOUT;
+        });
+        handling_urc = true;
+    }
+
+private:
+    got_line_cb handle_urc{nullptr};
+    got_line_cb handle_cmd{nullptr};
+    SignalGroup signal;
+    bool handling_urc {false};
+
+    command_result handle_data(uint8_t *data, size_t len);
+
+};
+
+class Factory: public ::esp_modem::dce_factory::Factory {
+public:
+
+    static std::unique_ptr<DCE> create(const esp_modem::dce_config *config,
+                                       std::shared_ptr<esp_modem::DTE> dte,
+                                       esp_netif_t *netif)
+    {
+        return build_generic_DCE<MyShinyModem, DCE, std::unique_ptr<DCE>>(config, std::move(dte), netif);
+    }
+
+};
+
+} // namespace Shiny
+
 /**
  * @brief Helper create method which employs the DCE factory for creating DCE objects templated by a custom module
  * @return unique pointer of the resultant DCE
  */
-std::unique_ptr<esp_modem::DCE> create_shiny_dce(const esp_modem::dce_config *config,
+std::unique_ptr<Shiny::DCE> create_shiny_dce(const esp_modem::dce_config *config,
         std::shared_ptr<esp_modem::DTE> dte,
-        esp_netif_t *netif)
-{
-    return esp_modem::dce_factory::Factory::build_unique<MyShinyModem>(config, std::move(dte), netif);
-}
+        esp_netif_t *netif);

components/esp_modem/examples/modem_console/sdkconfig.ci.usb_s2

@@ -1 +1,2 @@
+CONFIG_IDF_TARGET="esp32s2"
 CONFIG_EXAMPLE_SERIAL_CONFIG_USB=y

components/esp_modem/examples/modem_console/sdkconfig.ci.usb_s3

@@ -1 +1,2 @@
+CONFIG_IDF_TARGET="esp32s3"
 CONFIG_EXAMPLE_SERIAL_CONFIG_USB=y

components/esp_modem/examples/modem_tcp_client/CMakeLists.txt

@@ -0,0 +1,9 @@
+# The following lines of boilerplate have to be in your project's CMakeLists
+# in this exact order for cmake to work correctly
+cmake_minimum_required(VERSION 3.8)
+set(CMAKE_CXX_STANDARD 17)
+
+set(EXTRA_COMPONENT_DIRS "../..")
+
+include($ENV{IDF_PATH}/tools/cmake/project.cmake)
+project(modem_tcp_client)

components/esp_modem/examples/modem_tcp_client/README.md

@@ -0,0 +1,10 @@
+# Modem TCP client
+
+(See the README.md file in the upper level 'examples' directory for more information about examples.)
+
+## Overview
+This example demonstrates how to act as a MQTT client using modem's TCP commands (provided, the device supports "socket" related commands)
+
+### Supported IDF versions
+
+This example is supported from IDF `v4.4`.

components/esp_modem/examples/modem_tcp_client/main/CMakeLists.txt

@@ -0,0 +1,11 @@
+if (CONFIG_EXAMPLE_MODEM_DEVICE_BG96)
+    set(device_srcs sock_commands_bg96.cpp)
+elseif(CONFIG_EXAMPLE_MODEM_DEVICE_SIM7600)
+    set(device_srcs sock_commands_sim7600.cpp)
+endif()
+
+idf_component_register(SRCS "modem_client.cpp"
+                            "sock_dce.cpp"
+                            "${device_srcs}"
+                       INCLUDE_DIRS ".")
+target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")

components/esp_modem/examples/modem_tcp_client/main/Kconfig.projbuild

@@ -0,0 +1,96 @@
+menu "Example Configuration"
+
+    choice EXAMPLE_MODEM_DEVICE
+        prompt "Choose supported modem device (DCE)"
+        default EXAMPLE_MODEM_DEVICE_BG96
+        help
+            Select modem device connected to the ESP DTE.
+        config EXAMPLE_MODEM_DEVICE_BG96
+            bool "BG96"
+            help
+                Quectel BG96 is a series of LTE Cat M1/Cat NB1/EGPRS module.
+        config EXAMPLE_MODEM_DEVICE_SIM7600
+            bool "SIM7600"
+            help
+                SIM7600 is Multi-Band LTE-TDD/LTE-FDD/HSPA+ and GSM/GPRS/EDGE module
+    endchoice
+
+    config EXAMPLE_MODEM_APN
+        string "Set MODEM APN"
+        default "internet"
+        help
+            Set APN (Access Point Name), a logical name to choose data network
+
+    menu "UART Configuration"
+        config EXAMPLE_MODEM_UART_TX_PIN
+            int "TXD Pin Number"
+            default 25
+            range 0 31
+            help
+                Pin number of UART TX.
+
+        config EXAMPLE_MODEM_UART_RX_PIN
+            int "RXD Pin Number"
+            default 26
+            range 0 31
+            help
+                Pin number of UART RX.
+
+        config EXAMPLE_MODEM_UART_RTS_PIN
+            int "RTS Pin Number"
+            default 27
+            range 0 31
+            help
+                Pin number of UART RTS.
+
+        config EXAMPLE_MODEM_UART_CTS_PIN
+            int "CTS Pin Number"
+            default 23
+            range 0 31
+            help
+                Pin number of UART CTS.
+
+        config EXAMPLE_MODEM_UART_EVENT_TASK_STACK_SIZE
+            int "UART Event Task Stack Size"
+            range 2000 6000
+            default 2048
+            help
+                Stack size of UART event task.
+
+        config EXAMPLE_MODEM_UART_EVENT_TASK_PRIORITY
+            int "UART Event Task Priority"
+            range 3 22
+            default 5
+            help
+                Priority of UART event task.
+
+        config EXAMPLE_MODEM_UART_EVENT_QUEUE_SIZE
+            int "UART Event Queue Size"
+            range 10 40
+            default 30
+            help
+                Length of UART event queue.
+
+        config EXAMPLE_MODEM_UART_PATTERN_QUEUE_SIZE
+            int "UART Pattern Queue Size"
+            range 10 40
+            default 20
+            help
+                Length of UART pattern queue.
+
+        config EXAMPLE_MODEM_UART_TX_BUFFER_SIZE
+            int "UART TX Buffer Size"
+            range 256 2048
+            default 512
+            help
+                Buffer size of UART TX buffer.
+
+        config EXAMPLE_MODEM_UART_RX_BUFFER_SIZE
+            int "UART RX Buffer Size"
+            range 256 2048
+            default 1024
+            help
+                Buffer size of UART RX buffer.
+    endmenu
+
+endmenu

components/esp_modem/examples/modem_tcp_client/main/modem_client.cpp

@@ -0,0 +1,140 @@
+/*
+ * SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Unlicense OR CC0-1.0
+ */
+/* PPPoS Client Example
+
+   This example code is in the Public Domain (or CC0 licensed, at your option.)
+
+   Unless required by applicable law or agreed to in writing, this
+   software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+   CONDITIONS OF ANY KIND, either express or implied.
+*/
+#include <string>
+#include "freertos/FreeRTOS.h"
+#include "freertos/event_groups.h"
+#include "esp_netif.h"
+#include "esp_netif_ppp.h"
+#include "mqtt_client.h"
+#include "esp_modem_config.h"
+#include "cxx_include/esp_modem_api.hpp"
+#include "sock_dce.hpp"
+#include "esp_log.h"
+
+#define BROKER_URL "mqtt.eclipseprojects.io"
+
+static const char *TAG = "modem_client";
+static EventGroupHandle_t event_group = NULL;
+static const int CONNECT_BIT = BIT0;
+static const int GOT_DATA_BIT = BIT2;
+
+static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data)
+{
+    ESP_LOGD(TAG, "Event dispatched from event loop base=%s, event_id=%d", base, event_id);
+    esp_mqtt_event_handle_t event = (esp_mqtt_event_handle_t)event_data;
+    esp_mqtt_client_handle_t client = event->client;
+    int msg_id;
+    switch ((esp_mqtt_event_id_t)event_id) {
+    case MQTT_EVENT_CONNECTED:
+        ESP_LOGI(TAG, "MQTT_EVENT_CONNECTED");
+        msg_id = esp_mqtt_client_subscribe(client, "/topic/esp-pppos", 0);
+        ESP_LOGI(TAG, "sent subscribe successful, msg_id=%d", msg_id);
+        break;
+    case MQTT_EVENT_DISCONNECTED:
+        ESP_LOGI(TAG, "MQTT_EVENT_DISCONNECTED");
+        break;
+    case MQTT_EVENT_SUBSCRIBED:
+        ESP_LOGI(TAG, "MQTT_EVENT_SUBSCRIBED, msg_id=%d", event->msg_id);
+        msg_id = esp_mqtt_client_publish(client, "/topic/esp-pppos", "esp32-pppos", 0, 0, 0);
+        ESP_LOGI(TAG, "sent publish successful, msg_id=%d", msg_id);
+        break;
+    case MQTT_EVENT_UNSUBSCRIBED:
+        ESP_LOGI(TAG, "MQTT_EVENT_UNSUBSCRIBED, msg_id=%d", event->msg_id);
+        break;
+    case MQTT_EVENT_PUBLISHED:
+        ESP_LOGI(TAG, "MQTT_EVENT_PUBLISHED, msg_id=%d", event->msg_id);
+        break;
+    case MQTT_EVENT_DATA:
+        ESP_LOGI(TAG, "MQTT_EVENT_DATA");
+        printf("TOPIC=%.*s\r\n", event->topic_len, event->topic);
+        printf("DATA=%.*s\r\n", event->data_len, event->data);
+        xEventGroupSetBits(event_group, GOT_DATA_BIT);
+        break;
+    case MQTT_EVENT_ERROR:
+        ESP_LOGI(TAG, "MQTT_EVENT_ERROR");
+        break;
+    default:
+        ESP_LOGI(TAG, "MQTT other event id: %d", event->event_id);
+        break;
+    }
+}
+
+extern "C" void app_main(void)
+{
+
+    /* Init and register system/core components */
+    ESP_ERROR_CHECK(esp_netif_init());
+    ESP_ERROR_CHECK(esp_event_loop_create_default());
+
+    event_group = xEventGroupCreate();
+
+    /* Configure and create the UART DTE */
+    esp_modem_dte_config_t dte_config = ESP_MODEM_DTE_DEFAULT_CONFIG();
+    /* setup UART specific configuration based on kconfig options */
+    dte_config.uart_config.tx_io_num = CONFIG_EXAMPLE_MODEM_UART_TX_PIN;
+    dte_config.uart_config.rx_io_num = CONFIG_EXAMPLE_MODEM_UART_RX_PIN;
+    dte_config.uart_config.rts_io_num = CONFIG_EXAMPLE_MODEM_UART_RTS_PIN;
+    dte_config.uart_config.cts_io_num = CONFIG_EXAMPLE_MODEM_UART_CTS_PIN;
+    dte_config.uart_config.rx_buffer_size = CONFIG_EXAMPLE_MODEM_UART_RX_BUFFER_SIZE;
+    dte_config.uart_config.tx_buffer_size = CONFIG_EXAMPLE_MODEM_UART_TX_BUFFER_SIZE;
+    dte_config.uart_config.event_queue_size = CONFIG_EXAMPLE_MODEM_UART_EVENT_QUEUE_SIZE;
+    dte_config.task_stack_size = CONFIG_EXAMPLE_MODEM_UART_EVENT_TASK_STACK_SIZE * 2;
+    dte_config.task_priority = CONFIG_EXAMPLE_MODEM_UART_EVENT_TASK_PRIORITY;
+    dte_config.dte_buffer_size = CONFIG_EXAMPLE_MODEM_UART_RX_BUFFER_SIZE / 2;
+
+    auto dte = esp_modem::create_uart_dte(&dte_config);
+    assert(dte);
+
+    /* Configure the DCE */
+    esp_modem_dce_config_t dce_config = ESP_MODEM_DCE_DEFAULT_CONFIG(CONFIG_EXAMPLE_MODEM_APN);
+
+    /* create the DCE and initialize network manually (using AT commands) */
+    auto dce = sock_dce::create(&dce_config, std::move(dte));
+    if (!dce->init_network()) {
+        ESP_LOGE(TAG,  "Failed to setup network");
+        return;
+    }
+
+    dce->init_sock(8883);
+    esp_mqtt_client_config_t mqtt_config = {};
+#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 0, 0)
+    mqtt_config.broker.address.uri = "mqtts://127.0.0.1";
+    mqtt_config.session.message_retransmit_timeout = 10000;
+#else
+    mqtt_config.uri = "mqtt://127.0.0.1";
+    mqtt_config.message_retransmit_timeout = 10000;
+#endif
+    esp_mqtt_client_handle_t mqtt_client = esp_mqtt_client_init(&mqtt_config);
+    esp_mqtt_client_register_event(mqtt_client, static_cast<esp_mqtt_event_id_t>(ESP_EVENT_ANY_ID), mqtt_event_handler, NULL);
+    esp_mqtt_client_start(mqtt_client);
+    if (!dce->start(BROKER_URL, 8883)) {
+        ESP_LOGE(TAG, "Failed to start DCE");
+        return;
+    }
+    while (1) {
+        while (dce->perform_sock()) {
+            ESP_LOGV(TAG, "...performing");
+        }
+        ESP_LOGE(TAG, "Loop exit.. retrying");
+        // handle disconnections errors
+        if (!dce->init_network()) {
+            ESP_LOGE(TAG, "Failed to reinit network");
+            return;
+        }
+        if (!dce->start("test.mosquitto.org", 1883)) {
+            ESP_LOGI(TAG, "Network reinitialized, retrying");
+        }
+    }
+
+}

components/esp_modem/examples/modem_tcp_client/main/sock_commands.hpp

@@ -0,0 +1,25 @@
+/*
+ * SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+#pragma once
+
+#include "cxx_include/esp_modem_dte.hpp"
+#include "cxx_include/esp_modem_dce_module.hpp"
+#include "cxx_include/esp_modem_types.hpp"
+#include "socket_commands.inc"
+
+namespace sock_commands {
+
+//using namespace esp_modem;
+
+#define ESP_MODEM_DECLARE_DCE_COMMAND(name, return_type, num, ...) \
+        esp_modem::return_type name(esp_modem::CommandableIf *t, ## __VA_ARGS__);
+
+DECLARE_SOCK_COMMANDS(declare name(Commandable *p, ...);)
+
+#undef ESP_MODEM_DECLARE_DCE_COMMAND
+
+}

components/esp_modem/examples/modem_tcp_client/main/sock_commands_bg96.cpp

@@ -0,0 +1,348 @@
+/*
+ * SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+#include <charconv>
+#include <cstring>
+#include <sys/socket.h>
+#include "sock_commands.hpp"
+#include "cxx_include/esp_modem_command_library_utils.hpp"
+#include "sock_dce.hpp"
+
+static const char *TAG = "sock_commands";
+
+namespace sock_commands {
+
+using namespace esp_modem;
+
+command_result net_open(CommandableIf *t)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    std::string out;
+    auto ret = dce_commands::generic_get_string(t, "AT+QISTATE?\r", out, 1000);
+    if (ret != command_result::OK) {
+        return ret;
+    }
+    if (out.find("+QISTATE: 0") != std::string::npos) {
+        ESP_LOGV(TAG, "%s", out.data() );
+        ESP_LOGD(TAG, "Already there");
+        return command_result::FAIL;
+    } else if (out.empty()) {
+        return dce_commands::generic_command(t, "AT+QIACT=1\r", "OK", "ERROR", 150000);
+    }
+    return command_result::FAIL;
+}
+
+command_result net_close(CommandableIf *t)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    dce_commands::generic_command(t, "AT+QICLOSE=0\r", "OK", "ERROR", 10000);
+    esp_modem::Task::Delay(1000);
+    return dce_commands::generic_command(t, "AT+QIDEACT=1\r", "OK", "ERROR", 40000);
+}
+
+command_result tcp_open(CommandableIf *t, const std::string &host, int port, int timeout)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    std::string ip_open = R"(AT+QIOPEN=1,0,"TCP",")" + host + "\"," + std::to_string(port) + "\r";
+    auto ret = dce_commands::generic_command(t, ip_open, "+QIOPEN: 0,0", "ERROR", timeout);
+    if (ret != command_result::OK) {
+        ESP_LOGE(TAG, "%s Failed", __func__ );
+        return ret;
+    }
+    return command_result::OK;
+}
+
+command_result tcp_close(CommandableIf *t)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    return dce_commands::generic_command(t, "AT+QICLOSE=0\r", "OK", "ERROR", 10000);
+}
+
+command_result tcp_send(CommandableIf *t, uint8_t *data, size_t len)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    assert(0);      // Remove when fix done
+    return command_result::FAIL;
+}
+
+command_result tcp_recv(CommandableIf *t, uint8_t *data, size_t len, size_t &out_len)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    assert(0);      // Remove when fix done
+    return command_result::FAIL;
+}
+
+command_result get_ip(CommandableIf *t, std::string &ip)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    std::string out;
+    auto ret = dce_commands::generic_get_string(t, "AT+QIACT?\r", out, 5000);
+    if (ret != command_result::OK) {
+        return ret;
+    }
+    auto pos = out.find("+QIACT: 1");
+    auto property = 0;
+    while (pos != std::string::npos) {
+        // Looking for: +QIACT: <contextID>,<context_state>,<context_type>,<IP_address>
+        if (property++ == 3) {  // ip is after 3rd comma (as a 4rd property of QIACT string)
+            ip = out.substr(++pos);
+            // strip quotes if present
+            auto quote1 = ip.find('"');
+            auto quote2 = ip.rfind('"');
+            if (quote1 != std::string::npos && quote2 != std::string::npos) {
+                ip = ip.substr(quote1 + 1, quote2 - 1);
+            }
+            return command_result::OK;
+        }
+        pos = out.find(',', ++pos);
+    }
+    return command_result::FAIL;
+}
+
+} // sock_commands
+
+namespace sock_dce {
+
+void Responder::start_sending(size_t len)
+{
+    data_to_send = len;
+    send_stat = 0;
+    send_cmd("AT+QISEND=0," + std::to_string(len) + "\r");
+}
+
+void Responder::start_receiving(size_t len)
+{
+    send_cmd("AT+QIRD=0," + std::to_string(len) + "\r");
+}
+
+bool Responder::start_connecting(std::string host, int port)
+{
+    send_cmd(R"(AT+QIOPEN=1,0,"TCP",")" + host + "\"," + std::to_string(port) + "\r");
+    return true;
+}
+
+Responder::ret Responder::recv(uint8_t *data, size_t len)
+{
+    const int MIN_MESSAGE = 6;
+    size_t actual_len = 0;
+    auto *recv_data = (char *)data;
+    if (data_to_recv == 0) {
+        const std::string_view head = "+QIRD: ";
+        auto head_pos = std::search(recv_data, recv_data + len, head.begin(), head.end());
+        if (head_pos == nullptr) {
+            return ret::FAIL;
+        }
+
+        auto next_nl = (char *)memchr(head_pos + head.size(), '\n', MIN_MESSAGE);
+        if (next_nl == nullptr) {
+            return ret::FAIL;
+        }
+
+        if (std::from_chars(head_pos + head.size(), next_nl, actual_len).ec == std::errc::invalid_argument) {
+            ESP_LOGE(TAG, "cannot convert");
+            return ret::FAIL;
+        }
+
+        ESP_LOGD(TAG, "Received: actual len=%d", actual_len);
+        if (actual_len == 0) {
+            ESP_LOGD(TAG, "no data received");
+            return ret::FAIL;
+        }
+
+        if (actual_len > buffer_size) {
+            ESP_LOGE(TAG, "TOO BIG");
+            return ret::FAIL;
+        }
+
+        recv_data = next_nl + 1;
+        auto first_data_len = len - (recv_data - (char *)data) /* minus size of the command marker */;
+        if (actual_len > first_data_len) {
+            ::send(sock, recv_data, first_data_len, 0);
+            data_to_recv = actual_len - first_data_len;
+            return ret::NEED_MORE_DATA;
+        }
+        ::send(sock, recv_data, actual_len, 0);
+    } else if (data_to_recv > len) {    // continue sending
+        ::send(sock, recv_data, len, 0);
+        data_to_recv -= len;
+        return ret::NEED_MORE_DATA;
+    } else if (data_to_recv <= len) {    // last read -> looking for "OK" marker
+        ::send(sock, recv_data, data_to_recv, 0);
+        actual_len = data_to_recv;
+    }
+
+    // "OK" after the data
+    char *last_pos = nullptr;
+    if (actual_len + 1 + 2 /* OK */  > len) {
+        last_pos = (char *)memchr(recv_data + 1 + actual_len, 'O', MIN_MESSAGE);
+        if (last_pos == nullptr || last_pos[1] != 'K') {
+            data_to_recv = 0;
+            return ret::FAIL;
+        }
+    }
+    if (last_pos != nullptr && (char *)data + len - last_pos - 2 > MIN_MESSAGE) {
+        // check for async replies after the Recv header
+        std::string_view response((char *)last_pos + 2 /* OK */, (char *)data + len - last_pos);
+        check_async_replies(status::RECEIVING, response);
+    }
+    // check if some other data?
+    start_receiving(0);
+    data_to_recv = 0;
+    return ret::OK;
+}
+
+
+Responder::ret Responder::send(uint8_t *data, size_t len)
+{
+    if (send_stat < 3) {
+        if (memchr(data, '>', len) == NULL) {
+            if (send_stat++ < 2) {
+                return Responder::ret::NEED_MORE_DATA;
+            }
+            ESP_LOGE(TAG, "Missed >");
+            return ret::FAIL;
+        }
+        auto written = dte->write(&buffer[0], data_to_send);
+        if (written != data_to_send) {
+            ESP_LOGE(TAG, "written %d (%d)...", written, len);
+            return ret::FAIL;
+        }
+        data_to_send = 0;
+        send_stat = 3;
+    }
+    return Responder::ret::IN_PROGRESS;
+}
+
+Responder::ret Responder::send(std::string_view response)
+{
+    if (send_stat == 3) {
+        if (response.find("SEND OK") != std::string::npos) {
+            send_cmd("AT+QISEND=0,0\r");
+            send_stat++;
+            return ret::IN_PROGRESS;
+        } else if (response.find("SEND FAIL") != std::string::npos) {
+            ESP_LOGE(TAG, "Sending buffer full");
+            return ret::FAIL;
+        } else if (response.find("ERROR") != std::string::npos) {
+            ESP_LOGE(TAG, "Failed to sent");
+            return ret::FAIL;
+        }
+    } else if (send_stat == 4) {
+        constexpr std::string_view head = "+QISEND: ";
+        if (response.find(head) != std::string::npos) {
+            // Parsing +QISEND: <total_send_length>,<ackedbytes>,<unackedbytes>
+            size_t head_pos = response.find(head);
+            response = response.substr(head_pos + head.size());
+            int pos, property = 0;
+            int total = 0, ack = 0, unack = 0;
+            while ((pos = response.find(',')) != std::string::npos) {
+                auto next_comma = (char *)memchr(response.data(), ',', response.size());
+
+                // extract value
+                size_t value;
+                if (std::from_chars(response.data(), next_comma, value).ec == std::errc::invalid_argument) {
+                    ESP_LOGE(TAG, "cannot convert");
+                    return ret::FAIL;
+                }
+
+                switch (property++) {
+                case 0: total = value;
+                    break;
+                case 1: ack = value;
+                    break;
+                default:
+                    return ret::FAIL;
+                }
+                response = response.substr(pos + 1);
+            }
+            if (std::from_chars(response.data(), response.data() + pos, unack).ec == std::errc::invalid_argument) {
+                return ret::FAIL;
+            }
+
+            if (ack < total) {
+                ESP_LOGD(TAG, "all sending data are not ack (missing %d bytes acked)", (total - ack));
+                if (total - ack > 64) {
+                    ESP_LOGW(TAG, "Need a pause: missing %d bytes acked", (total - ack));
+                    return ret::NEED_MORE_TIME;
+                }
+            }
+            send_stat = 0;
+            return ret::OK;
+        } else if (response.find("ERROR") != std::string::npos) {
+            ESP_LOGE(TAG, "Failed to check sending");
+            return ret::FAIL;
+        }
+
+    }
+    return Responder::ret::IN_PROGRESS;
+}
+
+Responder::ret Responder::connect(std::string_view response)
+{
+    if (response.find("+QIOPEN: 0,0") != std::string::npos) {
+        ESP_LOGI(TAG, "Connected!");
+        return ret::OK;
+    }
+    if (response.find("ERROR") != std::string::npos) {
+        ESP_LOGE(TAG, "Failed to open");
+        return ret::FAIL;
+    }
+    return Responder::ret::IN_PROGRESS;
+}
+
+Responder::ret Responder::check_async_replies(status state, std::string_view &response)
+{
+    ESP_LOGD(TAG, "response %.*s", static_cast<int>(response.size()), response.data());
+    if (response.find("+QIURC: \"recv\",0") != std::string::npos) {
+        uint64_t data_ready = 1;
+        write(data_ready_fd, &data_ready, sizeof(data_ready));
+        ESP_LOGD(TAG, "Got data on modem!");
+    } else if (response.find("+QIRD: ") != std::string::npos) {
+        static constexpr std::string_view head = "+QIRD: ";
+        size_t head_pos = response.find(head);
+        // Parsing +QIURC: <total_receive_length>,<have_read_length>,<unread_length>
+        response = response.substr(head_pos + head.size());
+        int next_cr = response.find('\r');
+        if (next_cr != std::string::npos) {
+            response = response.substr(next_cr - 2, next_cr);
+            if (response.find(",0") != std::string::npos) {
+                ESP_LOGV(TAG, "Receiving done");
+            } else {
+                uint64_t data_ready = 1;
+                write(data_ready_fd, &data_ready, sizeof(data_ready));
+                ESP_LOGD(TAG, "Got data on modem!");
+            }
+        }
+    } else if (response.find("+QIURC: \"closed\",0") != std::string::npos) {
+        return ret::FAIL;
+    }
+    if (state == status::SENDING) {
+        return send(response);
+    } else if (state == status::CONNECTING) {
+        return connect(response);
+    }
+    return ret::IN_PROGRESS;
+}
+
+Responder::ret Responder::process_data(status state, uint8_t *data, size_t len)
+{
+    if (state == status::SENDING) {
+        return send(data, len);
+    }
+    if (state == status::RECEIVING) {
+        return recv(data, len);
+    }
+    return Responder::ret::IN_PROGRESS;
+}
+
+status Responder::pending()
+{
+    send_cmd("AT+QISEND=0,0\r");
+    return status::SENDING;
+}
+
+
+} // sock_dce

components/esp_modem/examples/modem_tcp_client/main/sock_commands_sim7600.cpp

@@ -0,0 +1,378 @@
+/*
+ * SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+#include <charconv>
+#include <cstring>
+#include <sys/socket.h>
+#include "sock_commands.hpp"
+#include "cxx_include/esp_modem_command_library_utils.hpp"
+#include "sock_dce.hpp"
+
+static const char *TAG = "sock_commands";
+
+namespace sock_commands {
+
+using namespace esp_modem;
+
+command_result net_open(CommandableIf *term)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    std::string response;
+    auto ret = dce_commands::generic_get_string(term, "AT+NETOPEN?\r", response, 1000);
+    if (ret != command_result::OK) {
+        return ret;
+    }
+    ESP_LOGV(TAG, "%s", response.data() );
+    if (response.find("+NETOPEN: 1") != std::string::npos) {
+        ESP_LOGD(TAG, "Already there");
+        ret = command_result::OK;
+    } else if (response.find("+NETOPEN: 0") != std::string::npos) {
+        ESP_LOGD(TAG, "Need to setup");
+        ret = dce_commands::generic_command(term, "AT+NETOPEN\r", "+NETOPEN: 1", "+NETOPEN: 0", 10000);
+    } else {
+        return command_result::FAIL;
+    }
+    if (ret != command_result::OK) {
+        return ret;
+    }
+    return dce_commands::generic_command(term, "AT+CIPRXGET=1\r", "OK", "ERROR", 5000);
+}
+
+command_result net_close(CommandableIf *term)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    return dce_commands::generic_command(term, "AT+NETCLOSE\r", "+NETCLOSE:", "ERROR", 30000);
+}
+
+command_result tcp_open(CommandableIf *term, const std::string &host, int port, int timeout)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    auto ret = dce_commands::generic_command(term, "AT+CIPRXGET=1\r", "OK", "ERROR", 50000);
+    if (ret != command_result::OK) {
+        ESP_LOGE(TAG, "Setting Rx mode failed!");
+        return ret;
+    }
+    ESP_LOGV(TAG, "%s", __func__ );
+    std::string ip_open = R"(AT+CIPOPEN=0,"TCP",")" + host + "\"," + std::to_string(port) + "\r";
+    ret = dce_commands::generic_command(term, ip_open, "+CIPOPEN: 0,0", "ERROR", timeout);
+    if (ret != command_result::OK) {
+        ESP_LOGE(TAG, "%s Failed", __func__ );
+        return ret;
+    }
+    return command_result::OK;
+}
+
+command_result tcp_close(CommandableIf *term)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    return dce_commands::generic_command(term, "AT+CIPCLOSE=0\r", "+CIPCLOSE:", "ERROR", 10000);
+}
+
+command_result tcp_send(CommandableIf *term, uint8_t *data, size_t len)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    std::string send = "AT+CIPSEND=0," + std::to_string(len) + "\r";
+    auto ret = term->command(send, [&](uint8_t *data, size_t len) {
+        std::string_view response((char *)data, len);
+        ESP_LOGI(TAG, "CIPSEND response %.*s", static_cast<int>(response.size()), response.data());
+        if (response.find('>') != std::string::npos) {
+            return command_result::OK;
+        }
+        return command_result::TIMEOUT;
+    }, 50000, '>');
+    if (ret != command_result::OK) {
+        return ret;
+    }
+    ret = command_result::TIMEOUT;
+    ESP_LOGW(TAG, "Before setting...");
+    term->on_read([&ret](uint8_t *cmd_data, size_t cmd_len) {
+        std::string_view response((char *)cmd_data, cmd_len);
+        ESP_LOGW(TAG, "CIPSEND response %.*s", static_cast<int>(response.size()), response.data());
+
+        if (response.find("+CIPSEND:") != std::string::npos) {
+            ret = command_result::OK;
+        } else if (response.find("ERROR") != std::string::npos) {
+            ret = command_result::FAIL;
+        }
+        return ret;
+    });
+    ESP_LOGW(TAG, "Before writing...");
+    auto written = term->write(data, len);
+    if (written != len) {
+        ESP_LOGE(TAG, "written %d (%d)...", written, len);
+        return command_result::FAIL;
+    }
+    uint8_t ctrl_z = '\x1A';
+    term->write(&ctrl_z, 1);
+    int count = 0;
+    while (ret == command_result::TIMEOUT && count++ < 1000 ) {
+        vTaskDelay(pdMS_TO_TICKS(1000));
+    }
+    term->on_read(nullptr);
+    return ret;
+}
+
+command_result tcp_recv(CommandableIf *term, uint8_t *data, size_t len, size_t &out_len)
+{
+    ESP_LOGV(TAG, "%s", __func__ );
+    std::string out;
+    auto ret = dce_commands::generic_get_string(term, "AT+CIPRXGET=4,0\r", out);
+    if (ret != command_result::OK) {
+        return ret;
+    }
+    constexpr std::string_view pattern = "+CIPRXGET: 4,0,";
+    if (out.find(pattern) == std::string::npos) {
+
+        return command_result::FAIL;
+    }
+    size_t data_len;
+    if (std::from_chars(out.data() + pattern.size(), out.data() + out.size(), data_len).ec == std::errc::invalid_argument) {
+        return command_result::FAIL;
+    }
+    ESP_LOGD(TAG, "size=%d", data_len);
+    if (data_len == 0) {
+        out_len = data_len;
+        return command_result::OK;
+    }
+    return term->command("AT+CIPRXGET=2,0,100\r", [&](uint8_t *cmd_data, size_t cmd_len) {
+        char pattern[] = "+CIPRXGET: 2,0,";
+        ESP_LOG_BUFFER_HEXDUMP(TAG, cmd_data, cmd_len, ESP_LOG_DEBUG);
+        char *pos = strstr((char *)cmd_data, pattern);
+        if (pos == nullptr) {
+            return command_result::FAIL;
+        }
+        auto p1 = memchr(pos + sizeof(pattern) - 1, ',', 4);
+        if (p1 == nullptr)  {
+            return command_result::FAIL;
+        }
+        *(char *)p1 = '\0';
+        size_t actual_len = atoi(pos + sizeof(pattern) - 1);
+        ESP_LOGD(TAG, "actual len=%d", actual_len);
+
+        pos = strchr((char *)p1 + 1, '\n');
+        if (pos == nullptr) {
+            ESP_LOGE(TAG, "not found");
+            return command_result::FAIL;
+        }
+        if (actual_len > len) {
+            ESP_LOGE(TAG, "TOO BIG");
+            return command_result::FAIL;
+        }
+        out_len = actual_len;
+        memcpy(data, pos + 1, actual_len);
+        pos = strstr((char *)pos + 1 + actual_len, "OK");
+        if (pos == nullptr) {
+            ESP_LOGE(TAG, "ok NOT FOUND");
+            return command_result::FAIL;
+        }
+        return command_result::OK;
+    }, 50000);
+}
+
+command_result get_ip(CommandableIf *term, std::string &ip)
+{
+    std::string resp;
+    auto ret = dce_commands::generic_get_string(term, "AT+IPADDR\r", resp, 5000);
+    if (ret != command_result::OK) {
+        return ret;
+    }
+    ip = resp;
+    return command_result::OK;
+}
+
+command_result set_rx_mode(CommandableIf *term, int mode)
+{
+    return dce_commands::generic_command(term, "AT+CIPRXGET=" + std::to_string(mode) + "\r", "OK", "ERROR", 5000);
+}
+
+} // sock_commands
+
+namespace sock_dce {
+
+void Responder::start_sending(size_t len)
+{
+    data_to_send = len;
+    send_stat = 0;
+    send_cmd("AT+CIPSEND=0," + std::to_string(len) + "\r");
+}
+
+void Responder::start_receiving(size_t len)
+{
+    send_cmd("AT+CIPRXGET=2,0," + std::to_string(len) + "\r");
+}
+
+bool Responder::start_connecting(std::string host, int port)
+{
+    send_cmd(R"(AT+CIPOPEN=0,"TCP",")" + host + "\"," + std::to_string(port) + "\r");
+    return true;
+}
+
+Responder::ret Responder::recv(uint8_t *data, size_t len)
+{
+    const int MIN_MESSAGE = 6;
+    size_t actual_len = 0;
+    auto *recv_data = (char *)data;
+    if (data_to_recv == 0) {
+        static constexpr std::string_view head = "+CIPRXGET: 2,0,";
+        auto head_pos = std::search(recv_data, recv_data + len, head.begin(), head.end());
+        if (head_pos == nullptr) {
+            return ret::FAIL;
+        }
+
+        if (head_pos - (char *)data > MIN_MESSAGE) {
+            // check for async replies before the Recv header
+            std::string_view response((char *)data, head_pos - (char *)data);
+            check_async_replies(status::RECEIVING, response);
+        }
+
+        auto next_comma = (char *)memchr(head_pos + head.size(), ',', MIN_MESSAGE);
+        if (next_comma == nullptr)  {
+            return ret::FAIL;
+        }
+        if (std::from_chars(head_pos + head.size(), next_comma, actual_len).ec == std::errc::invalid_argument) {
+            ESP_LOGE(TAG, "cannot convert");
+            return ret::FAIL;
+        }
+
+        auto next_nl = (char *)memchr(next_comma, '\n', 8 /* total_len size (~4) + markers */);
+        if (next_nl == nullptr) {
+            ESP_LOGE(TAG, "not found");
+            return ret::FAIL;
+        }
+        if (actual_len > buffer_size) {
+            ESP_LOGE(TAG, "TOO BIG");
+            return ret::FAIL;
+        }
+        size_t total_len = 0;
+        if (std::from_chars(next_comma + 1, next_nl - 1, total_len).ec == std::errc::invalid_argument) {
+            ESP_LOGE(TAG, "cannot convert");
+            return ret::FAIL;
+        }
+        read_again = (total_len > 0);
+        recv_data = next_nl + 1;
+        auto first_data_len = len - (recv_data - (char *)data) /* minus size of the command marker */;
+        if (actual_len > first_data_len) {
+            ::send(sock, recv_data, first_data_len, 0);
+            data_to_recv = actual_len - first_data_len;
+            return ret::NEED_MORE_DATA;
+        }
+        ::send(sock, recv_data, actual_len, 0);
+    } else if (data_to_recv > len) {    // continue sending
+        ::send(sock, recv_data, len, 0);
+        data_to_recv -= len;
+        return ret::NEED_MORE_DATA;
+    } else if (data_to_recv <= len) {    // last read -> looking for "OK" marker
+        ::send(sock, recv_data, data_to_recv, 0);
+        actual_len = data_to_recv;
+    }
+
+    // "OK" after the data
+    char *last_pos = nullptr;
+    if (actual_len + 1 + 2 /* OK */  > len) {
+        last_pos = (char *)memchr(recv_data + 1 + actual_len, 'O', MIN_MESSAGE);
+        if (last_pos == nullptr || last_pos[1] != 'K') {
+            data_to_recv = 0;
+            return ret::FAIL;
+        }
+    }
+    if (last_pos != nullptr && (char *)data + len - last_pos - 2 > MIN_MESSAGE) {
+        // check for async replies after the Recv header
+        std::string_view response((char *)last_pos + 2 /* OK */, (char *)data + len - last_pos - 2);
+        check_async_replies(status::RECEIVING, response);
+    }
+    data_to_recv = 0;
+    if (read_again) {
+        uint64_t data_ready = 1;
+        write(data_ready_fd, &data_ready, sizeof(data_ready));
+    }
+    return ret::OK;
+}
+
+Responder::ret Responder::send(uint8_t *data, size_t len)
+{
+    if (send_stat == 0) {
+        if (memchr(data, '>', len) == NULL) {
+            ESP_LOGE(TAG, "Missed >");
+            return ret::FAIL;
+        }
+        auto written = dte->write(&buffer[0], data_to_send);
+        if (written != data_to_send) {
+            ESP_LOGE(TAG, "written %d (%d)...", written, len);
+            return ret::FAIL;
+        }
+        data_to_send = 0;
+        uint8_t ctrl_z = '\x1A';
+        dte->write(&ctrl_z, 1);
+        send_stat++;
+        return ret::IN_PROGRESS;
+    }
+    return Responder::ret::IN_PROGRESS;
+}
+
+Responder::ret Responder::send(std::string_view response)
+{
+    if (send_stat == 1) {
+        if (response.find("+CIPSEND:") != std::string::npos) {
+            send_stat = 0;
+            return ret::OK;
+        }
+        if (response.find("ERROR") != std::string::npos) {
+            ESP_LOGE(TAG, "Failed to sent");
+            send_stat = 0;
+            return ret::FAIL;
+        }
+    }
+    return Responder::ret::IN_PROGRESS;
+}
+
+Responder::ret Responder::connect(std::string_view response)
+{
+    if (response.find("+CIPOPEN: 0,0") != std::string::npos) {
+        ESP_LOGI(TAG, "Connected!");
+        return ret::OK;
+    }
+    if (response.find("ERROR") != std::string::npos) {
+        ESP_LOGE(TAG, "Failed to open");
+        return ret::FAIL;
+    }
+    return Responder::ret::IN_PROGRESS;
+}
+
+Responder::ret Responder::check_async_replies(status state, std::string_view &response)
+{
+    ESP_LOGD(TAG, "response %.*s", static_cast<int>(response.size()), response.data());
+    if (response.find("+CIPRXGET: 1") != std::string::npos) {
+        uint64_t data_ready = 1;
+        write(data_ready_fd, &data_ready, sizeof(data_ready));
+        ESP_LOGD(TAG, "Got data on modem!");
+    }
+    if (state == status::SENDING) {
+        return send(response);
+    } else if (state == status::CONNECTING) {
+        return connect(response);
+    }
+    return ret::IN_PROGRESS;
+
+}
+
+Responder::ret Responder::process_data(status state, uint8_t *data, size_t len)
+{
+    if (state == status::SENDING) {
+        return send(data, len);
+    }
+    if (state == status::RECEIVING) {
+        return recv(data, len);
+    }
+    return Responder::ret::IN_PROGRESS;
+}
+
+status Responder::pending()
+{
+    return status::PENDING;
+}
+
+
+} // sock_dce

components/esp_modem/examples/modem_tcp_client/main/sock_dce.cpp

@@ -0,0 +1,326 @@
+/*
+ * SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+#include <charconv>
+#include <sys/socket.h>
+#include "esp_vfs.h"
+#include "esp_vfs_eventfd.h"
+
+#include "sock_dce.hpp"
+
+namespace sock_dce {
+
+constexpr auto const *TAG = "sock_dce";
+
+
+bool DCE::perform_sock()
+{
+    if (listen_sock == -1) {
+        ESP_LOGE(TAG, "Listening socket not ready");
+        close_sock();
+        return false;
+    }
+    if (sock == -1) {   // no active socket, need to accept one first
+        return accept_sock();
+    }
+
+    // we have a socket, let's check the status
+    struct timeval tv = {
+        .tv_sec = 0,
+        .tv_usec = 500000,
+    };
+    if (state == status::PENDING) {
+        vTaskDelay(pdMS_TO_TICKS(500));
+        state = at.pending();
+        return true;
+    }
+    fd_set fdset;
+    FD_ZERO(&fdset);
+    FD_SET(sock, &fdset);
+    FD_SET(data_ready_fd, &fdset);
+    int s = select(std::max(sock, data_ready_fd) + 1, &fdset, nullptr, nullptr, &tv);
+    if (s == 0) {
+        ESP_LOGV(TAG, "perform select timeout...");
+        return true;
+    } else if (s < 0) {
+        ESP_LOGE(TAG,  "select error %d", errno);
+        close_sock();
+        return false;
+    }
+    if (FD_ISSET(sock, &fdset) && !sock_to_at()) {
+        return false;
+    }
+    if (FD_ISSET(data_ready_fd, &fdset) && !at_to_sock()) {
+        return false;
+    }
+    return true;
+}
+
+void DCE::perform_at(uint8_t *data, size_t len)
+{
+    ESP_LOG_BUFFER_HEXDUMP(TAG, data, len, ESP_LOG_VERBOSE);
+    switch (at.process_data(state, data, len)) {
+    case Responder::ret::OK:
+        state = status::IDLE;
+        signal.set(IDLE);
+        return;
+    case Responder::ret::FAIL:
+        state = status::FAILED;
+        signal.set(IDLE);
+        return;
+    case Responder::ret::NEED_MORE_DATA:
+        return;
+    case Responder::ret::IN_PROGRESS:
+        break;
+    case Responder::ret::NEED_MORE_TIME:
+        state = status::PENDING;
+        return;
+    }
+    std::string_view response((char *)data, len);
+    switch (at.check_async_replies(state, response)) {
+    case Responder::ret::OK:
+        state = status::IDLE;
+        signal.set(IDLE);
+        return;
+    case Responder::ret::FAIL:
+        state = status::FAILED;
+        signal.set(IDLE);
+        return;
+    case Responder::ret::NEED_MORE_TIME:
+        state = status::PENDING;
+        return;
+    case Responder::ret::NEED_MORE_DATA:
+    case Responder::ret::IN_PROGRESS:
+        break;
+    }
+}
+
+void DCE::close_sock()
+{
+    if (sock > 0) {
+        close(sock);
+        sock = -1;
+    }
+    dte->on_read(nullptr);
+    const int retries = 5;
+    int i = 0;
+    while (net_close() != esp_modem::command_result::OK) {
+        if (i++ > retries) {
+            ESP_LOGE(TAG, "Failed to close network");
+            return;
+        }
+        esp_modem::Task::Delay(1000);
+    }
+}
+
+bool DCE::at_to_sock()
+{
+    uint64_t data;
+    read(data_ready_fd, &data, sizeof(data));
+    ESP_LOGD(TAG, "select read: modem data available %x", data);
+    if (!signal.wait(IDLE, 1000)) {
+        ESP_LOGE(TAG, "Failed to get idle");
+        close_sock();
+        return false;
+    }
+    if (state != status::IDLE) {
+        ESP_LOGE(TAG, "Unexpected state %d", state);
+        close_sock();
+        return false;
+    }
+    state = status::RECEIVING;
+    at.start_receiving(at.get_buf_len());
+    return true;
+}
+
+bool DCE::sock_to_at()
+{
+    ESP_LOGD(TAG,  "socket read: data available");
+    if (!signal.wait(IDLE, 1000)) {
+        ESP_LOGE(TAG,  "Failed to get idle");
+        close_sock();
+        return false;
+    }
+    if (state != status::IDLE) {
+        ESP_LOGE(TAG,  "Unexpected state %d", state);
+        close_sock();
+        return false;
+    }
+    state = status::SENDING;
+    int len = ::recv(sock, at.get_buf(), at.get_buf_len(), 0);
+    if (len < 0) {
+        ESP_LOGE(TAG,  "read error %d", errno);
+        close_sock();
+        return false;
+    } else if (len == 0) {
+        ESP_LOGE(TAG,  "EOF %d", errno);
+        close_sock();
+        return false;
+    }
+    ESP_LOG_BUFFER_HEXDUMP(TAG, at.get_buf(), len, ESP_LOG_VERBOSE);
+    at.start_sending(len);
+    return true;
+}
+
+bool DCE::accept_sock()
+{
+    struct timeval tv = {
+        .tv_sec = 0,
+        .tv_usec = 500000,
+    };
+    fd_set fdset;
+    FD_ZERO(&fdset);
+    FD_SET(listen_sock, &fdset);
+    int s = select(listen_sock + 1, &fdset, nullptr, nullptr, &tv);
+    if (s > 0 && FD_ISSET(listen_sock, &fdset)) {
+        struct sockaddr_in source_addr = {};
+        socklen_t addr_len = sizeof(source_addr);
+        sock = accept(listen_sock, (struct sockaddr *)&source_addr, &addr_len);
+        if (sock < 0) {
+            ESP_LOGE(TAG, "Unable to accept connection: errno %d", errno);
+            return false;
+        }
+        ESP_LOGD(TAG, "Socket accepted!");
+        FD_ZERO(&fdset);
+        return true;
+    } else if (s == 0) {
+        return true;
+    }
+    return false;
+}
+
+void DCE::init_sock(int port)
+{
+    esp_vfs_eventfd_config_t config = ESP_VFS_EVENTD_CONFIG_DEFAULT();
+    esp_vfs_eventfd_register(&config);
+
+    data_ready_fd = eventfd(0, EFD_SUPPORT_ISR);
+    assert(data_ready_fd > 0);
+
+    listen_sock = socket(AF_INET, SOCK_STREAM, IPPROTO_IP);
+    if (listen_sock < 0) {
+        ESP_LOGE(TAG, "Unable to create socket: errno %d", errno);
+        return;
+    }
+    int opt = 1;
+    setsockopt(listen_sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
+    ESP_LOGI(TAG, "Socket created");
+    struct sockaddr_in addr = { };
+    addr.sin_family = AF_INET;
+    addr.sin_port = htons(port);
+    addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+//    inet_aton("127.0.0.1", &addr.sin_addr);
+
+    int err = bind(listen_sock, (struct sockaddr *)&addr, sizeof(addr));
+    if (err != 0) {
+        ESP_LOGE(TAG, "Socket unable to bind: errno %d", errno);
+        return;
+    }
+    ESP_LOGI(TAG, "Socket bound, port %d", 1883);
+    err = listen(listen_sock, 1);
+    if (err != 0) {
+        ESP_LOGE(TAG, "Error occurred during listen: errno %d", errno);
+        return;
+    }
+
+}
+
+bool DCE::start(std::string host, int port)
+{
+    dte->on_read(nullptr);
+    tcp_close();
+    dte->on_read([this](uint8_t *data, size_t len) {
+        this->perform_at(data, len);
+        return esp_modem::command_result::TIMEOUT;
+    });
+    if (!at.start_connecting(host, port)) {
+        ESP_LOGE(TAG, "Unable to start connecting");
+        dte->on_read(nullptr);
+        return false;
+    }
+    state = status::CONNECTING;
+    return true;
+}
+
+bool DCE::init_network()
+{
+    dte->on_read(nullptr);
+    const int retries = 5;
+    int i = 0;
+    while (sync() != esp_modem::command_result::OK) {
+        if (i++ > retries) {
+            ESP_LOGE(TAG, "Failed to sync up");
+            return false;
+        }
+        esp_modem::Task::Delay(1000);
+    }
+    ESP_LOGD(TAG, "Modem in sync");
+    i = 0;
+    while (setup_data_mode() != true) {
+        if (i++ > retries) {
+            ESP_LOGE(TAG, "Failed to setup pdp/data");
+            return false;
+        }
+        esp_modem::Task::Delay(1000);
+    }
+    ESP_LOGD(TAG, "PDP configured");
+    i = 0;
+    while (net_open() != esp_modem::command_result::OK) {
+        if (i++ > retries) {
+            ESP_LOGE(TAG, "Failed to open network");
+            return false;
+        }
+        net_close();
+        esp_modem::Task::Delay(1000);
+    }
+    ESP_LOGD(TAG, "Network opened");
+    i = 0;
+    std::string ip_addr;
+    while (get_ip(ip_addr) != esp_modem::command_result::OK) {
+        if (i++ > retries) {
+            ESP_LOGE(TAG, "Failed obtain an IP address");
+            return false;
+        }
+        esp_modem::Task::Delay(5000);
+    }
+    ESP_LOGI(TAG, "Got IP %s", ip_addr.c_str());
+    return true;
+}
+
+
+class Factory: public ::esp_modem::dce_factory::Factory {
+public:
+    static std::unique_ptr<DCE> create(const esp_modem::dce_config *config, std::shared_ptr<esp_modem::DTE> dte)
+    {
+        return esp_modem::dce_factory::Factory::build_module_T<DCE, std::unique_ptr<DCE>>(config, std::move(dte));
+    }
+};
+
+std::unique_ptr<DCE> create(const esp_modem::dce_config *config, std::shared_ptr<esp_modem::DTE> dte)
+{
+    return Factory::create(config, std::move(dte));
+}
+
+// Helper macros to handle multiple arguments of declared API
+#define ARGS0
+#define ARGS1 , p1
+#define ARGS2 , p1 , p2
+#define ARGS3 , p1 , p2 , p3
+
+#define EXPAND_ARGS(x)  ARGS ## x
+#define ARGS(x)  EXPAND_ARGS(x)
+
+//
+// Repeat all declarations and forward to the AT commands defined in ::sock_commands namespace
+//
+#define ESP_MODEM_DECLARE_DCE_COMMAND(name, return_type, arg_nr, ...) \
+     esp_modem::return_type DCE::name(__VA_ARGS__) { return sock_commands::name(dte.get() ARGS(arg_nr) ); }
+
+DECLARE_SOCK_COMMANDS(return_type name(...) )
+
+#undef ESP_MODEM_DECLARE_DCE_COMMAND
+
+} // namespace sock_dce

components/esp_modem/examples/modem_tcp_client/main/sock_dce.hpp

@@ -0,0 +1,108 @@
+/*
+ * SPDX-FileCopyrightText: 2021-2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+#include "esp_modem_config.h"
+#include "cxx_include/esp_modem_api.hpp"
+#include <cxx_include/esp_modem_dce_factory.hpp>
+#include "socket_commands.inc"
+#include "sock_commands.hpp"
+
+#pragma  once
+
+namespace sock_dce {
+
+
+enum class status {
+    IDLE,
+    CONNECTING,
+    SENDING,
+    RECEIVING,
+    FAILED,
+    PENDING
+};
+
+class Responder {
+public:
+    enum class ret {
+        OK, FAIL, IN_PROGRESS, NEED_MORE_DATA, NEED_MORE_TIME
+    };
+    Responder(int &s, int &ready_fd, std::shared_ptr<esp_modem::DTE> &dte_arg):
+        sock(s), data_ready_fd(ready_fd), dte(dte_arg) {}
+    ret process_data(status state, uint8_t *data, size_t len);
+    ret check_async_replies(status state, std::string_view &response);
+
+    void start_sending(size_t len);
+    void start_receiving(size_t len);
+    bool start_connecting(std::string host, int port);
+    status pending();
+    uint8_t *get_buf()
+    {
+        return &buffer[0];
+    }
+    size_t get_buf_len()
+    {
+        return buffer_size;
+    }
+private:
+    static constexpr size_t buffer_size = 512;
+
+    ret recv(uint8_t *data, size_t len);
+    ret send(uint8_t *data, size_t len);
+    ret send(std::string_view response);
+    ret connect(std::string_view response);
+    void send_cmd(std::string_view command)
+    {
+        dte->write((uint8_t *) command.begin(), command.size());
+    }
+    std::array<uint8_t, buffer_size> buffer;
+    size_t data_to_recv = 0;
+    bool read_again = false;
+    int &sock;
+    int &data_ready_fd;
+    int send_stat = 0;
+    size_t data_to_send = 0;
+    std::shared_ptr<esp_modem::DTE> &dte;
+};
+
+class DCE : public ::esp_modem::GenericModule {
+    using esp_modem::GenericModule::GenericModule;
+public:
+
+#define ESP_MODEM_DECLARE_DCE_COMMAND(name, return_type, num, ...) \
+esp_modem::return_type name(__VA_ARGS__);
+
+    DECLARE_SOCK_COMMANDS(declare name(Commandable *p, ...);)
+
+#undef ESP_MODEM_DECLARE_DCE_COMMAND
+
+    bool init_network();
+    bool start(std::string host, int port);
+
+    void init_sock(int port);
+
+    bool perform_sock();
+
+private:
+    esp_modem::SignalGroup signal;
+
+    void close_sock();
+    bool accept_sock();
+    bool sock_to_at();
+    bool at_to_sock();
+
+    void perform_at(uint8_t *data, size_t len);
+
+    status state{status::IDLE};
+    static constexpr uint8_t IDLE = 1;
+    Responder at{sock, data_ready_fd, dte};
+    int sock {-1};
+    int listen_sock {-1};
+    int data_ready_fd {-1};
+};
+
+std::unique_ptr<DCE> create(const esp_modem::dce_config *config, std::shared_ptr<esp_modem::DTE> dte);
+
+}

components/esp_modem/examples/modem_tcp_client/main/socket_commands.inc

@@ -0,0 +1,58 @@
+// Copyright 2021-2022 Espressif Systems (Shanghai) PTE LTD
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+#pragma once
+
+#include "generate/esp_modem_command_declare_helper.inc"
+
+#define DECLARE_SOCK_COMMANDS(...) \
+/**
+ * @brief Opens network in AT command mode
+ * @return OK, FAIL or TIMEOUT
+ */ \
+ESP_MODEM_DECLARE_DCE_COMMAND(net_open, command_result, 0) \
+    \
+/**
+ * @brief Closes network in AT command mode
+ * @return OK, FAIL or TIMEOUT
+ */ \
+ESP_MODEM_DECLARE_DCE_COMMAND(net_close, command_result, 0) \
+    \
+/**
+ * @brief Opens a TCP connection
+ * @param[in] host Host name or IP address to connect to
+ * @param[in] port Port number
+ * @param[in] timeout Connection timeout
+ * @return OK, FAIL or TIMEOUT
+ */ \
+ESP_MODEM_DECLARE_DCE_COMMAND(tcp_open, command_result, 3, STRING_IN(p1, host), INT_IN(p2, port), INT_IN(p3, timeout)) \
+    \
+/**
+ * @brief Closes opened TCP socket
+ * @return OK, FAIL or TIMEOUT
+ */ \
+ESP_MODEM_DECLARE_DCE_COMMAND(tcp_close, command_result, 0) \
+    \
+/**
+ * @brief Gets modem IP address
+ * @param[out] addr String representation of modem's IP
+ * @return OK, FAIL or TIMEOUT
+ */ \
+ESP_MODEM_DECLARE_DCE_COMMAND(get_ip, command_result, 1, STRING_OUT(p1, addr)) \
+    \
+/**
+ * @brief Sets Rx mode
+ * @param[in] mode 0=auto, 1=manual
+ * @return OK, FAIL or TIMEOUT
+ */ \
+ESP_MODEM_DECLARE_DCE_COMMAND(set_rx_mode, command_result, 1, INT_IN(p1, mode))

components/esp_modem/examples/pppos_client/main/Kconfig.projbuild

@@ -15,12 +15,12 @@ menu "Example Configuration"
     endchoice
 
     choice EXAMPLE_MODEM_DEVICE
-        depends on EXAMPLE_SERIAL_CONFIG_UART
         prompt "Choose supported modem device (DCE)"
         default EXAMPLE_MODEM_DEVICE_BG96
         help
             Select modem device connected to the ESP DTE.
         config EXAMPLE_MODEM_DEVICE_SIM800
+            depends on EXAMPLE_SERIAL_CONFIG_UART
             bool "SIM800"
             help
                 SIMCom SIM800L is a GSM/GPRS module.
@@ -30,10 +30,12 @@ menu "Example Configuration"
             help
                 Quectel BG96 is a series of LTE Cat M1/Cat NB1/EGPRS module.
         config EXAMPLE_MODEM_DEVICE_SIM7000
+            depends on EXAMPLE_SERIAL_CONFIG_UART
             bool "SIM7000"
             help
                 SIM7000 is a Multi-Band LTE-FDD and GSM/GPRS/EDGE module.
         config EXAMPLE_MODEM_DEVICE_SIM7070
+            depends on EXAMPLE_SERIAL_CONFIG_UART
             bool "SIM7070"
             help
                 SIM7070 is Multi-Band CAT M and NB IoT module.
@@ -41,6 +43,11 @@ menu "Example Configuration"
             bool "SIM7600"
             help
                 SIM7600 is a Multi-Band LTE-TDD/LTE-FDD/HSPA+ and GSM/GPRS/EDGE module.
+        config EXAMPLE_MODEM_DEVICE_A7670
+            depends on EXAMPLE_SERIAL_CONFIG_USB
+            bool "A7670"
+            help
+                A7670X is Multi-Band LTE-FDD/LTE-TDD/GSM/GPRS/EDGE module.
     endchoice
 
     config EXAMPLE_MODEM_PPP_APN

components/esp_modem/examples/pppos_client/main/idf_component.yml

@@ -6,7 +6,7 @@ dependencies:
     version: "^0.1.23"
     override_path: "../../../"
   espressif/esp_modem_usb_dte:
-    version: "^1.0.0"
+    version: "^1.1.0"
     rules:
       - if: "idf_version >=4.4"
       - if: "target in [esp32s2, esp32s3]"

components/esp_modem/examples/pppos_client/main/pppos_client_main.c

@@ -208,14 +208,28 @@ void app_main(void)
 
 #elif defined(CONFIG_EXAMPLE_SERIAL_CONFIG_USB)
     while (1) {
+#if CONFIG_EXAMPLE_MODEM_DEVICE_BG96 == 1
         ESP_LOGI(TAG, "Initializing esp_modem for the BG96 module...");
-        struct esp_modem_usb_term_config usb_config = ESP_MODEM_DEFAULT_USB_CONFIG(0x2C7C, 0x0296, 2); // VID, PID and interface num of BG96 modem
+        struct esp_modem_usb_term_config usb_config = ESP_MODEM_BG96_USB_CONFIG();
+        esp_modem_dce_device_t usb_dev_type = ESP_MODEM_DCE_BG96;
+#elif CONFIG_EXAMPLE_MODEM_DEVICE_SIM7600 == 1
+        ESP_LOGI(TAG, "Initializing esp_modem for the SIM7600 module...");
+        struct esp_modem_usb_term_config usb_config = ESP_MODEM_SIM7600_USB_CONFIG();
+        esp_modem_dce_device_t usb_dev_type = ESP_MODEM_DCE_SIM7600;
+#elif CONFIG_EXAMPLE_MODEM_DEVICE_A7670 == 1
+        ESP_LOGI(TAG, "Initializing esp_modem for the A7670 module...");
+        struct esp_modem_usb_term_config usb_config = ESP_MODEM_A7670_USB_CONFIG();
+        esp_modem_dce_device_t usb_dev_type = ESP_MODEM_DCE_SIM7600;
+#else
+#error USB modem not selected
+#endif
         const esp_modem_dte_config_t dte_usb_config = ESP_MODEM_DTE_DEFAULT_USB_CONFIG(usb_config);
         ESP_LOGI(TAG, "Waiting for USB device connection...");
-        esp_modem_dce_t *dce = esp_modem_new_dev_usb(ESP_MODEM_DCE_BG96, &dte_usb_config, &dce_config, esp_netif);
+        esp_modem_dce_t *dce = esp_modem_new_dev_usb(usb_dev_type, &dte_usb_config, &dce_config, esp_netif);
         assert(dce);
         esp_modem_set_error_cb(dce, usb_terminal_error_handler);
-        vTaskDelay(pdMS_TO_TICKS(1000)); // Although the DTE should be ready after USB enumeration, sometimes it fails to respond without this delay
+        ESP_LOGI(TAG, "Modem connected, waiting 10 seconds for boot...");
+        vTaskDelay(pdMS_TO_TICKS(10000)); // Give DTE some time to boot
 
 #else
 #error Invalid serial connection to modem.

components/esp_modem/examples/pppos_client/sdkconfig.ci.sim800_c3

@@ -1,3 +1,4 @@
+CONFIG_IDF_TARGET="esp32c3"
 # Override some defaults to enable PPP
 CONFIG_LWIP_PPP_SUPPORT=y
 CONFIG_LWIP_PPP_NOTIFY_PHASE_SUPPORT=y

components/esp_modem/examples/pppos_client/sdkconfig.ci.usb_s2

@@ -0,0 +1,2 @@
+CONFIG_IDF_TARGET="esp32s2"
+CONFIG_EXAMPLE_SERIAL_CONFIG_USB=y

components/esp_modem/examples/pppos_client/sdkconfig.ci.usb_s3

@@ -0,0 +1,2 @@
+CONFIG_IDF_TARGET="esp32s3"
+CONFIG_EXAMPLE_SERIAL_CONFIG_USB=y

components/esp_modem/examples/simple_cmux_client/CMakeLists.txt

@@ -1,6 +1,7 @@
 # The following lines of boilerplate have to be in your project's CMakeLists
 # in this exact order for cmake to work correctly
-cmake_minimum_required(VERSION 3.5)
+cmake_minimum_required(VERSION 3.8)
+set(CMAKE_CXX_STANDARD 17)
 
 set(EXTRA_COMPONENT_DIRS "../..")
 

components/esp_modem/idf_component.yml

@@ -1,4 +1,4 @@
-version: "0.1.26"
+version: "1.0.0"
 description: esp modem
 url: https://github.com/espressif/esp-protocols/tree/master/components/esp_modem
 dependencies:

components/esp_modem/include/cxx_include/esp_modem_command_library.hpp

@@ -22,6 +22,19 @@ namespace dce_commands {
  * @{
  */
 
+/**
+ * @brief Generic AT command to be send with pass and fail phrases
+ *
+ * @param t Commandable object (anything that can accept commands)
+ * @param command Command to be sent do the commandable object
+ * @param pass_phrase String to be present in the reply to pass this command
+ * @param fail_phrase String to be present in the reply to fail this command
+ * @param timeout_ms Timeout in ms
+ */
+command_result generic_command(CommandableIf *t, const std::string &command,
+                               const std::string &pass_phrase,
+                               const std::string &fail_phrase, uint32_t timeout_ms);
+
 /**
  * @brief Declaration of all commands is generated from esp_modem_command_declare.inc
  */

components/esp_modem/include/cxx_include/esp_modem_dce_factory.hpp

@@ -69,6 +69,10 @@ public:
         ESP_MODEM_THROW_IF_FALSE(netif != nullptr, "Null netif");
     }
 
+    explicit Creator(std::shared_ptr<DTE> dte): dte(std::move(dte)), device(nullptr), netif(nullptr)
+    {
+    }
+
     ~Creator()
     {
         if (device != nullptr) {

components/esp_modem/include/cxx_include/esp_modem_dte.hpp

@@ -29,6 +29,13 @@ class CMux;
 * @{
 */
 
+struct DTE_Command {
+    DTE_Command(const std::string &cmd): data((uint8_t *)cmd.c_str()), len(cmd.length()) {}
+
+    uint8_t *data;
+    size_t len;
+};
+
 /**
  * DTE (Data Terminal Equipment) class
  */
@@ -39,9 +46,12 @@ public:
      * @brief Creates a DTE instance from the terminal
      * @param config DTE config structure
      * @param t unique-ptr to Terminal
+     * @param s unique-ptr to secondary Terminal
      */
     explicit DTE(const esp_modem_dte_config *config, std::unique_ptr<Terminal> t);
     explicit DTE(std::unique_ptr<Terminal> t);
+    explicit DTE(const esp_modem_dte_config *config, std::unique_ptr<Terminal> t, std::unique_ptr<Terminal> s);
+    explicit DTE(std::unique_ptr<Terminal> t, std::unique_ptr<Terminal> s);
 
     ~DTE() = default;
 
@@ -51,7 +61,9 @@ public:
      * @param len Data len to write
      * @return number of bytes written
      */
-    int write(uint8_t *data, size_t len);
+    int write(uint8_t *data, size_t len) override;
+
+    int write(DTE_Command command);
 
     /**
      * @brief Reading from the underlying terminal
@@ -67,6 +79,8 @@ public:
      */
     void set_read_cb(std::function<bool(uint8_t *data, size_t len)> f);
 
+    void on_read(got_line_cb on_data) override;
+
     /**
      * @brief Sets DTE error callback
      * @param f Function to be called on DTE error

components/esp_modem/include/cxx_include/esp_modem_types.hpp

@@ -27,8 +27,9 @@ namespace esp_modem {
  */
 enum class modem_mode {
     UNDEF,
-    COMMAND_MODE, /*!< Command mode -- the modem is supposed to send AT commands in this mode  */
+    COMMAND_MODE, /*!< Command mode -- the modem is supposed to send AT commands in this mode */
     DATA_MODE,    /*!< Data mode -- the modem communicates with network interface on PPP protocol */
+    DUAL_MODE,    /*!< Dual mode -- the modem has two real terminals. Data and commands work at the same time */
     CMUX_MODE,    /*!< CMUX (Multiplex mode) -- Simplified CMUX mode, which creates two virtual terminals,
                    *  assigning one solely to command interface and the other  to the data mode */
     CMUX_MANUAL_MODE,    /*!< Enter CMUX mode manually -- just creates two virtual terminals */
@@ -79,6 +80,9 @@ public:
      */
     virtual command_result command(const std::string &command, got_line_cb got_line, uint32_t time_ms, const char separator) = 0;
     virtual command_result command(const std::string &command, got_line_cb got_line, uint32_t time_ms) = 0;
+
+    virtual int write(uint8_t *data, size_t len) = 0;
+    virtual void on_read(got_line_cb on_data) = 0;
 };
 
 /**

components/esp_modem/include/esp_modem_config.h

@@ -84,6 +84,11 @@ struct esp_modem_dte_config {
     };
 };
 
+#if ESP_IDF_VERSION_MAJOR >= 5
+#define ESP_MODEM_DEFAULT_UART_CLK      UART_SCLK_DEFAULT
+#else
+#define ESP_MODEM_DEFAULT_UART_CLK      UART_SCLK_APB
+#endif
 
 /**
  * @brief ESP Modem DTE Default Configuration
@@ -92,15 +97,15 @@ struct esp_modem_dte_config {
 #define ESP_MODEM_DTE_DEFAULT_CONFIG() \
     {                                  \
         .dte_buffer_size = 512,        \
-        .task_stack_size = 4096, \
-        .task_priority = 5,      \
+        .task_stack_size = 4096,       \
+        .task_priority = 5,            \
         .uart_config = {               \
             .port_num = UART_NUM_1,                 \
             .data_bits = UART_DATA_8_BITS,          \
             .stop_bits = UART_STOP_BITS_1,          \
             .parity = UART_PARITY_DISABLE,          \
             .flow_control = ESP_MODEM_FLOW_CONTROL_NONE,\
-            .source_clk = UART_SCLK_APB,            \
+            .source_clk = ESP_MODEM_DEFAULT_UART_CLK,   \
             .baud_rate = 115200,                    \
             .tx_io_num = 25,                        \
             .rx_io_num = 26,                        \

components/esp_modem/include/generate/esp_modem_command_declare.inc

@@ -1,4 +1,4 @@
-// Copyright 2021 Espressif Systems (Shanghai) PTE LTD
+// Copyright 2021-2022 Espressif Systems (Shanghai) PTE LTD
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -11,36 +11,11 @@
 // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 // See the License for the specific language governing permissions and
 // limitations under the License.
+#pragma once
 
-#ifndef _ESP_MODEM_COMMAND_DECLARE_INC_
-#define _ESP_MODEM_COMMAND_DECLARE_INC_
+#include "esp_modem_command_declare_helper.inc"
 
 
-// Parameters
-// * handle different parameters for C++ and C API
-// * make parameter unique names, so they could be easily referenced and forwarded
-#define _ARG(param, name) param
-#define INT_IN(param, name) int  _ARG(param, name)
-#ifdef __cplusplus
-#include <string>
-#define STRING_IN(param, name) const std::string& _ARG(param, name)
-#define STRING_OUT(param, name) std::string& _ARG(param, name)
-#define BOOL_IN(param, name) const bool  _ARG(param, name)
-#define BOOL_OUT(param, name) bool&  _ARG(param, name)
-#define INT_OUT(param, name) int&  _ARG(param, name)
-#define INTEGER_LIST_IN(param, name) const int* _ARG(param, name)
-
-#define STRUCT_OUT(struct_name, p1)  struct_name& p1
-#else
-#define STRING_IN(param, name) const char* _ARG(param, name)
-#define STRING_OUT(param, name) char* _ARG(param, name)
-#define BOOL_IN(param, name) const bool _ARG(param, name)
-#define BOOL_OUT(param, name) bool* _ARG(param, name)
-#define INT_OUT(param, name) int* _ARG(param, name)
-#define INTEGER_LIST_IN(param, name) const int* _ARG(param, name)
-#define STRUCT_OUT(struct_name, p1)  esp_modem_ ## struct_name ## _t* p1
-#endif
-
 #define DECLARE_ALL_COMMAND_APIS(...) \
 /**
  * @brief Sends the initial AT sequence to sync up with the device
@@ -331,5 +306,3 @@ public:
 #endif
 
 #endif
-
-#endif // _ESP_MODEM_COMMAND_DECLARE_INC_

components/esp_modem/include/generate/esp_modem_command_declare_helper.inc

@@ -0,0 +1,26 @@
+
+
+// Parameters
+// * handle different parameters for C++ and C API
+// * make parameter unique names, so they could be easily referenced and forwarded
+#define _ARG(param, name) param
+#define INT_IN(param, name) int  _ARG(param, name)
+#ifdef __cplusplus
+#include <string>
+#define STRING_IN(param, name) const std::string& _ARG(param, name)
+#define STRING_OUT(param, name) std::string& _ARG(param, name)
+#define BOOL_IN(param, name) const bool  _ARG(param, name)
+#define BOOL_OUT(param, name) bool&  _ARG(param, name)
+#define INT_OUT(param, name) int&  _ARG(param, name)
+#define INTEGER_LIST_IN(param, name) const int* _ARG(param, name)
+
+#define STRUCT_OUT(struct_name, p1)  struct_name& p1
+#else
+#define STRING_IN(param, name) const char* _ARG(param, name)
+#define STRING_OUT(param, name) char* _ARG(param, name)
+#define BOOL_IN(param, name) const bool _ARG(param, name)
+#define BOOL_OUT(param, name) bool* _ARG(param, name)
+#define INT_OUT(param, name) int* _ARG(param, name)
+#define INTEGER_LIST_IN(param, name) const int* _ARG(param, name)
+#define STRUCT_OUT(struct_name, p1)  esp_modem_ ## struct_name ## _t* p1
+#endif

components/esp_modem/src/esp_modem_dce.cpp

@@ -88,6 +88,7 @@ bool DCE_Mode::set_unsafe(DTE *dte, ModuleIf *device, Netif &netif, modem_mode m
 {
     switch (m) {
     case modem_mode::UNDEF:
+    case modem_mode::DUAL_MODE: // Only DTE can be in Dual mode
         break;
     case modem_mode::COMMAND_MODE:
         if (mode == modem_mode::COMMAND_MODE || mode >= modem_mode::CMUX_MANUAL_MODE) {

components/esp_modem/src/esp_modem_dte.cpp

@@ -24,6 +24,16 @@ DTE::DTE(std::unique_ptr<Terminal> terminal):
     cmux_term(nullptr), primary_term(std::move(terminal)), secondary_term(primary_term),
     mode(modem_mode::UNDEF) {}
 
+DTE::DTE(const esp_modem_dte_config *config, std::unique_ptr<Terminal> t, std::unique_ptr<Terminal> s):
+    buffer(config->dte_buffer_size),
+    cmux_term(nullptr), primary_term(std::move(t)), secondary_term(std::move(s)),
+    mode(modem_mode::DUAL_MODE) {}
+
+DTE::DTE(std::unique_ptr<Terminal> t, std::unique_ptr<Terminal> s):
+    buffer(dte_default_buffer_size),
+    cmux_term(nullptr), primary_term(std::move(t)), secondary_term(std::move(s)),
+    mode(modem_mode::DUAL_MODE) {}
+
 command_result DTE::command(const std::string &command, got_line_cb got_line, uint32_t time_ms, const char separator)
 {
     Scoped<Lock> l(internal_lock);
@@ -104,9 +114,9 @@ bool DTE::set_mode(modem_mode m)
             return false;
         }
     }
-    // transitions (COMMAND|CMUX|UNDEF) -> DATA
+    // transitions (COMMAND|DUAL|CMUX|UNDEF) -> DATA
     if (m == modem_mode::DATA_MODE) {
-        if (mode == modem_mode::CMUX_MODE || mode == modem_mode::CMUX_MANUAL_MODE) {
+        if (mode == modem_mode::CMUX_MODE || mode == modem_mode::CMUX_MANUAL_MODE || mode == modem_mode::DUAL_MODE) {
             // mode stays the same, but need to swap terminals (as command has been switched)
             secondary_term.swap(primary_term);
         } else {
@@ -114,7 +124,7 @@ bool DTE::set_mode(modem_mode m)
         }
         return true;
     }
-    // transitions (DATA|CMUX|UNDEF) -> COMMAND
+    // transitions (DATA|DUAL|CMUX|UNDEF) -> COMMAND
     if (m == modem_mode::COMMAND_MODE) {
         if (mode == modem_mode::CMUX_MODE) {
             if (exit_cmux()) {
@@ -123,7 +133,7 @@ bool DTE::set_mode(modem_mode m)
             }
             mode = modem_mode::UNDEF;
             return false;
-        } if (mode == modem_mode::CMUX_MANUAL_MODE) {
+        } if (mode == modem_mode::CMUX_MANUAL_MODE || mode == modem_mode::DUAL_MODE) {
             return true;
         } else {
             mode = m;
@@ -192,6 +202,34 @@ int DTE::write(uint8_t *data, size_t len)
     return secondary_term->write(data, len);
 }
 
+int DTE::write(DTE_Command command)
+{
+    return primary_term->write(command.data, command.len);
+}
+
+void DTE::on_read(got_line_cb on_read_cb)
+{
+    if (on_read_cb == nullptr) {
+        primary_term->set_read_cb(nullptr);
+        internal_lock.unlock();
+        return;
+    }
+    internal_lock.lock();
+    primary_term->set_read_cb([this, on_read_cb](uint8_t *data, size_t len) {
+        if (!data) {
+            data = buffer.get();
+            len = primary_term->read(data, buffer.size);
+        }
+        auto res = on_read_cb(data, len);
+        if (res == command_result::OK || res == command_result::FAIL) {
+            primary_term->set_read_cb(nullptr);
+            internal_lock.unlock();
+            return true;
+        }
+        return false;
+    });
+}
+
 /**
  * Implemented here to keep all headers C++11 compliant
  */

components/esp_modem/src/esp_modem_netif.cpp

@@ -87,7 +87,7 @@ void Netif::start()
 {
     ppp_dte->set_read_cb([this](uint8_t *data, size_t len) -> bool {
         receive(data, len);
-        return false;
+        return true;
     });
     esp_netif_action_start(driver.base.netif, nullptr, 0, nullptr);
     signal.set(PPP_STARTED);

components/esp_modem/src/esp_modem_netif_linux.cpp

@@ -39,7 +39,7 @@ void Netif::start()
 {
     ppp_dte->set_read_cb([this](uint8_t *data, size_t len) -> bool {
         receive(data, len);
-        return false;
+        return true;
     });
     netif->transmit = esp_modem_dte_transmit;
     netif->ctx = (void *)this;

components/esp_modem/src/esp_modem_uart.cpp

@@ -125,9 +125,7 @@ void UartTerminal::task()
             case UART_DATA:
                 uart_get_buffered_data_len(uart.port, &len);
                 if (len && on_read) {
-                    if (on_read(nullptr, len)) {
-                        on_read = nullptr;
-                    }
+                    on_read(nullptr, len);
                 }
                 break;
             case UART_FIFO_OVF:

components/esp_mqtt_cxx/CHANGELOG.md

@@ -0,0 +1,7 @@
+# Changelog
+
+## [0.1.0](https://github.com/espressif/esp-protocols/commits/1407dfc)
+
+### Updated
+
+- Adds esp_mqtt_cxx component ([1407dfc](https://github.com/espressif/esp-protocols/commit/1407dfc))

components/esp_mqtt_cxx/CMakeLists.txt

@@ -0,0 +1,8 @@
+idf_build_get_property(target IDF_TARGET)
+
+idf_component_register(SRCS "esp_mqtt_cxx.cpp"
+                    INCLUDE_DIRS "include"
+                    REQUIRES mqtt
+                    )
+
+target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")

components/esp_mqtt_cxx/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

components/esp_mqtt_cxx/docs/Doxyfile

@@ -0,0 +1,73 @@
+# This is Doxygen configuration file
+#
+# Doxygen provides over 260 configuration statements
+# To make this file easier to follow,
+# it contains only statements that are non-default
+#
+# NOTE:
+# It is recommended not to change defaults unless specifically required
+# Test any changes how they affect generated documentation
+# Make sure that correct warnings are generated to flag issues with documented code
+#
+# For the complete list of configuration statements see:
+# http://doxygen.nl/manual/config.html
+
+
+PROJECT_NAME = "MQTT C++ client"
+
+## The 'INPUT' statement below is used as input by script 'gen-df-input.py'
+## to automatically generate API reference list files heder_file.inc
+## These files are placed in '_inc' directory
+## and used to include in API reference documentation
+
+INPUT = \
+    $(PROJECT_PATH)/include/esp_mqtt.hpp \
+    $(PROJECT_PATH)/include/esp_mqtt_client_config.hpp
+
+## Get warnings for functions that have no documentation for their parameters or return value
+##
+WARN_NO_PARAMDOC = YES
+
+## Enable preprocessing and remove __attribute__(...) expressions from the INPUT files
+##
+ENABLE_PREPROCESSING   = YES
+MACRO_EXPANSION        = YES
+EXPAND_ONLY_PREDEF     = YES
+PREDEFINED             = \
+    $(ENV_DOXYGEN_DEFINES) \
+    __DOXYGEN__=1 \
+    __attribute__(x)= \
+	_Static_assert()= \
+    IDF_DEPRECATED(X)= \
+    IRAM_ATTR= \
+    configSUPPORT_DYNAMIC_ALLOCATION=1 \
+    configSUPPORT_STATIC_ALLOCATION=1 \
+    configQUEUE_REGISTRY_SIZE=1 \
+    configUSE_RECURSIVE_MUTEXES=1 \
+    configTHREAD_LOCAL_STORAGE_DELETE_CALLBACKS=1 \
+    configNUM_THREAD_LOCAL_STORAGE_POINTERS=1 \
+    configUSE_APPLICATION_TASK_TAG=1 \
+    configTASKLIST_INCLUDE_COREID=1 \
+    "ESP_EVENT_DECLARE_BASE(x)=extern esp_event_base_t x"
+
+## Do not complain about not having dot
+##
+HAVE_DOT = NO
+
+## Generate XML that is required for Breathe
+##
+GENERATE_XML    = YES
+XML_OUTPUT      = xml
+
+GENERATE_HTML   = NO
+HAVE_DOT        = NO
+GENERATE_LATEX  = NO
+GENERATE_MAN    = YES
+
+## Skip distracting progress messages
+##
+QUIET = YES
+## Enable Section Tags for conditional documentation
+##
+ENABLED_SECTIONS += \
+    DOC_EXCLUDE_HEADER_SECTION   ## To conditionally remove doc sections from IDF source files without affecting documentation in upstream files.

components/esp_mqtt_cxx/docs/conf_common.py

@@ -0,0 +1,23 @@
+# flake8: noqa
+from esp_docs.conf_docs import *
+
+extensions += [
+    'sphinx_copybutton',
+    # Needed as a trigger for running doxygen
+    'esp_docs.esp_extensions.dummy_build_system',
+    'esp_docs.esp_extensions.run_doxygen',
+]
+
+# link roles config
+github_repo = 'espressif/esp-protocols'
+
+# context used by sphinx_idf_theme
+html_context['github_user'] = 'espressif'
+html_context['github_repo'] = 'esp-docs'
+
+# Extra options required by sphinx_idf_theme
+project_slug = 'esp-idf'  # >=5.0
+versions_url = 'https://github.com/espressif/esp-protocols/docs/docs_versions.js'
+
+idf_targets = ['esp32']
+languages = ['en']

components/esp_mqtt_cxx/docs/doxygen-known-warnings.txt

@@ -0,0 +1,25 @@
+esp_mqtt_client_config.hpp:line: warning: Compound idf::mqtt::BrokerConfiguration is not documented.
+esp_mqtt_client_config.hpp:line: warning: Compound idf::mqtt::ClientCredentials is not documented.
+esp_mqtt_client_config.hpp:line: warning: Compound idf::mqtt::Configuration is not documented.
+esp_mqtt_client_config.hpp:line: warning: Compound idf::mqtt::Connection is not documented.
+esp_mqtt_client_config.hpp:line: warning: Compound idf::mqtt::Event is not documented.
+esp_mqtt_client_config.hpp:line: warning: Compound idf::mqtt::LastWill is not documented.
+esp_mqtt_client_config.hpp:line: warning: Compound idf::mqtt::Session is not documented.
+esp_mqtt_client_config.hpp:line: warning: Compound idf::mqtt::Task is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member address (variable) of struct idf::mqtt::BrokerConfiguration is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member security (variable) of struct idf::mqtt::BrokerConfiguration is not documented.
+esp_mqtt.hpp:line: warning: Member operator()(esp_mqtt_client *client_handler) (function) of struct idf::mqtt::Client::MqttClientDeleter is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member username (variable) of struct idf::mqtt::ClientCredentials is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member authentication (variable) of struct idf::mqtt::ClientCredentials is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member client_id (variable) of struct idf::mqtt::ClientCredentials is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member event (variable) of struct idf::mqtt::Configuration is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member task (variable) of struct idf::mqtt::Configuration is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member session (variable) of struct idf::mqtt::Configuration is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member connection (variable) of struct idf::mqtt::Configuration is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member data (variable) of struct idf::mqtt::DER is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member len (variable) of struct idf::mqtt::DER is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member ds_data (variable) of struct idf::mqtt::DigitalSignatureData is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member data (variable) of struct idf::mqtt::Password is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member data (variable) of struct idf::mqtt::PEM is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member hint_key (variable) of struct idf::mqtt::PSK is not documented.
+esp_mqtt_client_config.hpp:line: warning: Member last_will (variable) of struct idf::mqtt::Session is not documented.

components/esp_mqtt_cxx/docs/en/conf.py

@@ -0,0 +1,24 @@
+# -*- coding: utf-8 -*-
+#
+# English Language RTD & Sphinx config file
+#
+# Uses ../conf_common.py for most non-language-specific settings.
+
+# Importing conf_common adds all the non-language-specific
+# parts to this conf module
+
+try:
+    from conf_common import *  # noqa: F403,F401
+except ImportError:
+    import os
+    import sys
+    sys.path.insert(0, os.path.abspath('../'))
+    from conf_common import *  # noqa: F403,F401
+
+# General information about the project.
+project = u'ESP-Protocols'
+copyright = u'2016 - 2023, Espressif Systems (Shanghai) Co., Ltd'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+language = 'en'

components/esp_mqtt_cxx/docs/en/index.rst

@@ -0,0 +1,132 @@
+ESP MQTT C++ client
+====================
+
+Overview
+--------
+The ESP MQTT client is a wrapper over the `esp_mqtt` client with the goal of providing a higher level API.
+
+Features
+--------
+   * Supports MQTT version 3.11
+   * Adds a Filter validation class for topic filters
+   * Split the event handlers to member functions
+
+Configuration
+-------------
+
+The current design uses exception as an error handling mechanism, therefore exceptions need to be enabled in menuconfig.
+
+Usage
+-----
+
+User code needs to inherit fromm :cpp:class:`idf::mqtt::Client` and provide overloads for the event handlers.
+
+.. note:: The handler is available to allow user code to interact directly with it in case of need. This member will likely be made private in the future once the class API stabilizes.
+
+
+.. doxygenclass:: idf::mqtt::Client
+    :members:
+    :protected-members:
+
+Event Handling
+--------------
+
+Events are dispatched throug calls to member functions each one dedicated to a type of event.
+
+Application Example
+-------------------
+
+* :example:`tcp <../examples/tcp>`
+* :example:`ssl <../examples/ssl>`
+
+API Reference
+-------------
+
+Header File
+^^^^^^^^^^^
+
+* :project_file:`include/esp_mqtt.hpp`
+
+Structures
+^^^^^^^^^^
+
+.. doxygenstruct:: idf::mqtt::MQTTException
+    :members:
+
+.. doxygenstruct:: idf::mqtt::Message
+    :members:
+
+Classes
+^^^^^^^
+
+
+.. doxygenclass:: idf::mqtt::Filter
+    :members:
+
+Header File
+^^^^^^^^^^^
+
+* :project_file:`include/esp_mqtt_client_config.hpp`
+
+Structures
+^^^^^^^^^^
+
+.. doxygenstruct:: idf::mqtt::Host
+    :members:
+
+.. doxygenstruct:: idf::mqtt::URI
+    :members:
+
+.. doxygenstruct:: idf::mqtt::BrokerAddress
+    :members:
+
+.. doxygenstruct:: idf::mqtt::PEM
+    :members:
+
+.. doxygenstruct:: idf::mqtt::DER
+    :members:
+
+.. doxygenstruct:: idf::mqtt::Insecure
+    :members:
+
+.. doxygenstruct:: idf::mqtt::GlobalCAStore
+    :members:
+
+.. doxygenstruct:: idf::mqtt::PSK
+    :members:
+
+.. doxygenstruct:: idf::mqtt::Password
+    :members:
+
+.. doxygenstruct:: idf::mqtt::ClientCertificate
+    :members:
+
+.. doxygenstruct:: idf::mqtt::SecureElement
+    :members:
+
+.. doxygenstruct:: idf::mqtt::DigitalSignatureData
+    :members:
+
+.. doxygenstruct:: idf::mqtt::BrokerConfiguration
+    :members:
+
+.. doxygenstruct:: idf::mqtt::ClientCredentials
+    :members:
+
+.. doxygenstruct:: idf::mqtt::Event
+    :members:
+
+.. doxygenstruct:: idf::mqtt::LastWill
+    :members:
+
+.. doxygenstruct:: idf::mqtt::Session
+    :members:
+
+.. doxygenstruct:: idf::mqtt::Task
+    :members:
+
+.. doxygenstruct:: idf::mqtt::Connection
+    :members:
+
+.. doxygenstruct:: idf::mqtt::Configuration
+    :members:

components/esp_mqtt_cxx/docs/generate_docs

@@ -0,0 +1,26 @@
+build-docs --target esp32 --language en
+
+cp -rf _build/en/esp32/html .
+rm -rf _build __pycache__
+
+# Modifes some version and target fields of index.html
+echo "<script type="text/javascript">
+window.onload =(function() {
+    var myAnchor = document.getElementById('version-select');
+    var mySpan = document.createElement('input');
+    mySpan.setAttribute('type', 'text');
+    mySpan.setAttribute('maxLength', '10');
+    mySpan.value = 'latest';
+    mySpan.setAttribute('disabled', true);
+    myAnchor.parentNode.replaceChild(mySpan, myAnchor);
+
+    var myAnchor = document.getElementById('target-select');
+    var mySpan = document.createElement('input');
+    mySpan.setAttribute('type', 'text');
+    mySpan.setAttribute('maxLength', '10');
+    mySpan.value = 'all targets';
+    mySpan.setAttribute('disabled', true);
+    myAnchor.parentNode.replaceChild(mySpan, myAnchor);
+
+})();
+</script>" >> html/index.html

components/esp_mqtt_cxx/esp_mqtt_cxx.cpp

@@ -0,0 +1,292 @@
+/*
+ * SPDX-FileCopyrightText: 2021-2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+#include <string>
+#include <algorithm>
+#include <stdexcept>
+
+#include "mqtt_client.h"
+#include "esp_log.h"
+
+#include "esp_mqtt.hpp"
+#include "esp_mqtt_client_config.hpp"
+
+namespace {
+
+// Helper for static assert.
+template<class T>
+constexpr bool always_false = false;
+
+template<class... Ts> struct overloaded : Ts... {
+    using Ts::operator()...;
+};
+template<class... Ts> overloaded(Ts...) -> overloaded<Ts...>;
+
+using namespace idf::mqtt;
+
+/*
+ *  This function is responsible for fill in the configurations for the broker related data
+ *  of mqtt_client_config_t
+ */
+void config_broker(esp_mqtt_client_config_t &mqtt_client_cfg, BrokerConfiguration const &broker)
+{
+    std::visit(overloaded{
+        [&mqtt_client_cfg](Host const & host)
+        {
+            mqtt_client_cfg.broker.address.hostname = host.address.c_str();
+            mqtt_client_cfg.broker.address.path = host.path.c_str();
+            mqtt_client_cfg.broker.address.transport = host.transport;
+        },
+        [&mqtt_client_cfg](URI const & uri)
+        {
+            mqtt_client_cfg.broker.address.uri = uri.address.c_str();
+        },
+        []([[maybe_unused ]]auto & unknown)
+        {
+            static_assert(always_false<decltype(unknown)>, "Missing type handler for variant handler");
+        }
+    },
+    broker.address.address);
+
+    std::visit(overloaded{
+        []([[maybe_unused]]Insecure const & insecure) {},
+        [&mqtt_client_cfg](GlobalCAStore const & use_global_store)
+        {
+            mqtt_client_cfg.broker.verification.use_global_ca_store = true;
+        },
+        [&mqtt_client_cfg](CryptographicInformation const & certificates)
+        {
+            std::visit(overloaded{
+                [&mqtt_client_cfg](PEM const & pem)
+                {
+                    mqtt_client_cfg.broker.verification.certificate = pem.data;
+                }, [&mqtt_client_cfg](DER const & der)
+                {
+                    mqtt_client_cfg.broker.verification.certificate = der.data;
+                    mqtt_client_cfg.broker.verification.certificate_len = der.len;
+                }}, certificates);
+        },
+        []([[maybe_unused]] PSK const & psk) {},
+        []([[maybe_unused]] auto & unknown)
+        {
+            static_assert(always_false<decltype(unknown)>, "Missing type handler for variant handler");
+        }
+    },
+    broker.security);
+    mqtt_client_cfg.broker.address.port = broker.address.port;
+}
+
+/*
+ *  This function is responsible for fill in the configurations for the client credentials related data
+ *  of mqtt_client_config_t
+ */
+void config_client_credentials(esp_mqtt_client_config_t &mqtt_client_cfg, ClientCredentials const &credentials)
+{
+    mqtt_client_cfg.credentials.client_id = credentials.client_id.has_value() ?  credentials.client_id.value().c_str() : nullptr ;
+    mqtt_client_cfg.credentials.username = credentials.username.has_value() ?  credentials.username.value().c_str() : nullptr ;
+    std::visit(overloaded{
+        [&mqtt_client_cfg](Password const & password)
+        {
+            mqtt_client_cfg.credentials.authentication.password = password.data.c_str();
+        },
+        [](ClientCertificate const & certificate) {},
+        [](SecureElement const & enable_secure_element) {},
+        []([[maybe_unused ]]auto & unknown)
+        {
+            static_assert(always_false<decltype(unknown)>, "Missing type handler for variant handler");
+        }
+    }, credentials.authentication);
+}
+
+esp_mqtt_client_config_t make_config(BrokerConfiguration const &broker, ClientCredentials const  &credentials, Configuration const &config)
+{
+    esp_mqtt_client_config_t mqtt_client_cfg{};
+    config_broker(mqtt_client_cfg, broker);
+    config_client_credentials(mqtt_client_cfg, credentials);
+    return mqtt_client_cfg;
+}
+}
+
+namespace idf::mqtt {
+
+Client::Client(BrokerConfiguration const &broker, ClientCredentials const  &credentials, Configuration const &config): Client(make_config(broker, credentials, config))  {}
+
+Client::Client(esp_mqtt_client_config_t const &config) :  handler(esp_mqtt_client_init(&config))
+{
+    if (handler == nullptr) {
+        throw MQTTException(ESP_FAIL);
+    };
+    CHECK_THROW_SPECIFIC(esp_mqtt_client_register_event(handler.get(), MQTT_EVENT_ANY, mqtt_event_handler, this), mqtt::MQTTException);
+    CHECK_THROW_SPECIFIC(esp_mqtt_client_start(handler.get()), mqtt::MQTTException);
+}
+
+
+void Client::mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data) noexcept
+{
+    ESP_LOGD(TAG, "Event dispatched from event loop base=%s, event_id=%d", base, event_id);
+    auto *event = static_cast<esp_mqtt_event_t *>(event_data);
+    auto &client = *static_cast<Client *>(handler_args);
+    switch (event->event_id) {
+    case MQTT_EVENT_CONNECTED:
+        ESP_LOGI(TAG, "MQTT_EVENT_CONNECTED");
+        client.on_connected(event);
+        break;
+    case MQTT_EVENT_DISCONNECTED:
+        ESP_LOGI(TAG, "MQTT_EVENT_DISCONNECTED");
+        client.on_disconnected(event);
+        break;
+
+    case MQTT_EVENT_SUBSCRIBED:
+        ESP_LOGI(TAG, "MQTT_EVENT_SUBSCRIBED, msg_id=%d", event->msg_id);
+        client.on_subscribed(event);
+        break;
+    case MQTT_EVENT_UNSUBSCRIBED:
+        ESP_LOGI(TAG, "MQTT_EVENT_UNSUBSCRIBED, msg_id=%d", event->msg_id);
+        client.on_unsubscribed(event);
+        break;
+    case MQTT_EVENT_PUBLISHED:
+        ESP_LOGI(TAG, "MQTT_EVENT_PUBLISHED, msg_id=%d", event->msg_id);
+        client.on_published(event);
+        break;
+    case MQTT_EVENT_DATA:
+        ESP_LOGI(TAG, "MQTT_EVENT_DATA");
+        client.on_data(event);
+        break;
+    case MQTT_EVENT_ERROR:
+        ESP_LOGI(TAG, "MQTT_EVENT_ERROR");
+        client.on_error(event);
+        break;
+    case MQTT_EVENT_BEFORE_CONNECT:
+        ESP_LOGI(TAG, "MQTT_EVENT_BEFORE_CONNECT");
+        client.on_before_connect(event);
+        break;
+    default:
+        ESP_LOGI(TAG, "Other event id:%d", event->event_id);
+        break;
+    }
+}
+
+void Client::on_error(esp_mqtt_event_handle_t const event)
+{
+    auto log_error_if_nonzero = [](const char *message, int error_code) {
+        if (error_code != 0) {
+            ESP_LOGE(TAG, "Last error %s: 0x%x", message, error_code);
+        }
+    };
+    if (event->error_handle->error_type == MQTT_ERROR_TYPE_TCP_TRANSPORT) {
+        log_error_if_nonzero("reported from esp-tls", event->error_handle->esp_tls_last_esp_err);
+        log_error_if_nonzero("reported from tls stack", event->error_handle->esp_tls_stack_err);
+        log_error_if_nonzero("captured as transport's socket errno",  event->error_handle->esp_transport_sock_errno);
+        ESP_LOGI(TAG, "Last errno string (%s)", strerror(event->error_handle->esp_transport_sock_errno));
+    }
+}
+void Client::on_disconnected(esp_mqtt_event_handle_t const event)
+{
+}
+void Client::on_subscribed(esp_mqtt_event_handle_t const event)
+{
+    printf("Subscribed to %.*s\r\n", event->topic_len, event->topic);
+}
+void Client::on_unsubscribed(esp_mqtt_event_handle_t const event)
+{
+}
+void Client::on_published(esp_mqtt_event_handle_t const event)
+{
+}
+void Client::on_before_connect(esp_mqtt_event_handle_t const event)
+{
+}
+void Client::on_connected(esp_mqtt_event_handle_t const event)
+{
+}
+void Client::on_data(esp_mqtt_event_handle_t const event)
+{
+}
+
+std::optional<MessageID> Client::subscribe(std::string const &topic, QoS qos)
+{
+    auto res = esp_mqtt_client_subscribe(handler.get(), topic.c_str(),
+                                         static_cast<int>(qos));
+    if (res < 0) {
+        return std::nullopt;
+    }
+    return MessageID{res};
+}
+
+bool is_valid(std::string::const_iterator first, std::string::const_iterator last)
+{
+    if (first == last) {
+        return false;
+    }
+    auto number = std::find(first, last, '#');
+    if (number != last) {
+        if (std::next(number) != last) {
+            return false;
+        }
+        if (*std::prev(number) != '/' && number != first) {
+            return false;
+        }
+    }
+
+    auto plus = std::find(first, last, '+');
+    if (plus != last) {
+        if (*(std::prev(plus)) != '/' && plus != first) {
+            return false;
+        }
+        if (std::next(plus) != last && *(std::next(plus)) != '/') {
+            return false;
+        }
+    }
+    return true;
+}
+
+Filter::Filter(std::string user_filter) : filter(std::move(user_filter))
+{
+    if (!is_valid(filter.begin(), filter.end())) {
+        throw std::domain_error("Forbidden Filter string");
+    }
+}
+
+[[nodiscard]] bool Filter::match(std::string::const_iterator topic_begin, std::string::const_iterator topic_end) const noexcept
+{
+    auto filter_begin = filter.begin();
+    auto filter_end = filter.end();
+    for (auto mismatch = std::mismatch(filter_begin, filter_end, topic_begin);
+            mismatch.first != filter.end() and mismatch.second != topic_end;
+            mismatch = std::mismatch(filter_begin, filter_end, topic_begin)) {
+        if (*mismatch.first != '#' and * mismatch.first != '+') {
+            return false;
+        }
+        if (*mismatch.first == '#') {
+            return true;
+        }
+        if (*mismatch.first == '+') {
+            filter_begin = advance(mismatch.first, filter_end);
+            topic_begin = advance(mismatch.second, topic_end);
+            if (filter_begin == filter_end and topic_begin != topic_end) {
+                return false;
+            }
+        }
+    }
+    return true;
+}
+const std::string &Filter::get()
+{
+    return filter;
+}
+
+[[nodiscard]] bool Filter::match(char *const begin, int size) const noexcept
+{
+    auto it = static_cast<std::string::const_iterator>(begin);
+    return match(it, it + size);
+}
+std::string::const_iterator Filter::advance(std::string::const_iterator begin, std::string::const_iterator end) const
+{
+    constexpr auto separator = '/';
+    return std::find(begin, end, separator);
+}
+
+}

components/esp_mqtt_cxx/examples/ssl/CMakeLists.txt

@@ -0,0 +1,14 @@
+# The following four lines of boilerplate have to be in your project's
+# CMakeLists in this exact order for cmake to work correctly
+cmake_minimum_required(VERSION 3.16)
+
+# (Not part of the boilerplate) This example uses an extra component for common
+# functions such as Wi-Fi and Ethernet connection.
+# The path to esp_mqtt_cxx is also added.
+set(EXTRA_COMPONENT_DIRS
+    "../../../../common_components/protocol_examples_common" "../../")
+
+include($ENV{IDF_PATH}/tools/cmake/project.cmake)
+project(mqtt_ssl_cxx)
+
+target_add_binary_data(mqtt_ssl_cxx.elf "main/mqtt_eclipseprojects_io.pem" TEXT)

components/esp_mqtt_cxx/examples/ssl/README.md

@@ -0,0 +1,2 @@
+| Supported Targets | ESP32 | ESP32-C2 | ESP32-C3 | ESP32-S2 | ESP32-S3 |
+| ----------------- | ----- | -------- | -------- | -------- | -------- |

components/esp_mqtt_cxx/examples/ssl/main/CMakeLists.txt

@@ -0,0 +1,3 @@
+idf_component_register(SRCS "mqtt_ssl_example.cpp"
+                    INCLUDE_DIRS ".")
+target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")

components/esp_mqtt_cxx/examples/ssl/main/Kconfig.projbuild

@@ -0,0 +1,9 @@
+menu "Example Configuration"
+
+    config BROKER_URI
+        string "Broker URL"
+        default "mqtts://mqtt.eclipseprojects.io:8883"
+        help
+            URL of the broker to connect to
+
+endmenu

components/esp_mqtt_cxx/examples/ssl/main/mqtt_eclipse_org.pem

@@ -0,0 +1,27 @@
+-----BEGIN CERTIFICATE-----
+MIIEkjCCA3qgAwIBAgIQCgFBQgAAAVOFc2oLheynCDANBgkqhkiG9w0BAQsFADA/
+MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
+DkRTVCBSb290IENBIFgzMB4XDTE2MDMxNzE2NDA0NloXDTIxMDMxNzE2NDA0Nlow
+SjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxIzAhBgNVBAMT
+GkxldCdzIEVuY3J5cHQgQXV0aG9yaXR5IFgzMIIBIjANBgkqhkiG9w0BAQEFAAOC
+AQ8AMIIBCgKCAQEAnNMM8FrlLke3cl03g7NoYzDq1zUmGSXhvb418XCSL7e4S0EF
+q6meNQhY7LEqxGiHC6PjdeTm86dicbp5gWAf15Gan/PQeGdxyGkOlZHP/uaZ6WA8
+SMx+yk13EiSdRxta67nsHjcAHJyse6cF6s5K671B5TaYucv9bTyWaN8jKkKQDIZ0
+Z8h/pZq4UmEUEz9l6YKHy9v6Dlb2honzhT+Xhq+w3Brvaw2VFn3EK6BlspkENnWA
+a6xK8xuQSXgvopZPKiAlKQTGdMDQMc2PMTiVFrqoM7hD8bEfwzB/onkxEz0tNvjj
+/PIzark5McWvxI0NHWQWM6r6hCm21AvA2H3DkwIDAQABo4IBfTCCAXkwEgYDVR0T
+AQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwfwYIKwYBBQUHAQEEczBxMDIG
+CCsGAQUFBzABhiZodHRwOi8vaXNyZy50cnVzdGlkLm9jc3AuaWRlbnRydXN0LmNv
+bTA7BggrBgEFBQcwAoYvaHR0cDovL2FwcHMuaWRlbnRydXN0LmNvbS9yb290cy9k
+c3Ryb290Y2F4My5wN2MwHwYDVR0jBBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAw
+VAYDVR0gBE0wSzAIBgZngQwBAgEwPwYLKwYBBAGC3xMBAQEwMDAuBggrBgEFBQcC
+ARYiaHR0cDovL2Nwcy5yb290LXgxLmxldHNlbmNyeXB0Lm9yZzA8BgNVHR8ENTAz
+MDGgL6AthitodHRwOi8vY3JsLmlkZW50cnVzdC5jb20vRFNUUk9PVENBWDNDUkwu
+Y3JsMB0GA1UdDgQWBBSoSmpjBH3duubRObemRWXv86jsoTANBgkqhkiG9w0BAQsF
+AAOCAQEA3TPXEfNjWDjdGBX7CVW+dla5cEilaUcne8IkCJLxWh9KEik3JHRRHGJo
+uM2VcGfl96S8TihRzZvoroed6ti6WqEBmtzw3Wodatg+VyOeph4EYpr/1wXKtx8/
+wApIvJSwtmVi4MFU5aMqrSDE6ea73Mj2tcMyo5jMd6jmeWUHK8so/joWUoHOUgwu
+X4Po1QYz+3dszkDqMp4fklxBwXRsW10KXzPMTZ+sOPAveyxindmjkW8lGy+QsRlG
+PfZ+G6Z6h7mjem0Y+iWlkYcV4PIWL1iwBi8saCbGS5jN2p8M+X+Q7UNKEkROb3N6
+KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg==
+-----END CERTIFICATE-----

components/esp_mqtt_cxx/examples/ssl/main/mqtt_eclipseprojects_io.pem

@@ -0,0 +1,30 @@
+-----BEGIN CERTIFICATE-----
+MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
+TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
+cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw
+WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
+RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
+AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP
+R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx
+sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm
+NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg
+Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG
+/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC
+AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB
+Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA
+FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw
+AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw
+Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB
+gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W
+PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl
+ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz
+CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm
+lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4
+avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2
+yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O
+yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids
+hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+
+HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv
+MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
+nLRbwHOoq7hHwg==
+-----END CERTIFICATE-----

components/esp_mqtt_cxx/examples/ssl/main/mqtt_ssl_example.cpp

@@ -0,0 +1,90 @@
+/*
+ * SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Unlicense OR CC0-1.0
+ */
+/* C++ MQTT (over TCP) Example
+
+   This example code is in the Public Domain (or CC0 licensed, at your option.)
+
+   Unless required by applicable law or agreed to in writing, this
+   software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+   CONDITIONS OF ANY KIND, either express or implied.
+*/
+
+#include <cstdint>
+#include <string>
+#include "esp_mqtt_client_config.hpp"
+#include "nvs_flash.h"
+#include "protocol_examples_common.h"
+
+
+#include "esp_log.h"
+#include "esp_mqtt.hpp"
+
+namespace {
+constexpr auto *TAG = "MQTT_EXAMPLE";
+
+extern const char mqtt_eclipse_org_pem_start[]   asm("_binary_mqtt_eclipseprojects_io_pem_start");
+extern const char mqtt_eclipse_org_pem_end[]   asm("_binary_mqtt_eclipseprojects_io_pem_end");
+
+class MyClient final : public idf::mqtt::Client {
+public:
+    using idf::mqtt::Client::Client;
+private:
+    void on_connected(esp_mqtt_event_handle_t const event) override
+    {
+        using idf::mqtt::QoS;
+        subscribe(messages.get());
+        subscribe(sent_load.get(), QoS::AtMostOnce);
+    }
+    void on_data(esp_mqtt_event_handle_t const event) override
+    {
+        if (messages.match(event->topic, event->topic_len)) {
+            ESP_LOGI(TAG, "Received in the messages topic");
+        }
+    }
+    idf::mqtt::Filter messages{std::string{"$SYS/broker/messages/received"}};
+    idf::mqtt::Filter sent_load{std::string{"$SYS/broker/load/+/sent"}};
+};
+}
+
+namespace mqtt = idf::mqtt;
+
+extern "C" void app_main(void)
+{
+    ESP_LOGI(TAG, "[APP] Startup..");
+    ESP_LOGI(TAG, "[APP] Free memory: %d bytes", esp_get_free_heap_size());
+    ESP_LOGI(TAG, "[APP] IDF version: %s", esp_get_idf_version());
+
+    esp_log_level_set("*", ESP_LOG_INFO);
+    esp_log_level_set("MQTT_CLIENT", ESP_LOG_VERBOSE);
+    esp_log_level_set("MQTT_EXAMPLE", ESP_LOG_VERBOSE);
+    esp_log_level_set("TRANSPORT_TCP", ESP_LOG_VERBOSE);
+    esp_log_level_set("TRANSPORT_SSL", ESP_LOG_VERBOSE);
+    esp_log_level_set("TRANSPORT", ESP_LOG_VERBOSE);
+    esp_log_level_set("OUTBOX", ESP_LOG_VERBOSE);
+
+    ESP_ERROR_CHECK(nvs_flash_init());
+    ESP_ERROR_CHECK(esp_netif_init());
+    ESP_ERROR_CHECK(esp_event_loop_create_default());
+
+    /* This helper function configures Wi-Fi or Ethernet, as selected in menuconfig.
+     * Read "Establishing Wi-Fi or Ethernet Connection" section in
+     * examples/protocols/README.md for more information about this function.
+     */
+    ESP_ERROR_CHECK(example_connect());
+
+    mqtt::BrokerConfiguration broker{
+        .address = {mqtt::URI{std::string{CONFIG_BROKER_URI}}},
+        .security = mqtt::CryptographicInformation{mqtt::PEM{mqtt_eclipse_org_pem_start}}
+    };
+    idf::mqtt::ClientCredentials credentials{};
+    idf::mqtt::Configuration config{};
+
+    MyClient client{broker, credentials, config};
+    while (true) {
+        constexpr TickType_t xDelay = 500 / portTICK_PERIOD_MS;
+        vTaskDelay( xDelay );
+    }
+}

components/esp_mqtt_cxx/examples/ssl/pytest_mqtt_ssl.py

@@ -0,0 +1,8 @@
+# SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
+# SPDX-License-Identifier: Unlicense OR CC0-1.0
+import pytest
+
+
+@pytest.mark.esp32
+def test_dummy() -> None:
+    pass

components/esp_mqtt_cxx/examples/ssl/sdkconfig.defaults

@@ -0,0 +1,21 @@
+# Enable C++ exceptions and set emergency pool size for exception objects
+CONFIG_COMPILER_CXX_EXCEPTIONS=y
+CONFIG_COMPILER_CXX_EXCEPTIONS_EMG_POOL_SIZE=1024
+CONFIG_EXAMPLE_CONNECT_ETHERNET=y
+CONFIG_EXAMPLE_CONNECT_WIFI=n
+CONFIG_EXAMPLE_USE_INTERNAL_ETHERNET=y
+CONFIG_EXAMPLE_ETH_PHY_IP101=y
+CONFIG_EXAMPLE_ETH_MDC_GPIO=23
+CONFIG_EXAMPLE_ETH_MDIO_GPIO=18
+CONFIG_EXAMPLE_ETH_PHY_RST_GPIO=5
+CONFIG_EXAMPLE_ETH_PHY_ADDR=1
+CONFIG_EXAMPLE_CONNECT_IPV6=y
+CONFIG_EXAMPLE_CONNECT_ETHERNET=y
+CONFIG_EXAMPLE_CONNECT_WIFI=n
+CONFIG_EXAMPLE_USE_INTERNAL_ETHERNET=y
+CONFIG_EXAMPLE_ETH_PHY_IP101=y
+CONFIG_EXAMPLE_ETH_MDC_GPIO=23
+CONFIG_EXAMPLE_ETH_MDIO_GPIO=18
+CONFIG_EXAMPLE_ETH_PHY_RST_GPIO=5
+CONFIG_EXAMPLE_ETH_PHY_ADDR=1
+CONFIG_EXAMPLE_CONNECT_IPV6=y

components/esp_mqtt_cxx/examples/tcp/CMakeLists.txt

@@ -0,0 +1,12 @@
+# The following four lines of boilerplate have to be in your project's CMakeLists
+# in this exact order for cmake to work correctly
+cmake_minimum_required(VERSION 3.16)
+
+# (Not part of the boilerplate)
+# This example uses an extra component for common functions such as Wi-Fi and Ethernet connection.
+# The path to esp_mqtt_cxx is also added.
+set(EXTRA_COMPONENT_DIRS
+    "../../../../common_components/protocol_examples_common" "../../")
+
+include($ENV{IDF_PATH}/tools/cmake/project.cmake)
+project(mqtt_tcp_cxx)

components/esp_mqtt_cxx/examples/tcp/README.md

@@ -0,0 +1,2 @@
+| Supported Targets | ESP32 | ESP32-C2 | ESP32-C3 | ESP32-S2 | ESP32-S3 |
+| ----------------- | ----- | -------- | -------- | -------- | -------- |

components/esp_mqtt_cxx/examples/tcp/main/CMakeLists.txt

@@ -0,0 +1,3 @@
+idf_component_register(SRCS "mqtt_tcp_example.cpp"
+                    INCLUDE_DIRS ".")
+target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")

components/esp_mqtt_cxx/examples/tcp/main/Kconfig.projbuild

@@ -0,0 +1,9 @@
+menu "Example Configuration"
+
+    config BROKER_URL
+        string "Broker URL"
+        default "mqtt://mqtt.eclipseprojects.io"
+        help
+            URL of the broker to connect to
+
+endmenu

components/esp_mqtt_cxx/examples/tcp/main/mqtt_tcp_example.cpp

@@ -0,0 +1,86 @@
+/*
+ * SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Unlicense OR CC0-1.0
+ */
+/* C++ MQTT (over TCP) Example
+
+   This example code is in the Public Domain (or CC0 licensed, at your option.)
+
+   Unless required by applicable law or agreed to in writing, this
+   software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+   CONDITIONS OF ANY KIND, either express or implied.
+*/
+
+#include "nvs_flash.h"
+#include "protocol_examples_common.h"
+
+#include "esp_log.h"
+#include "esp_mqtt.hpp"
+#include "esp_mqtt_client_config.hpp"
+
+namespace mqtt = idf::mqtt;
+
+namespace {
+constexpr auto *TAG = "MQTT_EXAMPLE";
+
+class MyClient final : public mqtt::Client {
+public:
+    using mqtt::Client::Client;
+
+private:
+    void on_connected(esp_mqtt_event_handle_t const event) override
+    {
+        using mqtt::QoS;
+        subscribe(messages.get());
+        subscribe(sent_load.get(), QoS::AtMostOnce);
+    }
+    void on_data(esp_mqtt_event_handle_t const event) override
+    {
+        if (messages.match(event->topic, event->topic_len)) {
+            ESP_LOGI(TAG, "Received in the messages topic");
+        }
+    }
+    mqtt::Filter messages{"$SYS/broker/messages/received"};
+    mqtt::Filter sent_load{"$SYS/broker/load/+/sent"};
+};
+}
+
+extern "C" void app_main(void)
+{
+    ESP_LOGI(TAG, "[APP] Startup..");
+    ESP_LOGI(TAG, "[APP] Free memory: %d bytes", esp_get_free_heap_size());
+    ESP_LOGI(TAG, "[APP] IDF version: %s", esp_get_idf_version());
+
+    esp_log_level_set("*", ESP_LOG_INFO);
+    esp_log_level_set("MQTT_CLIENT", ESP_LOG_VERBOSE);
+    esp_log_level_set("MQTT_EXAMPLE", ESP_LOG_VERBOSE);
+    esp_log_level_set("TRANSPORT_TCP", ESP_LOG_VERBOSE);
+    esp_log_level_set("TRANSPORT_SSL", ESP_LOG_VERBOSE);
+    esp_log_level_set("TRANSPORT", ESP_LOG_VERBOSE);
+    esp_log_level_set("OUTBOX", ESP_LOG_VERBOSE);
+
+    ESP_ERROR_CHECK(nvs_flash_init());
+    ESP_ERROR_CHECK(esp_netif_init());
+    ESP_ERROR_CHECK(esp_event_loop_create_default());
+
+    /* This helper function configures Wi-Fi or Ethernet, as selected in menuconfig.
+     * Read "Establishing Wi-Fi or Ethernet Connection" section in
+     * examples/protocols/README.md for more information about this function.
+     */
+    ESP_ERROR_CHECK(example_connect());
+
+    mqtt::BrokerConfiguration broker{
+        .address = {mqtt::URI{std::string{CONFIG_BROKER_URL}}},
+        .security =  mqtt::Insecure{}
+    };
+    mqtt::ClientCredentials credentials{};
+    mqtt::Configuration config{};
+
+    MyClient client{broker, credentials, config};
+
+    while (true) {
+        constexpr TickType_t xDelay = 500 / portTICK_PERIOD_MS;
+        vTaskDelay(xDelay);
+    }
+}

components/esp_mqtt_cxx/examples/tcp/pytest_mqtt_tcp.py

@@ -0,0 +1,8 @@
+# SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
+# SPDX-License-Identifier: Unlicense OR CC0-1.0
+import pytest
+
+
+@pytest.mark.esp32
+def test_dummy() -> None:
+    pass

components/esp_mqtt_cxx/examples/tcp/sdkconfig.defaults

@@ -0,0 +1,3 @@
+# Enable C++ exceptions and set emergency pool size for exception objects
+CONFIG_COMPILER_CXX_EXCEPTIONS=y
+CONFIG_COMPILER_CXX_EXCEPTIONS_EMG_POOL_SIZE=1024

components/esp_mqtt_cxx/idf_component.yml

@@ -0,0 +1,8 @@
+version: "0.1.0"
+description: esp mqtt cxx
+url: https://github.com/espressif/esp-protocols/tree/master/components/esp_mqtt_cxx
+dependencies:
+  espressif/esp-idf-cxx: "^1.0.0-beta"
+  # Required IDF version
+  idf:
+    version: ">=5.0"

components/esp_mqtt_cxx/include/esp_mqtt.hpp

@@ -0,0 +1,296 @@
+/*
+ * SPDX-FileCopyrightText: 2021-2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+#pragma once
+
+#include <string_view>
+#ifndef __cpp_exceptions
+#error MQTT class can only be used when __cpp_exceptions is enabled. Enable CONFIG_COMPILER_CXX_EXCEPTIONS in Kconfig
+#endif
+
+#include <optional>
+#include <variant>
+#include <utility>
+#include <memory>
+#include <string>
+#include "esp_exception.hpp"
+#include "esp_mqtt_client_config.hpp"
+#include "mqtt_client.h"
+
+namespace idf::mqtt {
+
+constexpr auto *TAG = "mqtt_client_cpp";
+
+struct MQTTException : ESPException {
+    using ESPException::ESPException;
+};
+
+/**
+ * @brief QoS for publish and subscribe
+ *
+ * Sets the QoS as:
+ * AtMostOnce  : Best effort delivery of messages. Message loss can occur.
+ * AtLeastOnce : Guaranteed delivery of messages. Duplicates can occur.
+ * ExactlyOnce : Guaranteed delivery of messages exactly once.
+ *
+ * @note
+ *  When subscribing to a topic the QoS means the maximum QoS that should be sent to
+ *  client on this topic
+ */
+enum class QoS { AtMostOnce = 0, AtLeastOnce = 1, ExactlyOnce = 2 };
+
+/**
+ * @brief Sets if a message must be retained.
+ *
+ * Retained messages are delivered to future subscribers that match the topic name.
+ *
+ */
+enum class Retain : bool { NotRetained = false, Retained = true };
+
+
+/**
+ * @brief Message class template to publish.
+ *
+ */
+template <typename T> struct Message {
+    T data; /*!< Data for publish. Should be a contiguous type*/
+    QoS qos = QoS::AtLeastOnce; /*!< QoS for the message*/
+    Retain retain = Retain::NotRetained; /*!< Retention mark for the message.*/
+};
+
+/**
+ * @brief Message type that holds std::string for data
+ *
+ */
+using StringMessage = Message<std::string>;
+
+[[nodiscard]] bool filter_is_valid(std::string::const_iterator first, std::string::const_iterator last);
+
+/**
+ * @brief Filter for mqtt topic subscription.
+ *
+ * Topic filter.
+ *
+ */
+class Filter {
+public:
+
+    /**
+     * @brief Constructs the topic filter from the user filter
+     * @throws std::domain_error if the filter is invalid.
+     *
+     * @param user_filter Filter to be used.
+     */
+    explicit Filter(std::string user_filter);
+
+
+    /**
+     * @brief Get the filter string used.
+     *
+     * @return Reference to the topic filter.
+     */
+    const std::string &get();
+
+    /**
+     * @brief Checks the filter string against a topic name.
+     *
+     * @param topic_begin Iterator to the beginning of the sequence.
+     * @param topic_end Iterator to the end of the sequence.
+     *
+     * @return true if the topic name match the filter
+     */
+    [[nodiscard]] bool match(std::string::const_iterator topic_begin, std::string::const_iterator topic_end) const noexcept;
+
+    /**
+     * @brief Checks the filter string against a topic name.
+     *
+     * @param topic topic name
+     *
+     * @return true if the topic name match the filter
+     */
+    [[nodiscard]] bool match(const std::string &topic) const noexcept;
+
+    /**
+     * @brief Checks the filter string against a topic name.
+     *
+     * @param begin Char array with topic name.
+     * @param size  Size of given topic name.
+     *
+     * @return true if the topic name match the filter
+     */
+    [[nodiscard]] bool match(char *begin, int size) const noexcept;
+
+
+private:
+
+    /**
+     * @brief Advance the topic to the next level.
+     *
+     * An mqtt topic ends with a /. This function is used to iterate in topic levels.
+     *
+     * @return Iterator to the start of the topic.
+     */
+    [[nodiscard]] std::string::const_iterator advance(std::string::const_iterator begin, std::string::const_iterator end) const;
+    std::string filter;
+};
+
+/**
+ * @brief Message identifier to track delivery.
+ *
+ */
+enum class MessageID : int {};
+
+/**
+ * @brief Base class for MQTT client
+ *
+ * Should be inherited to provide event handlers.
+ */
+class Client {
+public:
+    /**
+     * @brief Constructor of the client
+     *
+     * @param broker Configuration for broker connection
+     * @param credentials client credentials to be presented to the broker
+     * @param config Mqtt client configuration
+     */
+    Client(const BrokerConfiguration &broker, const ClientCredentials &credentials, const Configuration &config);
+
+    /**
+     * @brief Constructs Client using the same configuration used for
+     * `esp_mqtt_client`
+     * @param config config struct to `esp_mqtt_client`
+     */
+    Client(const esp_mqtt_client_config_t &config);
+
+    /**
+     * @brief Subscribe to topic
+     *
+     * @param topic_filter MQTT topic filter
+     * @param qos QoS subscription, defaulted as QoS::AtLeastOnce
+     *
+     * @return Optional MessageID. In case of failure std::nullopt is returned.
+     */
+    std::optional<MessageID> subscribe(const std::string &topic_filter, QoS qos = QoS::AtLeastOnce);
+
+    /**
+     * @brief publish message to topic
+     *
+     * @tparam Container Type for data container. Must be a contiguous memory.
+     * @param topic Topic name
+     * @param message Message struct containing data, qos and retain
+     * configuration.
+     *
+     * @return Optional MessageID. In case of failure std::nullopt is returned.
+     */
+    template <class Container>
+    std::optional<MessageID> publish(const std::string &topic, const Message<Container> &message)
+    {
+        return publish(topic, std::begin(message.data), std::end(message.data), message.qos, message.retain);
+    }
+
+    /**
+     * @brief publish message to topic
+     *
+     * @tparam InputIt Input data iterator type.
+     * @param topic Topic name
+     * @param first, last Iterator pair of data to publish
+     * @param qos Set qos message
+     * @param retain Set if message should be retained
+     *
+     * @return Optional MessageID. In case of failure std::nullopt is returned.
+     */
+    template <class InputIt>
+    std::optional<MessageID> publish(const std::string &topic, InputIt first, InputIt last, QoS qos = QoS::AtLeastOnce, Retain retain = Retain::NotRetained)
+    {
+        auto size = std::distance(first, last);
+        auto res =  esp_mqtt_client_publish(handler.get(), topic.c_str(), &(*first), size, static_cast<int>(qos),
+                                            static_cast<int>(retain));
+        if (res < 0) {
+            return std::nullopt;
+        }
+        return MessageID{res};
+    }
+
+    virtual ~Client() = default;
+
+protected:
+    /**
+     * @brief Helper type to be used as custom deleter for std::unique_ptr.
+     */
+    struct MqttClientDeleter {
+        void operator()(esp_mqtt_client *client_handler)
+        {
+            esp_mqtt_client_destroy(client_handler);
+        }
+    };
+
+    /**
+     * @brief Type of the handler for the underlying mqtt_client handler.
+     * It uses std::unique_ptr for lifetime management
+     */
+    using ClientHandler = std::unique_ptr<esp_mqtt_client, MqttClientDeleter>;
+    /**
+     * @brief esp_mqtt_client handler
+     *
+     */
+    ClientHandler handler;
+
+    /**
+    * @brief Called if there is an error event
+    *
+    * @param event mqtt event data
+    */
+    virtual void on_error(const esp_mqtt_event_handle_t event);
+    /**
+    * @brief Called if there is an disconnection event
+    *
+    * @param event mqtt event data
+    */
+    virtual void on_disconnected(const esp_mqtt_event_handle_t event);
+    /**
+    * @brief Called if there is an subscribed event
+    *
+    * @param event mqtt event data
+    */
+    virtual void on_subscribed(const esp_mqtt_event_handle_t event);
+    /**
+    * @brief Called if there is an unsubscribed event
+    *
+    * @param event mqtt event data
+    */
+    virtual void on_unsubscribed(const esp_mqtt_event_handle_t event);
+    /**
+    * @brief Called if there is an published event
+    *
+    * @param event mqtt event data
+    */
+    virtual void on_published(const esp_mqtt_event_handle_t event);
+    /**
+    * @brief Called if there is an before connect event
+    *
+    * @param event mqtt event data
+    */
+    virtual void on_before_connect(const esp_mqtt_event_handle_t event);
+    /**
+    * @brief Called if there is an connected event
+    *
+    * @param event mqtt event data
+    *
+    */
+    virtual void on_connected(const esp_mqtt_event_handle_t event) = 0;
+    /**
+    * @brief Called if there is an data event
+    *
+    * @param event mqtt event data
+    *
+    */
+    virtual void on_data(const esp_mqtt_event_handle_t event) = 0;
+private:
+    static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id,
+                                   void *event_data) noexcept;
+    void init(const esp_mqtt_client_config_t &config);
+};
+} // namespace idf::mqtt

components/esp_mqtt_cxx/include/esp_mqtt_client_config.hpp

@@ -0,0 +1,213 @@
+/*
+ * SPDX-FileCopyrightText: 2021-2023 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+#pragma once
+
+#include <cstddef>
+#include <optional>
+#include <string>
+#include <variant>
+#include <vector>
+
+#include "mqtt_client.h"
+
+namespace idf::mqtt {
+
+
+
+/**
+ * @brief Broker addresss
+ *
+ * Use this to set the broker without parsing the URI string.
+ *
+ */
+struct Host {
+    std::string address; /*!< Host name*/
+    std::string path;    /*!< Route path of the broker in host*/
+    esp_mqtt_transport_t transport; /*!< Transport scheme to use. */
+};
+
+/**
+ * @brief Broker addresss URI
+ *
+ * Use this to set the broker address using the URI.
+ *
+ */
+struct URI {
+    std::string address; /*!< Broker adddress URI*/
+};
+
+
+/**
+ * @brief Broker addresss.
+ *
+ */
+struct BrokerAddress {
+    std::variant<Host, URI> address; /*!< Address, defined by URI or Host struct */
+    uint32_t port = 0; /*!< Port used, defaults to 0 to select common port for the scheme used */
+};
+
+/**
+ * @brief PEM formated data
+ *
+ *  Store certificates, keys and cryptographic data.
+ *
+ */
+struct PEM {
+    const char *data;
+};
+
+/**
+ * @brief DER formated data
+ *
+ *  Store certificates, keys and cryptographic data.
+ *
+ */
+struct DER {
+    const char *data;
+    size_t len;
+};
+
+/**
+ * @brief Holds cryptography related information
+ *
+ *  Hold PEM or DER formated cryptographic data.
+ *
+ */
+using CryptographicInformation = std::variant<PEM, DER>;
+
+
+/**
+ * @brief Do not verify broker certificate.
+ *
+ * To be used when doing MQTT over TLS connection but not verify broker's certificates.
+ *
+ */
+struct Insecure {};
+
+/**
+ * @brief Use global CA store
+ *
+ * To be used when client should use the Global CA Store to get trusted certificates for the broker.
+ *
+ */
+struct GlobalCAStore {};
+
+/**
+ * @brief Use a pre shared key for broker authentication.
+ *
+ * To be used when client should use a PSK to authenticate the broker.
+ *
+ */
+struct PSK {
+    const struct psk_key_hint *hint_key;/*  Pointer to PSK struct defined in esp_tls.h to enable PSK authentication */
+};
+
+
+/**
+ * @brief Authentication method for Broker
+ *
+ * Selects the method for authentication based on the type it holds.
+ *
+ */
+using BrokerAuthentication = std::variant<Insecure, GlobalCAStore, CryptographicInformation, PSK>;
+
+/**
+ * @brief Password related data.
+ *
+ */
+struct Password {
+    std::string data;
+};
+
+/**
+ * @brief Data to authenticate client with certificates.
+ *
+ */
+struct ClientCertificate {
+    CryptographicInformation certificate; /*!< Certificate in PEM or DER format.*/
+    CryptographicInformation key; /*!< Key data in PEM or DER format.*/
+    std::optional<Password> key_password = std::nullopt; /*!< Optional password for key */
+};
+
+/**
+ * @brief Used to select usage of Secure Element
+ *
+ *  Enables the usage of the secure element present in ESP32-WROOM-32SE.
+ *
+ */
+struct SecureElement {};
+
+
+/**
+ * @brief Used to select usage of Digital Signature Peripheral.
+ *
+ *  Enables the usage of the Digital Signature hardware accelerator.
+ *
+ */
+struct DigitalSignatureData {
+    void *ds_data;                          /* carrier of handle for digital signature parameters */
+};
+
+using AuthenticationFactor = std::variant<Password, ClientCertificate, SecureElement>;
+
+struct BrokerConfiguration {
+    BrokerAddress address;
+    BrokerAuthentication security;
+};
+
+struct ClientCredentials {
+    std::optional<std::string> username;    // MQTT username
+    AuthenticationFactor authentication;
+    std::vector<std::string> alpn_protos;               /*!< List of supported application protocols to be used for ALPN */
+    /* default is ``ESP32_%CHIPID%`` where %CHIPID% are last 3 bytes of MAC address in hex format */
+    std::optional<std::string > client_id = std::nullopt;
+};
+
+struct Event {
+    mqtt_event_callback_t event_handle;     /*!< handle for MQTT events as a callback in legacy mode */
+    esp_event_loop_handle_t event_loop_handle; /*!< handle for MQTT event loop library */
+};
+
+struct LastWill {
+    const char *lwt_topic;                  /*!< LWT (Last Will and Testament) message topic (NULL by default) */
+    const char *lwt_msg;                    /*!< LWT message (NULL by default) */
+    int lwt_qos;                            /*!< LWT message qos */
+    int lwt_retain;                         /*!< LWT retained message flag */
+    int lwt_msg_len;                        /*!< LWT message length */
+};
+
+struct Session {
+    LastWill last_will;
+    int disable_clean_session;              /*!< mqtt clean session, default clean_session is true */
+    int keepalive;                          /*!< mqtt keepalive, default is 120 seconds */
+    bool disable_keepalive;                 /*!< Set disable_keepalive=true to turn off keep-alive mechanism, false by default (keepalive is active by default). Note: setting the config value `keepalive` to `0` doesn't disable keepalive feature, but uses a default keepalive period */
+    esp_mqtt_protocol_ver_t protocol_ver;   /*!< MQTT protocol version used for connection, defaults to value from menuconfig*/
+};
+
+struct Task {
+    int task_prio;                          /*!< MQTT task priority, default is 5, can be changed in ``make menuconfig`` */
+    int task_stack;                         /*!< MQTT task stack size, default is 6144 bytes, can be changed in ``make menuconfig`` */
+};
+
+struct Connection {
+    esp_mqtt_transport_t transport;         /*!< overrides URI transport */
+    int reconnect_timeout_ms;               /*!< Reconnect to the broker after this value in miliseconds if auto reconnect is not disabled (defaults to 10s) */
+    int network_timeout_ms;                 /*!< Abort network operation if it is not completed after this value, in milliseconds (defaults to 10s) */
+    int refresh_connection_after_ms;        /*!< Refresh connection after this value (in milliseconds) */
+    bool disable_auto_reconnect;            /*!< this mqtt client will reconnect to server (when errors/disconnect). Set disable_auto_reconnect=true to disable */
+};
+
+struct Configuration {
+    Event event;
+    Task task;
+    Session session;
+    Connection connection;
+    void *user_context;                     /*!< pass user context to this option, then can receive that context in ``event->user_context`` */
+    int buffer_size;                        /*!< size of MQTT send/receive buffer, default is 1024 (only receive buffer size if ``out_buffer_size`` defined) */
+    int out_buffer_size;                    /*!< size of MQTT output buffer. If not defined, both output and input buffers have the same size defined as ``buffer_size`` */
+};
+
+} // idf::mqtt

components/esp_websocket_client/CHANGELOG.md

@@ -0,0 +1,107 @@
+# Changelog
+
+## [1.0.0](https://github.com/espressif/esp-protocols/commits/996fef7)
+
+### Updated
+
+- esp_websocket_client: Updated version to 1.0.0 Updated tests to run agains release-v5.0 ([996fef7](https://github.com/espressif/esp-protocols/commit/996fef7))
+- esp_websocket_client: * Error handling improved to show status code from server * Added new API `esp_websocket_client_set_headers` * Dispatches 'WEBSOCKET_EVENT_BEFORE_CONNECT' event before tcp connection ([d047ff5](https://github.com/espressif/esp-protocols/commit/d047ff5))
+- unite all tags under common structure py test: update tags under common structure ([c6db3ea](https://github.com/espressif/esp-protocols/commit/c6db3ea))
+- websocket: Support HTTP basic authorization ([1b13448](https://github.com/espressif/esp-protocols/commit/1b13448))
+- Add task_name config option ([1d68884](https://github.com/espressif/esp-protocols/commit/1d68884))
+- Add websocket error messages ([d68624e](https://github.com/espressif/esp-protocols/commit/d68624e))
+- websocket: Added new API `esp_websocket_client_destroy_on_exit` ([f9b4790](https://github.com/espressif/esp-protocols/commit/f9b4790))
+- Added badges with version of components to the respective README files ([e4c8a59](https://github.com/espressif/esp-protocols/commit/e4c8a59))
+
+
+## [0.0.4](https://github.com/espressif/esp-protocols/commits/3330b96)
+
+### Updated
+
+- websocket: make `esp_websocket_client_send_with_opcode` a public API ([3330b96](https://github.com/espressif/esp-protocols/commit/3330b96))
+- websocket: updated example to use local websocket echo server ([55dc564](https://github.com/espressif/esp-protocols/commit/55dc564))
+- CI: Created a common requirements.txt ([23a537b](https://github.com/espressif/esp-protocols/commit/23a537b))
+- Examples: using pytest.ini from top level directory ([aee016d](https://github.com/espressif/esp-protocols/commit/aee016d))
+- CI: fixing the files to be complient with pre-commit hooks ([945bd17](https://github.com/espressif/esp-protocols/commit/945bd17))
+- websocket: updated example to show json data transfer ([3456781](https://github.com/espressif/esp-protocols/commit/3456781))
+
+
+## [0.0.3](https://github.com/espressif/esp-protocols/commits/5c245db)
+
+### Updated
+
+- esp_websocket_client: Upgraded version to 0.0.3 ([5c245db](https://github.com/espressif/esp-protocols/commit/5c245db))
+- CI: Fix build issues ([6e4e4fa](https://github.com/espressif/esp-protocols/commit/6e4e4fa))
+
+
+## [0.0.2](https://github.com/espressif/esp-protocols/commits/57afa38)
+
+### Features
+
+- Optimize memory size for websocket client init ([4cefcd3](https://github.com/espressif/esp-protocols/commit/4cefcd3))
+- allow users to attach CA bundle ([d56b5d9](https://github.com/espressif/esp-protocols/commit/d56b5d9))
+
+### Bug Fixes
+
+- Docs to refer esp-protocols ([91a177e](https://github.com/espressif/esp-protocols/commit/91a177e))
+
+### Updated
+
+- Bump asio/mdns/esp_websocket_client versions ([57afa38](https://github.com/espressif/esp-protocols/commit/57afa38))
+- ignore format warnings ([d66f9dc](https://github.com/espressif/esp-protocols/commit/d66f9dc))
+- Minor fixes here and there ([8fe2a3a](https://github.com/espressif/esp-protocols/commit/8fe2a3a))
+- CI: Added CI example run job ([76298ff](https://github.com/espressif/esp-protocols/commit/76298ff))
+- Implement websocket client connect error ([9e37f53](https://github.com/espressif/esp-protocols/commit/9e37f53))
+- Add methods to allow get/set of websocket client ping interval ([e55f54b](https://github.com/espressif/esp-protocols/commit/e55f54b))
+- esp_websocket_client: Expose frame fin flag in websocket event ([b72a9ae](https://github.com/espressif/esp-protocols/commit/b72a9ae))
+
+
+## [0.0.1](https://github.com/espressif/esp-protocols/commits/80c3cf0)
+
+### Updated
+
+- websocket: Initial version based on IDF 5.0 ([80c3cf0](https://github.com/espressif/esp-protocols/commit/80c3cf0))
+- freertos: Remove legacy data types ([b3c777a](https://github.com/espressif/esp-protocols/commit/b3c777a), [IDF@57fd78f](https://github.com/espressif/esp-idf/commit/57fd78f5baf93a368a82cf4b2e00ca17ffc09115))
+- websocket: Added configs `reconnect_timeout_ms` and `network_timeout_ms` ([8ce791e](https://github.com/espressif/esp-protocols/commit/8ce791e), [IDF#8263](https://github.com/espressif/esp-idf/issues/8263), [IDF@6c26d65](https://github.com/espressif/esp-idf/commit/6c26d6520311f83c2ebe852a487c36185a429a69))
+- Add http_parser (new component) dependency ([bece6e7](https://github.com/espressif/esp-protocols/commit/bece6e7), [IDF@8e94cf2](https://github.com/espressif/esp-idf/commit/8e94cf2bb1498e94045e73e649f1046111fc6f9f))
+- websocket: removed deprecated API "esp_websocket_client_send" ([46bd32d](https://github.com/espressif/esp-protocols/commit/46bd32d), [IDF@7f6ab93](https://github.com/espressif/esp-idf/commit/7f6ab93f7e52bddaf4c030d7337ea5574f33381d))
+- refactor (test_utils)!: separate file for memory check functions ([525c70c](https://github.com/espressif/esp-protocols/commit/525c70c), [IDF@16514f9](https://github.com/espressif/esp-idf/commit/16514f93f06cd833306459d615458536a9f2e5cd))
+- Build & config: Remove leftover files from the unsupported "make" build system ([19c0455](https://github.com/espressif/esp-protocols/commit/19c0455), [IDF@766aa57](https://github.com/espressif/esp-idf/commit/766aa5708443099f3f033b739cda0e1de101cca6))
+- transport: Add CONFI_WS_TRANSPORT for optimize the code size ([9118e0f](https://github.com/espressif/esp-protocols/commit/9118e0f), [IDF@8b02c90](https://github.com/espressif/esp-idf/commit/8b02c9026af32352c8c4ed23025fb42182db6cae))
+- ws_client: Fix const correctness in the API config structure ([fbdbd55](https://github.com/espressif/esp-protocols/commit/fbdbd55), [IDF@70b1247](https://github.com/espressif/esp-idf/commit/70b1247a47f4583fccd8a91bf6cc532e5741e632))
+- components: Remove repeated keep alive function by ssl layer function ([de7cd72](https://github.com/espressif/esp-protocols/commit/de7cd72), [IDF@c79a907](https://github.com/espressif/esp-idf/commit/c79a907e4fef0c54175ad5659bc0df45a40745c9))
+- components: Support bind socket to specified interface in esp_http_client and esp_websocket_client component ([4a608ec](https://github.com/espressif/esp-protocols/commit/4a608ec), [IDF@bead359](https://github.com/espressif/esp-idf/commit/bead3599abd875d746e64cd6749574ff2c155adb))
+- esp_websocket_client: Don't log the filename when logging "Websocket already stop" ([f0351ff](https://github.com/espressif/esp-protocols/commit/f0351ff), [IDF@10bde42](https://github.com/espressif/esp-idf/commit/10bde42551b479bd4bfccc9d3c6d983f8abe0b87))
+- websocket: Add websocket unit tests ([9219ff7](https://github.com/espressif/esp-protocols/commit/9219ff7), [IDF@cd01a0c](https://github.com/espressif/esp-idf/commit/cd01a0ca81ef2ba5648fd7712c9bf45bbf252339))
+- websockets: Set keepalive options after adding transport to the list ([86aa0b8](https://github.com/espressif/esp-protocols/commit/86aa0b8), [IDF@99805d8](https://github.com/espressif/esp-idf/commit/99805d880f41857702b3bbb35bc0dfaf7dec3aec))
+- websocket: Add configurable ping interval ([1933367](https://github.com/espressif/esp-protocols/commit/1933367), [IDF@9ff9137](https://github.com/espressif/esp-idf/commit/9ff9137e7a8b64e956c1c63e95a48f4049ad571e))
+- ws_transport: Add option to propagate control packets to the app ([95cf983](https://github.com/espressif/esp-protocols/commit/95cf983), [IDF#6307](https://github.com/espressif/esp-idf/issues/6307), [IDF@acc7bd2](https://github.com/espressif/esp-idf/commit/acc7bd2ca45c21033cbd02220a27c3c1ecdd5ad0))
+- Add options for esp_http_client and esp_websocket_client to support keepalive ([8a6c320](https://github.com/espressif/esp-protocols/commit/8a6c320), [IDF@b53e46a](https://github.com/espressif/esp-idf/commit/b53e46a68e8671c73e8aafe2602de5ff5a77e3db))
+- websocket: support mutual tls for websocket Closes https://github.com/espressif/esp-idf/issues/6059 ([d1dd6ec](https://github.com/espressif/esp-protocols/commit/d1dd6ec), [IDF#6059](https://github.com/espressif/esp-idf/issues/6059), [IDF@5ab774f](https://github.com/espressif/esp-idf/commit/5ab774f9d8e119fff56b566fa2f9bdad853bf701))
+- Whitespace: Automated whitespace fixes (large commit) ([d376480](https://github.com/espressif/esp-protocols/commit/d376480), [IDF@66fb5a2](https://github.com/espressif/esp-idf/commit/66fb5a29bbdc2482d67c52e6f66b303378c9b789))
+- Websocket client: avoid deadlock if stop called from event handler ([e90272c](https://github.com/espressif/esp-protocols/commit/e90272c), [IDF@c2bb076](https://github.com/espressif/esp-idf/commit/c2bb0762bb5c24cb170bc9c96fdadb86ae2f06e7))
+- tcp_transport: Added internal API for underlying socket, used for custom select on connection end for WS ([6d12d06](https://github.com/espressif/esp-protocols/commit/6d12d06), [IDF@5e9f8b5](https://github.com/espressif/esp-idf/commit/5e9f8b52e7a87371370205a387b2d94e5ac6cbf9))
+- ws_client: Added support for close frame, closing connection gracefully ([1455bc0](https://github.com/espressif/esp-protocols/commit/1455bc0), [IDF@b213f2c](https://github.com/espressif/esp-idf/commit/b213f2c6d3d78ba3a95005e3206d4ce370b8a649))
+- driver, http_client, web_socket, tcp_transport: remove __FILE__ from log messages ([01b4f64](https://github.com/espressif/esp-protocols/commit/01b4f64), [IDF#5637](https://github.com/espressif/esp-idf/issues/5637), [IDF@caaf62b](https://github.com/espressif/esp-idf/commit/caaf62bdad965e6b58bba74171986414057f6757))
+-  websocket_client : fix some issues for websocket client ([6ab0aea](https://github.com/espressif/esp-protocols/commit/6ab0aea), [IDF@341e480](https://github.com/espressif/esp-idf/commit/341e48057349d92c3b8afe5f9c0fcd0aa47500b0))
+- websocket: add configurable timeout for PONG not received ([b71c49c](https://github.com/espressif/esp-protocols/commit/b71c49c), [IDF@0049385](https://github.com/espressif/esp-idf/commit/0049385850daebfe2222c8f0526b896ffaeacdd9))
+- websocket client: the client now aborts the connection if send fails. ([f8e3ba7](https://github.com/espressif/esp-protocols/commit/f8e3ba7), [IDF@6bebfc8](https://github.com/espressif/esp-idf/commit/6bebfc84f3ed9c96bcb331fd0d5b0bbb26ce07a4))
+- ws_client: fix fragmented send setting proper opcodes ([7a5b2d5](https://github.com/espressif/esp-protocols/commit/7a5b2d5), [IDF#4974](https://github.com/espressif/esp-idf/issues/4974), [IDF@14992e6](https://github.com/espressif/esp-idf/commit/14992e62c5573d8b6076281f16b4fe11d6bc8f87))
+- esp32: add implementation of esp_timer based on TG0 LAC timer ([17281a5](https://github.com/espressif/esp-protocols/commit/17281a5), [IDF@739eb05](https://github.com/espressif/esp-idf/commit/739eb05bb97736b70507e7ebcfee58e670672d23))
+- tcp_transport/ws_client: websockets now correctly handle messages longer than buffer ([aec6a75](https://github.com/espressif/esp-protocols/commit/aec6a75), [IDF@ffeda30](https://github.com/espressif/esp-idf/commit/ffeda3003c92102d2d5b145c9adb3ea3105cbbda))
+- websocket: added missing event data ([a6be8e2](https://github.com/espressif/esp-protocols/commit/a6be8e2), [IDF@7c0e376](https://github.com/espressif/esp-idf/commit/7c0e3765ec009acaf2ef439e98895598b5fd9aaf))
+- Add User-Agent and additional headers to esp_websocket_client ([a48b0fa](https://github.com/espressif/esp-protocols/commit/a48b0fa), [IDF@9200250](https://github.com/espressif/esp-idf/commit/9200250f512146e348f84ebfc76f9e82e2070da2))
+- ws_client: fix handling timeouts by websocket client. ([1fcc001](https://github.com/espressif/esp-protocols/commit/1fcc001), [IDF#4316](https://github.com/espressif/esp-idf/issues/4316), [IDF@e1f9829](https://github.com/espressif/esp-idf/commit/e1f982921a08022ca4307900fc058ccacccd26d0))
+- websocket_client: fix locking mechanism in ws-client task and when sending data ([d0121b9](https://github.com/espressif/esp-protocols/commit/d0121b9), [IDF@7c5011f](https://github.com/espressif/esp-idf/commit/7c5011f411b7662feb50fd1e53114bec390d8c2e))
+- ws_client: fix for not sending ping responses, updated to pass events also for PING and PONG messages, added interfaces to send both binary and text data ([f55d839](https://github.com/espressif/esp-protocols/commit/f55d839), [IDF@abf9345](https://github.com/espressif/esp-idf/commit/abf9345b85559f4a922e8387f48336fb09994041))
+- websocket_client: fix URI parsing to include also query part in websocket connection path ([f5a26c4](https://github.com/espressif/esp-protocols/commit/f5a26c4), [IDF@271e6c4](https://github.com/espressif/esp-idf/commit/271e6c4c9c57ca6715c1435a71fe3974cd2b18b3))
+- ws_client: fixed posting to event loop with websocket timeout ([23f6a1d](https://github.com/espressif/esp-protocols/commit/23f6a1d), [IDF@5050506](https://github.com/espressif/esp-idf/commit/50505068c45fbe97611be9b7f2c30b8160cbb9e3))
+- ws_client: added subprotocol configuration option to websocket client ([2553d65](https://github.com/espressif/esp-protocols/commit/2553d65), [IDF@de6ea39](https://github.com/espressif/esp-idf/commit/de6ea396f17be820153da6acaf977c1bf11806fb))
+- ws_client: fixed path config issue when ws server configured using host and path instead of uri ([67949f9](https://github.com/espressif/esp-protocols/commit/67949f9), [IDF@c0ba9e1](https://github.com/espressif/esp-idf/commit/c0ba9e19fc6cff79f5760b991c259970bd4abeab))
+- ws_client: fixed transport config option when server address configured as host, port, transport rather then uri ([bfc88ab](https://github.com/espressif/esp-protocols/commit/bfc88ab), [IDF@adee25d](https://github.com/espressif/esp-idf/commit/adee25d90e100a169e959f94db23621f6ffab0e6))
+- esp_wifi: wifi support new event mechanism ([4d64495](https://github.com/espressif/esp-protocols/commit/4d64495), [IDF@003a987](https://github.com/espressif/esp-idf/commit/003a9872b7de69d799e9d37521cfbcaff9b37e85))
+- tools: Mass fixing of empty prototypes (for -Wstrict-prototypes) ([da74a4a](https://github.com/espressif/esp-protocols/commit/da74a4a), [IDF@afbaf74](https://github.com/espressif/esp-idf/commit/afbaf74007e89d016dbade4072bf2e7a3874139a))
+- ws_client: fix double delete issue in ws client initialization ([f718676](https://github.com/espressif/esp-protocols/commit/f718676), [IDF@9b507c4](https://github.com/espressif/esp-idf/commit/9b507c45c86cf491466d705cd7896c6f6e500d0d))
+- ws_client: removed dependency on internal tcp_transport header ([13a40d2](https://github.com/espressif/esp-protocols/commit/13a40d2), [IDF@d143356](https://github.com/espressif/esp-idf/commit/d1433564ecfc885f80a7a261a88ab87d227cf1c2))
+- examples: use new component registration api ([35d6f9a](https://github.com/espressif/esp-protocols/commit/35d6f9a), [IDF@6771eea](https://github.com/espressif/esp-idf/commit/6771eead80534c51efb2033c04769ef5893b4838))
+- esp_websocket_client: Add websocket client component ([f3a0586](https://github.com/espressif/esp-protocols/commit/f3a0586), [IDF#2829](https://github.com/espressif/esp-idf/issues/2829), [IDF@2a2d932](https://github.com/espressif/esp-idf/commit/2a2d932cfe2404057c71bc91d9d9416200e67a03))