bootloader: ensure security eFuses are always burnt

Previously security eFuses were only burnt if the flash was not encrypted yet. To enhance robustness of the security eFuse settings their correct setup should be verified on each bootup. Else it would be possible for an already encrypted ESP to be reflashed with firmware containing updated, more restrictive eFuse settings without them ever being applied. Additionally this change enables easy, secure use of ESPs with host sidee flash preencryption. Flash preencryption by the host computer performing the programming procedure can speed up the programming process by a great deal since the flash no longer needs to be read, erased and written again by the bootloader self-encryption routines. Additionally it avoids bricking of ESPs through interruption of the self-ecnryption procedure. Without this change the host would have to set up all fuses in the ESP correctly by itself, duplicating the fuse configuration code already present in the bootloader and creating additional maintenance burden for the host software if anything about the fuse setup logic changes. This commit changes the security eFuse configuration logic to always burn any configured security eFuses on bootup, regardless of current flash encryption status.
No more noreturn on asserts
2022-07-19 12:42:55 +02:00 · 2022-07-11 15:07:23 +02:00 · 2022-07-11 15:07:16 +02:00 · 2022-05-24 11:48:32 +02:00 · 2022-05-24 11:48:32 +02:00 · 2022-05-24 11:48:32 +02:00
4469 changed files with 85436 additions and 232132 deletions
--- a/.flake8
+++ b/.flake8
@@ -149,17 +149,18 @@ exclude =
        components/tinyusb,
        components/unity/unity,
        components/spiffs/spiffs,
+        examples/build_system/cmake/import_lib/main/lib/tinyxml2,
        examples/peripherals/secure_element/atecc608_ecdsa/components/esp-cryptoauthlib,
    # autogenerated scripts
        components/protocomm/python/constants_pb2.py,
        components/protocomm/python/sec0_pb2.py,
        components/protocomm/python/sec1_pb2.py,
-        components/protocomm/python/sec2_pb2.py,
        components/protocomm/python/session_pb2.py,
        components/wifi_provisioning/python/wifi_scan_pb2.py,
        components/wifi_provisioning/python/wifi_config_pb2.py,
        components/wifi_provisioning/python/wifi_constants_pb2.py,
        components/esp_local_ctrl/python/esp_local_ctrl_pb2.py,
+        examples/provisioning/legacy/custom_config/components/custom_provisioning/python/custom_config_pb2.py,

 per-file-ignores =
    # Sphinx conf.py files use star imports to setup config variables
--- a/.github/workflows/docker.yml
+++ b/.github/workflows/docker.yml
@@ -1,87 +0,0 @@
-name: docker
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-
-on:
-  push:
-    branches:
-      - 'master'
-      - 'release/*'
-    tags:
-      - 'v*.*'
-
-env:
-  # Platforms to build the image for
-  BUILD_PLATFORMS: linux/amd64,linux/arm64
-  DOCKERHUB_REPO: ${{ github.repository_owner }}/idf
-
-jobs:
-  docker:
-    # Disable the job in forks
-    if: ${{ github.repository_owner == 'espressif' }}
-
-    runs-on: ubuntu-latest
-    steps:
-      # Depending on the branch/tag, set CLONE_BRANCH_OR_TAG variable (used in the Dockerfile
-      # as a build arg) and TAG_NAME (used when tagging the image).
-      #
-      # The following 3 steps cover the alternatives (tag, release branch, master branch):
-      - name: Set variables (tags)
-        if: ${{ github.ref_type == 'tag' }}
-        run: |
-          echo "CLONE_BRANCH_OR_TAG=$GITHUB_REF_NAME" >> $GITHUB_ENV
-          echo "TAG_NAME=$GITHUB_REF_NAME" >> $GITHUB_ENV
-      - name: Set variables (release branches)
-        if: ${{ github.ref_type == 'branch' && startsWith(github.ref_name, 'release/') }}
-        run: |
-          echo "CLONE_BRANCH_OR_TAG=$GITHUB_REF_NAME" >> $GITHUB_ENV
-          echo "TAG_NAME=release-${GITHUB_REF_NAME##release/}" >> $GITHUB_ENV
-      - name: Set variables (main branch)
-        if: ${{ github.ref_type == 'branch' && github.ref_name == 'master' }}
-        run: |
-          echo "CLONE_BRANCH_OR_TAG=master" >> $GITHUB_ENV
-          echo "TAG_NAME=latest" >> $GITHUB_ENV
-
-      # Display the variables set above, just in case.
-      - name: Check variables
-        run: |
-          echo "CLONE_BRANCH_OR_TAG: $CLONE_BRANCH_OR_TAG"
-          echo "CHECKOUT_REF: $CHECKOUT_REF"
-          echo "TAG_NAME: $TAG_NAME"
-
-      # The following steps are the standard boilerplate from
-      # https://github.com/marketplace/actions/build-and-push-docker-images
-      - name: Checkout
-        uses: actions/checkout@v3
-      - name: Login to Docker Hub
-        uses: docker/login-action@v2
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-      - name: Set up QEMU for multiarch builds
-        uses: docker/setup-qemu-action@v2
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
-      - name: Build and push
-        uses: docker/build-push-action@v3
-        with:
-          context: tools/docker
-          push: true
-          tags: ${{ env.DOCKERHUB_REPO }}:${{ env.TAG_NAME }}
-          platforms: ${{ env.BUILD_PLATFORMS }}
-          build-args: |
-            IDF_CLONE_URL=${{ github.server_url }}/${{ github.repository }}.git
-            IDF_CLONE_BRANCH_OR_TAG=${{ env.CLONE_BRANCH_OR_TAG }}
-
-      - name: Update Docker Hub repository description (master branch)
-        if: ${{ github.ref_type == 'branch' && github.ref_name == 'master' }}
-        uses: peter-evans/dockerhub-description@v3
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          # Token based authentication is not supported here:
-          # https://github.com/peter-evans/dockerhub-description/issues/10
-          # https://github.com/docker/roadmap/issues/115#issuecomment-891694974
-          password: ${{ secrets.DOCKERHUB_PASSWORD }}
-          repository: ${{ env.DOCKERHUB_REPO }}
-          readme-filepath: ./tools/docker/README.md
--- a/.gitignore
+++ b/.gitignore
@@ -21,20 +21,15 @@ GPATH
 # MacOS directory files
 .DS_Store

-# cache dir
-.cache/
-
 # Components Unit Test Apps files
-components/**/build/
-components/**/build_*_*/
+components/**/build
 components/**/sdkconfig
 components/**/sdkconfig.old

 # Example project files
-examples/**/build/
-examples/**/build_esp*_*/
 examples/**/sdkconfig
 examples/**/sdkconfig.old
+examples/**/build

 # Doc build artifacts
 docs/_build/
@@ -48,7 +43,7 @@ docs/_static/NotoSansSC-Regular.otf
 tools/unit-test-app/sdkconfig
 tools/unit-test-app/sdkconfig.old
 tools/unit-test-app/build
-tools/unit-test-app/build_*_*/
+tools/unit-test-app/builds
 tools/unit-test-app/output
 tools/unit-test-app/test_configs

@@ -56,8 +51,7 @@ tools/unit-test-app/test_configs
 log_ut_cmake

 # test application build files
-tools/test_apps/**/build/
-tools/test_apps/**/build_*_*/
+tools/test_apps/**/build
 tools/test_apps/**/sdkconfig
 tools/test_apps/**/sdkconfig.old

--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,5 +1,4 @@
 stages:
-  - upload_cache
  - pre_check
  - build
  - assign_test
@@ -54,10 +53,6 @@ variables:
  BATCH_BUILD: "1"
  V: "0"
  CHECKOUT_REF_SCRIPT: "$CI_PROJECT_DIR/tools/ci/checkout_project_ref.py"
-  PYTHON_VER: 3.7.10
-
-  CLANG_TIDY_RUNNER_PROJ: 2107  # idf/clang-tidy-runner
-  IDF_BUILD_APPS_PROJ: 2818  # fuhanxi/idf-build-apps

  # Docker images
  BOT_DOCKER_IMAGE_TAG: ":latest"
@@ -79,6 +74,7 @@ variables:
  TEST_ENV_CONFIG_REPO: "https://gitlab-ci-token:${BOT_TOKEN}@${CI_SERVER_HOST}:${CI_SERVER_PORT}/qa/ci-test-runner-configs.git"
  CI_AUTO_TEST_SCRIPT_REPO_URL: "https://gitlab-ci-token:${BOT_TOKEN}@${CI_SERVER_HOST}:${CI_SERVER_PORT}/qa/auto_test_script.git"
  CI_AUTO_TEST_SCRIPT_REPO_BRANCH: "ci/v4.1"
+  PYTEST_EMBEDDED_VERSION: "0.7.0"

  # cache python dependencies
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
@@ -89,7 +85,7 @@ variables:
  CI_PYTHON_CONSTRAINT_BRANCH: ""

  # Update the filename for a specific ESP-IDF release. It is used only with CI_PYTHON_CONSTRAINT_BRANCH.
-  CI_PYTHON_CONSTRAINT_FILE: "espidf.constraints.v5.1.txt"
+  CI_PYTHON_CONSTRAINT_FILE: "espidf.constraints.v5.0.txt"

  # Set this variable to repository name of a Python tool you wish to install and test in the context of ESP-IDF CI.
  # Keep the variable empty when not used.
@@ -101,16 +97,12 @@ variables:
  CI_PYTHON_TOOL_BRANCH: ""

 cache:
+  key: pip-cache
+  paths:
+    - .cache/pip
  # pull only for most of the use cases since it's cache dir.
-  # Only set "push" policy for "upload_cache" stage jobs
-  - key: pip-cache
-    paths:
-      - .cache/pip
-    policy: pull
-  - key: submodule-cache
-    paths:
-      - .cache/submodule_archives
-    policy: pull
+  # Only set "pull-push" policy for "scan_test" since it would **possibly** install all pypi packages
+  policy: pull

 .setup_tools_unless_target_test: &setup_tools_unless_target_test |
  if [[ -n "$IDF_DONT_USE_MIRRORS" ]]; then
@@ -142,15 +134,9 @@ cache:
    rm -rf ${CI_PYTHON_TOOL_REPO}
  fi

-.set_include_nightly_run: &set_include_nightly_run |
-  if echo "$CI_MERGE_REQUEST_LABELS" | egrep "^([^,\n\r]+,)*include_nightly_run(,[^,\n\r]+)*$"; then
-    export INCLUDE_NIGHTLY_RUN="1"
-  fi
-
 before_script:
  - source tools/ci/utils.sh
  - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-  - *set_include_nightly_run
  - source tools/ci/setup_python.sh
  - add_gitlab_ssh_keys
  - source tools/ci/configure_ci_environment.sh
@@ -168,7 +154,6 @@ before_script:
    - echo "Not setting up GitLab key, not fetching submodules"
    - source tools/ci/utils.sh
    - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-    - *set_include_nightly_run
    - source tools/ci/setup_python.sh
    - source tools/ci/configure_ci_environment.sh
    - *download_test_python_contraint_file
@@ -179,14 +164,12 @@ before_script:
    - echo "Only load utils.sh"
    - source tools/ci/utils.sh
    - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-    - *set_include_nightly_run

 .before_script_integration_test:
  before_script:
    # need to install esptool, needn't install idf python env
  - source tools/ci/utils.sh
  - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-  - *set_include_nightly_run
  - source tools/ci/setup_python.sh
  - add_gitlab_ssh_keys
  - fetch_submodules
@@ -202,7 +185,6 @@ before_script:
  before_script:
    - source tools/ci/utils.sh
    - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-    - *set_include_nightly_run
    - export IDF_TOOLS_PATH="${HOME}/.espressif_runner_${CI_RUNNER_ID}_${CI_CONCURRENT_ID}"
    - *download_test_python_contraint_file
    - $IDF_PATH/tools/idf_tools.py install-python-env
@@ -211,7 +193,7 @@ before_script:
    # This adds tools (compilers) and the version-specific Python environment to PATH
    - *setup_tools_unless_target_test
    # Install packages required by CI scripts into IDF Python environment
-    - pip install -r $IDF_PATH/tools/requirements/requirements.ci.txt
+    - pip install -r $IDF_PATH/tools/ci/python_packages/ttfw_idf/requirements.txt
    - source tools/ci/configure_ci_environment.sh
    # Part of tools/ci/setup_python.sh; we don't use pyenv on macOS, so can't run the rest of the script.
    - export PYTHONPATH="$IDF_PATH/tools:$IDF_PATH/tools/ci/python_packages:$PYTHONPATH"
@@ -221,31 +203,22 @@ before_script:
  before_script:
    - source tools/ci/utils.sh
    - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-    - *set_include_nightly_run
    - source tools/ci/setup_python.sh
    - add_gitlab_ssh_keys
    - source tools/ci/configure_ci_environment.sh
    - *setup_tools_unless_target_test
    - fetch_submodules
    - *download_test_python_contraint_file
-    # only need ci and pytest related packages, ttfw requirements would be mocked if not installed
-    - $IDF_PATH/tools/idf_tools.py install-python-env --features pytest,ci
-    - eval "$($IDF_PATH/tools/idf_tools.py export)"  # use idf venv instead
-
-.before_script_build_jobs:
-  before_script:
-    - source tools/ci/utils.sh
-    - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-    - *set_include_nightly_run
-    - source tools/ci/setup_python.sh
-    - add_gitlab_ssh_keys
-    - source tools/ci/configure_ci_environment.sh
-    - *setup_tools_unless_target_test
-    - fetch_submodules
-    - *download_test_python_contraint_file
-    # only need ci and pytest related packages, ttfw requirements would be mocked if not installed
-    - $IDF_PATH/tools/idf_tools.py install-python-env --features pytest,ci
-    - eval "$($IDF_PATH/tools/idf_tools.py export)"  # use idf venv instead
+    - $IDF_PATH/tools/idf_tools.py install-python-env
+    # TODO: remove this, IDFCI-1207
+    - pip install esptool -c ~/.espressif/${CI_PYTHON_CONSTRAINT_FILE}
+    - pip install
+      "pytest-embedded-serial-esp~=$PYTEST_EMBEDDED_VERSION"
+      "pytest-embedded-idf~=$PYTEST_EMBEDDED_VERSION"
+      "pytest-embedded-qemu~=$PYTEST_EMBEDDED_VERSION"
+      pytest-rerunfailures
+      scapy
+      google-api-python-client
    - export EXTRA_CFLAGS=${PEDANTIC_CFLAGS}
    - export EXTRA_CXXFLAGS=${PEDANTIC_CXXFLAGS}

@@ -260,7 +233,6 @@ default:

 include:
  - '.gitlab/ci/rules.yml'
-  - '.gitlab/ci/upload_cache.yml'
  - '.gitlab/ci/docs.yml'
  - '.gitlab/ci/static-code-analysis.yml'
  - '.gitlab/ci/pre_check.yml'
--- a/.gitlab/CODEOWNERS
+++ b/.gitlab/CODEOWNERS
@@ -69,15 +69,17 @@

 /components/app_trace/                @esp-idf-codeowners/tools
 /components/app_update/               @esp-idf-codeowners/system @esp-idf-codeowners/app-utilities
+/components/asio/                     @esp-idf-codeowners/network
 /components/bootloader*/              @esp-idf-codeowners/system @esp-idf-codeowners/security
 /components/bootloader_support/bootloader_flash/    @esp-idf-codeowners/peripherals
 /components/bt/                       @esp-idf-codeowners/bluetooth
 /components/cmock/                    @esp-idf-codeowners/system
-/components/console/                  @esp-idf-codeowners/system @esp-idf-codeowners/app-utilities/console
+/components/console/                  @esp-idf-codeowners/system @esp-idf-codeowners/app-utilities
 /components/cxx/                      @esp-idf-codeowners/system
 /components/driver/                   @esp-idf-codeowners/peripherals
 /components/efuse/                    @esp-idf-codeowners/system
-/components/esp_adc/                  @esp-idf-codeowners/peripherals
+/components/esp32*/                   @esp-idf-codeowners/system
+/components/esp_adc_cal/              @esp-idf-codeowners/peripherals
 /components/esp_common/               @esp-idf-codeowners/system
 /components/esp_eth/                  @esp-idf-codeowners/network
 /components/esp_event/                @esp-idf-codeowners/system
@@ -93,7 +95,6 @@
 /components/esp_netif/                @esp-idf-codeowners/network
 /components/esp_phy/                  @esp-idf-codeowners/bluetooth @esp-idf-codeowners/wifi @esp-idf-codeowners/ieee802154
 /components/esp_pm/                   @esp-idf-codeowners/power-management @esp-idf-codeowners/bluetooth @esp-idf-codeowners/wifi
-/components/esp_psram/                @esp-idf-codeowners/peripherals @esp-idf-codeowners/system
 /components/esp_ringbuf/              @esp-idf-codeowners/system
 /components/esp_rom/                  @esp-idf-codeowners/system @esp-idf-codeowners/bluetooth @esp-idf-codeowners/wifi
 /components/esp_serial_slave_link/    @esp-idf-codeowners/peripherals
@@ -115,7 +116,8 @@
 /components/linux/                    @esp-idf-codeowners/system
 /components/log/                      @esp-idf-codeowners/system
 /components/lwip/                     @esp-idf-codeowners/lwip
-/components/mbedtls/                  @esp-idf-codeowners/app-utilities/mbedtls @esp-idf-codeowners/security
+/components/mbedtls/                  @esp-idf-codeowners/app-utilities @esp-idf-codeowners/security
+/components/mdns/                     @esp-idf-codeowners/network
 /components/mqtt/                     @esp-idf-codeowners/network
 /components/newlib/                   @esp-idf-codeowners/system @esp-idf-codeowners/tools
 /components/nvs_flash/                @esp-idf-codeowners/storage
@@ -123,7 +125,7 @@
 /components/partition_table/          @esp-idf-codeowners/system
 /components/perfmon/                  @esp-idf-codeowners/tools
 /components/protobuf-c/               @esp-idf-codeowners/app-utilities
-/components/protocomm/                @esp-idf-codeowners/app-utilities/provisioning
+/components/protocomm/                @esp-idf-codeowners/app-utilities
 /components/pthread/                  @esp-idf-codeowners/system
 /components/riscv/                    @esp-idf-codeowners/system
 /components/sdmmc/                    @esp-idf-codeowners/storage
@@ -138,7 +140,7 @@
 /components/usb/                      @esp-idf-codeowners/peripherals/usb
 /components/vfs/                      @esp-idf-codeowners/storage
 /components/wear_levelling/           @esp-idf-codeowners/storage
-/components/wifi_provisioning/        @esp-idf-codeowners/app-utilities/provisioning
+/components/wifi_provisioning/        @esp-idf-codeowners/app-utilities
 /components/wpa_supplicant/           @esp-idf-codeowners/wifi
 /components/xtensa/                   @esp-idf-codeowners/system

@@ -147,7 +149,7 @@
 /docs/**/api-reference/network/         @esp-idf-codeowners/network
 /docs/**/api-reference/peripherals/     @esp-idf-codeowners/peripherals
 /docs/**/api-reference/protocols/       @esp-idf-codeowners/network @esp-idf-codeowners/app-utilities
-/docs/**/api-reference/provisioning/    @esp-idf-codeowners/app-utilities/provisioning
+/docs/**/api-reference/provisioning/    @esp-idf-codeowners/app-utilities
 /docs/**/api-reference/storage/         @esp-idf-codeowners/storage
 /docs/**/api-reference/system/          @esp-idf-codeowners/system
 /docs/**/security/                      @esp-idf-codeowners/security
@@ -158,7 +160,6 @@
 /examples/bluetooth/          @esp-idf-codeowners/bluetooth
 /examples/build_system/       @esp-idf-codeowners/build-config
 /examples/common_components/  @esp-idf-codeowners/system
-/examples/custom_bootloader/  @esp-idf-codeowners/system
 /examples/cxx/                @esp-idf-codeowners/system
 /examples/ethernet/           @esp-idf-codeowners/network
 /examples/get-started/        @esp-idf-codeowners/system
@@ -168,20 +169,21 @@
 /examples/peripherals/        @esp-idf-codeowners/peripherals
 /examples/peripherals/usb/    @esp-idf-codeowners/peripherals @esp-idf-codeowners/peripherals/usb
 /examples/protocols/          @esp-idf-codeowners/network @esp-idf-codeowners/app-utilities
-/examples/provisioning/       @esp-idf-codeowners/app-utilities/provisioning
+/examples/provisioning/       @esp-idf-codeowners/app-utilities
 /examples/security/           @esp-idf-codeowners/security
 /examples/storage/            @esp-idf-codeowners/storage
 /examples/system/             @esp-idf-codeowners/system
-/examples/system/ota/         @esp-idf-codeowners/app-utilities
 /examples/wifi/               @esp-idf-codeowners/wifi
 /examples/zigbee/             @esp-idf-codeowners/ieee802154

 /tools/                       @esp-idf-codeowners/tools
+/tools/*_apps.py              @esp-idf-codeowners/ci
 /tools/ble/                   @esp-idf-codeowners/app-utilities
 /tools/catch/                 @esp-idf-codeowners/ci
 /tools/ci/                    @esp-idf-codeowners/ci
 /tools/cmake/                 @esp-idf-codeowners/build-config
 /tools/esp_prov/              @esp-idf-codeowners/app-utilities
+/tools/find_build_apps/       @esp-idf-codeowners/ci
 /tools/idf_size_yaml/         @esp-idf-codeowners/peripherals
 /tools/kconfig*/              @esp-idf-codeowners/build-config
 /tools/ldgen/                 @esp-idf-codeowners/build-config
--- a/.gitlab/ci/README.md
+++ b/.gitlab/ci/README.md
@@ -19,13 +19,6 @@
    - [Functions](#functions)
      - [CI Job Related](#ci-job-related)
      - [Shell Script Related](#shell-script-related)
-  - [Manifest File to Control the Build/Test apps](#manifest-file-to-control-the-buildtest-apps)
-    - [Grammar](#grammar)
-      - [Operands](#operands)
-      - [Operators](#operators)
-      - [Limitation:](#limitation)
-    - [How does it work?](#how-does-it-work)
-    - [Example](#example)

 ## General Workflow

@@ -231,93 +224,3 @@ To run these commands in shell script locally, place `source tools/ci/utils.sh`
 - `info`: log in green color
 - `run_cmd`: run the command with duration seconds info
 - `retry_failed`: run the command with duration seconds info, retry when failed
-
-## Manifest File to Control the Build/Test apps
-
-`.build-test-rules.yml` file is a manifest file to control if the CI is running the build and test job or not. The Supported Targets table in `README.md` for apps would be auto-generated by `pre-commit` from the app's `.build-test-rules.yml`.
-
-### Grammar
-
-#### Operands
-
- Variables start with `SOC_`. The value would be parsed from components/soc/[TARGET]/include/soc/*_caps.h
- `IDF_TARGET`
- `INCLUDE_DEFAULT` (The default value of officially supported targets is 1, otherwise is 0)
- String, must be double-quoted. e.g. `"esp32"`, `"12345"`
- Integer, support decimal and hex. e.g. `1`, `0xAB`
- List with String and Integer inside, the type could be mixed. e.g. `["esp32", 1]`
-
-#### Operators
-
- `==`, `!=`, `>`, `>=`, `<`, `<=`
- `and`, `or`
- `in`, `not in` with list
- parentheses
-
-#### Limitation:
-
- all operators are binary operator. For more than two operands, you may use nested parentheses trick. For example,
-  - `A == 1 or (B == 2 and C in [1,2,3])`
-  - `(A == 1 and B == 2) or (C not in ["3", "4", 5])`
-
-### How does it work?
-
-By default, we enable build and test jobs for supported targets. In other words, if an app supports all supported targets, it does not need to be added in a manifest file. The manifest files are files that set the violation rules for apps.
-
-three rules (disable rules are calculated after the `enable` rule):
- enable: run CI build/test jobs for targets that match any of the specified conditions only
- disable: will not run CI build/test jobs for targets that match any of the specified conditions
- disable_test: will not run CI test jobs for targets that match any of the specified conditions
-
-Each key is a test folder. Will apply to all folders inside.
-
-If one sub folder is in a special case, you can overwrite the rules for this folder by adding another entry for this folder itself. Each folder's rules are standalone, and will not inherit its parent's rules. (YAML inheritance is too complicated for reading)
-
-For example in the following codeblock, only `disable` rule exists in `examples/foo/bar`. It's unaware of its parent's `enable` rule. 
-
-```yaml
-examples/foo:
-  enable:
-    - if: IDF_TARGET == "esp32"
-
-examples/foo/bar:
-  disable:
-    - if: IDF_TARGET == "esp32s2"
-```
-
-### Example
-
-```yaml
-examples/foo:
-  enable:
-    - if IDF_TARGET in ["esp32", 1, 2, 3]
-    - if IDF_TARGET not in ["4", "5", 6]
-  # should be run under all targets!
-
-examples/bluetooth:
-  disable:  # disable both build and tests jobs
-    - if: SOC_BT_SUPPORTED != 1
-    # reason is optional if there's no `temporary: true`
-  disable_test:
-    - if: IDF_TARGET == "esp32"
-      temporary: true
-      reason: lack of ci runners  # required when `temporary: true`
-
-examples/bluetooth/test_foo:
-  # each folder's settings are standalone
-  disable:
-    - if: IDF_TARGET == "esp32s2"
-      temporary: true
-      reason: no idea
-  # unlike examples/bluetooth, the apps under this folder would not be build nor test for "no idea" under target esp32s2
-
-examples/get-started/hello_world:
-  enable:
-    - if: IDF_TARGET == "linux"
-      reason: this one only supports linux!
-
-examples/get-started/blink:
-  enable:
-    - if: INCLUDE_DEFAULT == 1 or IDF_TARGET == "linux"
-      reason: This one supports all supported targets and linux
-```
--- a/.gitlab/ci/assign-test.yml
+++ b/.gitlab/ci/assign-test.yml
@@ -7,12 +7,12 @@
    SUBMODULES_TO_FETCH: "none"
  artifacts:
    paths:
-      - ${TEST_DIR}/test_configs/
-      - artifact_index.json
+      - ${TEST_DIR}/test_configs
+      - ${BUILD_DIR}/artifact_index.json
    when: always
    expire_in: 1 week
  script:
-    - run_cmd python tools/ci/python_packages/ttfw_idf/IDFAssignTest.py $TEST_TYPE $TEST_DIR -c $CI_TARGET_TEST_CONFIG_FILE -o $TEST_DIR/test_configs
+    - python tools/ci/python_packages/ttfw_idf/IDFAssignTest.py $TEST_TYPE $TEST_DIR -c $CI_TARGET_TEST_CONFIG_FILE -o $TEST_DIR/test_configs

 assign_example_test:
  extends:
@@ -25,9 +25,6 @@ assign_example_test:
    - job: build_examples_cmake_esp32s2
      artifacts: false
      optional: true
-    - job: build_examples_cmake_esp32c2
-      artifacts: false
-      optional: true
    - job: build_examples_cmake_esp32c3
      artifacts: false
      optional: true
@@ -36,7 +33,8 @@ assign_example_test:
      optional: true
  variables:
    TEST_TYPE: example_test
-    TEST_DIR: examples
+    TEST_DIR: ${CI_PROJECT_DIR}/examples
+    BUILD_DIR: ${CI_PROJECT_DIR}/build_examples

 assign_custom_test:
  extends:
@@ -49,9 +47,6 @@ assign_custom_test:
    - job: build_test_apps_esp32s2
      artifacts: false
      optional: true
-    - job: build_test_apps_esp32c2
-      artifacts: false
-      optional: true
    - job: build_test_apps_esp32c3
      artifacts: false
      optional: true
@@ -60,7 +55,8 @@ assign_custom_test:
      optional: true
  variables:
    TEST_TYPE: custom_test
-    TEST_DIR: tools/test_apps
+    TEST_DIR: ${CI_PROJECT_DIR}/tools/test_apps
+    BUILD_DIR: ${CI_PROJECT_DIR}/build_test_apps

 assign_unit_test:
  extends:
@@ -71,15 +67,16 @@ assign_unit_test:
      optional: true
    - job: build_esp_idf_tests_cmake_esp32s2
      optional: true
-    - job: build_esp_idf_tests_cmake_esp32c2
-      optional: true
    - job: build_esp_idf_tests_cmake_esp32c3
      optional: true
    - job: build_esp_idf_tests_cmake_esp32s3
      optional: true
  variables:
    TEST_TYPE: unit_test
-    TEST_DIR: components/idf_test/unit_test
+    TEST_DIR: ${CI_PROJECT_DIR}/components/idf_test/unit_test
+    BUILD_DIR: ${CI_PROJECT_DIR}/tools/unit-test-app/builds
+  script:
+    - python tools/ci/python_packages/ttfw_idf/IDFAssignTest.py $TEST_TYPE $TEST_DIR -c $CI_TARGET_TEST_CONFIG_FILE -o $TEST_DIR/test_configs

 assign_integration_test:
  extends:
--- a/.gitlab/ci/build.yml
+++ b/.gitlab/ci/build.yml
@@ -15,178 +15,138 @@
 .build_pytest_template:
  extends:
    - .build_template
-    - .before_script_build_jobs
+    - .before_script_pytest
  dependencies:  # set dependencies to null to avoid missing artifacts issue
  needs:
    - job: fast_template_app
      artifacts: false
+  variables:
+    PYTHON_VER: 3.7.10
  artifacts:
    paths:
      - "**/build*/size.json"
-      - "**/build*/build_log.txt"
+      - "**/build*/build.log"
      - "**/build*/*.bin"
      - "**/build*/*.elf"
      - "**/build*/*.map"
      - "**/build*/flasher_args.json"
-      - "**/build*/flash_project_args"
      - "**/build*/config/sdkconfig.json"
      - "**/build*/bootloader/*.bin"
      - "**/build*/partition_table/*.bin"
      - $SIZE_INFO_LOCATION
    when: always
    expire_in: 3 days
-  script:
-    # CI specific options start from "--collect-size-info xxx". could ignore when running locally
-    - run_cmd python tools/ci/ci_build_apps.py $TEST_DIR -v
-      -t $IDF_TARGET
-      --pytest-apps
-      --collect-size-info $SIZE_INFO_LOCATION
-      --parallel-count ${CI_NODE_TOTAL:-1}
-      --parallel-index ${CI_NODE_INDEX:-1}

 build_pytest_examples_esp32:
  extends:
    - .build_pytest_template
    - .rules:build:example_test-esp32
-  parallel: 3
-  variables:
-    IDF_TARGET: esp32
-    TEST_DIR: examples
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py examples --target esp32 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_examples_esp32s2:
  extends:
    - .build_pytest_template
    - .rules:build:example_test-esp32s2
-  parallel: 3
-  variables:
-    IDF_TARGET: esp32s2
-    TEST_DIR: examples
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py examples --target esp32s2 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_examples_esp32s3:
  extends:
    - .build_pytest_template
    - .rules:build:example_test-esp32s3
-  parallel: 3
-  variables:
-    IDF_TARGET: esp32s3
-    TEST_DIR: examples
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py examples --target esp32s3 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_examples_esp32c3:
  extends:
    - .build_pytest_template
    - .rules:build:example_test-esp32c3
-  parallel: 3
-  variables:
-    IDF_TARGET: esp32c3
-    TEST_DIR: examples
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py examples --target esp32c3 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_examples_esp32c2:
  extends:
    - .build_pytest_template
    - .rules:build:example_test-esp32c2
-  variables:
-    IDF_TARGET: esp32c2
-    TEST_DIR: examples
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py examples --target esp32c2 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_components_esp32:
  extends:
    - .build_pytest_template
    - .rules:build:component_ut-esp32
-  parallel: 2
-  variables:
-    IDF_TARGET: esp32
-    TEST_DIR: components
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py components --target esp32 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_components_esp32s2:
  extends:
    - .build_pytest_template
    - .rules:build:component_ut-esp32s2
-  variables:
-    IDF_TARGET: esp32s2
-    TEST_DIR: components
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py components --target esp32s2 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_components_esp32s3:
  extends:
    - .build_pytest_template
    - .rules:build:component_ut-esp32s3
-  variables:
-    IDF_TARGET: esp32s3
-    TEST_DIR: components
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py components --target esp32s3 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_components_esp32c3:
  extends:
    - .build_pytest_template
    - .rules:build:component_ut-esp32c3
-  variables:
-    IDF_TARGET: esp32c3
-    TEST_DIR: components
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py components --target esp32c3 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_components_esp32c2:
  extends:
    - .build_pytest_template
    - .rules:build:component_ut-esp32c2
-  variables:
-    IDF_TARGET: esp32c2
-    TEST_DIR: components
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py components --target esp32c2 --size-info $SIZE_INFO_LOCATION -vv

 build_non_test_components_apps:
  extends:
-    - .build_cmake_template
+    - .build_template
+    - .build_test_apps_template
    - .rules:build:component_ut
-  script:
-    - set_component_ut_vars
-    # CI specific options start from "--collect-size-info xxx". could ignore when running locally
-    - run_cmd python tools/ci/ci_build_apps.py $COMPONENT_UT_DIRS -v
-      -t all
-      --collect-size-info $SIZE_INFO_LOCATION
-      --collect-app-info list_job_${CI_NODE_INDEX:-1}.json
-      --parallel-count ${CI_NODE_TOTAL:-1}
-      --parallel-index ${CI_NODE_INDEX:-1}
+  variables:
+    IDF_TARGET: all
+    TEST_PREFIX: component_ut
+    TEST_RELATIVE_DIR: component_ut

 build_pytest_test_apps_esp32:
  extends:
    - .build_pytest_template
    - .rules:build:custom_test-esp32
-  variables:
-    IDF_TARGET: esp32
-    TEST_DIR: tools/test_apps
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py tools/test_apps --target esp32 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_test_apps_esp32s2:
  extends:
    - .build_pytest_template
    - .rules:build:custom_test-esp32s2
-  variables:
-    IDF_TARGET: esp32s2
-    TEST_DIR: tools/test_apps
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py tools/test_apps --target esp32s2 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_test_apps_esp32s3:
  extends:
    - .build_pytest_template
    - .rules:build:custom_test-esp32s3
-  variables:
-    IDF_TARGET: esp32s3
-    TEST_DIR: tools/test_apps
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py tools/test_apps --target esp32s3 --size-info $SIZE_INFO_LOCATION -vv

 build_pytest_test_apps_esp32c3:
  extends:
    - .build_pytest_template
    - .rules:build:custom_test-esp32c3
-  variables:
-    IDF_TARGET: esp32c3
-    TEST_DIR: tools/test_apps
-
-build_pytest_test_apps_esp32c2:
-  extends:
-    - .build_pytest_template
-    - .rules:build:custom_test-esp32c2
-  variables:
-    IDF_TARGET: esp32c2
-    TEST_DIR: tools/test_apps
+  script:
+    - run_cmd python tools/ci/build_pytest_apps.py tools/test_apps --target esp32c3 --size-info $SIZE_INFO_LOCATION -vv

 .build_template_app_template:
-  extends:
-    - .build_template
-    - .before_script_build_jobs
+  extends: .build_template
  variables:
    LOG_PATH: "${CI_PROJECT_DIR}/log_template_app"
    BUILD_PATH: "${CI_PROJECT_DIR}/build_template_app"
@@ -208,6 +168,8 @@ build_pytest_test_apps_esp32c2:
    # using on esp-idf. If it doesn't exist then just stick to the default branch
    - python $CHECKOUT_REF_SCRIPT esp-idf-template esp-idf-template
    - export PATH="$IDF_PATH/tools:$PATH"
+    - export EXTRA_CFLAGS=${PEDANTIC_CFLAGS}
+    - export EXTRA_CXXFLAGS=${PEDANTIC_CXXFLAGS}
    # Only do the default cmake build for each target, remaining part are done in the build_template_app job
    - tools/ci/build_template_app.sh ${BUILD_COMMAND_ARGS}

@@ -248,12 +210,6 @@ build_ssc_esp32s2:
  variables:
    TARGET_NAME: "ESP32S2"

-build_ssc_esp32c2:
-  extends: .build_ssc_template
-  parallel: 2
-  variables:
-    TARGET_NAME: "ESP32C2"
-
 build_ssc_esp32c3:
  extends: .build_ssc_template
  parallel: 3
@@ -267,45 +223,34 @@ build_ssc_esp32s3:
    TARGET_NAME: "ESP32S3"

 .build_esp_idf_tests_cmake_template:
-  extends:
-    - .build_template
-    - .before_script_build_jobs
+  extends: .build_template
  dependencies:  # set dependencies to null to avoid missing artifacts issue
  needs:
    - job: fast_template_app
      artifacts: false
+    - scan_tests
  artifacts:
    paths:
-      - "**/build*/size.json"
-      - "**/build*/build_log.txt"
-      - "**/build*/*.bin"
-      - "**/build*/*.elf"
-      - "**/build*/*.map"
-      - "**/build*/flasher_args.json"
-      - "**/build*/flash_project_args"
-      - "**/build*/config/sdkconfig.json"
-      - "**/build*/sdkconfig"
-      - "**/build*/bootloader/*.bin"
-      - "**/build*/partition_table/*.bin"
-      - list_job_*.json
-      - $SIZE_INFO_LOCATION
+      - tools/unit-test-app/output/${IDF_TARGET}
+      - tools/unit-test-app/builds/*.json
+      - tools/unit-test-app/builds/${IDF_TARGET}/*/size.json
      - components/idf_test/unit_test/*.yml
+      - $LOG_PATH
+      - $SIZE_INFO_LOCATION
    when: always
    expire_in: 4 days
  variables:
+    LOG_PATH: "$CI_PROJECT_DIR/log_ut_cmake"
+    BUILD_PATH: ${CI_PROJECT_DIR}/tools/unit-test-app/builds
+    OUTPUT_PATH: ${CI_PROJECT_DIR}/tools/unit-test-app/output
+    BUILD_SYSTEM: "cmake"
+    TEST_TYPE: "unit_test"
+    PYTHON_VER: 3.7.10
    LDGEN_CHECK_MAPPING: 1
  script:
-    # CI specific options start from "--collect-size-info xxx". could ignore when running locally
-    - run_cmd python tools/ci/ci_build_apps.py tools/unit-test-app -v
-      -t $IDF_TARGET
-      --config "configs/*="
-      --copy-sdkconfig
-      --preserve-all
-      --collect-size-info $SIZE_INFO_LOCATION
-      --collect-app-info list_job_${CI_NODE_INDEX:-1}.json
-      --parallel-count ${CI_NODE_TOTAL:-1}
-      --parallel-index ${CI_NODE_INDEX:-1}
-    - run_cmd python tools/unit-test-app/tools/UnitTestParser.py tools/unit-test-app ${CI_NODE_INDEX:-1}
+    - ${IDF_PATH}/tools/ci/find_apps_build_apps.sh
+    - cd $CI_PROJECT_DIR/tools/unit-test-app
+    - python tools/UnitTestParser.py ${BUILD_PATH} ${CI_NODE_INDEX:-1}

 build_esp_idf_tests_cmake_esp32:
  extends:
@@ -330,13 +275,6 @@ build_esp_idf_tests_cmake_esp32s3:
  variables:
    IDF_TARGET: esp32s3

-build_esp_idf_tests_cmake_esp32c2:
-  extends:
-    - .build_esp_idf_tests_cmake_template
-    - .rules:build:unit_test-esp32c2
-  variables:
-    IDF_TARGET: esp32c2
-
 build_esp_idf_tests_cmake_esp32c3:
  extends:
    - .build_esp_idf_tests_cmake_template
@@ -344,143 +282,141 @@ build_esp_idf_tests_cmake_esp32c3:
  variables:
    IDF_TARGET: esp32c3

-.build_cmake_template:
+build_esp_idf_tests_cmake_esp32c2:
  extends:
-    - .build_template
-    - .before_script_build_jobs
-  dependencies: # set dependencies to null to avoid missing artifacts issue
+    - .build_esp_idf_tests_cmake_template
+    - .rules:build:unit_test-esp32c2
+  variables:
+    IDF_TARGET: esp32c2
+
+.build_examples_template:
+  extends: .build_template
+  dependencies:  # set dependencies to null to avoid missing artifacts issue
  needs:
    - job: fast_template_app
      artifacts: false
+    - scan_tests
  variables:
+    TEST_PREFIX: examples
+    TEST_RELATIVE_DIR: examples
+    SCAN_TEST_JSON: ${CI_PROJECT_DIR}/${TEST_RELATIVE_DIR}/test_configs/scan_${IDF_TARGET}_${BUILD_SYSTEM}.json
+    TEST_TYPE: example_test
+    LOG_PATH: ${CI_PROJECT_DIR}/log_${TEST_PREFIX}
+    BUILD_PATH: ${CI_PROJECT_DIR}/build_${TEST_PREFIX}
+    PYTHON_VER: 3.7.10
    LDGEN_CHECK_MAPPING: 1
+  script:
+    # it's not possible to build 100% out-of-tree and have the "artifacts"
+    # mechanism work, but this is the next best thing
+    - ${IDF_PATH}/tools/ci/find_apps_build_apps.sh
+
+.build_examples_cmake_template:
+  extends: .build_examples_template
  artifacts:
    paths:
-      - "**/build*/size.json"
-      - "**/build*/build_log.txt"
-      - "**/build*/*.bin"
-      - "**/build*/*.elf"
-      - "**/build*/*.map"
-      - "**/build*/flasher_args.json"
-      - "**/build*/flash_project_args"
-      - "**/build*/config/sdkconfig.json"
-      - "**/build*/sdkconfig"
-      - "**/build*/bootloader/*.bin"
-      - "**/build*/partition_table/*.bin"
-      - list_job_*.json
+      - build_${TEST_PREFIX}/list.json
+      - build_${TEST_PREFIX}/list_job_*.json
+      - build_${TEST_PREFIX}/*/*/*/sdkconfig
+      - build_${TEST_PREFIX}/*/*/*/build/size.json
+      - build_${TEST_PREFIX}/*/*/*/build/*.bin
+      - build_${TEST_PREFIX}/*/*/*/build/*.elf
+      - build_${TEST_PREFIX}/*/*/*/build/*.map
+      - build_${TEST_PREFIX}/*/*/*/build/flasher_args.json
+      - build_${TEST_PREFIX}/*/*/*/build/bootloader/*.bin
+      - build_${TEST_PREFIX}/*/*/*/build/partition_table/*.bin
+      - $LOG_PATH
      - $SIZE_INFO_LOCATION
    when: always
    expire_in: 4 days
-  script:
-    # CI specific options start from "--collect-size-info xxx". could ignore when running locally
-    - run_cmd python tools/ci/ci_build_apps.py $TEST_DIR -v
-      -t $IDF_TARGET
-      --copy-sdkconfig
-      --collect-size-info $SIZE_INFO_LOCATION
-      --collect-app-info list_job_${CI_NODE_INDEX:-1}.json
-      --parallel-count ${CI_NODE_TOTAL:-1}
-      --parallel-index ${CI_NODE_INDEX:-1}
-      --extra-preserve-dirs
-        examples/bluetooth/esp_ble_mesh/ble_mesh_console
-        examples/bluetooth/hci/controller_hci_uart_esp32
-        examples/wifi/iperf
+  variables:
+    BUILD_SYSTEM: cmake

 build_examples_cmake_esp32:
  extends:
-    - .build_cmake_template
+    - .build_examples_cmake_template
    - .rules:build:example_test-esp32
  parallel: 12
  variables:
    IDF_TARGET: esp32
-    TEST_DIR: examples

 build_examples_cmake_esp32s2:
  extends:
-    - .build_cmake_template
+    - .build_examples_cmake_template
    - .rules:build:example_test-esp32s2
  parallel: 8
  variables:
    IDF_TARGET: esp32s2
-    TEST_DIR: examples

 build_examples_cmake_esp32s3:
  extends:
-    - .build_cmake_template
+    - .build_examples_cmake_template
    - .rules:build:example_test-esp32s3
  parallel: 8
  variables:
    IDF_TARGET: esp32s3
-    TEST_DIR: examples
-
-build_examples_cmake_esp32c2:
-  extends:
-    - .build_cmake_template
-    - .rules:build:example_test-esp32c2
-  parallel: 8
-  variables:
-    IDF_TARGET: esp32c2
-    TEST_DIR: examples

 build_examples_cmake_esp32c3:
  extends:
-    - .build_cmake_template
+    - .build_examples_cmake_template
    - .rules:build:example_test-esp32c3
  parallel: 8
  variables:
    IDF_TARGET: esp32c3
-    TEST_DIR: examples

 build_examples_cmake_esp32h2:
  extends:
-    - .build_cmake_template
+    - .build_examples_cmake_template
    - .rules:build:example_test-esp32h2
  variables:
    IDF_TARGET: esp32h2
-    TEST_DIR: examples
+
+.build_test_apps_template:
+  extends: .build_examples_cmake_template
+  variables:
+    TEST_PREFIX: test_apps
+    TEST_RELATIVE_DIR: tools/test_apps
+    TEST_TYPE: custom_test
+  script:
+    - ${IDF_PATH}/tools/ci/find_apps_build_apps.sh

 build_test_apps_esp32:
  extends:
-    - .build_cmake_template
+    - .build_test_apps_template
    - .rules:build:custom_test-esp32
  parallel: 2
  variables:
    IDF_TARGET: esp32
-    TEST_DIR: tools/test_apps

 build_test_apps_esp32s2:
  extends:
-    - .build_cmake_template
+    - .build_test_apps_template
    - .rules:build:custom_test-esp32s2
  parallel: 2
  variables:
    IDF_TARGET: esp32s2
-    TEST_DIR: tools/test_apps

 build_test_apps_esp32s3:
  extends:
-    - .build_cmake_template
+    - .build_test_apps_template
    - .rules:build:custom_test-esp32s3
  parallel: 2
  variables:
    IDF_TARGET: esp32s3
-    TEST_DIR: tools/test_apps

 build_test_apps_esp32c3:
  extends:
-    - .build_cmake_template
+    - .build_test_apps_template
    - .rules:build:custom_test-esp32c3
  parallel: 2
  variables:
    IDF_TARGET: esp32c3
-    TEST_DIR: tools/test_apps

 build_test_apps_esp32c2:
  extends:
-    - .build_cmake_template
+    - .build_test_apps_template
    - .rules:build:custom_test-esp32c2
  variables:
    IDF_TARGET: esp32c2
-    TEST_DIR: tools/test_apps

 .test_build_system_template:
  stage: host_test
--- a/.gitlab/ci/dependencies/dependencies.yml
+++ b/.gitlab/ci/dependencies/dependencies.yml
@@ -156,64 +156,6 @@ build:integration_test:
    - "build:component_ut-{0}"
    - build:target_test

-# To reduce the specific runners' usage.
-# Do not create these jobs by default patterns on development branches
-# Can be triggered by labels or related changes
-"test:{0}-{1}-{2}":
-  matrix:
-    - *target_test
-    - *all_targets
-    - - bt        # example_test_005
-      - wifi      # example_test_002, example_test*wifi*
-      - ethernet  # example_test*ethernet*
-      - sdio      # UT_044, UT_045
-  patterns:
-    - "{0}-{1}-{2}"
-    - "{0}-{2}"
-    - "target_test-{2}"
-  labels:
-    - "{0}_{1}"
-    - "{0}"
-    - target_test
-  included_in:
-    - "build:{0}-{1}"
-    - "build:{0}"
-    - build:target_test
-
-# For example_test*flash_encryption_wifi_high_traffic jobs
-# set `INCLUDE_NIGHTLY_RUN` variable when triggered on development branches
-"test:example_test-{0}-include_nightly_run-rule":
-  matrix:
-    - - esp32
-      - esp32c3
-  specific_rules:
-    - "if-example_test-ota-include_nightly_run-rule"
-  included_in:
-    - "build:example_test-{0}"
-    - "build:example_test"
-    - build:target_test
-
-# due to the lack of runners, c2 tests will only be triggered by label
-"test:{0}-esp32c2":
-  matrix:
-    - *target_test
-  labels:
-    - "{0}_esp32c2"
-  included_in:
-    - "build:{0}"
-    - "build:{0}-esp32c2"
-    - build:target_test
-
-# overwrite the esp32c2 component ut again
-"test:component_ut-esp32c2": # component_ut will trigger by unit_test as well, since now we have 2 kinds of UT
-  labels:
-    - "component_ut_esp32c2"
-    - "unit_test_esp32c2"
-  included_in:
-    - build:component_ut
-    - "build:component_ut-esp32c2"
-    - build:target_test
-
 # due to the lack of runners, s3 tests will only be triggered by label
 "test:{0}-esp32s3":
  matrix:
--- a/.gitlab/ci/dependencies/generate_rules.py
+++ b/.gitlab/ci/dependencies/generate_rules.py
@@ -82,7 +82,6 @@ class RulesWriter:
    RULE_LABEL_TEMPLATE = '    - <<: *if-label-{0}'
    RULE_PATTERN_TEMPLATE = '    - <<: *if-dev-push\n' \
                            '      changes: *patterns-{0}'
-    SPECIFIC_RULE_TEMPLATE = '    - <<: *{0}'
    RULES_TEMPLATE = inspect.cleandoc(r"""
    .rules:{0}:
      rules:
@@ -157,8 +156,6 @@ class RulesWriter:
                continue
            if 'included_in' in v:
                for item in _list(v['included_in']):
-                    if 'specific_rules' in v:
-                        res[item]['specific_rules'].update(_list(v['specific_rules']))
                    if 'labels' in v:
                        res[item]['labels'].update(_list(v['labels']))
                    if 'patterns' in v:
@@ -213,12 +210,6 @@ class RulesWriter:
                _rules.append(self.RULE_PROTECTED)
            if name.startswith('test:'):
                _rules.append(self.RULE_BUILD_ONLY)
-
-            for specific_rule in cfg['specific_rules']:
-                if f'.{specific_rule}' in self.rules_cfg:
-                    _rules.append(self.SPECIFIC_RULE_TEMPLATE.format(specific_rule))
-                else:
-                    print('WARNING: specific_rule {} not exists'.format(specific_rule))
            for label in cfg['labels']:
                _rules.append(self.RULE_LABEL_TEMPLATE.format(label))
            for pattern in cfg['patterns']:
--- a/.gitlab/ci/deploy.yml
+++ b/.gitlab/ci/deploy.yml
@@ -45,7 +45,7 @@ deploy_test_result:
  extends:
    - .deploy_job_template
    - .before_script_minimal
-    - .rules:ref:master-always
+    - .rules:ref:master-schedule-always
  image: $CI_DOCKER_REGISTRY/bot-env:1
  dependencies: []
  tags:
--- a/.gitlab/ci/docs.yml
+++ b/.gitlab/ci/docs.yml
@@ -57,12 +57,14 @@ check_docs_lang_sync:

 .build_docs_template:
  image: $ESP_IDF_DOC_ENV_IMAGE
+  variables:
+    PYTHON_VER: 3.7.10
  tags:
    - build_docs
  dependencies: []
  script:
    - cd docs
-    - pip install -U -r $IDF_PATH/tools/requirements/requirements.docs.txt
+    - pip install -U -r requirements.txt
    - build-docs -t $DOCTGT -bs $DOC_BUILDERS -l $DOCLANG build
  parallel:
    matrix:
@@ -71,12 +73,14 @@ check_docs_lang_sync:

 check_docs_gh_links:
  image: $ESP_IDF_DOC_ENV_IMAGE
+  variables:
+    PYTHON_VER: 3.7.10
  extends:
    - .pre_check_job_template
    - .doc-rules:build:docs
  script:
    - cd docs
-    - pip install -U -r $IDF_PATH/tools/requirements/requirements.docs.txt
+    - pip install -U -r requirements.txt
    - build-docs gh-linkcheck

 # stage: build_doc
@@ -152,6 +156,7 @@ build_docs_pdf:
    - .before_script_no_sync_submodule
  image: $ESP_IDF_DOC_ENV_IMAGE
  variables:
+    PYTHON_VER: 3.7.10
    DOCS_BUILD_DIR: "${IDF_PATH}/docs/_build/"
    PYTHONUNBUFFERED: 1
  stage: test_deploy
@@ -161,7 +166,7 @@ build_docs_pdf:
  script:
    - add_doc_server_ssh_keys $DOCS_DEPLOY_PRIVATEKEY $DOCS_DEPLOY_SERVER $DOCS_DEPLOY_SERVER_USER
    - export GIT_VER=$(git describe --always ${PIPELINE_COMMIT_SHA} --)
-    - pip install -U -r $IDF_PATH/tools/requirements/requirements.docs.txt
+    - pip install -U -r docs/requirements.txt
    - deploy-docs

 # stage: test_deploy
@@ -227,5 +232,5 @@ check_doc_links:
  allow_failure: true
  script:
    - cd docs
-    - pip install -U -r $IDF_PATH/tools/requirements/requirements.docs.txt
+    - pip install -U -r requirements.txt
    - build-docs -t $DOCTGT -l $DOCLANG linkcheck
--- a/.gitlab/ci/host-test.yml
+++ b/.gitlab/ci/host-test.yml
@@ -2,17 +2,12 @@
  extends: .rules:test:host_test
  stage: host_test
  image: $ESP_ENV_IMAGE
+  variables:
+    PYTHON_VER: 3.7.10
  tags:
    - host_test
  dependencies: []
-  # run host_test jobs immediately, only after upload cache
-  needs:
-    - job: upload-pip-cache
-      optional: true
-      artifacts: false
-    - job: upload-submodules-cache
-      optional: true
-      artifacts: false
+  needs: []  # run host_test jobs immediately

 test_nvs_on_host:
  extends: .host_test_template
@@ -104,6 +99,17 @@ test_reproducible_build:
    # check no crashes found
    - test -z "$(ls out/crashes/)" || exit 1

+test_mdns_fuzzer_on_host:
+  extends: .host_fuzzer_test_template
+  variables:
+    FUZZER_TEST_DIR: components/mdns/test_afl_fuzz_host
+
+test_mdns_no_serv_fuzzer_on_host:
+  extends: .host_fuzzer_test_template
+  variables:
+    FUZZER_TEST_DIR: components/mdns/test_afl_fuzz_host
+    FUZZER_PARAMS: MDNS_NO_SERVICES=on
+
 test_lwip_dns_fuzzer_on_host:
  extends: .host_fuzzer_test_template
  variables:
@@ -352,13 +358,6 @@ test_nvs_page:
    - idf.py build
    - build/test_nvs_page_host.elf

-test_mqtt_on_host:
-  extends: .host_test_template
-  script:
-    - cd ${IDF_PATH}/components/mqtt/host_test
-    - idf.py build
-    - LSAN_OPTIONS=verbosity=1:log_threads=1 build/host_mqtt_client_test.elf
-
 test_log:
  extends: .host_test_template
  script:
@@ -439,11 +438,8 @@ test_gen_soc_caps_kconfig:
 test_pytest_qemu:
  extends:
    - .host_test_template
-    - .before_script_build_jobs
+    - .before_script_pytest
  image: $QEMU_IMAGE
  script:
-    - run_cmd python tools/ci/ci_build_apps.py . -vv
-      --target esp32
-      --pytest-apps
-      -m qemu
+    - run_cmd python tools/ci/build_pytest_apps.py . --target esp32 -m qemu -vv
    - pytest --target esp32 -m qemu --embedded-services idf,qemu
--- a/.gitlab/ci/pre_check.yml
+++ b/.gitlab/ci/pre_check.yml
@@ -51,6 +51,8 @@ check_python_style:
  extends:
    - .pre_check_base_template
    - .rules:patterns:python-files
+  variables:
+    PYTHON_VER: 3.7.10
  artifacts:
    when: on_failure
    paths:
@@ -61,6 +63,8 @@ check_python_style:

 test_check_kconfigs:
  extends: .pre_check_job_template
+  variables:
+    PYTHON_VER: 3.7.10
  artifacts:
    when: on_failure
    paths:
@@ -97,6 +101,8 @@ check_fuzzer_compilation:
     - make MODE=dhcp_server
     - make MODE=dhcp_client
     - make MODE=dns
+     - cd ${IDF_PATH}/components/mdns/test_afl_fuzz_host
+     - make

 check_public_headers:
  extends:
@@ -105,11 +111,7 @@ check_public_headers:
  tags:
    - build
  script:
-    - IDF_TARGET=esp32 python tools/ci/check_public_headers.py --jobs 4 --prefix xtensa-esp32-elf-
-    - IDF_TARGET=esp32s2 python tools/ci/check_public_headers.py --jobs 4 --prefix xtensa-esp32s2-elf-
-    - IDF_TARGET=esp32s3 python tools/ci/check_public_headers.py --jobs 4 --prefix xtensa-esp32s3-elf-
-    - IDF_TARGET=esp32c3 python tools/ci/check_public_headers.py --jobs 4 --prefix riscv32-esp-elf-
-    - IDF_TARGET=esp32c2 python tools/ci/check_public_headers.py --jobs 4 --prefix riscv32-esp-elf-
+    - python tools/ci/check_public_headers.py --jobs 4 --prefix xtensa-esp32-elf-

 check_chip_support_components:
  extends:
@@ -144,14 +146,44 @@ check_esp_err_to_name:
    - ./gen_esp_err_to_name.py
    - git diff --exit-code -- ../components/esp_common/src/esp_err_to_name.c || { echo 'Differences found. Please run gen_esp_err_to_name.py and commit the changes.'; exit 1; }

-check_esp_system:
+scan_tests:
  extends:
    - .pre_check_base_template
-    - .rules:build
+    - .before_script_pytest
+    - .rules:build:target_test
+  image: $TARGET_TEST_ENV_IMAGE
  tags:
-    - build
+    - scan_test  # since this job is used for uploading the cache, the runner tags should be unique
+  artifacts:
+    paths:
+      - $EXAMPLE_TEST_OUTPUT_DIR
+      - $TEST_APPS_OUTPUT_DIR
+      - $COMPONENT_UT_OUTPUT_DIR
+    expire_in: 1 week
+  cache:
+    key: pip-cache
+    paths:
+      - .cache/pip
+    # pull only for most of the use cases since it's cache dir.
+    # Only set "pull-push" policy for "scan_test" since it would **possibly** install all pypi packages
+    policy: pull-push
+  variables:
+    EXAMPLE_TEST_DIR: ${CI_PROJECT_DIR}/examples
+    EXAMPLE_TEST_OUTPUT_DIR: ${CI_PROJECT_DIR}/examples/test_configs
+    TEST_APPS_TEST_DIR: ${CI_PROJECT_DIR}/tools/test_apps
+    TEST_APPS_OUTPUT_DIR: ${CI_PROJECT_DIR}/tools/test_apps/test_configs
+    COMPONENT_UT_OUTPUT_DIR: ${CI_PROJECT_DIR}/component_ut/test_configs
+    CI_SCAN_TESTS_PY: ${CI_PROJECT_DIR}/tools/ci/python_packages/ttfw_idf/CIScanTests.py
+    EXTRA_TEST_DIRS: >-
+      examples/bluetooth/esp_ble_mesh/ble_mesh_console
+      examples/bluetooth/hci/controller_hci_uart_esp32
+      examples/wifi/iperf
+    EXTRA_EVALUATE_ARGS: '--evaluate-parallel-count --config "sdkconfig.ci=default" --config "sdkconfig.ci.*=" --config "=default"'
  script:
-    - python components/esp_system/check_system_init_priorities.py
+    - run_cmd python $CI_SCAN_TESTS_PY example_test $EXAMPLE_TEST_DIR -b cmake --exclude examples/build_system/idf_as_lib -c $CI_TARGET_TEST_CONFIG_FILE -o $EXAMPLE_TEST_OUTPUT_DIR --extra_test_dirs $EXTRA_TEST_DIRS $EXTRA_EVALUATE_ARGS
+    - run_cmd python $CI_SCAN_TESTS_PY test_apps $TEST_APPS_TEST_DIR -c $CI_TARGET_TEST_CONFIG_FILE -o $TEST_APPS_OUTPUT_DIR $EXTRA_EVALUATE_ARGS
+    - set_component_ut_vars
+    - run_cmd python $CI_SCAN_TESTS_PY component_ut $COMPONENT_UT_DIRS --exclude $COMPONENT_UT_EXCLUDES -c $CI_TARGET_TEST_CONFIG_FILE -o $COMPONENT_UT_OUTPUT_DIR --combine-all-targets --except-targets linux $EXTRA_EVALUATE_ARGS

 # For release tag pipelines only, make sure the tag was created with 'git tag -a' so it will update
 # the version returned by 'git describe'
@@ -175,10 +207,3 @@ check_commit_msg:
    - git log -n10 --oneline ${PIPELINE_COMMIT_SHA}
    # commit start with "WIP: " need to be squashed before merge
    - 'git log --pretty=%s origin/master..${PIPELINE_COMMIT_SHA} -- | grep -i "^WIP:" && exit 1 || exit 0'
-
-check_test_scripts_build_test_rules:
-  extends:
-    - .pre_check_job_template
-    - .before_script_build_jobs
-  script:
-    - python tools/ci/check_build_test_rules.py check-test-scripts examples/ tools/test_apps components
--- a/.gitlab/ci/rules.yml
+++ b/.gitlab/ci/rules.yml
--- a/.gitlab/ci/static-code-analysis.yml
+++ b/.gitlab/ci/static-code-analysis.yml
@@ -10,6 +10,7 @@ clang_tidy_check:
    when: always
    expire_in: 1 day
  variables:
+    CLANG_TIDY_RUNNER_PROJ: 2107  # idf/clang-tidy-runner
    CLANG_TIDY_DIRS_TXT: ${CI_PROJECT_DIR}/tools/ci/clang_tidy_dirs.txt
    RULES_FILE: ${CI_PROJECT_DIR}/tools/ci/static-analysis-rules.yml
    OUTPUT_DIR: ${CI_PROJECT_DIR}/clang_tidy_reports
--- a/.gitlab/ci/target-test.yml
+++ b/.gitlab/ci/target-test.yml
@@ -1,21 +1,9 @@
-.target_test_template:
+.pytest_template:
  image: $TARGET_TEST_ENV_IMAGE
  stage: target_test
  timeout: 1 hour
-  variables:
-    GIT_DEPTH: 1
-    SUBMODULES_TO_FETCH: "none"
-  cache:
-    # Usually do not need submodule-cache in target_test
-    - key: pip-cache
-      paths:
-        - .cache/pip
-      policy: pull
-
-.pytest_template:
-  extends:
-    - .target_test_template
-    - .before_script_pytest
+  extends: .before_script_pytest
+  tags: [$TARGET, $ENV_MARKER]
  artifacts:
    when: always
    paths:
@@ -26,14 +14,7 @@
    expire_in: 1 week
  script:
    - retry_failed git clone $KNOWN_FAILURE_CASES_REPO known_failure_cases
-    # get runner env config file
-    - retry_failed git clone $TEST_ENV_CONFIG_REPO
-    - python $CHECKOUT_REF_SCRIPT ci-test-runner-configs ci-test-runner-configs
-    # using runner tags as markers to filter the test cases
-    # Runner tags are comma separated, replace the comma with " and " for markers
-    - job_tags=$(python tools/ci/python_packages/gitlab_api.py get_job_tags $CI_PROJECT_ID --job_id $CI_JOB_ID)
-    - markers=$(echo $job_tags | sed -e "s/,/ and /g")
-    - run_cmd pytest $TEST_DIR -m \"${markers}\" --junitxml=XUNIT_RESULT.xml --known-failure-cases-file known_failure_cases/known_failure_cases.txt
+    - pytest $TEST_DIR --target $TARGET -m $ENV_MARKER --junitxml=XUNIT_RESULT.xml --known-failure-cases-file known_failure_cases/known_failure_cases.txt

 .pytest_examples_dir_template:
  extends: .pytest_template
@@ -46,7 +27,9 @@ example_test_pytest_esp32_generic:
    - .rules:test:example_test-esp32
  needs:
    - build_pytest_examples_esp32
-  tags: [ esp32, generic ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: generic

 example_test_pytest_esp32_ir_transceiver:
  extends:
@@ -54,7 +37,9 @@ example_test_pytest_esp32_ir_transceiver:
    - .rules:test:example_test-esp32
  needs:
    - build_pytest_examples_esp32
-  tags: [ esp32, ir_transceiver ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: ir_transceiver

 example_test_pytest_esp32s2_generic:
  extends:
@@ -62,7 +47,9 @@ example_test_pytest_esp32s2_generic:
    - .rules:test:example_test-esp32s2
  needs:
    - build_pytest_examples_esp32s2
-  tags: [ esp32s2, generic ]
+  variables:
+    TARGET: ESP32S2
+    ENV_MARKER: generic

 example_test_pytest_esp32s3_generic:
  extends:
@@ -70,23 +57,9 @@ example_test_pytest_esp32s3_generic:
    - .rules:test:example_test-esp32s3
  needs:
    - build_pytest_examples_esp32s3
-  tags: [ esp32s3, generic ]
-
-example_test_pytest_esp32c2_generic:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32c2
-  needs:
-    - build_pytest_examples_esp32c2
-  tags: [ esp32c2, generic, xtal_40mhz ]
-
-example_test_pytest_esp32c2_26mhz:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32c2
-  needs:
-    - build_pytest_examples_esp32c2
-  tags: [ esp32c2, xtal_26mhz ]
+  variables:
+    TARGET: ESP32S3
+    ENV_MARKER: generic

 example_test_pytest_esp32c3_generic:
  extends:
@@ -94,7 +67,9 @@ example_test_pytest_esp32c3_generic:
    - .rules:test:example_test-esp32c3
  needs:
    - build_pytest_examples_esp32c3
-  tags: [ esp32c3, generic ]
+  variables:
+    TARGET: ESP32C3
+    ENV_MARKER: generic

 example_test_pytest_esp32c3_flash_suspend:
  extends:
@@ -102,79 +77,29 @@ example_test_pytest_esp32c3_flash_suspend:
    - .rules:test:example_test-esp32c3
  needs:
    - build_pytest_examples_esp32c3
-  tags: [ esp32c3, flash_suspend ]
+  variables:
+    TARGET: ESP32C3
+    ENV_MARKER: flash_suspend

 example_test_pytest_esp32_ethernet_ota:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32-ethernet
-  needs:
-    - build_pytest_examples_esp32
-  tags: [ esp32, ethernet_ota ]
-
-example_test_pytest_esp32_wifi_high_traffic:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32-wifi
-  needs:
-    - build_pytest_examples_esp32
-  tags: [ esp32, wifi_high_traffic ]
-
-example_test_pytest_esp32_flash_encryption_wifi_high_traffic:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32-include_nightly_run-rule
-  needs:
-    - build_pytest_examples_esp32
-  tags: [ esp32, flash_encryption_wifi_high_traffic ]
-
-example_test_pytest_esp32c3_flash_encryption_wifi_high_traffic:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32c3-include_nightly_run-rule
-  needs:
-    - build_pytest_examples_esp32c3
-  tags: [ esp32c3, flash_encryption_wifi_high_traffic ]
-
-example_test_pytest_esp32_ethernet:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32-ethernet
-  needs:
-    - build_pytest_examples_esp32
-  tags: [ esp32, ethernet]
-
-example_test_pytest_esp32_8mb_flash:
  extends:
    - .pytest_examples_dir_template
    - .rules:test:example_test-esp32
  needs:
    - build_pytest_examples_esp32
-  tags: [ esp32, ethernet_flash_8m ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: ethernet_ota

-example_test_pytest_esp32_wifi_ap:
+example_test_pytest_esp32_wifi:
  extends:
    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32-wifi
+    - .rules:test:example_test-esp32
  needs:
    - build_pytest_examples_esp32
-  tags: [ esp32, wifi_ap ]
-
-example_test_pytest_esp32_wifi_router:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32-wifi
-  needs:
-    - build_pytest_examples_esp32
-  tags: [ esp32, wifi_router ]
-
-example_test_pytest_esp32_wifi_wlan:
-  extends:
-    - .pytest_examples_dir_template
-    - .rules:test:example_test-esp32-wifi
-  needs:
-    - build_pytest_examples_esp32
-  tags: [ esp32, wifi_wlan ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: wifi

 example_test_pytest_esp32_ethernet_ip101:
  extends:
@@ -182,7 +107,9 @@ example_test_pytest_esp32_ethernet_ip101:
    - .rules:test:example_test-esp32
  needs:
    - build_pytest_examples_esp32
-  tags: [ esp32, ip101 ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: ip101

 example_test_pytest_esp32_flash_encryption:
  extends:
@@ -190,7 +117,9 @@ example_test_pytest_esp32_flash_encryption:
    - .rules:test:example_test-esp32
  needs:
    - build_pytest_examples_esp32
-  tags: [ esp32, flash_encryption ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: flash_encryption

 example_test_pytest_esp32_multi_dut_generic:
  extends:
@@ -198,7 +127,9 @@ example_test_pytest_esp32_multi_dut_generic:
    - .rules:test:example_test-esp32
  needs:
    - build_pytest_examples_esp32
-  tags: [ esp32, multi_dut_generic ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: multi_dut_generic

 example_test_pytest_esp32c3_flash_encryption:
  extends:
@@ -206,7 +137,9 @@ example_test_pytest_esp32c3_flash_encryption:
    - .rules:test:example_test-esp32c3
  needs:
    - build_pytest_examples_esp32c3
-  tags: [ esp32c3, flash_encryption ]
+  variables:
+    TARGET: ESP32C3
+    ENV_MARKER: flash_encryption

 .pytest_components_dir_template:
  extends: .pytest_template
@@ -219,7 +152,9 @@ component_ut_pytest_esp32_generic:
    - .rules:test:component_ut-esp32
  needs:
    - build_pytest_components_esp32
-  tags: [ esp32, generic ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: generic

 component_ut_pytest_esp32_ip101:
  extends:
@@ -227,7 +162,9 @@ component_ut_pytest_esp32_ip101:
    - .rules:test:component_ut-esp32
  needs:
    - build_pytest_components_esp32
-  tags: [ esp32, ip101 ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: ip101

 component_ut_pytest_esp32_lan8720:
  extends:
@@ -235,15 +172,9 @@ component_ut_pytest_esp32_lan8720:
    - .rules:labels-protected:lan8720  # FIXME: IDFCI-1176
  needs:
    - build_pytest_components_esp32
-  tags: [ esp32, lan8720 ]
-
-component_ut_pytest_esp32_flash_encryption:
-  extends:
-    - .pytest_components_dir_template
-    - .rules:test:component_ut-esp32
-  needs:
-    - build_pytest_components_esp32
-  tags: [ esp32, flash_encryption ]
+  variables:
+    TARGET: ESP32
+    ENV_MARKER: lan8720

 component_ut_pytest_esp32s2_generic:
  extends:
@@ -251,7 +182,9 @@ component_ut_pytest_esp32s2_generic:
    - .rules:test:component_ut-esp32s2
  needs:
    - build_pytest_components_esp32s2
-  tags: [ esp32s2, generic ]
+  variables:
+    TARGET: ESP32S2
+    ENV_MARKER: generic

 component_ut_pytest_esp32s3_generic:
  extends:
@@ -259,7 +192,9 @@ component_ut_pytest_esp32s3_generic:
    - .rules:test:component_ut-esp32s3
  needs:
    - build_pytest_components_esp32s3
-  tags: [ esp32s3, generic ]
+  variables:
+    TARGET: ESP32S3
+    ENV_MARKER: generic

 component_ut_pytest_esp32s3_octal_psram:
  extends:
@@ -267,39 +202,9 @@ component_ut_pytest_esp32s3_octal_psram:
    - .rules:test:component_ut-esp32s3
  needs:
    - build_pytest_components_esp32s3
-  tags: [ esp32s3, octal_psram ]
-
-component_ut_pytest_esp32s3_flash_encryption_f4r8:
-  extends:
-    - .pytest_components_dir_template
-    - .rules:test:component_ut-esp32s3
-  needs:
-    - build_pytest_components_esp32s3
-  tags: [ esp32s3, flash_encryption_f4r8 ]
-
-component_ut_pytest_esp32s3_flash_encryption_f8r8:
-  extends:
-    - .pytest_components_dir_template
-    - .rules:test:component_ut-esp32s3
-  needs:
-    - build_pytest_components_esp32s3
-  tags: [ esp32s3, flash_encryption_f8r8 ]
-
-component_ut_pytest_esp32c2_generic:
-  extends:
-    - .pytest_components_dir_template
-    - .rules:test:component_ut-esp32c2
-  needs:
-    - build_pytest_components_esp32c2
-  tags: [ esp32c2, generic, xtal_40mhz ]
-
-component_ut_pytest_esp32c2_xtal_26mhz:
-  extends:
-    - .pytest_components_dir_template
-    - .rules:test:component_ut-esp32c2
-  needs:
-    - build_pytest_components_esp32c2
-  tags: [ esp32c2, xtal_26mhz ]
+  variables:
+    TARGET: ESP32S3
+    ENV_MARKER: octal_psram

 component_ut_pytest_esp32c3_generic:
  extends:
@@ -307,15 +212,9 @@ component_ut_pytest_esp32c3_generic:
    - .rules:test:component_ut-esp32c3
  needs:
    - build_pytest_components_esp32c3
-  tags: [ esp32c3, generic ]
-
-component_ut_pytest_esp32c3_flash_encryption:
-  extends:
-    - .pytest_components_dir_template
-    - .rules:test:component_ut-esp32c3
-  needs:
-    - build_pytest_components_esp32c3
-  tags: [ esp32c3, flash_encryption ]
+  variables:
+    TARGET: ESP32C3
+    ENV_MARKER: generic

 .pytest_test_apps_dir_template:
  extends: .pytest_template
@@ -328,8 +227,9 @@ test_app_test_pytest_esp32_generic:
    - .rules:test:custom_test-esp32
  needs:
    - build_pytest_test_apps_esp32
-  tags: [ esp32, generic ]
  variables:
+    TARGET: ESP32
+    ENV_MARKER: generic
    SETUP_TOOLS: "1"  # need gdb

 test_app_test_pytest_esp32s2_generic:
@@ -338,8 +238,9 @@ test_app_test_pytest_esp32s2_generic:
    - .rules:test:custom_test-esp32s2
  needs:
    - build_pytest_test_apps_esp32s2
-  tags: [ esp32s2, generic ]
  variables:
+    TARGET: ESP32S2
+    ENV_MARKER: generic
    SETUP_TOOLS: "1"  # need gdb

 test_app_test_pytest_esp32s3_generic:
@@ -348,23 +249,9 @@ test_app_test_pytest_esp32s3_generic:
    - .rules:test:custom_test-esp32s3
  needs:
    - build_pytest_test_apps_esp32s3
-  tags: [ esp32s3, generic ]
-
-test_app_test_pytest_esp32c2_generic:
-  extends:
-    - .pytest_test_apps_dir_template
-    - .rules:test:custom_test-esp32c2
-  needs:
-    - build_pytest_test_apps_esp32c2
-  tags: [ esp32c2, generic, xtal_40mhz ]
-
-test_app_test_pytest_esp32c2_26mhz:
-  extends:
-    - .pytest_test_apps_dir_template
-    - .rules:test:custom_test-esp32c2
-  needs:
-    - build_pytest_test_apps_esp32c2
-  tags: [ esp32c2, xtal_26mhz ]
+  variables:
+    TARGET: ESP32S3
+    ENV_MARKER: generic

 test_app_test_pytest_esp32c3_generic:
  extends:
@@ -372,7 +259,9 @@ test_app_test_pytest_esp32c3_generic:
    - .rules:test:custom_test-esp32c3
  needs:
    - build_pytest_test_apps_esp32c3
-  tags: [ esp32c3, generic ]
+  variables:
+    TARGET: ESP32C3
+    ENV_MARKER: generic

 test_app_test_pytest_esp32s2_usb_host:
  extends:
@@ -380,31 +269,9 @@ test_app_test_pytest_esp32s2_usb_host:
    - .rules:test:custom_test-esp32s2
  needs:
    - build_pytest_test_apps_esp32s2
-  tags: [ esp32s2, usb_host ]
-
-test_app_test_pytest_esp32s3_mspi_f8r8:
-  extends:
-    - .pytest_test_apps_dir_template
-    - .rules:test:custom_test-esp32s3
-  needs:
-    - build_pytest_test_apps_esp32s3
-  tags: [ esp32s3, MSPI_F8R8 ]
-
-test_app_test_pytest_esp32s3_mspi_f4r8:
-  extends:
-    - .pytest_test_apps_dir_template
-    - .rules:test:custom_test-esp32s3
-  needs:
-    - build_pytest_test_apps_esp32s3
-  tags: [ esp32s3, MSPI_F4R8 ]
-
-test_app_test_pytest_esp32s3_mspi_f4r4:
-  extends:
-    - .pytest_test_apps_dir_template
-    - .rules:test:custom_test-esp32s3
-  needs:
-    - build_pytest_test_apps_esp32s3
-  tags: [ esp32s3, MSPI_F4R4 ]
+  variables:
+    TARGET: ESP32S2
+    ENV_MARKER: usb_host

 # for parallel jobs, CI_JOB_NAME will be "job_name index/total" (for example, "IT_001 1/2")
 # we need to convert to pattern "job_name_index.yml"
@@ -414,8 +281,9 @@ test_app_test_pytest_esp32s3_mspi_f4r4:
  CONFIG_FILE="${CONFIG_FILE_PATH}/${JOB_FULL_NAME}.yml"

 .target_test_job_template:
-  extends:
-    - .target_test_template
+  stage: target_test
+  timeout: 1 hour
+  image: $TARGET_TEST_ENV_IMAGE
  artifacts:
    when: always
    paths:
@@ -430,6 +298,7 @@ test_app_test_pytest_esp32s3_mspi_f4r4:
    TEST_FW_PATH: "$CI_PROJECT_DIR/tools/tiny-test-fw"
    LOG_PATH: "$CI_PROJECT_DIR/TEST_LOGS"
    ENV_FILE: "$CI_PROJECT_DIR/ci-test-runner-configs/$CI_RUNNER_DESCRIPTION/EnvConfig.yml"
+    SUBMODULES_TO_FETCH: "none"
  script:
    - *define_config_file_name
    # first test if config file exists, if not exist, exit 0
@@ -452,13 +321,20 @@ test_app_test_pytest_esp32s3_mspi_f4r4:
    TEST_CASE_PATH: "$CI_PROJECT_DIR/examples"
    CONFIG_FILE_PATH: "${CI_PROJECT_DIR}/examples/test_configs"

+.example_debug_template:
+  extends:
+    - .example_test_template
+    - .rules:test:example_test-esp32
+  variables:
+    SUBMODULES_TO_FETCH: "all"
+
 test_weekend_mqtt:
  extends:
    - .test_app_esp32_template
    - .rules:labels:weekend_test
  tags:
    - ESP32
-    - ethernet_router
+    - Example_EthKitV1
  script:
    - export MQTT_PUBLISH_TEST=1
    - export TEST_PATH=$CI_PROJECT_DIR/tools/test_apps/protocols/mqtt/publish_connect_test
@@ -475,11 +351,6 @@ test_weekend_mqtt:
    - .example_test_template
    - .rules:test:example_test-esp32s2

-.example_test_esp32c2_template:
-  extends:
-    - .example_test_template
-    - .rules:test:example_test-esp32c2
-
 .example_test_esp32c3_template:
  extends:
    - .example_test_template
@@ -490,6 +361,19 @@ test_weekend_mqtt:
    - .example_test_template
    - .rules:test:example_test-esp32s3

+example_test_001A:
+  extends: .example_test_esp32_template
+  tags:
+    - ESP32
+    - Example_WIFI
+
+example_test_001B:
+  extends: .example_test_esp32_template
+  parallel: 2
+  tags:
+    - ESP32
+    - Example_EthKitV1
+
 example_test_001B_V3:
  extends: .example_test_esp32_template
  tags:
@@ -503,29 +387,43 @@ example_test_001C:
    - ESP32
    - Example_GENERIC

-example_test_protocols:
-  extends:
-    - .example_test_esp32_template
-    - .rules:test:example_test-esp32-wifi
+example_test_001D:
+  extends: .example_test_esp32_template
  tags:
    - ESP32
-    - wifi_router
+    - Example_8Mflash_Ethernet
+
+example_test_OTA:
+  extends: .example_test_esp32_template
+  tags:
+    - ESP32
+    - EXAMPLE_ETH_OTA
+
+example_test_protocols:
+  extends: .example_test_esp32_template
+  parallel: 2
+  tags:
+    - ESP32
+    - Example_WIFI_Protocols
+
+# This job is only triggered by env var `NIGHTLY_RUN`, please do NOT remove
+example_test_esp32_WIFI_OTA:
+  extends: .example_test_esp32_template
+  tags:
+    - ESP32
+    - Example_WIFI_OTA

 example_test_002:
-  extends:
-    - .example_test_esp32_template
-    - .rules:test:example_test-esp32-wifi
+  extends: .example_test_esp32_template
  tags:
    - ESP32
    - Example_ShieldBox_Basic

-example_test_ethernet_router:
-  extends:
-    - .example_test_esp32_template
-    - .rules:test:example_test-esp32-ethernet
+example_test_ethernet:
+  extends: .example_test_esp32_template
  tags:
    - ESP32
-    - ethernet_router
+    - Example_Ethernet

 .example_test_003:
  extends: .example_test_esp32_template
@@ -546,9 +444,7 @@ example_test_004B:
    - Example_TWAI2

 example_test_005:
-  extends:
-    - .example_test_esp32_template
-    - .rules:test:example_test-esp32-bt
+  extends: .example_test_esp32_template
  tags:
    - ESP32
    - Example_WIFI_BT
@@ -567,6 +463,12 @@ example_test_007:
    - ESP32
    - Example_I2C_CCS811_SENSOR

+example_test_008B:
+  extends: .example_test_esp32_template
+  tags:
+    - ESP32
+    - Example_Flash_Encryption_OTA
+
 example_test_009:
  extends: .example_test_esp32_template
  tags:
@@ -583,10 +485,12 @@ example_test_010:
    - Example_ExtFlash

 example_test_011:
-  extends: .example_test_esp32_template
+  extends: .example_debug_template
  tags:
    - ESP32
    - Example_T2_RS485
+  variables:
+    SETUP_TOOLS: "1"

 example_test_013:
  extends: .example_test_esp32_template
@@ -594,6 +498,18 @@ example_test_013:
    - ESP32
    - UT_T1_SDMODE

+example_test_014:
+  extends: .example_test_esp32_template
+  tags:
+    - ESP32
+    - 8Mpsram
+
+example_test_015:
+  extends: .example_test_esp32_template
+  tags:
+    - ESP32
+    - Example_PPP
+
 example_test_016:
  extends: .example_test_esp32_template
  tags:
@@ -613,6 +529,12 @@ example_test_C3_GENERIC:
    - ESP32C3
    - Example_GENERIC

+example_test_C3_FLASH_ENC_OTA:
+  extends: .example_test_esp32c3_template
+  tags:
+    - ESP32C3
+    - Example_Flash_Encryption_OTA_WiFi
+
 example_test_ESP32_SDSPI:
  extends: .example_test_esp32_template
  tags:
@@ -636,7 +558,6 @@ example_test_ESP32C3_SDSPI:
  needs:
    - assign_custom_test
  variables:
-    GIT_DEPTH: 1
    TEST_CASE_PATH: "$CI_PROJECT_DIR/tools/test_apps"
    CONFIG_FILE_PATH: "${CI_PROJECT_DIR}/tools/test_apps/test_configs"

@@ -650,11 +571,6 @@ example_test_ESP32C3_SDSPI:
    - .test_app_template
    - .rules:test:custom_test-esp32s2

-.test_app_esp32c2_template:
-  extends:
-    - .test_app_template
-    - .rules:test:custom_test-esp32c2
-
 .test_app_esp32c3_template:
  extends:
    - .test_app_template
@@ -673,11 +589,23 @@ test_app_test_001:
  variables:
    SETUP_TOOLS: "1"

+test_app_test_002:
+  extends: .test_app_esp32_template
+  tags:
+    - ESP32
+    - Example_WIFI
+
 test_app_test_eth:
  extends: .test_app_esp32_template
  tags:
    - ESP32
-    - ethernet_router
+    - Example_EthKitV1
+
+test_app_test_003:
+  extends: .test_app_esp32_template
+  tags:
+    - ESP32
+    - Example_PPP

 test_app_test_004:
  extends: .test_app_esp32s2_template
@@ -685,43 +613,41 @@ test_app_test_004:
    - ESP32S2
    - Example_GENERIC

-test_app_test_esp32c2_generic:
-  extends: .test_app_esp32c2_template
-  tags:
-    - ESP32C2
-    - Example_GENERIC
-    - xtal_40mhz
-
-test_app_test_esp32c2_26mhz:
-  extends: .test_app_esp32c2_template
-  tags:
-    - ESP32C2
-    - xtal_26mhz
-
 test_app_test_005:
  extends: .test_app_esp32c3_template
  tags:
    - ESP32C3
    - Example_GENERIC

-test_app_test_006:
-  extends: .test_app_esp32s3_template
-  tags:
-    - ESP32S3
-    - Example_GENERIC
-
 test_app_test_esp32_generic:
  extends: .test_app_esp32_template
  tags:
    - ESP32
    - Example_GENERIC

+test_app_test_flash_psram_f4r4:
+  extends: .test_app_esp32s3_template
+  tags:
+    - ESP32S3
+    - MSPI_F4R4
+
+test_app_test_flash_psram_f4r8:
+  extends: .test_app_esp32s3_template
+  tags:
+    - ESP32S3
+    - MSPI_F4R8
+
+test_app_test_flash_psram_f8r8:
+  extends: .test_app_esp32s3_template
+  tags:
+    - ESP32S3
+    - MSPI_F8R8
+
 .unit_test_template:
  extends: .target_test_job_template
  needs: # the assign already needs all the build jobs
    - assign_unit_test
  variables:
-    GIT_DEPTH: 1
    TEST_CASE_PATH: "$CI_PROJECT_DIR/tools/unit-test-app"
    CONFIG_FILE_PATH: "${CI_PROJECT_DIR}/components/idf_test/unit_test/test_configs"

@@ -740,11 +666,6 @@ test_app_test_esp32_generic:
    - .unit_test_template
    - .rules:test:unit_test-esp32s3

-.unit_test_esp32c2_template:
-  extends:
-    - .unit_test_template
-    - .rules:test:unit_test-esp32c2
-
 .unit_test_esp32c3_template:
  extends:
    - .unit_test_template
@@ -796,7 +717,7 @@ UT_006:

 UT_007:
  extends: .unit_test_esp32_template
-  parallel: 2
+  parallel: 5
  tags:
    - ESP32_IDF
    - UT_T1_1
@@ -836,6 +757,13 @@ UT_020:
    - Example_SPI_Multi_device
    - psram

+UT_021:
+  extends: .unit_test_esp32_template
+  tags:
+    - ESP32_IDF
+    - psram
+    - UT_T1_FlashEncryption
+
 UT_022:
  extends: .unit_test_esp32_template
  tags:
@@ -850,6 +778,12 @@ UT_028:
    - UT_T2_1
    - psram

+UT_031:
+  extends: .unit_test_esp32_template
+  tags:
+    - ESP32_IDF
+    - UT_T1_FlashEncryption
+
 UT_033:
  extends: .unit_test_esp32_template
  tags:
@@ -900,17 +834,13 @@ UT_043:
    - psram

 UT_044:
-  extends:
-    - .unit_test_esp32_template
-    - .rules:test:unit_test-esp32-sdio
+  extends: .unit_test_esp32_template
  tags:
    - ESP32_IDF
    - UT_SDIO

 UT_045:
-  extends:
-    - .unit_test_esp32_template
-    - .rules:test:unit_test-esp32-sdio
+  extends: .unit_test_esp32_template
  tags:
    - ESP32_IDF
    - UT_SDIO
@@ -924,7 +854,7 @@ UT_046:

 UT_047:
  extends: .unit_test_esp32s2_template
-  parallel: 6
+  parallel: 13
  tags:
    - ESP32S2_IDF
    - UT_T1_1
@@ -941,24 +871,9 @@ UT_S2_SDSPI:
    - ESP32S2_IDF
    - UT_T1_SPIMODE

-UT_C2:
-  extends: .unit_test_esp32c2_template
-  parallel: 21
-  tags:
-    - ESP32C2_IDF
-    - UT_T1_1
-    - xtal_40mhz
-
-UT_C2_26M:
-  extends: .unit_test_esp32c2_template
-  tags:
-    - ESP32C2_IDF
-    - UT_T1_1
-    - xtal_26mhz
-
 UT_C3:
  extends: .unit_test_esp32c3_template
-  parallel: 31
+  parallel: 34
  tags:
    - ESP32C3_IDF
    - UT_T1_1
@@ -976,6 +891,12 @@ UT_C3_SPI_DUAL:
    - ESP32C3_IDF
    - Example_SPI_Multi_device

+UT_C3_FLASH_ENC:
+  extends: .unit_test_esp32c3_template
+  tags:
+    - ESP32C3_IDF
+    - UT_T1_FlashEncryption
+
 UT_C3_I2C:
  extends: .unit_test_esp32c3_template
  tags:
@@ -996,19 +917,27 @@ UT_C3_SDSPI:

 UT_S3:
  extends: .unit_test_esp32s3_template
-  parallel: 29
+  parallel: 33
  tags:
    - ESP32S3_IDF
    - UT_T1_1

 UT_S3_QUAD_PSRAM:
  extends: .unit_test_esp32s3_template
-  parallel: 4
+  parallel: 33
  tags:
    - ESP32S3_IDF
    - UT_T1_1
    - quad_psram

+UT_S3_OCTAL_PSRAM:
+  extends: .unit_test_esp32s3_template
+  parallel: 33
+  tags:
+    - ESP32S3_IDF
+    - UT_T1_1
+    - octal_psram
+
 UT_S3_SPI_DUAL:
  extends: .unit_test_esp32s3_template
  tags:
--- a/.gitlab/ci/upload_cache.yml
+++ b/.gitlab/ci/upload_cache.yml
@@ -1,53 +0,0 @@
-# pull only for most of the use cases for cache
-# only set "push" policy for the jobs under this file.
-# The cache would be updated when files matched specified patterns changes.
-
-.upload_cache_template:
-  stage: upload_cache
-  image: $ESP_ENV_IMAGE
-
-upload-pip-cache:
-  extends:
-    - .upload_cache_template
-    - .rules:patterns:python-cache
-  tags:
-    - $GEO
-    - build
-  cache:
-    key: pip-cache
-    paths:
-      - .cache/pip
-    policy: push
-  before_script: []
-  script:
-    - source tools/ci/utils.sh
-    - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-    - source tools/ci/setup_python.sh
-    - rm -rf .cache/pip  # clear old packages
-    - $IDF_PATH/tools/idf_tools.py install-python-env --features pytest,ci
-  parallel:
-    matrix:
-      - GEO: [ 'shiny',  'brew' ]
-
-upload-submodules-cache:
-  extends:
-    - .upload_cache_template
-    - .rules:patterns:submodule
-  tags:
-    - $GEO
-    - build
-  cache:
-    key: submodule-cache
-    paths:
-      - .cache/submodule_archives
-    policy: push
-  before_script: []
-  script:
-    - source tools/ci/utils.sh
-    - is_based_on_commits $REQUIRED_ANCESTOR_COMMITS
-    - source tools/ci/setup_python.sh
-    - rm -rf .cache/submodule_archives  # clear old submodule archives
-    - fetch_submodules
-  parallel:
-    matrix:
-      - GEO: [ 'shiny',  'brew' ]
--- a/.gitmodules
+++ b/.gitmodules
@@ -5,7 +5,7 @@

 [submodule "components/bt/controller/lib_esp32"]
 	path = components/bt/controller/lib_esp32
-	url = ../../espressif/esp32-bt-lib.git
+        url = ../../espressif/esp32-bt-lib.git

 [submodule "components/bootloader/subproject/components/micro-ecc/micro-ecc"]
 	path = components/bootloader/subproject/components/micro-ecc/micro-ecc
@@ -23,6 +23,10 @@
 	path = components/mbedtls/mbedtls
 	url = ../../espressif/mbedtls.git

+[submodule "components/asio/asio"]
+	path = components/asio/asio
+	url = ../../espressif/asio.git
+
 [submodule "components/lwip/lwip"]
 	path = components/lwip/lwip
 	url = ../../espressif/esp-lwip.git
@@ -39,6 +43,10 @@
 	path = components/unity/unity
 	url = ../../ThrowTheSwitch/Unity.git

+[submodule "examples/build_system/cmake/import_lib/main/lib/tinyxml2"]
+	path = examples/build_system/cmake/import_lib/main/lib/tinyxml2
+	url = ../../leethomason/tinyxml2.git
+
 [submodule "components/bt/host/nimble/nimble"]
 	path = components/bt/host/nimble/nimble
 	url = ../../espressif/esp-nimble.git
@@ -82,7 +90,3 @@
 [submodule "components/bt/controller/lib_esp32h2/esp32h2-bt-lib"]
 	path = components/bt/controller/lib_esp32h2/esp32h2-bt-lib
 	url = ../../espressif/esp32h2-bt-lib.git
-
-[submodule "components/bt/controller/lib_esp32c2/esp32c2-bt-lib"]
-	path = components/bt/controller/lib_esp32c2/esp32c2-bt-lib
-	url = ../../espressif/esp32c2-bt-lib.git
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -12,16 +12,12 @@ repos:
        # 2 - any file matching *test*/*expected* (for host tests, if possible use this naming pattern always)
        # 3 - any directory named 'testdata'
        # 4 - IDF monitor test data
-        # 5 - protobuf auto-generated files
        exclude: &whitespace_excludes |
          (?x)^(
            .+\.(md|rst|map|bin)|
            .+test.*\/.*expected.*|
            .+\/testdata\/.+|
-            .+test_idf_monitor\/tests\/.+|
-            .*_pb2.py|
-            .*.pb-c.h|
-            .*.pb-c.c
+            .+test_idf_monitor\/tests\/.+
          )$
      - id: end-of-file-fixer
        exclude: *whitespace_excludes
@@ -39,10 +35,6 @@ repos:
    hooks:
      - id: isort
        name: isort (python)
-        exclude: >
-          (?x)^(
-              .*_pb2.py
-          )$
  - repo: local
    hooks:
      - id: check-executables
@@ -105,13 +97,21 @@ repos:
          - 'mypy==0.940'
          - 'mypy-extensions==0.4.3'
          - 'types-setuptools==57.4.14'
-          - 'types-PyYAML==0.1.9'
-        exclude: >
-          (?x)^(
-              .*_pb2.py
-          )$
        language: python
        types: [python]
+      - id: check-copyright
+        name: Check copyright notices
+        entry: tools/ci/check_copyright.py --verbose --replace
+        additional_dependencies:
+          - 'comment_parser == 1.2.3'
+          - 'thefuzz == 0.19.0'
+          - 'thefuzz[speedup] == 0.19.0; sys_platform != "win32"'
+          # don't depend on python-Levenshtein on Windows, as it requires Microsoft C++ Build Tools to install
+          - 'pyyaml == 5.4.1'
+          - 'pathspec == 0.9.0'
+        language: python
+        files: \.(py|c|h|cpp|hpp|ld|s|S)$
+        require_serial: true
      - id: check-requirement-files
        name: Check requirement files
        entry: tools/ci/check_requirement_files.py
@@ -136,27 +136,8 @@ repos:
        pass_filenames: false
        additional_dependencies:
          - pyparsing
-      - id: check-all-apps-readmes
-        name: Check if all apps readme files match given .build-test-rules.yml files. Modify the supported target tables
-        entry: tools/ci/check_build_test_rules.py check-readmes
-        language: python
-        files: 'tools/test_apps/.+|examples/.+|components/.+'
-        additional_dependencies:
-          - idf_build_apps
-      - id: sort-build-test-rules-ymls
-        name: sort .build-test-rules.yml files
-        entry: tools/ci/check_build_test_rules.py sort-yaml
-        language: python
-        files: '\.build-test-rules\.yml'
-        additional_dependencies:
-          - ruamel.yaml
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.0.1
    hooks:
      - id: file-contents-sorter
        files: 'tools\/ci\/(executable-list\.txt|mypy_ignore_list\.txt|check_copyright_ignore\.txt)'
-  - repo: https://github.com/espressif/check-copyright/
-    rev: v1.0.1
-    hooks:
-      - id: check-copyright
-        args: ['--ignore', 'tools/ci/check_copyright_ignore.txt', '--config', 'tools/ci/check_copyright_config.yaml']
--- a/.pylintrc
+++ b/.pylintrc
@@ -152,7 +152,6 @@ disable=print-statement,
        too-many-statements,
        ungrouped-imports,  # since we have isort in pre-commit
        no-name-in-module,  # since we have flake8 to check this
-        too-many-instance-attributes,

 # Enable the message, report, category or checker with the given id(s). You can
 # either give multiple identifier separated by comma (,) or put this option
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 3.16)
+cmake_minimum_required(VERSION 3.5)
 project(esp-idf C CXX ASM)

 if(CMAKE_CURRENT_LIST_DIR STREQUAL CMAKE_SOURCE_DIR)
@@ -62,6 +62,23 @@ if(CONFIG_COMPILER_SAVE_RESTORE_LIBCALLS)
    list(APPEND compile_options "-msave-restore")
 endif()

+if(CONFIG_COMPILER_DISABLE_GCC8_WARNINGS)
+    list(APPEND compile_options "-Wno-parentheses"
+                                "-Wno-sizeof-pointer-memaccess"
+                                "-Wno-clobbered")
+
+    list(APPEND compile_options "-Wno-format-overflow"
+                                "-Wno-stringop-truncation"
+                                "-Wno-misleading-indentation"
+                                "-Wno-cast-function-type"
+                                "-Wno-implicit-fallthrough"
+                                "-Wno-unused-const-variable"
+                                "-Wno-switch-unreachable"
+                                "-Wno-format-truncation"
+                                "-Wno-memset-elt-size"
+                                "-Wno-int-in-bool-context")
+endif()
+
 if(CMAKE_C_COMPILER_ID MATCHES "GNU")
    list(APPEND c_compile_options "-Wno-old-style-declaration")
 endif()
--- a/51
+++ b/51
@@ -71,7 +71,7 @@ mainmenu "Espressif IoT Development Framework Configuration"
    choice IDF_TARGET_ESP32H2_BETA_VERSION
        prompt "ESP32-H2 beta version"
        depends on IDF_TARGET_ESP32H2
-        default IDF_TARGET_ESP32H2_BETA_VERSION_2
+        default IDF_TARGET_ESP32H2_BETA_VERSION_1
        help
            Currently ESP32-H2 has several beta versions for internal use only.
            Select the one that matches your chip model.
@@ -200,48 +200,6 @@ mainmenu "Espressif IoT Development Framework Configuration"
                If enabled, this disables the linking of binary libraries in the application build. Note
                that after enabling this Wi-Fi/Bluetooth will not work.

-        config APP_COMPATIBLE_PRE_V2_1_BOOTLOADERS
-            bool "App compatible with bootloaders before ESP-IDF v2.1"
-            select APP_COMPATIBLE_PRE_V3_1_BOOTLOADERS
-            depends on IDF_TARGET_ESP32
-            default n
-            help
-                Bootloaders before ESP-IDF v2.1 did less initialisation of the
-                system clock. This setting needs to be enabled to build an app
-                which can be booted by these older bootloaders.
-
-                If this setting is enabled, the app can be booted by any bootloader
-                from IDF v1.0 up to the current version.
-
-                If this setting is disabled, the app can only be booted by bootloaders
-                from IDF v2.1 or newer.
-
-                Enabling this setting adds approximately 1KB to the app's IRAM usage.
-
-        config APP_COMPATIBLE_PRE_V3_1_BOOTLOADERS
-            bool "App compatible with bootloader and partition table before ESP-IDF v3.1"
-            depends on IDF_TARGET_ESP32
-            default n
-            help
-                Partition tables before ESP-IDF V3.1 do not contain an MD5 checksum
-                field, and the bootloader before ESP-IDF v3.1 cannot read a partition
-                table that contains an MD5 checksum field.
-
-                Enable this option only if your app needs to boot on a bootloader and/or
-                partition table that was generated from a version *before* ESP-IDF v3.1.
-
-                If this option and Flash Encryption are enabled at the same time, and any
-                data partitions in the partition table are marked Encrypted, then the
-                partition encrypted flag should be manually verified in the app before accessing
-                the partition (see CVE-2021-27926).
-
-        config APP_INIT_CLK
-            bool
-            depends on IDF_TARGET_ESP32
-            default y if APP_COMPATIBLE_PRE_V2_1_BOOTLOADERS
-            default y if APP_BUILD_TYPE_ELF_RAM
-
-
    endmenu # Build type

    source "$COMPONENT_KCONFIGS_PROJBUILD_SOURCE_FILE"
@@ -466,6 +424,13 @@ mainmenu "Espressif IoT Development Framework Configuration"

                This option can be enabled for RISC-V targets only.

+        config COMPILER_DISABLE_GCC8_WARNINGS
+            bool "Disable new warnings introduced in GCC 6 - 8"
+            default "n"
+            help
+                Enable this option if using GCC 6 or newer, and wanting to disable warnings which don't appear with
+                GCC 5.
+
        config COMPILER_DUMP_RTL_FILES
            bool "Dump RTL files during compilation"
            help
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -1,9 +0,0 @@
-# Security Policy
-
-## Supported Versions
-
-Please refer to https://docs.espressif.com/projects/esp-idf/en/latest/esp32/versions.html#support-periods for more details on ESP-IDF supported versions and support period policy.
-
-## Reporting a Vulnerability
-
-If you think you have found a security vulnerability in Espressif solutions (including ESP-IDF), then please send an email to our Bug Bounty team at bugbounty@espressif.com. Please do **NOT** create a public GitHub issue.
--- a/components/app_trace/app_trace.c
+++ b/components/app_trace/app_trace.c
@@ -5,11 +5,9 @@
 */

 #include <string.h>
-#include "esp_cpu.h"
 #include "esp_log.h"
 #include "esp_app_trace.h"
 #include "esp_app_trace_port.h"
-#include "esp_private/startup_internal.h"

 #ifdef CONFIG_APPTRACE_DEST_UART0
 #define ESP_APPTRACE_DEST_UART_NUM 0
@@ -46,7 +44,7 @@ esp_err_t esp_apptrace_init(void)
    void *hw_data = NULL;

    // 'esp_apptrace_init()' is called on every core, so ensure to do main initialization only once
-    if (esp_cpu_get_core_id() == 0) {
+    if (cpu_hal_get_core_id() == 0) {
        memset(&s_trace_channels, 0, sizeof(s_trace_channels));
        hw = esp_apptrace_jtag_hw_get(&hw_data);
        ESP_APPTRACE_LOGD("HW interface %p", hw);
@@ -77,11 +75,6 @@ esp_err_t esp_apptrace_init(void)
    return ESP_OK;
 }

-ESP_SYSTEM_INIT_FN(esp_apptrace_init, ESP_SYSTEM_INIT_ALL_CORES, 115)
-{
-    return esp_apptrace_init();
-}
-
 void esp_apptrace_down_buffer_config(uint8_t *buf, uint32_t size)
 {
    esp_apptrace_channel_t *ch;
--- a/components/app_trace/app_trace_membufs_proto.c
+++ b/components/app_trace/app_trace_membufs_proto.c
@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: 2021-2022 Espressif Systems (Shanghai) CO LTD
+ * SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
 *
 * SPDX-License-Identifier: Apache-2.0
 */
@@ -8,7 +8,6 @@
 #include <string.h>
 #include "sdkconfig.h"
 #include "esp_log.h"
-#include "esp_cpu.h"
 #include "esp_app_trace_membufs_proto.h"

 /** Trace data header. Every user data chunk is prepended with this header.
@@ -260,8 +259,8 @@ static inline uint8_t *esp_apptrace_membufs_wait4buf(esp_apptrace_membufs_proto_

 static inline uint8_t *esp_apptrace_membufs_pkt_start(uint8_t *ptr, uint16_t size)
 {
-    // it is safe to use esp_cpu_get_core_id() in macro call because arg is used only once inside it
-    ((esp_tracedata_hdr_t *)ptr)->block_sz = ESP_APPTRACE_USR_BLOCK_CORE(esp_cpu_get_core_id()) | size;
+    // it is safe to use cpu_hal_get_core_id() in macro call because arg is used only once inside it
+    ((esp_tracedata_hdr_t *)ptr)->block_sz = ESP_APPTRACE_USR_BLOCK_CORE(cpu_hal_get_core_id()) | size;
    ((esp_tracedata_hdr_t *)ptr)->wr_sz = 0;
    return ptr + sizeof(esp_tracedata_hdr_t);
 }
--- a/components/app_trace/app_trace_util.c
+++ b/components/app_trace/app_trace_util.c
@@ -62,21 +62,13 @@ esp_err_t esp_apptrace_lock_take(esp_apptrace_lock_t *lock, esp_apptrace_tmo_t *
    while (1) {
        //Todo: Replace the current locking mechanism and int_state with portTRY_ENTER_CRITICAL() instead.
        // do not overwrite lock->int_state before we actually acquired the mux
-#if CONFIG_FREERTOS_SMP
-        unsigned int_state = portDISABLE_INTERRUPTS();
-#else
        unsigned int_state = portSET_INTERRUPT_MASK_FROM_ISR();
-#endif
        bool success = spinlock_acquire(&lock->mux, 0);
        if (success) {
            lock->int_state = int_state;
            return ESP_OK;
        }
-#if CONFIG_FREERTOS_SMP
-        portRESTORE_INTERRUPTS(int_state);
-#else
        portCLEAR_INTERRUPT_MASK_FROM_ISR(int_state);
-#endif
        // we can be preempted from this place till the next call (above) to portSET_INTERRUPT_MASK_FROM_ISR()
        res = esp_apptrace_tmo_check(tmo);
        if (res != ESP_OK) {
@@ -93,11 +85,7 @@ esp_err_t esp_apptrace_lock_give(esp_apptrace_lock_t *lock)
    // after call to the following func we can not be sure that lock->int_state
    // is not overwritten by other CPU who has acquired the mux just after we released it. See esp_apptrace_lock_take().
    spinlock_release(&lock->mux);
-#if CONFIG_FREERTOS_SMP
-    portRESTORE_INTERRUPTS(int_state);
-#else
    portCLEAR_INTERRUPT_MASK_FROM_ISR(int_state);
-#endif
    return ESP_OK;
 }

--- a/components/app_trace/include/esp_app_trace_util.h
+++ b/components/app_trace/include/esp_app_trace_util.h
@@ -174,7 +174,7 @@ void esp_apptrace_log_unlock(void);
 #define ESP_APPTRACE_LOG_LEV( _L_, level, format, ... )   \
    do { \
        if (LOG_LOCAL_LEVEL >= level) { \
-            ESP_APPTRACE_LOG(LOG_FORMAT(_L_, format), esp_log_early_timestamp(), TAG, ##__VA_ARGS__); \
+            ESP_APPTRACE_LOG(LOG_EARLY_FORMAT(_L_, format, TAG), ##__VA_ARGS__); \
        } \
    } while(0)

--- a/components/app_trace/linker.lf
+++ b/components/app_trace/linker.lf
@@ -4,10 +4,6 @@ entries:
    app_trace (noflash)
    port_uart (noflash)
    app_trace_util (noflash)
-    if APPTRACE_MEMBUFS_APPTRACE_PROTO_ENABLE:
-      app_trace_membufs_proto (noflash)
-    if APPTRACE_DEST_JTAG = y:
-      port (noflash)
    if APPTRACE_SV_ENABLE = y:
      SEGGER_SYSVIEW (noflash)
      SEGGER_RTT_esp (noflash)
--- a/components/app_trace/port/port_uart.c
+++ b/components/app_trace/port/port_uart.c
@@ -6,7 +6,6 @@

 #include "soc/soc.h"
 #include "esp_log.h"
-#include "esp_cpu.h"
 #include "esp_app_trace_port.h"

 #include "driver/uart.h"
@@ -106,7 +105,7 @@ static esp_err_t esp_apptrace_uart_unlock(esp_apptrace_uart_data_t *hw_data)

 static inline void esp_apptrace_uart_hw_init(void)
 {
-    ESP_APPTRACE_LOGI("Initialized UART on CPU%d", esp_cpu_get_core_id());
+    ESP_APPTRACE_LOGI("Initialized UART on CPU%d", cpu_hal_get_core_id());
 }


@@ -200,7 +199,7 @@ static const int APP_TRACE_UART_RX_BUF_SIZE = 4024;

 static esp_err_t esp_apptrace_uart_init(esp_apptrace_uart_data_t *hw_data)
 {
-    int core_id = esp_cpu_get_core_id();
+    int core_id = cpu_hal_get_core_id();
    if (core_id == 0) {
        hw_data->tx_data_buff = (uint8_t *)heap_caps_malloc(APP_TRACE_MAX_TX_BUFF_UART, MALLOC_CAP_INTERNAL|MALLOC_CAP_8BIT);
        if (hw_data->tx_data_buff == NULL){
--- a/components/app_trace/port/riscv/port.c
+++ b/components/app_trace/port/riscv/port.c
@@ -4,7 +4,6 @@
 * SPDX-License-Identifier: Apache-2.0
 */

-#include "esp_cpu.h"
 #include "esp_log.h"
 #include "esp_app_trace_membufs_proto.h"
 #include "esp_app_trace_port.h"
@@ -37,7 +36,7 @@ typedef struct {
 #define ESP_APPTRACE_RISCV_HOST_DATA             (1 << 22)
 #define ESP_APPTRACE_RISCV_HOST_CONNECT          (1 << 23)

-#define ESP_APPTRACE_RISCV_INITED(_hw_)          ((_hw_)->inited & (1 << 0/*esp_cpu_get_core_id()*/))
+#define ESP_APPTRACE_RISCV_INITED(_hw_)          ((_hw_)->inited & (1 << 0/*cpu_hal_get_core_id()*/))

 static esp_err_t esp_apptrace_riscv_init(esp_apptrace_riscv_data_t *hw_data);
 static esp_err_t esp_apptrace_riscv_flush(esp_apptrace_riscv_data_t *hw_data, esp_apptrace_tmo_t *tmo);
@@ -95,7 +94,7 @@ esp_apptrace_hw_t *esp_apptrace_jtag_hw_get(void **data)
   e.g. OpenOCD flasher stub use own implementation of it. */
 __attribute__((weak)) int esp_apptrace_advertise_ctrl_block(void *ctrl_block_addr)
 {
-    if (!esp_cpu_dbgr_is_attached()) {
+    if (!esp_cpu_in_ocd_debug_mode()) {
        return 0;
    }
    return (int) semihosting_call_noerrno(ESP_SEMIHOSTING_SYS_APPTRACE_INIT, (long*)ctrl_block_addr);
@@ -140,7 +139,7 @@ static esp_err_t esp_apptrace_riscv_unlock(esp_apptrace_riscv_data_t *hw_data)

 static esp_err_t esp_apptrace_riscv_init(esp_apptrace_riscv_data_t *hw_data)
 {
-    int core_id = esp_cpu_get_core_id();
+    int core_id = cpu_hal_get_core_id();

    if (hw_data->inited == 0) {
        esp_apptrace_mem_block_t mem_blocks_cfg[2];
@@ -254,7 +253,7 @@ static bool esp_apptrace_riscv_host_is_connected(esp_apptrace_riscv_data_t *hw_d
    if (!ESP_APPTRACE_RISCV_INITED(hw_data)) {
        return false;
    }
-    return s_tracing_ctrl[esp_cpu_get_core_id()].ctrl & ESP_APPTRACE_RISCV_HOST_CONNECT ? true : false;
+    return s_tracing_ctrl[cpu_hal_get_core_id()].ctrl & ESP_APPTRACE_RISCV_HOST_CONNECT ? true : false;
 }

 static esp_err_t esp_apptrace_riscv_flush_nolock(esp_apptrace_riscv_data_t *hw_data, uint32_t min_sz, esp_apptrace_tmo_t *tmo)
@@ -298,13 +297,13 @@ static inline void esp_apptrace_riscv_buffer_swap_lock(void)
    // HACK: in this case host will set breakpoint just after ESP_APPTRACE_RISCV_CTRL_REG update,
    // here we set address to set bp at
    // enter ERI update critical section
-    s_tracing_ctrl[esp_cpu_get_core_id()].stat = (uint32_t)&__esp_apptrace_riscv_updated;
+    s_tracing_ctrl[cpu_hal_get_core_id()].stat = (uint32_t)&__esp_apptrace_riscv_updated;
 }

 static __attribute__((noinline)) void esp_apptrace_riscv_buffer_swap_unlock(void)
 {
    // exit ERI update critical section
-    s_tracing_ctrl[esp_cpu_get_core_id()].stat = 0;
+    s_tracing_ctrl[cpu_hal_get_core_id()].stat = 0;
    // TODO: currently host sets breakpoint, use break instruction to stop;
    // it will allow to use ESP_APPTRACE_RISCV_STAT_REG for other purposes
    asm volatile (
@@ -318,13 +317,13 @@ static esp_err_t esp_apptrace_riscv_buffer_swap_start(uint32_t curr_block_id)

    esp_apptrace_riscv_buffer_swap_lock();

-    uint32_t ctrl_reg = s_tracing_ctrl[esp_cpu_get_core_id()].ctrl;
+    uint32_t ctrl_reg = s_tracing_ctrl[cpu_hal_get_core_id()].ctrl;
    uint32_t host_connected = ESP_APPTRACE_RISCV_HOST_CONNECT & ctrl_reg;
    if (host_connected) {
        uint32_t acked_block = ESP_APPTRACE_RISCV_BLOCK_ID_GET(ctrl_reg);
        uint32_t host_to_read = ESP_APPTRACE_RISCV_BLOCK_LEN_GET(ctrl_reg);
        if (host_to_read != 0 || acked_block != (curr_block_id & ESP_APPTRACE_RISCV_BLOCK_ID_MSK)) {
-            ESP_APPTRACE_LOGD("[%d]: Can not switch %x %d %x %x/%lx", esp_cpu_get_core_id(), ctrl_reg, host_to_read, acked_block,
+            ESP_APPTRACE_LOGD("[%d]: Can not switch %x %d %x %x/%lx", cpu_hal_get_core_id(), ctrl_reg, host_to_read, acked_block,
                curr_block_id & ESP_APPTRACE_RISCV_BLOCK_ID_MSK, curr_block_id);
            res = ESP_ERR_NO_MEM;
            goto _on_err;
@@ -338,9 +337,9 @@ _on_err:

 static esp_err_t esp_apptrace_riscv_buffer_swap_end(uint32_t new_block_id, uint32_t prev_block_len)
 {
-    uint32_t ctrl_reg = s_tracing_ctrl[esp_cpu_get_core_id()].ctrl;
+    uint32_t ctrl_reg = s_tracing_ctrl[cpu_hal_get_core_id()].ctrl;
    uint32_t host_connected = ESP_APPTRACE_RISCV_HOST_CONNECT & ctrl_reg;
-    s_tracing_ctrl[esp_cpu_get_core_id()].ctrl = ESP_APPTRACE_RISCV_BLOCK_ID(new_block_id) |
+    s_tracing_ctrl[cpu_hal_get_core_id()].ctrl = ESP_APPTRACE_RISCV_BLOCK_ID(new_block_id) |
              host_connected | ESP_APPTRACE_RISCV_BLOCK_LEN(prev_block_len);
    esp_apptrace_riscv_buffer_swap_unlock();
    return ESP_OK;
@@ -354,7 +353,7 @@ static esp_err_t esp_apptrace_riscv_buffer_swap(uint32_t new_block_id)

 static bool esp_apptrace_riscv_host_data_pending(void)
 {
-    uint32_t ctrl_reg = s_tracing_ctrl[esp_cpu_get_core_id()].ctrl;
+    uint32_t ctrl_reg = s_tracing_ctrl[cpu_hal_get_core_id()].ctrl;
    // ESP_APPTRACE_LOGV("%s() 0x%x", __func__, ctrl_reg);
    return (ctrl_reg & ESP_APPTRACE_RISCV_HOST_DATA) ? true : false;
 }
--- a/components/app_trace/port/xtensa/port.c
+++ b/components/app_trace/port/xtensa/port.c
@@ -148,6 +148,7 @@
 // time exceeds specified timeout value operation is canceled and ESP_ERR_TIMEOUT code is returned.
 #include "sdkconfig.h"
 #include "soc/soc.h"
+#include "soc/dport_access.h"
 #include "soc/dport_reg.h"
 #include "soc/tracemem_config.h"
 #if CONFIG_IDF_TARGET_ESP32S2 || CONFIG_IDF_TARGET_ESP32S3
@@ -155,7 +156,6 @@
 #endif
 #include "eri.h"
 #include "esp_private/trax.h"
-#include "esp_cpu.h"
 #include "esp_log.h"
 #include "esp_app_trace_membufs_proto.h"
 #include "esp_app_trace_port.h"
@@ -174,7 +174,7 @@
 #define ESP_APPTRACE_TRAX_HOST_DATA             (1 << 22)
 #define ESP_APPTRACE_TRAX_HOST_CONNECT          (1 << 23)

-#define ESP_APPTRACE_TRAX_INITED(_hw_)          ((_hw_)->inited & (1 << esp_cpu_get_core_id()))
+#define ESP_APPTRACE_TRAX_INITED(_hw_)          ((_hw_)->inited & (1 << cpu_hal_get_core_id()))

 #define ESP_APPTRACE_TRAX_BLOCK_SIZE            (0x4000UL)

@@ -272,7 +272,7 @@ static inline void esp_apptrace_trax_hw_init(void)
    // must be read by host before any transfer using TRAX
    eri_write(ESP_APPTRACE_TRAX_STAT_REG, 0);

-    ESP_APPTRACE_LOGI("Initialized TRAX on CPU%d", esp_cpu_get_core_id());
+    ESP_APPTRACE_LOGI("Initialized TRAX on CPU%d", cpu_hal_get_core_id());
 }

 static inline void esp_apptrace_trax_select_memory_block(int block_num)
@@ -311,7 +311,7 @@ static inline void esp_apptrace_trax_memory_enable(void)

 static esp_err_t esp_apptrace_trax_init(esp_apptrace_trax_data_t *hw_data)
 {
-    int core_id = esp_cpu_get_core_id();
+    int core_id = cpu_hal_get_core_id();

    // 'esp_apptrace_trax_init()' is called on every core, so ensure to do main initialization only once
    if (core_id == 0) {
@@ -498,7 +498,7 @@ static esp_err_t esp_apptrace_trax_buffer_swap_start(uint32_t curr_block_id)
        uint32_t acked_block = ESP_APPTRACE_TRAX_BLOCK_ID_GET(ctrl_reg);
        uint32_t host_to_read = ESP_APPTRACE_TRAX_BLOCK_LEN_GET(ctrl_reg);
        if (host_to_read != 0 || acked_block != (curr_block_id & ESP_APPTRACE_TRAX_BLOCK_ID_MSK)) {
-            ESP_APPTRACE_LOGD("HC[%d]: Can not switch %x %d %x %x/%lx", esp_cpu_get_core_id(), ctrl_reg, host_to_read, acked_block,
+            ESP_APPTRACE_LOGD("HC[%d]: Can not switch %x %d %x %x/%lx", cpu_hal_get_core_id(), ctrl_reg, host_to_read, acked_block,
                curr_block_id & ESP_APPTRACE_TRAX_BLOCK_ID_MSK, curr_block_id);
            res = ESP_ERR_NO_MEM;
            goto _on_err;
--- a/components/app_trace/sys_view/Sample/OS/SEGGER_SYSVIEW_FreeRTOS.h
+++ b/components/app_trace/sys_view/Sample/OS/SEGGER_SYSVIEW_FreeRTOS.h
@@ -342,12 +342,12 @@ define away all of the tracing macros.
  #define traceTASK_SWITCHED_IN()                   if(prvGetTCBFromHandle(NULL) == xTaskGetIdleTaskHandle()) {           \
                                                      SEGGER_SYSVIEW_OnIdle();                                          \
                                                    } else {                                                            \
-                                                      SEGGER_SYSVIEW_OnTaskStartExec((U32)pxCurrentTCB[esp_cpu_get_core_id()]); \
+                                                      SEGGER_SYSVIEW_OnTaskStartExec((U32)pxCurrentTCB[cpu_hal_get_core_id()]); \
                                                    }
 #else
  #define traceTASK_SWITCHED_IN()                   {                                                                   \
-                                                      if (memcmp(pxCurrentTCB[esp_cpu_get_core_id()]->pcTaskName, "IDLE", 5) != 0) { \
-                                                        SEGGER_SYSVIEW_OnTaskStartExec((U32)pxCurrentTCB[esp_cpu_get_core_id()]);    \
+                                                      if (memcmp(pxCurrentTCB[cpu_hal_get_core_id()]->pcTaskName, "IDLE", 5) != 0) { \
+                                                        SEGGER_SYSVIEW_OnTaskStartExec((U32)pxCurrentTCB[cpu_hal_get_core_id()]);    \
                                                      } else {                                                          \
                                                        SEGGER_SYSVIEW_OnIdle();                                        \
                                                      }                                                                 \
@@ -357,8 +357,8 @@ define away all of the tracing macros.
 #define traceMOVED_TASK_TO_READY_STATE(pxTCB)       SEGGER_SYSVIEW_OnTaskStartReady((U32)pxTCB)
 #define traceREADDED_TASK_TO_READY_STATE(pxTCB)

-#define traceMOVED_TASK_TO_DELAYED_LIST()           SEGGER_SYSVIEW_OnTaskStopReady((U32)pxCurrentTCB[esp_cpu_get_core_id()],  (1u << 2))
-#define traceMOVED_TASK_TO_OVERFLOW_DELAYED_LIST()  SEGGER_SYSVIEW_OnTaskStopReady((U32)pxCurrentTCB[esp_cpu_get_core_id()],  (1u << 2))
+#define traceMOVED_TASK_TO_DELAYED_LIST()           SEGGER_SYSVIEW_OnTaskStopReady((U32)pxCurrentTCB[cpu_hal_get_core_id()],  (1u << 2))
+#define traceMOVED_TASK_TO_OVERFLOW_DELAYED_LIST()  SEGGER_SYSVIEW_OnTaskStopReady((U32)pxCurrentTCB[cpu_hal_get_core_id()],  (1u << 2))
 #define traceMOVED_TASK_TO_SUSPENDED_LIST(pxTCB)    SEGGER_SYSVIEW_OnTaskStopReady((U32)pxTCB,         ((3u << 3) | 3))

 #define traceISR_EXIT_TO_SCHEDULER()                SEGGER_SYSVIEW_RecordExitISRToScheduler()
--- a/components/app_trace/sys_view/esp/SEGGER_RTT_esp.c
+++ b/components/app_trace/sys_view/esp/SEGGER_RTT_esp.c
@@ -12,8 +12,6 @@

 #include "esp_app_trace.h"
 #include "esp_log.h"
-#include "esp_cpu.h"
-#include "esp_private/startup_internal.h"

 const static char *TAG = "segger_rtt";

@@ -159,7 +157,7 @@ unsigned SEGGER_RTT_WriteSkipNoLock(unsigned BufferIndex, const void* pBuffer, u
  uint8_t event_id = *pbuf;
 #if CONFIG_APPTRACE_SV_DEST_UART
  if (
-    (APPTRACE_SV_DEST_CPU != esp_cpu_get_core_id()) &&
+    (APPTRACE_SV_DEST_CPU != cpu_hal_get_core_id()) &&
    (
      (event_id == SYSVIEW_EVTID_ISR_ENTER) ||
      (event_id == SYSVIEW_EVTID_ISR_EXIT) ||
@@ -190,7 +188,7 @@ unsigned SEGGER_RTT_WriteSkipNoLock(unsigned BufferIndex, const void* pBuffer, u
      return 0;
  }
 #if CONFIG_APPTRACE_SV_DEST_JTAG
-  if (esp_cpu_get_core_id()) { // dual core specific code
+  if (cpu_hal_get_core_id()) { // dual core specific code
    // use the highest - 1 bit of event ID to indicate core ID
    // the highest bit can not be used due to event ID encoding method
    // this reduces supported ID range to [0..63] (for 1 byte IDs) plus [128..16383] (for 2 bytes IDs)
@@ -290,17 +288,4 @@ int SEGGER_RTT_ConfigDownBuffer(unsigned BufferIndex, const char* sName, void* p
  return 0;
 }

-/*************************** Init hook ****************************
- *
- * This init function is placed here because this port file will be
- * linked whenever SystemView is used.
- */
-
-ESP_SYSTEM_INIT_FN(sysview_init, BIT(0), 120)
-{
-    SEGGER_SYSVIEW_Conf();
-    return ESP_OK;
-}
-
-
 /*************************** End of file ****************************/
--- a/components/app_trace/test/CMakeLists.txt
+++ b/components/app_trace/test/CMakeLists.txt
@@ -1,4 +1,3 @@
 idf_component_register(SRC_DIRS "."
                    PRIV_INCLUDE_DIRS "."
                    PRIV_REQUIRES cmock driver)
-target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")
--- a/components/app_trace/test/test_trace.c
+++ b/components/app_trace/test/test_trace.c
@@ -12,7 +12,6 @@
 #include "driver/gptimer.h"
 #include "esp_intr_alloc.h"
 #include "esp_rom_sys.h"
-#include "esp_cpu.h"
 #include "freertos/FreeRTOS.h"
 #include "freertos/semphr.h"
 #include "freertos/task.h"
@@ -55,7 +54,7 @@ const static char *TAG = "esp_apptrace_test";
 #define ESP_APPTRACE_TEST_LOG_LEVEL( _L_, level, format, ... )   \
    do { \
        if (LOG_LOCAL_LEVEL >= level) { \
-            ESP_APPTRACE_TEST_LOG(LOG_FORMAT(_L_, format), esp_log_early_timestamp(), TAG, ##__VA_ARGS__); \
+            ESP_APPTRACE_TEST_LOG(LOG_EARLY_FORMAT(_L_, format, TAG), ##__VA_ARGS__); \
        } \
    } while(0)

@@ -180,7 +179,7 @@ static void esp_apptrace_dummy_task(void *p)

    int i = 0;
    while (!arg->stop) {
-        ESP_APPTRACE_TEST_LOGD("%x: dummy task work %d.%d", xTaskGetCurrentTaskHandle(), esp_cpu_get_core_id(), i++);
+        ESP_APPTRACE_TEST_LOGD("%x: dummy task work %d.%d", xTaskGetCurrentTaskHandle(), cpu_hal_get_core_id(), i++);
        if (tmo_ticks) {
            vTaskDelay(tmo_ticks);
        }
@@ -211,7 +210,7 @@ static void esp_apptrace_test_task(void *p)
            .resolution_hz = 1000000,
        };
        TEST_ESP_OK(gptimer_new_timer(&timer_config, &arg->timers[i].gptimer));
-        *(uint32_t *)arg->timers[i].data.buf = ((uint32_t)arg->timers[i].gptimer) | (1 << 31) | (esp_cpu_get_core_id() ? 0x1 : 0);
+        *(uint32_t *)arg->timers[i].data.buf = ((uint32_t)arg->timers[i].gptimer) | (1 << 31) | (cpu_hal_get_core_id() ? 0x1 : 0);
        ESP_APPTRACE_TEST_LOGI("%x: start timer %x period %u us", xTaskGetCurrentTaskHandle(), arg->timers[i].gptimer, arg->timers[i].data.period);
        gptimer_alarm_config_t alarm_config = {
            .reload_count = 0,
@@ -227,7 +226,7 @@ static void esp_apptrace_test_task(void *p)
        TEST_ESP_OK(gptimer_start(arg->timers[i].gptimer));
    }

-    *(uint32_t *)arg->data.buf = (uint32_t)xTaskGetCurrentTaskHandle() | (esp_cpu_get_core_id() ? 0x1 : 0);
+    *(uint32_t *)arg->data.buf = (uint32_t)xTaskGetCurrentTaskHandle() | (cpu_hal_get_core_id() ? 0x1 : 0);
    arg->data.wr_cnt = 0;
    arg->data.wr_err = 0;
    while (!arg->stop) {
@@ -653,7 +652,7 @@ static void esp_logtrace_task(void *p)
        ESP_LOGI(TAG, "%p: sample print 4 %c", xTaskGetCurrentTaskHandle(), ((i & 0xFF) % 95) + 32);
        ESP_LOGI(TAG, "%p: sample print 5 %f", xTaskGetCurrentTaskHandle(), 1.0);
        ESP_LOGI(TAG, "%p: sample print 6 %f", xTaskGetCurrentTaskHandle(), 3.45);
-        ESP_LOGI(TAG, "%p: logtrace task work %d.%d", xTaskGetCurrentTaskHandle(), esp_cpu_get_core_id(), i);
+        ESP_LOGI(TAG, "%p: logtrace task work %d.%d", xTaskGetCurrentTaskHandle(), cpu_hal_get_core_id(), i);
        if (++i == 10000) {
            break;
        }
--- a/components/app_update/CMakeLists.txt
+++ b/components/app_update/CMakeLists.txt
@@ -48,37 +48,34 @@ if(NOT BOOTLOADER_BUILD)
        add_dependencies(flash blank_ota_data)
        add_dependencies(encrypted-flash blank_ota_data)

-        set(otatool_py "${python}" "${COMPONENT_DIR}/otatool.py")
-
-        set(esptool_args --esptool-args before=${CONFIG_ESPTOOLPY_BEFORE} after=${CONFIG_ESPTOOLPY_AFTER})
-        set(otatool_args --partition-table-file ${PARTITION_CSV_PATH})
-        list(APPEND otatool_args --partition-table-offset ${PARTITION_TABLE_OFFSET})
+        set(otatool_py ${python} ${COMPONENT_DIR}/otatool.py)

+        set(esptool_args "--esptool-args;before=${CONFIG_ESPTOOLPY_BEFORE};after=${CONFIG_ESPTOOLPY_AFTER}")
+        set(otatool_args "--partition-table-file;${PARTITION_CSV_PATH}"
+                         "--partition-table-offset;${PARTITION_TABLE_OFFSET}")
        idf_component_get_property(esptool_py_dir esptool_py COMPONENT_DIR)

        add_custom_target(read-otadata DEPENDS "${PARTITION_CSV_PATH}"
            COMMAND ${CMAKE_COMMAND}
-            -D "IDF_PATH=${idf_path}"
-            -D "SERIAL_TOOL=${otatool_py}"
-            -D "SERIAL_TOOL_ARGS=${esptool_args};${otatool_args};read_otadata"
-            -D "WORKING_DIRECTORY=${build_dir}"
+            -D IDF_PATH="${idf_path}"
+            -D SERIAL_TOOL="${otatool_py}"
+            -D SERIAL_TOOL_ARGS="${esptool_args};${otatool_args};read_otadata"
+            -D WORKING_DIRECTORY="${build_dir}"
            -P ${esptool_py_dir}/run_serial_tool.cmake
-            WORKING_DIRECTORY "${build_dir}"
+            WORKING_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}
            USES_TERMINAL
-            VERBATIM
            )
        add_deprecated_target_alias(read_otadata read-otadata)

        add_custom_target(erase-otadata DEPENDS "${PARTITION_CSV_PATH}"
            COMMAND ${CMAKE_COMMAND}
-            -D "IDF_PATH=${idf_path}"
-            -D "SERIAL_TOOL=${otatool_py}"
-            -D "SERIAL_TOOL_ARGS=${esptool_args};${otatool_args};erase_otadata"
-            -D "WORKING_DIRECTORY=${build_dir}"
+            -D IDF_PATH="${idf_path}"
+            -D SERIAL_TOOL="${otatool_py}"
+            -D SERIAL_TOOL_ARGS="${esptool_args};${otatool_args};erase_otadata"
+            -D WORKING_DIRECTORY="${build_dir}"
            -P ${esptool_py_dir}/run_serial_tool.cmake
-            WORKING_DIRECTORY "${build_dir}"
+            WORKING_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}
            USES_TERMINAL
-            VERBATIM
            )
        add_deprecated_target_alias(erase_otadata erase-otadata)

--- a/components/app_update/esp_ota_ops.c
+++ b/components/app_update/esp_ota_ops.c
@@ -14,10 +14,11 @@

 #include "esp_err.h"
 #include "esp_partition.h"
-#include "spi_flash_mmap.h"
+#include "esp_spi_flash.h"
 #include "esp_image_format.h"
 #include "esp_secure_boot.h"
 #include "esp_flash_encrypt.h"
+#include "esp_spi_flash.h"
 #include "sdkconfig.h"

 #include "esp_ota_ops.h"
@@ -30,20 +31,6 @@
 #include "esp_efuse.h"
 #include "esp_attr.h"

-#if CONFIG_IDF_TARGET_ESP32
-#include "esp32/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32S2
-#include "esp32s2/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32C3
-#include "esp32c3/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32S3
-#include "esp32s3/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32H2
-#include "esp32h2/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32C2
-#include "esp32c2/rom/secure_boot.h"
-#endif
-
 #define SUB_TYPE_ID(i) (i & 0x0F)

 /* Partial_data is word aligned so no reallocation is necessary for encrypted flash write */
@@ -912,7 +899,7 @@ esp_err_t esp_ota_revoke_secure_boot_public_key(esp_ota_secure_boot_public_key_i
    }

    esp_err_t ret;
-    esp_secure_boot_key_digests_t trusted_keys;
+    ets_secure_boot_key_digests_t trusted_keys;
    ret = esp_secure_boot_read_key_digests(&trusted_keys);
    if (ret != ESP_OK) {
        ESP_LOGE(TAG, "Could not read the secure boot key digests from efuse. Aborting..");
--- a/components/app_update/test/CMakeLists.txt
+++ b/components/app_update/test/CMakeLists.txt
@@ -2,4 +2,3 @@ idf_component_register(SRC_DIRS "."
                       PRIV_INCLUDE_DIRS "."
                       PRIV_REQUIRES cmock test_utils app_update bootloader_support nvs_flash driver
                      )
-target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")
--- a/components/app_update/test/test_switch_ota.c
+++ b/components/app_update/test/test_switch_ota.c
@@ -32,13 +32,9 @@

 #include "driver/gpio.h"
 #include "esp_sleep.h"
-#include "test_utils.h"


-#if !TEMPORARY_DISABLED_FOR_TARGETS(ESP32C2)
-//IDF-5131
 RTC_DATA_ATTR static int boot_count = 0;
-
 static const char *TAG = "ota_test";

 /* @brief Copies a current app to next partition using handle.
@@ -825,7 +821,6 @@ static void test_flow6(void)
 // 3 Stage: run OTA0    -> check it -> erase OTA_DATA for next tests    -> PASS
 TEST_CASE_MULTIPLE_STAGES("Switching between factory, OTA0 using esp_ota_write_with_offset", "[app_update][timeout=90][reset=DEEPSLEEP_RESET, DEEPSLEEP_RESET]", start_test, test_flow6, test_flow6);

-//IDF-5145
 TEST_CASE("Test bootloader_common_get_sha256_of_partition returns ESP_ERR_IMAGE_INVALID when image is ivalid", "[partitions]")
 {
    const esp_partition_t *cur_app = esp_ota_get_running_partition();
@@ -847,4 +842,3 @@ TEST_CASE("Test bootloader_common_get_sha256_of_partition returns ESP_ERR_IMAGE_
    TEST_ESP_ERR(ESP_ERR_IMAGE_INVALID, bootloader_common_get_sha256_of_partition(other_app->address, other_app->size, other_app->type, sha_256_other_app));
    TEST_ASSERT_EQUAL_MEMORY_MESSAGE(sha_256_cur_app, sha_256_other_app, sizeof(sha_256_cur_app), "must be the same");
 }
-#endif //!TEMPORARY_DISABLED_FOR_TARGETS(ESP32C2)
--- a/components/asio/CMakeLists.txt
+++ b/components/asio/CMakeLists.txt
@@ -0,0 +1,44 @@
+if(NOT CONFIG_LWIP_IPV6 AND NOT CMAKE_BUILD_EARLY_EXPANSION)
+    # note: the component is still included in the build so it can become visible again in config
+    # without needing to re-run CMake. However no source or header files are built.
+    message(STATUS "IPV6 support is disabled so the asio component will not be built")
+    idf_component_register()
+    return()
+endif()
+
+set(asio_sources "asio/asio/src/asio.cpp")
+
+if(CONFIG_ASIO_SSL_SUPPORT)
+    if(CONFIG_ASIO_USE_ESP_OPENSSL)
+        list(APPEND asio_sources
+                "port/src/asio_ssl_impl.cpp"
+                "port/mbedtls/src/mbedtls_context.cpp"
+                "port/mbedtls/src/mbedtls_engine.cpp")
+        set(asio_priv_includes "port/mbedtls/include")
+    endif()
+
+    if(CONFIG_ASIO_USE_ESP_WOLFSSL)
+        list(APPEND asio_sources
+                "asio/asio/src/asio_ssl.cpp")
+    endif()
+endif()
+
+idf_component_register(SRCS ${asio_sources}
+                    INCLUDE_DIRS "asio/asio/include" "port/include"
+                    PRIV_INCLUDE_DIRS ${asio_priv_includes}
+                    REQUIRES lwip)
+
+if(CONFIG_ASIO_SSL_SUPPORT)
+    if(CONFIG_ASIO_USE_ESP_WOLFSSL)
+        idf_component_get_property(wolflib esp-wolfssl COMPONENT_LIB)
+        idf_component_get_property(wolfdir esp-wolfssl COMPONENT_DIR)
+
+        target_link_libraries(${COMPONENT_LIB} PUBLIC ${wolflib})
+        target_include_directories(${COMPONENT_LIB} PUBLIC ${wolfdir}/wolfssl/wolfssl)
+    endif()
+
+    if(CONFIG_ASIO_USE_ESP_OPENSSL)
+        idf_component_get_property(mbedtls mbedtls COMPONENT_LIB)
+        target_link_libraries(${COMPONENT_LIB} PUBLIC ${mbedtls})
+    endif()
+endif()
--- a/components/asio/Kconfig
+++ b/components/asio/Kconfig
@@ -0,0 +1,36 @@
+menu "ESP-ASIO"
+    visible if LWIP_IPV6
+
+    config ASIO_SSL_SUPPORT
+        bool "Enable SSL/TLS support of ASIO"
+        default n
+        help
+            Enable support for basic SSL/TLS features, available for mbedTLS/OpenSSL
+            as well as wolfSSL TLS library.
+
+    choice ASIO_SSL_LIBRARY_CHOICE
+        prompt "Choose SSL/TLS library for ESP-TLS (See help for more Info)"
+        default ASIO_USE_ESP_OPENSSL
+        depends on ASIO_SSL_SUPPORT
+        help
+            The ASIO support multiple backend TLS libraries. Currently the mbedTLS with a thin ESP-OpenSSL
+            port layer (default choice) and WolfSSL are supported.
+            Different TLS libraries may support different features and have different resource
+            usage. Consult the ESP-TLS documentation in ESP-IDF Programming guide for more details.
+        config ASIO_USE_ESP_OPENSSL
+            bool "esp-openssl"
+        config ASIO_USE_ESP_WOLFSSL
+            depends on TLS_STACK_WOLFSSL
+            bool "wolfSSL (License info in wolfSSL directory README)"
+    endchoice
+
+    config ASIO_SSL_BIO_SIZE
+        int "Size of BIO object"
+        default 1024
+        depends on ASIO_SSL_SUPPORT
+        help
+            Size in bytes of SSL-BIO implementation.
+            Reducing the BIO size saves more RAM, but may slow down input output operations due to
+            fragmentation.
+
+endmenu
--- a/components/asio/asio
+++ b/components/asio/asio
--- a/components/asio/port/include/esp_asio_config.h
+++ b/components/asio/port/include/esp_asio_config.h
@@ -0,0 +1,45 @@
+/*
+ * SPDX-FileCopyrightText: 2018-2021 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+#ifndef _ESP_ASIO_CONFIG_H_
+#define _ESP_ASIO_CONFIG_H_
+
+//
+// Enabling exceptions only when they are enabled in menuconfig
+//
+# include <sdkconfig.h>
+# ifndef CONFIG_COMPILER_CXX_EXCEPTIONS
+#  define ASIO_NO_EXCEPTIONS
+# endif   // CONFIG_COMPILER_CXX_EXCEPTIONS
+
+# ifndef CONFIG_COMPILER_RTTI
+#  define ASIO_NO_TYPEID
+# endif   // CONFIG_COMPILER_RTTI
+
+//
+// LWIP compatibility inet and address macros/functions
+//
+# define LWIP_COMPAT_SOCKET_INET 1
+# define LWIP_COMPAT_SOCKET_ADDR 1
+
+//
+// Specific ASIO feature flags
+//
+# define ASIO_DISABLE_SERIAL_PORT
+# define ASIO_SEPARATE_COMPILATION
+# define ASIO_STANDALONE
+# define ASIO_HAS_PTHREADS
+
+# ifdef CONFIG_ASIO_USE_ESP_OPENSSL
+#  define ASIO_USE_ESP_OPENSSL
+#  define OPENSSL_NO_ENGINE
+#  define ASIO_SSL_DETAIL_OPENSSL_TYPES_HPP
+#  include "openssl_stub.hpp"
+
+# elif CONFIG_ASIO_USE_ESP_WOLFSSL
+#  define ASIO_USE_WOLFSSL
+# endif   // CONFIG_ASIO_USE_ESP_OPENSSL
+
+#endif // _ESP_ASIO_CONFIG_H_
--- a/components/asio/port/include/esp_exception.h
+++ b/components/asio/port/include/esp_exception.h
@@ -0,0 +1,30 @@
+/*
+ * SPDX-FileCopyrightText: 2018-2021 Espressif Systems (Shanghai) CO LTD
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ */
+#ifndef _ESP_EXCEPTION_H_
+#define _ESP_EXCEPTION_H_
+
+//
+// This exception stub is enabled only if exceptions are disabled in menuconfig
+//
+#if !defined(CONFIG_COMPILER_CXX_EXCEPTIONS) && defined (ASIO_NO_EXCEPTIONS)
+
+#include "esp_log.h"
+
+//
+// asio exception stub
+//
+namespace asio {
+namespace detail {
+template <typename Exception>
+void throw_exception(const Exception& e)
+{
+  ESP_LOGE("esp32_asio_exception", "Caught exception: %s!", e.what());
+  abort();
+}
+}}
+#endif // CONFIG_COMPILER_CXX_EXCEPTIONS==1 && defined (ASIO_NO_EXCEPTIONS)
+
+#endif // _ESP_EXCEPTION_H_
--- a/components/asio/port/include/openssl_stub.hpp
+++ b/components/asio/port/include/openssl_stub.hpp
@@ -0,0 +1,46 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+
+#pragma once
+
+//
+// Supply OpenSSL macros and flags for asio-ssl header files
+//
+#define OPENSSL_VERSION_NUMBER 0x10100001L
+
+#define SSL_R_SHORT_READ 219
+#define SSL_OP_ALL 0
+#define SSL_OP_SINGLE_DH_USE 0
+#define SSL_OP_NO_COMPRESSION 0
+
+#define SSL_OP_NO_SSLv2					0x01000000L
+#define SSL_OP_NO_SSLv3					0x02000000L
+#define SSL_OP_NO_TLSv1					0x04000000L
+
+#define SSL_VERIFY_NONE                 0x00
+#define SSL_VERIFY_PEER                 0x01
+#define SSL_VERIFY_FAIL_IF_NO_PEER_CERT 0x02
+#define SSL_VERIFY_CLIENT_ONCE          0x04
+
+//
+// Implement asio-ssl layer with these three classes in asio::ssl::mbedtls
+//
+namespace asio {
+namespace ssl {
+namespace mbedtls {
+
+class engine;
+class bio;
+class shared_ctx;
+} } } // namespace asio::ssl::mbedtls
+
+//
+// Supply OpenSSL types as aliases to mbedtls classes
+//
+using X509_STORE_CTX=void;
+using BIO=asio::ssl::mbedtls::bio;
+using SSL_CTX=asio::ssl::mbedtls::shared_ctx;
+using SSL=asio::ssl::mbedtls::engine;
--- a/components/asio/port/mbedtls/include/mbedtls_bio.hpp
+++ b/components/asio/port/mbedtls/include/mbedtls_bio.hpp
@@ -0,0 +1,113 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
+
+#include "asio/ssl/context_base.hpp"
+#include "asio/ssl/context.hpp"
+#include "sdkconfig.h"
+
+namespace asio {
+namespace ssl {
+namespace mbedtls {
+
+class bio {
+    static constexpr int BIO_SIZE = CONFIG_ASIO_SSL_BIO_SIZE;
+    static constexpr int BIO_FLAGS_READ = 1;
+    static constexpr int BIO_FLAGS_WRITE = 2;
+
+public:
+    int write(const void *buf, int len)
+    {
+        if (buf == nullptr || len <= 0) {
+            // not an error, just empty operation (as in openssl/bio)
+            return 0;
+        }
+        int remaining = size_ - offset_;
+        if (remaining <= 0) {
+            flags_ |= BIO_FLAGS_WRITE;
+            return -1;
+        }
+        int len_to_write = len > remaining ? remaining : len;
+        std::memcpy(&data_[offset_], buf, len_to_write);
+        offset_ += len_to_write;
+        dlen_ = offset_;
+        if (len_to_write == len) {
+            flags_ &= ~BIO_FLAGS_WRITE;
+        }
+        return len_to_write;
+    }
+
+    int read(void *buf, int len)
+    {
+        if (buf == nullptr || len <= 0) {
+            // not an error, just empty operation (as in openssl/bio)
+            return 0;
+        }
+        int remaining = peer_->dlen_ - peer_->roffset_;
+        if (remaining <= 0) {
+            flags_ |= BIO_FLAGS_READ;
+            return -1;
+        }
+        int len_to_read = remaining > len ? len : remaining;
+        std::memcpy(buf, &peer_->data_[peer_->roffset_], len_to_read);
+        peer_->roffset_ += len_to_read;
+        if (len_to_read == len) {
+            flags_ &= ~BIO_FLAGS_READ;
+        }
+        if (peer_->offset_) {
+            // shift data back to the beginning of the buffer
+            std::memmove(&peer_->data_[0], &peer_->data_[peer_->roffset_], peer_->offset_ - peer_->roffset_);
+            peer_->offset_ -= peer_->roffset_;
+            peer_->roffset_ = 0;
+            peer_->dlen_ = peer_->offset_;
+        }
+        return len_to_read;
+    }
+
+    size_t wpending() const
+    {
+        return dlen_ - roffset_;
+    }
+
+    size_t ctrl_pending()
+    {
+        return peer_->dlen_ - peer_->roffset_;
+    }
+
+    bool should_write() const
+    {
+        return flags_ & BIO_FLAGS_WRITE;
+    }
+
+    bool should_read() const
+    {
+        return flags_ & BIO_FLAGS_READ;
+    }
+
+    static std::pair<std::shared_ptr<bio>, std::shared_ptr<bio>>  new_pair(const char* error_location)
+    {
+        auto b1 = std::shared_ptr<bio>(new (std::nothrow) bio);
+        auto b2 = std::shared_ptr<bio>(new (std::nothrow) bio);
+        if (b1 == nullptr || b2 == nullptr) {
+            throw_alloc_failure(error_location);
+        } else {
+            b1->peer_ = b2;
+            b2->peer_ = b1;
+        }
+        return std::make_pair(b1, b2);
+    }
+
+private:
+    std::array<uint8_t, BIO_SIZE> data_ {};
+    size_t size_ {BIO_SIZE};
+    std::shared_ptr<bio> peer_ {nullptr};
+    int dlen_ {0};
+    size_t offset_ {0};
+    size_t roffset_ {0};
+    size_t flags_ {0};
+};
+
+} } } // namespace asio::ssl::mbedtls
--- a/components/asio/port/mbedtls/include/mbedtls_context.hpp
+++ b/components/asio/port/mbedtls/include/mbedtls_context.hpp
@@ -0,0 +1,105 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
+
+#include "asio/ssl/context_base.hpp"
+#include "asio/ssl/context.hpp"
+
+namespace asio {
+namespace error {
+
+const asio::error_category& get_mbedtls_category();
+} // namespace error
+
+namespace ssl {
+namespace mbedtls {
+
+void throw_alloc_failure(const char* location);
+
+const char *error_message(int error_code);
+
+enum class container {
+    CERT, CA_CERT, PRIVKEY
+};
+
+template <typename T, typename... Args>
+inline T* create(const char * location, Args &&... args)
+{
+    T* t = new (std::nothrow) T(std::forward<Args>(args)...);
+    if (t == nullptr)
+    {
+        throw_alloc_failure(location);
+    }
+    return t;
+}
+
+class context {
+public:
+    explicit context(context_base::method m): method_(m), options_(0) {}
+
+    const unsigned char *data(container c) const
+    {
+        switch (c) {
+            case container::CERT:
+                return static_cast<const unsigned char *>(cert_chain_.data());
+            case container::CA_CERT:
+                return static_cast<const unsigned char *>(ca_cert_.data());
+            case container::PRIVKEY:
+                return static_cast<const unsigned char *>(private_key_.data());
+        }
+        return nullptr;
+    }
+
+    std::size_t size(container c) const
+    {
+        switch (c) {
+            case container::CERT:
+                return cert_chain_.size();
+            case container::CA_CERT:
+                return ca_cert_.size();
+            case container::PRIVKEY:
+                return private_key_.size();
+        }
+        return 0;
+    }
+
+    context_base::method method_;
+    asio::ssl::context::options options_;
+    const_buffer cert_chain_;
+    const_buffer private_key_;
+    const_buffer ca_cert_;
+};
+
+/**
+ * @brief Wrapper class around SSL_CTX so we can easily create
+ * a shared pointer to the context without throwing the default exception.
+ * This is useful, as we can use asio::detail::throw_error for allocation errors.
+ */
+class shared_ctx {
+public:
+    static SSL_CTX *create(const char* location, context_base::method m)
+    {
+        auto wrapped = asio::ssl::mbedtls::create<shared_ctx>(location, m);
+        if (wrapped->ctx_ == nullptr)
+        {
+            throw_alloc_failure(location);
+        }
+        return wrapped;
+    }
+
+    std::shared_ptr<mbedtls::context> get() const
+    {
+        return ctx_;
+    }
+
+    explicit shared_ctx(context_base::method m)
+        :ctx_(std::shared_ptr<context>(new (std::nothrow) context(m))) { }
+
+private:
+    std::shared_ptr<mbedtls::context> ctx_;
+};
+
+} } } // namespace asio::ssl::mbedtls
--- a/components/asio/port/mbedtls/include/mbedtls_engine.hpp
+++ b/components/asio/port/mbedtls/include/mbedtls_engine.hpp
@@ -0,0 +1,293 @@
+//
+// SPDX-FileCopyrightText: 2021-2022 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
+
+#include "mbedtls/ssl.h"
+#include "mbedtls/entropy.h"
+#include "mbedtls/ctr_drbg.h"
+#include "mbedtls/error.h"
+#include "mbedtls/esp_debug.h"
+#include "esp_log.h"
+
+namespace asio {
+namespace ssl {
+namespace mbedtls {
+
+const char *error_message(int error_code)
+{
+    static char error_buf[100];
+    mbedtls_strerror(error_code, error_buf, sizeof(error_buf));
+    return error_buf;
+}
+
+void throw_alloc_failure(const char* location)
+{
+    asio::error_code ec( MBEDTLS_ERR_SSL_ALLOC_FAILED, asio::error::get_mbedtls_category());
+    asio::detail::throw_error(ec, location);
+}
+
+namespace error_codes {
+
+bool is_error(int ret)
+{
+    return  ret < 0 && ret != MBEDTLS_ERR_SSL_WANT_READ && ret != MBEDTLS_ERR_SSL_WANT_WRITE;
+}
+
+static bool want_write(int ret)
+{
+    return  ret == MBEDTLS_ERR_SSL_WANT_WRITE;
+}
+
+static bool want_read(int ret)
+{
+    return  ret == MBEDTLS_ERR_SSL_WANT_READ;
+}
+
+} // namespace error_codes
+
+enum rw_state {
+    IDLE, READING, WRITING, CLOSED
+};
+
+class engine {
+public:
+    explicit engine(std::shared_ptr<context> ctx): ctx_(std::move(ctx)),
+        bio_(bio::new_pair("mbedtls-engine")), state_(IDLE), verify_mode_(0) {}
+
+    void set_verify_mode(asio::ssl::verify_mode mode)
+    {
+        verify_mode_ = mode;
+    }
+
+    bio* ext_bio() const
+    {
+        return bio_.second.get();
+    }
+
+    rw_state get_state() const
+    {
+        return state_;
+    }
+
+    int shutdown()
+    {
+        int ret = mbedtls_ssl_close_notify(&impl_.ssl_);
+        if (ret) {
+            impl::print_error("mbedtls_ssl_close_notify", ret);
+        }
+        state_ = CLOSED;
+        return ret;
+    }
+
+    int connect()
+    {
+        return handshake(true);
+    }
+
+    int accept()
+    {
+        return handshake(false);
+    }
+
+    int write(const void *buffer, int len)
+    {
+        int ret = impl_.write(buffer, len);
+        state_ = ret == len ? IDLE: WRITING;
+        return ret;
+    }
+
+    int read(void *buffer, int len)
+    {
+        int ret = impl_.read(buffer, len);
+        state_ = ret == len ? IDLE: READING;
+        return ret;
+    }
+
+private:
+    int handshake(bool is_client_not_server)
+    {
+        if (impl_.before_handshake()) {
+            impl_.configure(ctx_.get(), is_client_not_server, impl_verify_mode(is_client_not_server));
+        }
+        return do_handshake();
+    }
+
+    static int bio_read(void *ctx, unsigned char *buf, size_t len)
+    {
+        auto bio = static_cast<BIO*>(ctx);
+        int read = bio->read(buf, len);
+        if (read <= 0 && bio->should_read()) {
+            return MBEDTLS_ERR_SSL_WANT_READ;
+        }
+        return read;
+    }
+
+    static int bio_write(void *ctx, const unsigned char *buf, size_t len)
+    {
+        auto bio = static_cast<BIO*>(ctx);
+        int written = bio->write(buf, len);
+        if (written <= 0 && bio->should_write()) {
+            return MBEDTLS_ERR_SSL_WANT_WRITE;
+        }
+        return written;
+    }
+
+    int do_handshake()
+    {
+        int ret = 0;
+        mbedtls_ssl_set_bio(&impl_.ssl_, bio_.first.get(), bio_write, bio_read, nullptr);
+
+        while (impl_.ssl_.MBEDTLS_PRIVATE(state) != MBEDTLS_SSL_HANDSHAKE_OVER) {
+            ret = mbedtls_ssl_handshake_step(&impl_.ssl_);
+
+            if (ret != 0) {
+                if (ret != MBEDTLS_ERR_SSL_WANT_READ && ret != MBEDTLS_ERR_SSL_WANT_WRITE) {
+                    impl::print_error("mbedtls_ssl_handshake_step", ret);
+                }
+                if (ret == MBEDTLS_ERR_SSL_WANT_READ) {
+                    state_ = READING;
+                } else if (ret == MBEDTLS_ERR_SSL_WANT_WRITE) {
+                    state_ = WRITING;
+                }
+                break;
+            }
+        }
+        return ret;
+    }
+
+    // Converts OpenSSL verification mode to mbedtls enum
+    int impl_verify_mode(bool is_client_not_server) const
+    {
+        int mode = MBEDTLS_SSL_VERIFY_UNSET;
+        if (is_client_not_server) {
+            if (verify_mode_ & SSL_VERIFY_PEER)
+                mode = MBEDTLS_SSL_VERIFY_REQUIRED;
+            else if (verify_mode_ == SSL_VERIFY_NONE)
+                mode = MBEDTLS_SSL_VERIFY_NONE;
+        } else {
+            if (verify_mode_ & SSL_VERIFY_FAIL_IF_NO_PEER_CERT)
+                mode = MBEDTLS_SSL_VERIFY_REQUIRED;
+            else if (verify_mode_ & SSL_VERIFY_PEER)
+                mode = MBEDTLS_SSL_VERIFY_OPTIONAL;
+            else if (verify_mode_ == SSL_VERIFY_NONE)
+                mode = MBEDTLS_SSL_VERIFY_NONE;
+        }
+        return mode;
+    }
+
+    struct impl {
+        static void print_error(const char* function, int error_code)
+        {
+            constexpr const char *TAG="mbedtls-engine-impl";
+            ESP_LOGE(TAG, "%s() returned -0x%04X", function, -error_code);
+            ESP_LOGI(TAG, "-0x%04X: %s", -error_code, error_message(error_code));
+        }
+
+        bool before_handshake() const
+        {
+            return ssl_.MBEDTLS_PRIVATE(state) == 0;
+        }
+
+        int write(const void *buffer, int len)
+        {
+            int ret = mbedtls_ssl_write(&ssl_, static_cast<const unsigned char *>(buffer), len);
+            if (ret < 0 && ret != MBEDTLS_ERR_SSL_WANT_WRITE) {
+                print_error("mbedtls_ssl_write", ret);
+            }
+            return ret;
+        }
+
+        int read(void *buffer, int len)
+        {
+            int ret = mbedtls_ssl_read(&ssl_, static_cast<unsigned char *>(buffer), len);
+            if (ret < 0 && ret != MBEDTLS_ERR_SSL_WANT_READ) {
+                print_error("mbedtls_ssl_read", ret);
+            }
+            return ret;
+        }
+
+        impl()
+        {
+            const unsigned char pers[] = "asio ssl";
+            mbedtls_ssl_init(&ssl_);
+            mbedtls_ssl_config_init(&conf_);
+            mbedtls_ctr_drbg_init(&ctr_drbg_);
+#ifdef CONFIG_MBEDTLS_DEBUG
+            mbedtls_esp_enable_debug_log(&conf_, CONFIG_MBEDTLS_DEBUG_LEVEL);
+#endif
+            mbedtls_entropy_init(&entropy_);
+            mbedtls_ctr_drbg_seed(&ctr_drbg_, mbedtls_entropy_func, &entropy_, pers, sizeof(pers));
+            mbedtls_x509_crt_init(&public_cert_);
+            mbedtls_pk_init(&pk_key_);
+            mbedtls_x509_crt_init(&ca_cert_);
+        }
+
+        bool configure(context *ctx, bool is_client_not_server, int mbedtls_verify_mode)
+        {
+            mbedtls_x509_crt_init(&public_cert_);
+            mbedtls_pk_init(&pk_key_);
+            mbedtls_x509_crt_init(&ca_cert_);
+            int ret = mbedtls_ssl_config_defaults(&conf_, is_client_not_server ? MBEDTLS_SSL_IS_CLIENT: MBEDTLS_SSL_IS_SERVER,
+                                                  MBEDTLS_SSL_TRANSPORT_STREAM, MBEDTLS_SSL_PRESET_DEFAULT);
+            if (ret) {
+                print_error("mbedtls_ssl_config_defaults", ret);
+                return false;
+            }
+            mbedtls_ssl_conf_rng(&conf_, mbedtls_ctr_drbg_random, &ctr_drbg_);
+            mbedtls_ssl_conf_authmode(&conf_, mbedtls_verify_mode);
+            if (ctx->cert_chain_.size() > 0 && ctx->private_key_.size() > 0) {
+                ret = mbedtls_x509_crt_parse(&public_cert_, ctx->data(container::CERT), ctx->size(container::CERT));
+                if (ret < 0) {
+                    print_error("mbedtls_x509_crt_parse", ret);
+                    return false;
+                }
+                ret = mbedtls_pk_parse_key(&pk_key_, ctx->data(container::PRIVKEY), ctx->size(container::PRIVKEY),
+                                           nullptr, 0, mbedtls_ctr_drbg_random, &ctr_drbg_);
+                if (ret < 0) {
+                    print_error("mbedtls_pk_parse_keyfile", ret);
+                    return false;
+                }
+                ret = mbedtls_ssl_conf_own_cert(&conf_, &public_cert_, &pk_key_);
+                if (ret) {
+                    print_error("mbedtls_ssl_conf_own_cert", ret);
+                    return false;
+                }
+            }
+
+            if (ctx->ca_cert_.size() > 0) {
+                ret = mbedtls_x509_crt_parse(&ca_cert_, ctx->data(container::CA_CERT), ctx->size(container::CA_CERT));
+                if (ret < 0) {
+                    print_error("mbedtls_x509_crt_parse", ret);
+                    return false;
+                }
+                mbedtls_ssl_conf_ca_chain(&conf_, &ca_cert_, nullptr);
+            } else {
+                mbedtls_ssl_conf_ca_chain(&conf_, nullptr, nullptr);
+            }
+            ret = mbedtls_ssl_setup(&ssl_, &conf_);
+            if (ret) {
+                print_error("mbedtls_ssl_setup", ret);
+                return false;
+            }
+            return true;
+        }
+        mbedtls_ssl_context ssl_{};
+        mbedtls_entropy_context entropy_{};
+        mbedtls_ctr_drbg_context ctr_drbg_{};
+        mbedtls_ssl_config conf_{};
+        mbedtls_x509_crt public_cert_{};
+        mbedtls_pk_context pk_key_{};
+        mbedtls_x509_crt ca_cert_{};
+    };
+
+    impl impl_{};
+    std::shared_ptr<context> ctx_;
+    std::pair<std::shared_ptr<bio>, std::shared_ptr<bio>> bio_;
+    enum rw_state state_;
+    asio::ssl::verify_mode verify_mode_;
+};
+
+} } } // namespace asio::ssl::mbedtls
--- a/components/asio/port/mbedtls/include/mbedtls_error.hpp
+++ b/components/asio/port/mbedtls/include/mbedtls_error.hpp
@@ -0,0 +1,55 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
+
+#include "asio/detail/config.hpp"
+#include "asio/ssl/error.hpp"
+#include "asio/ssl/detail/openssl_init.hpp"
+#include "mbedtls_context.hpp"
+
+namespace asio {
+namespace error {
+namespace detail {
+
+class mbedtls_category : public asio::error_category
+{
+public:
+    const char* name() const ASIO_ERROR_CATEGORY_NOEXCEPT
+    {
+        return "asio.ssl";
+    }
+
+    std::string message(int value) const
+    {
+        const char* s = asio::ssl::mbedtls::error_message(value);
+        return s ? s : "asio.mbedtls error";
+    }
+};
+
+} // namespace detail
+
+const asio::error_category& get_mbedtls_category()
+{
+    static detail::mbedtls_category instance;
+    return instance;
+}
+
+const asio::error_category& get_ssl_category()
+{
+    return asio::error::get_mbedtls_category();
+}
+
+} // namespace error
+
+namespace ssl {
+namespace error {
+
+const asio::error_category& get_stream_category()
+{
+    return asio::error::get_mbedtls_category();
+}
+
+} } } // namespace asio::ssl::error
--- a/components/asio/port/mbedtls/include/openssl/conf.h
+++ b/components/asio/port/mbedtls/include/openssl/conf.h
@@ -0,0 +1,6 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
--- a/components/asio/port/mbedtls/include/openssl/dh.h
+++ b/components/asio/port/mbedtls/include/openssl/dh.h
@@ -0,0 +1,6 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
--- a/components/asio/port/mbedtls/include/openssl/err.h
+++ b/components/asio/port/mbedtls/include/openssl/err.h
@@ -0,0 +1,6 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
--- a/components/asio/port/mbedtls/include/openssl/rsa.h
+++ b/components/asio/port/mbedtls/include/openssl/rsa.h
@@ -0,0 +1,6 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
--- a/components/asio/port/mbedtls/include/openssl/ssl.h
+++ b/components/asio/port/mbedtls/include/openssl/ssl.h
@@ -0,0 +1,8 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
+
+#include "openssl_stub.hpp"
--- a/components/asio/port/mbedtls/include/openssl/x509v3.h
+++ b/components/asio/port/mbedtls/include/openssl/x509v3.h
@@ -0,0 +1,6 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+#pragma once
--- a/components/asio/port/mbedtls/src/mbedtls_context.cpp
+++ b/components/asio/port/mbedtls/src/mbedtls_context.cpp
@@ -0,0 +1,115 @@
+//
+// SPDX-FileCopyrightText: 2005 Voipster / Indrek dot Juhani at voipster dot com
+// SPDX-FileCopyrightText: 2005-2019 Christopher M. Kohlhoff (chris at kohlhoff dot com)
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+// SPDX-FileContributor: 2021 Espressif Systems (Shanghai) CO LTD
+//
+
+#include "asio/detail/config.hpp"
+#include "openssl_stub.hpp"
+#include <cstring>
+#include "asio/detail/throw_error.hpp"
+#include "asio/error.hpp"
+#include "asio/ssl/context.hpp"
+#include "asio/ssl/error.hpp"
+#include "mbedtls_context.hpp"
+
+
+namespace asio {
+namespace ssl {
+
+
+context::context(context::method m)
+        : handle_(0)
+{
+    handle_ = mbedtls::shared_ctx::create("mbedtls-context", m);
+    set_options(no_compression);
+}
+
+context::context(context&& other)
+{
+    handle_ = other.handle_;
+    other.handle_ = 0;
+}
+
+context& context::operator=(context&& other)
+{
+    context tmp(ASIO_MOVE_CAST(context)(*this));
+    handle_ = other.handle_;
+    other.handle_ = 0;
+    return *this;
+}
+
+context::~context()
+{
+    delete handle_;
+}
+
+context::native_handle_type context::native_handle()
+{
+    return handle_;
+}
+
+void context::set_options(context::options o)
+{
+    asio::error_code ec;
+    set_options(o, ec);
+    asio::detail::throw_error(ec, "set_options");
+}
+
+ASIO_SYNC_OP_VOID context::set_options(
+        context::options o, asio::error_code& ec)
+{
+    handle_->get()->options_ = o;
+    ec = asio::error_code();
+    ASIO_SYNC_OP_VOID_RETURN(ec);
+}
+
+void context::add_certificate_authority(const const_buffer& ca)
+{
+    asio::error_code ec;
+    add_certificate_authority(ca, ec);
+    asio::detail::throw_error(ec, "add_certificate_authority");
+}
+
+ASIO_SYNC_OP_VOID context::add_certificate_authority(
+        const const_buffer& ca, asio::error_code& ec)
+{
+    handle_->get()->ca_cert_ = ca;
+    ASIO_SYNC_OP_VOID_RETURN(asio::error_code());
+}
+
+void context::use_certificate_chain(const const_buffer& chain)
+{
+    asio::error_code ec;
+    use_certificate_chain(chain, ec);
+    asio::detail::throw_error(ec, "use_certificate_chain");
+}
+
+ASIO_SYNC_OP_VOID context::use_certificate_chain(
+        const const_buffer& chain, asio::error_code& ec)
+{
+    handle_->get()->cert_chain_ = chain;
+    ASIO_SYNC_OP_VOID_RETURN(asio::error_code());
+}
+
+void context::use_private_key(
+        const const_buffer& private_key, context::file_format format)
+{
+    asio::error_code ec;
+    use_private_key(private_key, format, ec);
+    asio::detail::throw_error(ec, "use_private_key");
+}
+
+ASIO_SYNC_OP_VOID context::use_private_key(
+        const const_buffer& private_key, context::file_format format,
+        asio::error_code& ec)
+{
+    handle_->get()->private_key_ = private_key;
+    ASIO_SYNC_OP_VOID_RETURN(asio::error_code());
+}
+
+} // namespace ssl
+} // namespace asio
--- a/components/asio/port/mbedtls/src/mbedtls_engine.cpp
+++ b/components/asio/port/mbedtls/src/mbedtls_engine.cpp
@@ -0,0 +1,208 @@
+//
+// SPDX-FileCopyrightText: 2003-2019 Christopher M. Kohlhoff (chris at kohlhoff dot com)
+//
+// SPDX-License-Identifier: BSL-1.0
+//
+// SPDX-FileContributor: 2021 Espressif Systems (Shanghai) CO LTD
+//
+
+#include "asio/detail/config.hpp"
+#include "openssl_stub.hpp"
+#include "asio/detail/throw_error.hpp"
+#include "asio/error.hpp"
+#include "asio/ssl/detail/engine.hpp"
+#include "asio/ssl/error.hpp"
+#include "asio/ssl/verify_context.hpp"
+#include "mbedtls_context.hpp"
+#include "mbedtls_bio.hpp"
+#include "mbedtls_error.hpp"
+#include "mbedtls_engine.hpp"
+
+
+namespace asio {
+namespace ssl {
+namespace detail {
+
+
+engine::engine(SSL_CTX* context)
+        : ssl_(nullptr)
+{
+    ssl_ = mbedtls::create<mbedtls::engine>("mbedtls-engine", context->get());
+}
+
+engine::~engine()
+{
+    delete ssl_;
+}
+
+SSL* engine::native_handle()
+{
+    return ssl_;
+}
+
+asio::error_code engine::set_verify_mode(
+        verify_mode v, asio::error_code& ec)
+{
+    ssl_->set_verify_mode(v);
+    return {};
+}
+
+engine::want engine::handshake(
+        stream_base::handshake_type type, asio::error_code& ec)
+{
+    return perform((type == asio::ssl::stream_base::client)
+                   ? &engine::do_connect : &engine::do_accept, 0, 0, ec, 0);
+}
+
+engine::want engine::shutdown(asio::error_code& ec)
+{
+    return perform(&engine::do_shutdown, 0, 0, ec, 0);
+}
+
+engine::want engine::write(const asio::const_buffer& data,
+                           asio::error_code& ec, std::size_t& bytes_transferred)
+{
+    if (data.size() == 0)
+    {
+        ec = asio::error_code();
+        return engine::want_nothing;
+    }
+
+    return perform(&engine::do_write,
+                   const_cast<void*>(data.data()),
+                   data.size(), ec, &bytes_transferred);
+}
+
+engine::want engine::read(const asio::mutable_buffer& data,
+                          asio::error_code& ec, std::size_t& bytes_transferred)
+{
+    if (data.size() == 0)
+    {
+        ec = asio::error_code();
+        return engine::want_nothing;
+    }
+
+    return perform(&engine::do_read, data.data(),
+                   data.size(), ec, &bytes_transferred);
+}
+
+asio::mutable_buffer engine::get_output(
+        const asio::mutable_buffer& data)
+{
+    int length = ssl_->ext_bio()->read(data.data(), static_cast<int>(data.size()));
+
+    return asio::buffer(data,
+                        length > 0 ? static_cast<std::size_t>(length) : 0);
+}
+
+asio::const_buffer engine::put_input(
+        const asio::const_buffer& data)
+{
+    int length = ssl_->ext_bio()->write(data.data(), static_cast<int>(data.size()));
+
+    return asio::buffer(data +
+                        (length > 0 ? static_cast<std::size_t>(length) : 0));
+}
+
+const asio::error_code& engine::map_error_code(
+        asio::error_code& ec) const
+{
+    // We only want to map the error::eof code.
+    if (ec != asio::error::eof)
+        return ec;
+
+    // If there's data yet to be read, it's an error.
+    if (ssl_->ext_bio()->wpending())
+    {
+        ec = asio::ssl::error::stream_truncated;
+        return ec;
+    }
+
+    // Otherwise, the peer should have negotiated a proper shutdown.
+    if (ssl_->shutdown() != 0)
+    {
+        ec = asio::ssl::error::stream_truncated;
+    }
+
+    return ec;
+}
+
+// This is a simplified implementation of a generic ssl io operation
+// original implementation using openssl's SSL object is in asio/include/asio/ssl/detail/impl/engine.ipp
+engine::want engine::perform(int (engine::* op)(void*, std::size_t),
+                             void* data, std::size_t length, asio::error_code& ec,
+                             std::size_t* bytes_transferred)
+{
+    std::size_t pending_output_before = ssl_->ext_bio()->ctrl_pending();
+    int result = (this->*op)(data, length);
+
+    std::size_t pending_output_after = ssl_->ext_bio()->ctrl_pending();
+
+    if (mbedtls::error_codes::is_error(result))
+    {
+        ec = asio::error_code(result, asio::error::get_mbedtls_category());
+        return pending_output_after > pending_output_before ? want_output : want_nothing;
+    }
+
+    if (result == 0)
+    {
+        return pending_output_after > pending_output_before
+               ? want_output : want_nothing;
+    }
+
+    if (result > 0 && bytes_transferred)
+        *bytes_transferred = static_cast<std::size_t>(result);
+
+    if (mbedtls::error_codes::want_write(result))
+    {
+        ec = asio::error_code();
+        return want_output_and_retry;
+    }
+    else if (pending_output_after > pending_output_before)
+    {
+        ec = asio::error_code();
+        return result > 0 ? want_output : want_output_and_retry;
+    }
+    else if (mbedtls::error_codes::want_read(result))
+    {
+        ec = asio::error_code();
+        return want_input_and_retry;
+    }
+    else if (ssl_->get_state() == mbedtls::CLOSED)
+    {
+        ec = asio::error::eof;
+        return want_nothing;
+    }
+
+    ec = asio::error_code();
+    return want_nothing;
+}
+
+int engine::do_accept(void*, std::size_t)
+{
+    return ssl_->accept();
+}
+
+int engine::do_connect(void*, std::size_t)
+{
+    return ssl_->connect();
+}
+
+int engine::do_shutdown(void*, std::size_t)
+{
+    return ssl_->shutdown();
+}
+
+int engine::do_read(void* data, std::size_t length)
+{
+    return ssl_->read(data, length < INT_MAX ? static_cast<int>(length) : INT_MAX);
+}
+
+int engine::do_write(void* data, std::size_t length)
+{
+    return ssl_->write(data, length < INT_MAX ? static_cast<int>(length) : INT_MAX);
+}
+
+} // namespace detail
+} // namespace ssl
+} // namespace asio
--- a/components/asio/port/src/asio_ssl_impl.cpp
+++ b/components/asio/port/src/asio_ssl_impl.cpp
@@ -0,0 +1,20 @@
+//
+// SPDX-FileCopyrightText: 2021 Espressif Systems (Shanghai) CO LTD
+//
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#include "asio/detail/config.hpp"
+#include "asio/ssl/detail/openssl_init.hpp"
+
+namespace asio {
+namespace ssl {
+namespace detail {
+
+// No OpenSSL in this implementation, instance is nullptr
+asio::detail::shared_ptr<openssl_init_base::do_init> openssl_init_base::instance()
+{
+    return nullptr;
+}
+
+} } } // namespace asio::ssl::detail
--- a/components/bootloader/Kconfig.projbuild
+++ b/components/bootloader/Kconfig.projbuild
@@ -323,7 +323,7 @@ menu "Bootloader config"
        # options, allowing to turn on "allow insecure options" and have secure boot with
        # "skip validation when existing deep sleep". Keeping this to avoid a breaking change,
        # but - as noted in help - it invalidates the integrity of Secure Boot checks
-        depends on SOC_RTC_FAST_MEM_SUPPORTED && ((SECURE_BOOT && SECURE_BOOT_INSECURE) || !SECURE_BOOT)
+        depends on (SECURE_BOOT && SECURE_BOOT_INSECURE) || !SECURE_BOOT
        default n
        help
            This option disables the normal validation of an image coming out of
@@ -379,7 +379,6 @@ menu "Bootloader config"

    config BOOTLOADER_RESERVE_RTC_SIZE
        hex
-        depends on SOC_RTC_FAST_MEM_SUPPORTED
        default 0x10 if BOOTLOADER_SKIP_VALIDATE_IN_DEEP_SLEEP || BOOTLOADER_CUSTOM_RESERVE_RTC
        default 0
        help
@@ -391,7 +390,6 @@ menu "Bootloader config"

    config BOOTLOADER_CUSTOM_RESERVE_RTC
        bool "Reserve RTC FAST memory for custom purposes"
-        depends on SOC_RTC_FAST_MEM_SUPPORTED
        default n
        help
            This option allows the customer to place data in the RTC FAST memory,
@@ -476,11 +474,6 @@ menu "Security features"
        bool
        default y if SECURE_BOOT_V2_ENABLED && SECURE_BOOT_V2_RSA_SUPPORTED

-    config SECURE_BOOT_FLASH_ENC_KEYS_BURN_TOGETHER
-        bool
-        default y if SOC_EFUSE_CONSISTS_OF_ONE_KEY_BLOCK && SECURE_BOOT && SECURE_FLASH_ENC_ENABLED
-        # ESP32-C2 has one key block for SB and FE keys. These keys must be burned at the same time.
-
    config SECURE_SIGNED_APPS_NO_SECURE_BOOT
        bool "Require signed app images"
        depends on !SECURE_BOOT
@@ -774,29 +767,21 @@ menu "Security features"
    choice SECURE_FLASH_ENCRYPTION_KEYSIZE
        bool "Size of generated AES-XTS key"
        default SECURE_FLASH_ENCRYPTION_AES128
-        depends on SOC_FLASH_ENCRYPTION_XTS_AES_OPTIONS && SECURE_FLASH_ENC_ENABLED
+        depends on SOC_FLASH_ENCRYPTION_XTS_AES_256 && SECURE_FLASH_ENC_ENABLED
        help
            Size of generated AES-XTS key.

-            - AES-128 uses a 256-bit key (32 bytes) derived from 128 bits (16 bytes) burned in half Efuse key block.
-              Internally, it calculates SHA256(128 bits)
-            - AES-128 uses a 256-bit key (32 bytes) which occupies one Efuse key block.
-            - AES-256 uses a 512-bit key (64 bytes) which occupies two Efuse key blocks.
+            AES-128 uses a 256-bit key (32 bytes) which occupies one Efuse key block.
+            AES-256 uses a 512-bit key (64 bytes) which occupies two Efuse key blocks.

            This setting is ignored if either type of key is already burned to Efuse before the first boot.
            In this case, the pre-burned key is used and no new key is generated.

-        config SECURE_FLASH_ENCRYPTION_AES128_DERIVED
-            bool "AES-128 key derived from 128 bits (SHA256(128 bits))"
-            depends on SOC_FLASH_ENCRYPTION_XTS_AES_128_DERIVED
-
        config SECURE_FLASH_ENCRYPTION_AES128
            bool "AES-128 (256-bit key)"
-            depends on SOC_FLASH_ENCRYPTION_XTS_AES_128 && !(IDF_TARGET_ESP32C2 && SECURE_BOOT)

        config SECURE_FLASH_ENCRYPTION_AES256
            bool "AES-256 (512-bit key)"
-            depends on SOC_FLASH_ENCRYPTION_XTS_AES_256
    endchoice

    choice SECURE_FLASH_ENCRYPTION_MODE
@@ -820,12 +805,12 @@ menu "Security features"

        config SECURE_FLASH_ENCRYPTION_MODE_RELEASE
            bool "Release"
-            select PARTITION_TABLE_MD5 if !APP_COMPATIBLE_PRE_V3_1_BOOTLOADERS
+            select PARTITION_TABLE_MD5 if !ESP32_COMPATIBLE_PRE_V3_1_BOOTLOADERS

    endchoice

    menu "Potentially insecure options"
-        visible if SECURE_FLASH_ENCRYPTION_MODE_DEVELOPMENT || SECURE_BOOT_INSECURE || SECURE_SIGNED_ON_UPDATE_NO_SECURE_BOOT # NOERROR
+        visible if SECURE_FLASH_ENCRYPTION_MODE_DEVELOPMENT || SECURE_BOOT_INSECURE

        # NOTE: Options in this menu NEED to have SECURE_BOOT_INSECURE
        # and/or SECURE_FLASH_ENCRYPTION_MODE_DEVELOPMENT in "depends on", as the menu
@@ -862,7 +847,7 @@ menu "Security features"

        config SECURE_BOOT_ALLOW_SHORT_APP_PARTITION
            bool "Allow app partition length not 64KB aligned"
-            depends on SECURE_BOOT_INSECURE || SECURE_SIGNED_ON_UPDATE_NO_SECURE_BOOT
+            depends on SECURE_BOOT_INSECURE
            help
                If not set (default), app partition size must be a multiple of 64KB. App images are padded to 64KB
                length, and the bootloader checks any trailing bytes after the signature (before the next 64KB
--- a/components/bootloader/subproject/CMakeLists.txt
+++ b/components/bootloader/subproject/CMakeLists.txt
@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 3.16)
+cmake_minimum_required(VERSION 3.5)

 if(NOT SDKCONFIG)
    message(FATAL_ERROR "Bootloader subproject expects the SDKCONFIG variable to be passed "
--- a/components/bootloader/subproject/main/bootloader_start.c
+++ b/components/bootloader/subproject/main/bootloader_start.c
@@ -5,7 +5,6 @@
 */
 #include <stdbool.h>
 #include "esp_log.h"
-#include "esp_rom_sys.h"
 #include "bootloader_init.h"
 #include "bootloader_utility.h"
 #include "bootloader_common.h"
@@ -80,7 +79,7 @@ static int selected_boot_partition(const bootloader_state_t *bs)
    if (boot_index == INVALID_INDEX) {
        return boot_index; // Unrecoverable failure (not due to corrupt ota data or bad partition contents)
    }
-    if (esp_rom_get_reset_reason(0) != RESET_REASON_CORE_DEEP_SLEEP) {
+    if (bootloader_common_get_reset_reason(0) != DEEPSLEEP_RESET) {
        // Factory firmware.
 #ifdef CONFIG_BOOTLOADER_FACTORY_RESET
        bool reset_level = false;
--- a/components/bootloader/subproject/main/ld/esp32/bootloader.ld
+++ b/components/bootloader/subproject/main/ld/esp32/bootloader.ld
@@ -119,6 +119,8 @@ SECTIONS
    *(.sdata)
    *(.sdata.*)
    *(.gnu.linkonce.s.*)
+    *(.sdata2)
+    *(.sdata2.*)
    *(.gnu.linkonce.s2.*)
    *(.jcr)
    _data_end = ABSOLUTE(.);
@@ -131,7 +133,6 @@ SECTIONS
    *(.rodata.*)
    *(.gnu.linkonce.r.*)
    *(.rodata1)
-    *(.sdata2 .sdata2.*)
    __XT_EXCEPTION_TABLE_ = ABSOLUTE(.);
    *(.xt_except_table)
    *(.gcc_except_table)
@@ -190,23 +191,4 @@ SECTIONS
    _etext = .;
  } > iram_seg

-  /** This section will be used by the debugger and disassembler to get more information
-   * about raw data present in the code.
-   * Indeed, it may be required to add some padding at some points in the code
-   * in order to align a branch/jump destination on a particular bound.
-   * Padding these instructions will generate null bytes that shall be
-   * interpreted as data, and not code by the debugger or disassembler.
-   * This section will only be present in the ELF file, not in the final binary
-   * For more details, check GCC-212
-   */
-  .xt.prop 0 :
-  {
-    KEEP (*(.xt.prop .gnu.linkonce.prop.*))
-  }
-
-  .xt.lit 0 :
-  {
-    KEEP (*(.xt.lit .gnu.linkonce.p.*))
-  }
-
 }
--- a/components/bootloader/subproject/main/ld/esp32c2/bootloader.ld
+++ b/components/bootloader/subproject/main/ld/esp32c2/bootloader.ld
@@ -5,41 +5,15 @@
 */
 /** Simplified memory map for the bootloader.
 *  Make sure the bootloader can load into main memory without overwriting itself.
- *
- *  ESP32-C2 ROM static data usage is as follows:
- *  - 0x3fccb264 - 0x3fcdcb70: Shared buffers, used in UART/USB/SPI download mode only
- *  - 0x3fcdcb70 - 0x3fcdeb70: PRO CPU stack, can be reclaimed as heap after RTOS startup
- *  - 0x3fcdeb70 - 0x3fce0000: ROM .bss and .data (not easily reclaimable)
- *
- *  The 2nd stage bootloader can take space up to the end of ROM shared
- *  buffers area (0x3fcdcb70).
+ *  We put 2nd bootloader in the high address space (before ROM stack/data/bss).
+ *  See memory usage for ROM bootloader at the end of this file.
 */

-/* The offset between Dbus and Ibus. Used to convert between 0x403xxxxx and 0x3fcxxxxx addresses. */
-iram_dram_offset = 0x6e0000;
-
-/* We consider 0x3fcdcb70 to be the last usable address for 2nd stage bootloader stack overhead, dram_seg,
- * and work out iram_seg and iram_loader_seg addresses from there, backwards.
- */
-
-/* These lengths can be adjusted, if necessary: */
-bootloader_usable_dram_end = 0x3fcdcb70;
-bootloader_stack_overhead = 0x2000; /* For safety margin between bootloader data section and startup stacks */
-bootloader_dram_seg_len = 0x5000;
-bootloader_iram_loader_seg_len = 0x7000;
-bootloader_iram_seg_len = 0x2000;
-
-/* Start of the lower region is determined by region size and the end of the higher region */
-bootloader_dram_seg_end = bootloader_usable_dram_end - bootloader_stack_overhead;
-bootloader_dram_seg_start = bootloader_dram_seg_end - bootloader_dram_seg_len;
-bootloader_iram_loader_seg_start = bootloader_dram_seg_start - bootloader_iram_loader_seg_len + iram_dram_offset;
-bootloader_iram_seg_start = bootloader_iram_loader_seg_start - bootloader_iram_seg_len;
-
 MEMORY
 {
-  iram_seg (RWX) :                  org = bootloader_iram_seg_start, len = bootloader_iram_seg_len
-  iram_loader_seg (RWX) :           org = bootloader_iram_loader_seg_start, len = bootloader_iram_loader_seg_len
-  dram_seg (RW) :                   org = bootloader_dram_seg_start, len = bootloader_dram_seg_len
+  iram_seg (RWX) :                  org = 0x403AE000, len = 0x2000
+  iram_loader_seg (RWX) :           org = 0x403B0000, len = 0x6000
+  dram_seg (RW) :                   org = 0x3FCD6000, len = 0x4000
 }

 /* Default entry point: */
@@ -133,6 +107,8 @@ SECTIONS
    *(.sdata)
    *(.sdata.*)
    *(.gnu.linkonce.s.*)
+    *(.sdata2)
+    *(.sdata2.*)
    *(.gnu.linkonce.s2.*)
    *(.jcr)
    _data_end = ABSOLUTE(.);
@@ -145,7 +121,6 @@ SECTIONS
    *(.rodata.*)
    *(.gnu.linkonce.r.*)
    *(.rodata1)
-    *(.sdata2 .sdata2.* .srodata .srodata.*)
    __XT_EXCEPTION_TABLE_ = ABSOLUTE(.);
    *(.xt_except_table)
    *(.gcc_except_table)
@@ -209,34 +184,17 @@ SECTIONS
 /**
 *  Appendix: Memory Usage of ROM bootloader
 *
- * 0x3fccb264 ------------------> _dram0_0_start
- *            |               |
- *            |               |
- *            |               |   1. Large buffers that are only used in certain boot modes, see shared_buffers.h
- *            |               |
- *            |               |
- * 0x3fcdcb70 ------------------> __stack_sentry
- *            |               |
- *            |               |   2. Startup pro cpu stack (freed when IDF app is running)
- *            |               |
- * 0x3fcdeb70 ------------------> __stack (pro cpu)
- *            |               |
- *            |               |
- *            |               |   3. Shared memory only used in startup code or nonos/early boot*
- *            |               |      (can be freed when IDF runs)
- *            |               |
- *            |               |
- * 0x3fcdf4bc ------------------> _dram0_rtos_reserved_start
- *            |               |
- *            |               |
- *            |               |   4. Shared memory used in startup code and when IDF runs
- *            |               |
- *            |               |
- * 0x3fcdfa70 ------------------> _dram0_rtos_reserved_end
- *            |               |
- * 0x3fcdfa74 ------------------> _data_start_interface
- *            |               |
- *            |               |   5. End of DRAM is the 'interface' data with constant addresses (ECO compatible)
- *            |               |
- * 0x3fce0000 ------------------> _data_end_interface
+ *  +--------+--------------+------+ 0x3FCC_B0B0
+ *  |        ^              |
+ *  |        |              |
+ *  |        | data/bss     |
+ *  |        |              |
+ *  |        v              |
+ *  +------------------------------+ 0x3FCD_C9C0
+ *  |        ^              |
+ *  |        |              |
+ *  |        | stack        |
+ *  |        |              |
+ *  |        v              |
+ *  +------------------------------+ 0x3FCD_E9C0
 */
--- a/components/bootloader/subproject/main/ld/esp32c3/bootloader.ld
+++ b/components/bootloader/subproject/main/ld/esp32c3/bootloader.ld
@@ -5,41 +5,15 @@
 */
 /** Simplified memory map for the bootloader.
 *  Make sure the bootloader can load into main memory without overwriting itself.
- *
- *  ESP32-C3 ROM static data usage is as follows:
- *  - 0x3fccae00 - 0x3fcdc710: Shared buffers, used in UART/USB/SPI download mode only
- *  - 0x3fcdc710 - 0x3fcde710: PRO CPU stack, can be reclaimed as heap after RTOS startup
- *  - 0x3fcde710 - 0x3fce0000: ROM .bss and .data (not easily reclaimable)
- *
- *  The 2nd stage bootloader can take space up to the end of ROM shared
- *  buffers area (0x3fcdc710).
+ *  We put 2nd bootloader in the high address space (before ROM stack/data/bss).
+ *  See memory usage for ROM bootloader at the end of this file.
 */

-/* The offset between Dbus and Ibus. Used to convert between 0x403xxxxx and 0x3fcxxxxx addresses. */
-iram_dram_offset = 0x700000;
-
-/* We consider 0x3fcdc710 to be the last usable address for 2nd stage bootloader stack overhead, dram_seg,
- * and work out iram_seg and iram_loader_seg addresses from there, backwards.
- */
-
-/* These lengths can be adjusted, if necessary: */
-bootloader_usable_dram_end = 0x3fcdc710;
-bootloader_stack_overhead = 0x2000; /* For safety margin between bootloader data section and startup stacks */
-bootloader_dram_seg_len = 0x5000;
-bootloader_iram_loader_seg_len = 0x7000;
-bootloader_iram_seg_len = 0x2000;
-
-/* Start of the lower region is determined by region size and the end of the higher region */
-bootloader_dram_seg_end = bootloader_usable_dram_end - bootloader_stack_overhead;
-bootloader_dram_seg_start = bootloader_dram_seg_end - bootloader_dram_seg_len;
-bootloader_iram_loader_seg_start = bootloader_dram_seg_start - bootloader_iram_loader_seg_len + iram_dram_offset;
-bootloader_iram_seg_start = bootloader_iram_loader_seg_start - bootloader_iram_seg_len;
-
 MEMORY
 {
-  iram_seg (RWX) :                  org = bootloader_iram_seg_start, len = bootloader_iram_seg_len
-  iram_loader_seg (RWX) :           org = bootloader_iram_loader_seg_start, len = bootloader_iram_loader_seg_len
-  dram_seg (RW) :                   org = bootloader_dram_seg_start, len = bootloader_dram_seg_len
+  iram_seg (RWX) :                  org = 0x403CE000, len = 0x1600
+  iram_loader_seg (RWX) :           org = 0x403CF600, len = 0x6A00
+  dram_seg (RW) :                   org = 0x3FCD6000, len = 0x4000
 }

 /* Default entry point: */
@@ -133,6 +107,8 @@ SECTIONS
    *(.sdata)
    *(.sdata.*)
    *(.gnu.linkonce.s.*)
+    *(.sdata2)
+    *(.sdata2.*)
    *(.gnu.linkonce.s2.*)
    *(.jcr)
    _data_end = ABSOLUTE(.);
@@ -145,7 +121,6 @@ SECTIONS
    *(.rodata.*)
    *(.gnu.linkonce.r.*)
    *(.rodata1)
-    *(.sdata2 .sdata2.* .srodata .srodata.*)
    __XT_EXCEPTION_TABLE_ = ABSOLUTE(.);
    *(.xt_except_table)
    *(.gcc_except_table)
@@ -210,34 +185,17 @@ SECTIONS
 /**
 *  Appendix: Memory Usage of ROM bootloader
 *
- * 0x3fccae00 ------------------> _dram0_0_start
- *            |               |
- *            |               |
- *            |               |   1. Large buffers that are only used in certain boot modes, see shared_buffers.h
- *            |               |
- *            |               |
- * 0x3fcdc710 ------------------> __stack_sentry
- *            |               |
- *            |               |   2. Startup pro cpu stack (freed when IDF app is running)
- *            |               |
- * 0x3fcde710 ------------------> __stack (pro cpu)
- *            |               |
- *            |               |
- *            |               |   3. Shared memory only used in startup code or nonos/early boot*
- *            |               |      (can be freed when IDF runs)
- *            |               |
- *            |               |
- * 0x3fcdf060 ------------------> _dram0_rtos_reserved_start
- *            |               |
- *            |               |
- *            |               |   4. Shared memory used in startup code and when IDF runs
- *            |               |
- *            |               |
- * 0x3fcdf664 ------------------> _dram0_rtos_reserved_end
- *            |               |
- * 0x3fcdf830 ------------------> _data_start_interface
- *            |               |
- *            |               |   5. End of DRAM is the 'interface' data with constant addresses (ECO compatible)
- *            |               |
- * 0x3fce0000 ------------------> _data_end_interface
+ *  +--------+--------------+------+ 0x3FCC_AE00
+ *  |        ^              |
+ *  |        |              |
+ *  |        | data/bss     |
+ *  |        |              |
+ *  |        v              |
+ *  +------------------------------+ 0x3FCD_C710
+ *  |        ^              |
+ *  |        |              |
+ *  |        | stack        |
+ *  |        |              |
+ *  |        v              |
+ *  +------------------------------+ 0x3FCD_E710
 */
--- a/components/bootloader/subproject/main/ld/esp32h2/bootloader.ld
+++ b/components/bootloader/subproject/main/ld/esp32h2/bootloader.ld
@@ -5,41 +5,15 @@
 */
 /** Simplified memory map for the bootloader.
 *  Make sure the bootloader can load into main memory without overwriting itself.
- *
- *  ESP32-H2 ROM static data usage is as follows:
- *  - 0x3fccb900 - 0x3fcdd210: Shared buffers, used in UART/USB/SPI download mode only
- *  - 0x3fcdd210 - 0x3fcdf210: PRO CPU stack, can be reclaimed as heap after RTOS startup
- *  - 0x3fcdf210 - 0x3fce0000: ROM .bss and .data (not easily reclaimable)
- *
- *  The 2nd stage bootloader can take space up to the end of ROM shared
- *  buffers area (0x3fce9704). For alignment purpose we shall use value (0x3fce9700).
+ *  We put 2nd bootloader in the high address space (before ROM stack/data/bss).
+ *  See memory usage for ROM bootloader at the end of this file.
 */

-/* The offset between Dbus and Ibus. Used to convert between 0x403xxxxx and 0x3fcxxxxx addresses. */
-iram_dram_offset = 0x700000;
-
-/* We consider 0x3fce9700 to be the last usable address for 2nd stage bootloader stack overhead, dram_seg,
- * and work out iram_seg and iram_loader_seg addresses from there, backwards.
- */
-
-/* These lengths can be adjusted, if necessary: */
-bootloader_usable_dram_end = 0x3fcdd120;
-bootloader_stack_overhead = 0x2000; /* For safety margin between bootloader data section and startup stacks */
-bootloader_dram_seg_len = 0x5000;
-bootloader_iram_loader_seg_len = 0x7000;
-bootloader_iram_seg_len = 0x2000;
-
-/* Start of the lower region is determined by region size and the end of the higher region */
-bootloader_dram_seg_end = bootloader_usable_dram_end - bootloader_stack_overhead;
-bootloader_dram_seg_start = bootloader_dram_seg_end - bootloader_dram_seg_len;
-bootloader_iram_loader_seg_start = bootloader_dram_seg_start - bootloader_iram_loader_seg_len + iram_dram_offset;
-bootloader_iram_seg_start = bootloader_iram_loader_seg_start - bootloader_iram_seg_len;
-
 MEMORY
 {
-  iram_seg (RWX) :                  org = bootloader_iram_seg_start, len = bootloader_iram_seg_len
-  iram_loader_seg (RWX) :           org = bootloader_iram_loader_seg_start, len = bootloader_iram_loader_seg_len
-  dram_seg (RW) :                   org = bootloader_dram_seg_start, len = bootloader_dram_seg_len
+  iram_seg (RWX) :                  org = 0x403CE000, len = 0x2000
+  iram_loader_seg (RWX) :           org = 0x403D0000, len = 0x6000
+  dram_seg (RW) :                   org = 0x3FCD6000, len = 0x4000
 }

 /* Default entry point: */
@@ -131,6 +105,8 @@ SECTIONS
    *(.sdata)
    *(.sdata.*)
    *(.gnu.linkonce.s.*)
+    *(.sdata2)
+    *(.sdata2.*)
    *(.gnu.linkonce.s2.*)
    *(.jcr)
    _data_end = ABSOLUTE(.);
@@ -143,7 +119,6 @@ SECTIONS
    *(.rodata.*)
    *(.gnu.linkonce.r.*)
    *(.rodata1)
-    *(.sdata2 .sdata2.* .srodata .srodata.*)
    __XT_EXCEPTION_TABLE_ = ABSOLUTE(.);
    *(.xt_except_table)
    *(.gcc_except_table)
@@ -207,34 +182,17 @@ SECTIONS
 /**
 *  Appendix: Memory Usage of ROM bootloader
 *
- * 0x3fccb81c ------------------> _dram0_0_start
- *            |               |
- *            |               |
- *            |               |   1. Large buffers that are only used in certain boot modes, see shared_buffers.h
- *            |               |
- *            |               |
- * 0x3fcdd120 ------------------> __stack_sentry
- *            |               |
- *            |               |   2. Startup pro cpu stack (freed when IDF app is running)
- *            |               |
- * 0x3fcdf120 ------------------> __stack (pro cpu)
- *            |               |
- *            |               |
- *            |               |   3. Shared memory only used in startup code or nonos/early boot*
- *            |               |      (can be freed when IDF runs)
- *            |               |
- *            |               |
- * 0x3fcdfa6c ------------------> _dram0_rtos_reserved_start
- *            |               |
- *            |               |
- *            |               |   4. Shared memory used in startup code and when IDF runs
- *            |               |
- *            |               |
- * 0x3fcdfe40 ------------------> _dram0_rtos_reserved_end
- *            |               |
- * 0x3fcdfe4c ------------------> _data_start_interface
- *            |               |
- *            |               |   5. End of DRAM is the 'interface' data with constant addresses (ECO compatible)
- *            |               |
- * 0x3fce0000 ------------------> _data_end_interface
+ *  +--------+--------------+------+ 0x3FCC_B900
+ *  |        ^              |
+ *  |        |              |
+ *  |        | data/bss     |
+ *  |        |              |
+ *  |        v              |
+ *  +------------------------------+ 0x3FCD_D210
+ *  |        ^              |
+ *  |        |              |
+ *  |        | stack        |
+ *  |        |              |
+ *  |        v              |
+ *  +------------------------------+ 0x3FCD_F210
 */
--- a/components/bootloader/subproject/main/ld/esp32s2/bootloader.ld
+++ b/components/bootloader/subproject/main/ld/esp32s2/bootloader.ld
@@ -108,6 +108,8 @@ SECTIONS
    *(.sdata)
    *(.sdata.*)
    *(.gnu.linkonce.s.*)
+    *(.sdata2)
+    *(.sdata2.*)
    *(.gnu.linkonce.s2.*)
    *(.jcr)
    _data_end = ABSOLUTE(.);
@@ -120,7 +122,6 @@ SECTIONS
    *(.rodata.*)
    *(.gnu.linkonce.r.*)
    *(.rodata1)
-    *(.sdata2 .sdata2.*)
    __XT_EXCEPTION_TABLE_ = ABSOLUTE(.);
    *(.xt_except_table)
    *(.gcc_except_table)
@@ -179,23 +180,4 @@ SECTIONS
    _etext = .;
  } > iram_seg

-  /** This section will be used by the debugger and disassembler to get more information
-   * about raw data present in the code.
-   * Indeed, it may be required to add some padding at some points in the code
-   * in order to align a branch/jump destination on a particular bound.
-   * Padding these instructions will generate null bytes that shall be
-   * interpreted as data, and not code by the debugger or disassembler.
-   * This section will only be present in the ELF file, not in the final binary
-   * For more details, check GCC-212
-   */
-  .xt.prop 0 :
-  {
-    KEEP (*(.xt.prop .gnu.linkonce.prop.*))
-  }
-
-  .xt.lit 0 :
-  {
-    KEEP (*(.xt.lit .gnu.linkonce.p.*))
-  }
-
 }
--- a/components/bootloader/subproject/main/ld/esp32s3/bootloader.ld
+++ b/components/bootloader/subproject/main/ld/esp32s3/bootloader.ld
@@ -5,52 +5,17 @@
 */
 /** Simplified memory map for the bootloader.
 *  Make sure the bootloader can load into main memory without overwriting itself.
- *
- *  ESP32-S3 ROM static data usage is as follows:
- *  - 0x3fcd7e00 - 0x3fce9704: Shared buffers, used in UART/USB/SPI download mode only
- *  - 0x3fce9710 - 0x3fceb710: PRO CPU stack, can be reclaimed as heap after RTOS startup
- *  - 0x3fceb710 - 0x3fced710: APP CPU stack, can be reclaimed as heap after RTOS startup
- *  - 0x3fced710 - 0x3fcf0000: ROM .bss and .data (not easily reclaimable)
- *
- *  The 2nd stage bootloader can take space up to the end of ROM shared
- *  buffers area (0x3fce9704). For alignment purpose we shall use value (0x3fce9700).
+ *  We put 2nd bootloader in the high address space (before ROM stack/data/bss).
+ *  See memory usage for ROM bootloader at the end of this file.
 */

-/* The offset between Dbus and Ibus. Used to convert between 0x403xxxxx and 0x3fcxxxxx addresses. */
-iram_dram_offset = 0x6f0000;
-
-/* We consider 0x3fce9700 to be the last usable address for 2nd stage bootloader stack overhead, dram_seg,
- * and work out iram_seg and iram_loader_seg addresses from there, backwards.
- */
-
-/* These lengths can be adjusted, if necessary: */
-bootloader_usable_dram_end = 0x3fce9700;
-bootloader_stack_overhead = 0x2000; /* For safety margin between bootloader data section and startup stacks */
-bootloader_dram_seg_len = 0x4000;
-bootloader_iram_loader_seg_len = 0x7000;
-bootloader_iram_seg_len = 0x3000;
-
-/* Start of the lower region is determined by region size and the end of the higher region */
-bootloader_dram_seg_end = bootloader_usable_dram_end - bootloader_stack_overhead;
-bootloader_dram_seg_start = bootloader_dram_seg_end - bootloader_dram_seg_len;
-bootloader_iram_loader_seg_start = bootloader_dram_seg_start - bootloader_iram_loader_seg_len + iram_dram_offset;
-bootloader_iram_seg_start = bootloader_iram_loader_seg_start - bootloader_iram_seg_len;
-
 MEMORY
 {
-  iram_seg (RWX) :                  org = bootloader_iram_seg_start, len = bootloader_iram_seg_len
-  iram_loader_seg (RWX) :           org = bootloader_iram_loader_seg_start, len = bootloader_iram_loader_seg_len
-  dram_seg (RW) :                   org = bootloader_dram_seg_start, len = bootloader_dram_seg_len
+  iram_seg (RWX) :                  org = 0x403B6000, len = 0x3000
+  iram_loader_seg (RWX) :           org = 0x403B9000, len = 0x7000
+  dram_seg (RW) :                   org = 0x3FCD0000, len = 0x4000
 }

-/* The app may use RAM for static allocations up to the start of iram_loader_seg.
- * If you have changed something above and this assert fails:
- * 1. Check what the new value of bootloader_iram_loader_seg start is.
- * 2. Update the value in this assert.
- * 3. Update SRAM_IRAM_END in components/esp_system/ld/esp32s3/memory.ld.in to the same value.
- */
-ASSERT(bootloader_iram_loader_seg_start == 0x403cc700, "bootloader_iram_loader_seg_start inconsistent with SRAM_IRAM_END");
-
 /* Default entry point: */
 ENTRY(call_start_cpu0);

@@ -142,6 +107,8 @@ SECTIONS
    *(.sdata)
    *(.sdata.*)
    *(.gnu.linkonce.s.*)
+    *(.sdata2)
+    *(.sdata2.*)
    *(.gnu.linkonce.s2.*)
    *(.jcr)
    _data_end = ABSOLUTE(.);
@@ -154,7 +121,6 @@ SECTIONS
    *(.rodata.*)
    *(.gnu.linkonce.r.*)
    *(.rodata1)
-    *(.sdata2 .sdata2.*)
    __XT_EXCEPTION_TABLE_ = ABSOLUTE(.);
    *(.xt_except_table)
    *(.gcc_except_table)
@@ -213,62 +179,29 @@ SECTIONS
    _etext = .;
  } > iram_seg

-  /** This section will be used by the debugger and disassembler to get more information
-   * about raw data present in the code.
-   * Indeed, it may be required to add some padding at some points in the code
-   * in order to align a branch/jump destination on a particular bound.
-   * Padding these instructions will generate null bytes that shall be
-   * interpreted as data, and not code by the debugger or disassembler.
-   * This section will only be present in the ELF file, not in the final binary
-   * For more details, check GCC-212
-   */
-  .xt.prop 0 :
-  {
-    KEEP (*(.xt.prop .gnu.linkonce.prop.*))
-  }
-
-  .xt.lit 0 :
-  {
-    KEEP (*(.xt.lit .gnu.linkonce.p.*))
-  }
-
 }

+
 /**
 *  Appendix: Memory Usage of ROM bootloader
 *
- * 0x3fcd7e00 ------------------> _dram0_0_start
- *            |               |
- *            |               |
- *            |               |   1. Large buffers that are only used in certain boot modes, see shared_buffers.h
- *            |               |
- *            |               |
- * 0x3fce9710 ------------------> __stack_sentry
- *            |               |
- *            |               |   2. Startup pro cpu stack (freed when IDF app is running)
- *            |               |
- * 0x3fceb710 ------------------> __stack (pro cpu)
- *            |               |
- *            |               |      Startup app cpu stack
- *            |               |
- * 0x3fced710 ------------------> __stack_app (app cpu)
- *            |               |
- *            |               |
- *            |               |   3. Shared memory only used in startup code or nonos/early boot*
- *            |               |      (can be freed when IDF runs)
- *            |               |
- *            |               |
- * 0x3fceee34 ------------------> _dram0_rtos_reserved_start
- *            |               |
- *            |               |
- *            |               |   4. Shared memory used in startup code and when IDF runs
- *            |               |
- *            |               |
- * 0x3fcef770 ------------------> _dram0_rtos_reserved_end
- *            |               |
- * 0x3fcef81c ------------------> _data_start_interface
- *            |               |
- *            |               |   5. End of DRAM is the 'interface' data with constant addresses (ECO compatible)
- *            |               |
- * 0x3fcf0000 ------------------> _data_end_interface
+ *  +--------+--------------+------+ 0x3FCD_8000
+ *  |        ^              |
+ *  |        |              |
+ *  |        | data/bss     |
+ *  |        |              |
+ *  |        v              |
+ *  +------------------------------+ 0x3FCE_9910
+ *  |        ^              |
+ *  |        |              |
+ *  |        | stack (pro)  |
+ *  |        |              |
+ *  |        v              |
+ *  +------------------------------+ 0x3FCE_B910
+ *  |        ^              |
+ *  |        |              |
+ *  |        | stack (app)  |
+ *  |        |              |
+ *  |        v              |
+ *  +--------+--------------+------+ 0x3FCE_D910
 */
--- a/components/bootloader_support/CMakeLists.txt
+++ b/components/bootloader_support/CMakeLists.txt
@@ -18,7 +18,7 @@ set(srcs

 if(BOOTLOADER_BUILD)
    set(include_dirs "include" "bootloader_flash/include"
-        "private_include")
+        "include_bootloader")
    set(priv_requires micro-ecc spi_flash efuse)
    list(APPEND srcs
    "src/bootloader_init.c"
@@ -35,7 +35,7 @@ else()
    list(APPEND srcs
        "src/idf/bootloader_sha.c")
    set(include_dirs "include" "bootloader_flash/include")
-    set(priv_include_dirs "private_include")
+    set(priv_include_dirs "include_bootloader")
    # heap is required for `heap_memory_layout.h` header
    set(priv_requires spi_flash mbedtls efuse app_update heap)
 endif()
@@ -148,5 +148,3 @@ endif()
 if(BOOTLOADER_BUILD)
    target_link_libraries(${COMPONENT_LIB} INTERFACE "-u abort")
 endif()
-
-target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")
--- a/components/bootloader_support/bootloader_flash/include/bootloader_flash.h
+++ b/components/bootloader_support/bootloader_flash/include/bootloader_flash.h
@@ -7,8 +7,7 @@

 #include <stdint.h>
 #include <esp_err.h>
-#include "spi_flash_mmap.h" /* including in bootloader for error values */
-#include "esp_private/spi_flash_os.h"
+#include <esp_spi_flash.h> /* including in bootloader for error values */
 #include "sdkconfig.h"
 #include "soc/soc_caps.h"
 #include "bootloader_flash_override.h"
@@ -50,12 +49,6 @@ esp_err_t bootloader_flash_xmc_startup(void);
  */
 esp_err_t  __attribute__((weak)) bootloader_flash_unlock(void);

-/**
- * @brief Reset the flash chip (66H + 99H).
- *
- * @return ESP_OK if success, otherwise ESP_FAIL.
- */
-esp_err_t bootloader_flash_reset_chip(void);

 #ifdef __cplusplus
 }
--- a/components/bootloader_support/bootloader_flash/include/bootloader_flash_priv.h
+++ b/components/bootloader_support/bootloader_flash/include/bootloader_flash_priv.h
@@ -10,7 +10,7 @@
 #include <stdbool.h>
 #include <stdint.h>
 #include <esp_err.h>
-#include <spi_flash_mmap.h> /* including in bootloader for error values */
+#include <esp_spi_flash.h> /* including in bootloader for error values */
 #include "sdkconfig.h"
 #include "bootloader_flash.h"

@@ -21,18 +21,20 @@ extern "C" {
 #define FLASH_SECTOR_SIZE 0x1000
 #define FLASH_BLOCK_SIZE 	0x10000
 #define MMAP_ALIGNED_MASK 	0x0000FFFF
-#define MMU_FLASH_MASK    (~(SPI_FLASH_MMU_PAGE_SIZE - 1))

+//This will be replaced with a kconfig, TODO: IDF-3821
+#define MMU_PAGE_SIZE                   0x10000
+#define MMU_FLASH_MASK                  (~(MMU_PAGE_SIZE - 1))
 /**
- * MMU mapping must always be in the unit of a SPI_FLASH_MMU_PAGE_SIZE
+ * MMU mapping must always be in the unit of a MMU_PAGE_SIZE
 * This macro is a helper for you to get needed page nums to be mapped. e.g.:
- * Let's say SPI_FLASH_MMU_PAGE_SIZE is 64KB.
+ * Let's say MMU_PAGE_SIZE is 64KB.
 * - v_start = 0x4200_0004
 * - size = 4 * 64KB
 *
 * You should map from 0x4200_0000, then map 5 pages.
 */
-#define GET_REQUIRED_MMU_PAGES(size, v_start)    ((size + (v_start - (v_start & MMU_FLASH_MASK)) + SPI_FLASH_MMU_PAGE_SIZE - 1) / SPI_FLASH_MMU_PAGE_SIZE)
+#define GET_REQUIRED_MMU_PAGES(size, v_start)    ((size + (v_start - (v_start & MMU_FLASH_MASK)) + MMU_PAGE_SIZE - 1) / MMU_PAGE_SIZE)

 /* SPI commands (actual on-wire commands not SPI controller bitmasks)
   Suitable for use with the bootloader_execute_flash_command static function.
@@ -42,7 +44,6 @@ extern "C" {
 #define CMD_WRSR2      0x31 /* Not all SPI flash uses this command */
 #define CMD_WRSR3      0x11 /* Not all SPI flash uses this command */
 #define CMD_WREN       0x06
-#define CMD_WRENVSR    0x50 /* Flash write enable for volatile SR bits */
 #define CMD_WRDI       0x04
 #define CMD_RDSR       0x05
 #define CMD_RDSR2      0x35 /* Not all SPI flash uses this command */
@@ -51,8 +52,6 @@ extern "C" {
 #define CMD_RDSFDP     0x5A /* Read the SFDP of the flash */
 #define CMD_WRAP       0x77 /* Set burst with wrap command */
 #define CMD_RESUME     0x7A /* Resume command to clear flash suspend bit */
-#define CMD_RESETEN    0x66
-#define CMD_RESET      0x99


 /* Provide a Flash API for bootloader_support code,
@@ -175,14 +174,6 @@ uint32_t bootloader_flash_read_sfdp(uint32_t sfdp_addr, unsigned int miso_byte_n
 */
 void bootloader_enable_wp(void);

-/**
- * @brief Once this function is called,
- * any on-going internal operations will be terminated and the device will return to its default power-on
- * state and lose all the current volatile settings, such as Volatile Status Register bits, Write Enable Latch
- * (WEL) status, Program/Erase Suspend status, etc.
- */
-void bootloader_spi_flash_reset(void);
-
 #ifdef __cplusplus
 }
 #endif
--- a/components/bootloader_support/bootloader_flash/src/bootloader_flash.c
+++ b/components/bootloader_support/bootloader_flash/src/bootloader_flash.c
@@ -23,9 +23,6 @@
 #   define SPIFLASH SPIMEM1
 #endif

-// This dependency will be removed in the future.  IDF-5025
-#include "esp_flash.h"
-
 #include "esp_rom_spiflash.h"

 #ifdef CONFIG_EFUSE_VIRTUAL_KEEP_IN_FLASH
@@ -46,7 +43,7 @@
 #define ESP_BOOTLOADER_SPIFLASH_QE_SR1_2BYTE     BIT9   // QE position when you write 16 bits at one time.

 #ifndef BOOTLOADER_BUILD
-/* Normal app version maps to spi_flash_mmap.h operations...
+/* Normal app version maps to esp_spi_flash.h operations...
 */
 static const char *TAG = "bootloader_mmap";

@@ -85,31 +82,33 @@ void bootloader_munmap(const void *mapping)
 esp_err_t bootloader_flash_read(size_t src, void *dest, size_t size, bool allow_decrypt)
 {
    if (allow_decrypt && esp_flash_encryption_enabled()) {
-        return esp_flash_read_encrypted(NULL, src, dest, size);
+        return spi_flash_read_encrypted(src, dest, size);
    } else {
-        return esp_flash_read(NULL, dest, src, size);
+        return spi_flash_read(src, dest, size);
    }
 }

 esp_err_t bootloader_flash_write(size_t dest_addr, void *src, size_t size, bool write_encrypted)
 {
    if (write_encrypted && !ENCRYPTION_IS_VIRTUAL) {
-        return esp_flash_write_encrypted(NULL, dest_addr, src, size);
+#if CONFIG_IDF_TARGET_ESP32
+        return spi_flash_write_encrypted(dest_addr, src, size);
+#else
+        return esp_rom_spiflash_write_encrypted(dest_addr, src, size);
+#endif
    } else {
-        return esp_flash_write(NULL, src, dest_addr, size);
+        return spi_flash_write(dest_addr, src, size);
    }
 }

 esp_err_t bootloader_flash_erase_sector(size_t sector)
 {
-    // Will de-dependency IDF-5025
-    return esp_flash_erase_region(NULL, sector * SPI_FLASH_SEC_SIZE, SPI_FLASH_SEC_SIZE);
+    return spi_flash_erase_sector(sector);
 }

 esp_err_t bootloader_flash_erase_range(uint32_t start_addr, uint32_t size)
 {
-    // Will de-dependency IDF-5025
-    return esp_flash_erase_region(NULL, start_addr, size);
+    return spi_flash_erase_range(start_addr, size);
 }

 #else //BOOTLOADER_BUILD
@@ -138,11 +137,7 @@ static const char *TAG = "bootloader_flash";
   63th block for bootloader_flash_read
 */
 #define MMU_BLOCK0_VADDR  SOC_DROM_LOW
-#ifdef SOC_MMU_PAGE_SIZE_CONFIGURABLE
-#define MMU_SIZE          (DRAM0_CACHE_ADDRESS_HIGH(SPI_FLASH_MMU_PAGE_SIZE) - DRAM0_CACHE_ADDRESS_LOW - SPI_FLASH_MMU_PAGE_SIZE) // This mmu size means that the mmu size to be mapped
-#else
-#define MMU_SIZE          (DRAM0_CACHE_ADDRESS_HIGH - DRAM0_CACHE_ADDRESS_LOW - SPI_FLASH_MMU_PAGE_SIZE) // This mmu size means that the mmu size to be mapped
-#endif
+#define MMU_SIZE          (0x3f0000)
 #define MMU_BLOCK63_VADDR (MMU_BLOCK0_VADDR + MMU_SIZE)
 #define FLASH_READ_VADDR MMU_BLOCK63_VADDR
 #endif
@@ -201,7 +196,7 @@ const void *bootloader_mmap(uint32_t src_paddr, uint32_t size)
 #if CONFIG_IDF_TARGET_ESP32
    uint32_t count = GET_REQUIRED_MMU_PAGES(size, src_paddr);
    int e = cache_flash_mmu_set(0, 0, MMU_BLOCK0_VADDR, src_paddr_aligned, 64, count);
-    ESP_EARLY_LOGV(TAG, "after mapping, starting from paddr=0x%08x and vaddr=0x%08x, 0x%x bytes are mapped", src_paddr_aligned, MMU_BLOCK0_VADDR, count * SPI_FLASH_MMU_PAGE_SIZE);
+    ESP_EARLY_LOGV(TAG, "after mapping, starting from paddr=0x%08x and vaddr=0x%08x, 0x%x bytes are mapped", src_paddr_aligned, MMU_BLOCK0_VADDR, count * MMU_PAGE_SIZE);
    if (e != 0) {
        ESP_EARLY_LOGE(TAG, "cache_flash_mmu_set failed: %d\n", e);
        Cache_Read_Enable(0);
@@ -308,12 +303,12 @@ static esp_err_t bootloader_flash_read_allow_decrypt(size_t src_addr, void *dest
            //---------------Do mapping------------------------
            ESP_EARLY_LOGD(TAG, "mmu set block paddr=0x%08x (was 0x%08x)", map_at, current_read_mapping);
 #if CONFIG_IDF_TARGET_ESP32
-            //Should never fail if we only map a SPI_FLASH_MMU_PAGE_SIZE to the vaddr starting from FLASH_READ_VADDR
+            //Should never fail if we only map a MMU_PAGE_SIZE to the vaddr starting from FLASH_READ_VADDR
            int e = cache_flash_mmu_set(0, 0, FLASH_READ_VADDR, map_at, 64, 1);
            assert(e == 0);
 #else
            uint32_t actual_mapped_len = 0;
-            mmu_hal_map_region(0, MMU_TARGET_FLASH0, FLASH_READ_VADDR, map_at, SPI_FLASH_MMU_PAGE_SIZE - 1, &actual_mapped_len);
+            mmu_hal_map_region(0, MMU_TARGET_FLASH0, FLASH_READ_VADDR, map_at, MMU_PAGE_SIZE - 1, &actual_mapped_len);
 #endif
            current_read_mapping = map_at;

@@ -603,12 +598,6 @@ uint32_t IRAM_ATTR bootloader_read_flash_id(void)
    return id;
 }

-void bootloader_spi_flash_reset(void)
-{
-    bootloader_execute_flash_command(CMD_RESETEN, 0, 0, 0);
-    bootloader_execute_flash_command(CMD_RESET, 0, 0, 0);
-}
-
 #if SOC_CACHE_SUPPORT_WRAP
 esp_err_t bootloader_flash_wrap_set(spi_flash_wrap_mode_t mode)
 {
@@ -741,44 +730,3 @@ esp_err_t IRAM_ATTR bootloader_flash_xmc_startup(void)
 }

 #endif //XMC_SUPPORT
-
-FORCE_INLINE_ATTR void bootloader_mspi_reset(void)
-{
-#if CONFIG_IDF_TARGET_ESP32
-    SPI1.slave.sync_reset = 0;
-    SPI0.slave.sync_reset = 0;
-    SPI1.slave.sync_reset = 1;
-    SPI0.slave.sync_reset = 1;
-    SPI1.slave.sync_reset = 0;
-    SPI0.slave.sync_reset = 0;
-#else
-    SPIMEM1.ctrl2.sync_reset = 0;
-    SPIMEM0.ctrl2.sync_reset = 0;
-    SPIMEM1.ctrl2.sync_reset = 1;
-    SPIMEM0.ctrl2.sync_reset = 1;
-    SPIMEM1.ctrl2.sync_reset = 0;
-    SPIMEM0.ctrl2.sync_reset = 0;
-#endif
-}
-
-esp_err_t IRAM_ATTR bootloader_flash_reset_chip(void)
-{
-    bootloader_mspi_reset();
-    // Seems that sync_reset cannot make host totally idle.'
-    // Sending an extra(useless) command to make the host idle in order to send reset command.
-    bootloader_execute_flash_command(0x05, 0, 0, 0);
-#if CONFIG_IDF_TARGET_ESP32
-    if (SPI1.ext2.st != 0)
-#elif CONFIG_IDF_TARGET_ESP32S2 || CONFIG_IDF_TARGET_ESP32S3
-    if (SPIMEM1.fsm.st != 0)
-#else
-    if (SPIMEM1.fsm.spi0_mst_st != 0)
-#endif
-    {
-        return ESP_FAIL;
-    }
-    bootloader_execute_flash_command(0x66, 0, 0, 0);
-    bootloader_execute_flash_command(0x99, 0, 0, 0);
-
-    return ESP_OK;
-}
--- a/components/bootloader_support/bootloader_flash/src/bootloader_flash_config_esp32.c
+++ b/components/bootloader_support/bootloader_flash/src/bootloader_flash_config_esp32.c
@@ -17,7 +17,6 @@
 #include "soc/spi_reg.h"
 #include "soc/soc_caps.h"
 #include "soc/soc_pins.h"
-#include "hal/efuse_hal.h"
 #include "hal/gpio_hal.h"
 #include "flash_qio_mode.h"
 #include "bootloader_common.h"
@@ -176,7 +175,7 @@ int bootloader_flash_get_wp_pin(void)
        return ESP32_D2WD_WP_GPIO;
    case EFUSE_RD_CHIP_VER_PKG_ESP32PICOD4:
        /* Same package IDs are used for ESP32-PICO-V3 and ESP32-PICO-D4, silicon version differentiates */
-        chip_ver = efuse_hal_get_major_chip_version();
+        chip_ver = bootloader_common_get_chip_revision();
        return (chip_ver < 3) ? ESP32_D2WD_WP_GPIO : ESP32_PICO_V3_GPIO;
    case EFUSE_RD_CHIP_VER_PKG_ESP32PICOV302:
        return ESP32_PICO_V3_GPIO;
--- a/components/bootloader_support/include/bootloader_common.h
+++ b/components/bootloader_support/include/bootloader_common.h
@@ -8,12 +8,27 @@
 #include "esp_flash_partitions.h"
 #include "esp_image_format.h"
 #include "esp_app_format.h"
+// [refactor-todo]: we shouldn't expose ROM header files in a public API header, remove them in v5.0
+// Tracked in IDF-1968
+#if CONFIG_IDF_TARGET_ESP32
+#include "esp32/rom/rtc.h"
+#elif CONFIG_IDF_TARGET_ESP32S2
+#include "esp32s2/rom/rtc.h"
+#elif CONFIG_IDF_TARGET_ESP32S3
+#include "esp32s3/rom/rtc.h"
+#elif CONFIG_IDF_TARGET_ESP32C3
+#include "esp32c3/rom/rtc.h"
+#elif CONFIG_IDF_TARGET_ESP32H2
+#include "esp32h2/rom/rtc.h"
+#elif CONFIG_IDF_TARGET_ESP32C2
+#include "esp32c2/rom/rtc.h"
+#endif

 #ifdef __cplusplus
 extern "C" {
 #endif

-// Type of hold a GPIO in low state
+/// Type of hold a GPIO in low state
 typedef enum {
    GPIO_LONG_HOLD  = 1,    /*!< The long hold GPIO */
    GPIO_SHORT_HOLD = -1,   /*!< The short hold GPIO */
@@ -106,6 +121,15 @@ bool bootloader_common_label_search(const char *list, char *label);
 */
 void bootloader_configure_spi_pins(int drv);

+/**
+ * @brief Get flash CS IO
+ *
+ * Can be determined by eFuse values, or the default value
+ *
+ * @return Flash CS IO
+ */
+uint8_t bootloader_flash_get_cs_io(void);
+
 /**
 * @brief Calculates a sha-256 for a given partition or returns a appended digest.
 *
@@ -165,6 +189,13 @@ int bootloader_common_select_otadata(const esp_ota_select_entry_t *two_otadata,
 */
 esp_err_t bootloader_common_get_partition_description(const esp_partition_pos_t *partition, esp_app_desc_t *app_desc);

+/**
+ * @brief Get chip revision
+ *
+ * @return Chip revision number
+ */
+uint8_t bootloader_common_get_chip_revision(void);
+
 /**
 * @brief Get chip package
 *
@@ -172,6 +203,14 @@ esp_err_t bootloader_common_get_partition_description(const esp_partition_pos_t
 */
 uint32_t bootloader_common_get_chip_ver_pkg(void);

+/**
+ * @brief Query reset reason
+ *
+ * @param cpu_no CPU number
+ * @return reset reason enumeration
+ */
+RESET_REASON bootloader_common_get_reset_reason(int cpu_no);
+
 /**
 * @brief Check if the image (bootloader and application) has valid chip ID and revision
 *
--- a/components/bootloader_support/include/bootloader_util.h
+++ b/components/bootloader_support/include/bootloader_util.h
@@ -6,9 +6,7 @@

 #pragma once

-#include <stdbool.h>
-#include <stdint.h>
-#include <assert.h>
+#include <stddef.h>

 #ifdef __cplusplus
 extern "C" {
@@ -24,11 +22,11 @@ extern "C" {
 * @return true iff [start1; end1) overlaps [start2; end2)
 */
 static inline bool bootloader_util_regions_overlap(
-    const intptr_t start1, const intptr_t end1,
-    const intptr_t start2, const intptr_t end2)
+        const intptr_t start1, const intptr_t end1,
+        const intptr_t start2, const intptr_t end2)
 {
-    assert(end1 > start1);
-    assert(end2 > start2);
+    assert(end1>start1);
+    assert(end2>start2);
    return (end1 > start2 && end2 > start1);
 }

--- a/components/bootloader_support/include/esp_flash_encrypt.h
+++ b/components/bootloader_support/include/esp_flash_encrypt.h
@@ -9,7 +9,7 @@
 #include "esp_attr.h"
 #include "esp_err.h"
 #ifndef BOOTLOADER_BUILD
-#include "spi_flash_mmap.h"
+#include "esp_spi_flash.h"
 #endif
 #include "hal/efuse_ll.h"
 #include "sdkconfig.h"
@@ -85,49 +85,6 @@ bool esp_flash_encryption_enabled(void);
 */
 esp_err_t esp_flash_encrypt_check_and_update(void);

-/** @brief Returns the Flash Encryption state and prints it
- *
- * @return True  - Flash Encryption is enabled
- *         False - Flash Encryption is not enabled
- */
-bool esp_flash_encrypt_state(void);
-
-/** @brief Checks if the first initialization was done
- *
- * If the first initialization was done then FLASH_CRYPT_CNT != 0
- *
- * @return true - the first initialization was done
- *         false - the first initialization was NOT done
- */
-bool esp_flash_encrypt_initialized_once(void);
-
-/** @brief The first initialization of Flash Encryption key and related eFuses
- *
- * @return ESP_OK if all operations succeeded
- */
-esp_err_t esp_flash_encrypt_init(void);
-
-/** @brief Encrypts flash content
- *
- * @return ESP_OK if all operations succeeded
- */
-esp_err_t esp_flash_encrypt_contents(void);
-
-/** @brief Activates Flash encryption on the chip
- *
- * It burns FLASH_CRYPT_CNT eFuse based on the CONFIG_SECURE_FLASH_ENCRYPTION_MODE_RELEASE option.
- *
- * @return ESP_OK if all operations succeeded
- */
-esp_err_t esp_flash_encrypt_enable(void);
-
-/** @brief Returns True if the write protection of FLASH_CRYPT_CNT is set
- *
- * @param print_error Print error if it is write protected
- *
- * @return true - if FLASH_CRYPT_CNT is write protected
- */
-bool esp_flash_encrypt_is_write_protected(bool print_error);

 /** @brief Encrypt-in-place a block of flash sectors
 *
--- a/components/bootloader_support/include/esp_secure_boot.h
+++ b/components/bootloader_support/include/esp_secure_boot.h
@@ -8,13 +8,32 @@
 #include <stdbool.h>
 #include <esp_err.h>
 #include "soc/efuse_periph.h"
-#include "soc/soc_caps.h"
 #include "esp_image_format.h"
 #include "esp_rom_efuse.h"
 #include "sdkconfig.h"
 #include "esp_rom_crc.h"
 #include "hal/efuse_ll.h"

+#if CONFIG_IDF_TARGET_ESP32
+#include "esp32/rom/efuse.h"
+#include "esp32/rom/secure_boot.h"
+#elif CONFIG_IDF_TARGET_ESP32S2
+#include "esp32s2/rom/efuse.h"
+#include "esp32s2/rom/secure_boot.h"
+#elif CONFIG_IDF_TARGET_ESP32C3
+#include "esp32c3/rom/efuse.h"
+#include "esp32c3/rom/secure_boot.h"
+#elif CONFIG_IDF_TARGET_ESP32S3
+#include "esp32s3/rom/efuse.h"
+#include "esp32s3/rom/secure_boot.h"
+#elif CONFIG_IDF_TARGET_ESP32H2
+#include "esp32h2/rom/efuse.h"
+#include "esp32h2/rom/secure_boot.h"
+#elif CONFIG_IDF_TARGET_ESP32C2
+#include "esp32c2/rom/efuse.h"
+#include "esp32c2/rom/secure_boot.h"
+#endif
+
 #ifdef CONFIG_SECURE_BOOT_V1_ENABLED
 #if !defined(CONFIG_SECURE_SIGNED_ON_BOOT) || !defined(CONFIG_SECURE_SIGNED_ON_UPDATE) || !defined(CONFIG_SECURE_SIGNED_APPS)
 #error "internal sdkconfig error, secure boot should always enable all signature options"
@@ -198,10 +217,34 @@ esp_err_t esp_secure_boot_verify_ecdsa_signature_block(const esp_secure_boot_sig
 * Each image can have one or more signature blocks (up to SECURE_BOOT_NUM_BLOCKS). Each signature block includes a public key.
 */
 typedef struct {
-    uint8_t key_digests[SOC_EFUSE_SECURE_BOOT_KEY_DIGESTS][ESP_SECURE_BOOT_DIGEST_LEN];    /* SHA of the public key components in the signature block */
+    uint8_t key_digests[SECURE_BOOT_NUM_BLOCKS][ESP_SECURE_BOOT_DIGEST_LEN];    /* SHA of the public key components in the signature block */
    unsigned num_digests;                                       /* Number of valid digests, starting at index 0 */
 } esp_image_sig_public_key_digests_t;

+/** @brief Verify the secure boot signature block for Secure Boot V2.
+ *
+ *  Performs RSA-PSS or ECDSA verification of the SHA-256 image based on the public key
+ *  in the signature block, compared against the public key digest stored in efuse.
+ *
+ * Similar to esp_secure_boot_verify_signature(), but can be used when the digest is precalculated.
+ * @param sig_block Pointer to signature block data
+ * @param image_digest Pointer to 32 byte buffer holding SHA-256 hash.
+ * @param verified_digest Pointer to 32 byte buffer that will receive verified digest if verification completes. (Used during bootloader implementation only, result is invalid otherwise.)
+ *
+ */
+esp_err_t esp_secure_boot_verify_sbv2_signature_block(const ets_secure_boot_signature_t *sig_block, const uint8_t *image_digest, uint8_t *verified_digest);
+
+/** @brief Legacy function to verify RSA secure boot signature block for Secure Boot V2.
+ *
+ * @note This is kept for backward compatibility. It internally calls esp_secure_boot_verify_sbv2_signature_block.
+ *
+ * @param sig_block Pointer to RSA signature block data
+ * @param image_digest Pointer to 32 byte buffer holding SHA-256 hash.
+ * @param verified_digest Pointer to 32 byte buffer that will receive verified digest if verification completes. (Used during bootloader implementation only, result is invalid otherwise.)
+ *
+ */
+esp_err_t esp_secure_boot_verify_rsa_signature_block(const ets_secure_boot_signature_t *sig_block, const uint8_t *image_digest, uint8_t *verified_digest);
+
 #endif // !CONFIG_IDF_TARGET_ESP32 || CONFIG_ESP32_REV_MIN_3

 /** @brief Legacy ECDSA verification function
--- a/components/bootloader_support/include_bootloader/bootloader_config.h
+++ b/components/bootloader_support/include_bootloader/bootloader_config.h
--- a/components/bootloader_support/include_bootloader/bootloader_console.h
+++ b/components/bootloader_support/include_bootloader/bootloader_console.h
--- a/components/bootloader_support/include_bootloader/bootloader_init.h
+++ b/components/bootloader_support/include_bootloader/bootloader_init.h
--- a/components/bootloader_support/include_bootloader/bootloader_sha.h
+++ b/components/bootloader_support/include_bootloader/bootloader_sha.h
--- a/components/bootloader_support/include_bootloader/bootloader_soc.h
+++ b/components/bootloader_support/include_bootloader/bootloader_soc.h
--- a/components/bootloader_support/include_bootloader/bootloader_utility.h
+++ b/components/bootloader_support/include_bootloader/bootloader_utility.h
--- a/components/bootloader_support/private_include/bootloader_signature.h
+++ b/components/bootloader_support/private_include/bootloader_signature.h
@@ -1,52 +0,0 @@
-/*
- * SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD
- *
- * SPDX-License-Identifier: Apache-2.0
- */
-#pragma once
-
-#include "sdkconfig.h"
-#include <esp_err.h>
-#include <stdint.h>
-
-#if CONFIG_IDF_TARGET_ESP32
-#include "esp32/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32S2
-#include "esp32s2/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32C3
-#include "esp32c3/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32S3
-#include "esp32s3/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32H2
-#include "esp32h2/rom/secure_boot.h"
-#elif CONFIG_IDF_TARGET_ESP32C2
-#include "esp32c2/rom/secure_boot.h"
-#endif
-
-#if !CONFIG_IDF_TARGET_ESP32 || CONFIG_ESP32_REV_MIN_3
-
-/** @brief Verify the secure boot signature block for Secure Boot V2.
- *
- *  Performs RSA-PSS or ECDSA verification of the SHA-256 image based on the public key
- *  in the signature block, compared against the public key digest stored in efuse.
- *
- * Similar to esp_secure_boot_verify_signature(), but can be used when the digest is precalculated.
- * @param sig_block Pointer to signature block data
- * @param image_digest Pointer to 32 byte buffer holding SHA-256 hash.
- * @param verified_digest Pointer to 32 byte buffer that will receive verified digest if verification completes. (Used during bootloader implementation only, result is invalid otherwise.)
- *
- */
-esp_err_t esp_secure_boot_verify_sbv2_signature_block(const ets_secure_boot_signature_t *sig_block, const uint8_t *image_digest, uint8_t *verified_digest);
-
-/** @brief Legacy function to verify RSA secure boot signature block for Secure Boot V2.
- *
- * @note This is kept for backward compatibility. It internally calls esp_secure_boot_verify_sbv2_signature_block.
- *
- * @param sig_block Pointer to RSA signature block data
- * @param image_digest Pointer to 32 byte buffer holding SHA-256 hash.
- * @param verified_digest Pointer to 32 byte buffer that will receive verified digest if verification completes. (Used during bootloader implementation only, result is invalid otherwise.)
- *
- */
-esp_err_t esp_secure_boot_verify_rsa_signature_block(const ets_secure_boot_signature_t *sig_block, const uint8_t *image_digest, uint8_t *verified_digest);
-
-#endif
--- a/components/bootloader_support/src/bootloader_clock_init.c
+++ b/components/bootloader_support/src/bootloader_clock_init.c
@@ -9,7 +9,7 @@
 #include "hal/efuse_hal.h"
 #include "soc/rtc_cntl_reg.h"
 #if CONFIG_IDF_TARGET_ESP32
-#include "hal/clk_tree_ll.h"
+#include "soc/dport_reg.h"
 #endif
 #include "esp_rom_sys.h"
 #include "esp_rom_uart.h"
@@ -32,32 +32,23 @@ __attribute__((weak)) void bootloader_clock_configure(void)
     * document). For rev. 0, switch to 240 instead if it has been enabled
     * previously.
     */
-    if (efuse_hal_get_major_chip_version() == 0 &&
-            clk_ll_cpu_get_freq_mhz_from_pll() == CLK_LL_PLL_240M_FREQ_MHZ) {
+    if (efuse_hal_get_chip_revision() == 0 &&
+            DPORT_REG_GET_FIELD(DPORT_CPU_PER_CONF_REG, DPORT_CPUPERIOD_SEL) == DPORT_CPUPERIOD_SEL_240) {
        cpu_freq_mhz = 240;
    }
 #elif CONFIG_IDF_TARGET_ESP32H2
    cpu_freq_mhz = 64;
 #endif

-    if (esp_rom_get_reset_reason(0) != RESET_REASON_CPU0_SW || rtc_clk_apb_freq_get() < APB_CLK_FREQ) {
+    if (rtc_clk_apb_freq_get() < APB_CLK_FREQ || esp_rom_get_reset_reason(0) != RESET_REASON_CPU0_SW) {
        rtc_clk_config_t clk_cfg = RTC_CLK_CONFIG_DEFAULT();
 #if CONFIG_IDF_TARGET_ESP32
        clk_cfg.xtal_freq = CONFIG_ESP32_XTAL_FREQ;
 #endif
-#if CONFIG_IDF_TARGET_ESP32C2
-        clk_cfg.xtal_freq = CONFIG_ESP32C2_XTAL_FREQ;
-#endif
-        /* For other chips, there is no XTAL_FREQ choice */
+        /* ESP32-S2 doesn't have XTAL_FREQ choice, always 40MHz */
        clk_cfg.cpu_freq_mhz = cpu_freq_mhz;
-        clk_cfg.slow_clk_src = rtc_clk_slow_src_get();
-        if (clk_cfg.slow_clk_src == SOC_RTC_SLOW_CLK_SRC_INVALID) {
-            clk_cfg.slow_clk_src = SOC_RTC_SLOW_CLK_SRC_RC_SLOW;
-        }
-        clk_cfg.fast_clk_src = rtc_clk_fast_src_get();
-        if (clk_cfg.fast_clk_src == SOC_RTC_FAST_CLK_SRC_INVALID) {
-            clk_cfg.fast_clk_src = SOC_RTC_FAST_CLK_SRC_XTAL_DIV;
-        }
+        clk_cfg.slow_freq = rtc_clk_slow_freq_get();
+        clk_cfg.fast_freq = rtc_clk_fast_freq_get();
        rtc_clk_init(clk_cfg);
    }

--- a/components/bootloader_support/src/bootloader_common.c
+++ b/components/bootloader_support/src/bootloader_common.c
@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: 2018-2022 Espressif Systems (Shanghai) CO LTD
+ * SPDX-FileCopyrightText: 2018-2021 Espressif Systems (Shanghai) CO LTD
 *
 * SPDX-License-Identifier: Apache-2.0
 */
@@ -13,6 +13,7 @@
 #include "esp_rom_crc.h"
 #include "esp_rom_gpio.h"
 #include "esp_rom_sys.h"
+#include "esp_rom_efuse.h"
 #include "esp_flash_partitions.h"
 #include "bootloader_flash_priv.h"
 #include "bootloader_common.h"
@@ -186,3 +187,20 @@ void bootloader_common_vddsdio_configure(void)
    }
 #endif // CONFIG_BOOTLOADER_VDDSDIO_BOOST
 }
+
+RESET_REASON bootloader_common_get_reset_reason(int cpu_no)
+{
+    return (RESET_REASON)esp_rom_get_reset_reason(cpu_no);
+}
+
+uint8_t bootloader_flash_get_cs_io(void)
+{
+    uint8_t cs_io;
+    const uint32_t spiconfig = esp_rom_efuse_get_flash_gpio_info();
+    if (spiconfig == ESP_ROM_EFUSE_FLASH_DEFAULT_SPI) {
+        cs_io = SPI_CS0_GPIO_NUM;
+    } else {
+        cs_io = (spiconfig >> 18) & 0x3f;
+    }
+    return cs_io;
+}
--- a/components/bootloader_support/src/bootloader_common_loader.c
+++ b/components/bootloader_support/src/bootloader_common_loader.c
@@ -17,7 +17,6 @@
 #include "soc/gpio_periph.h"
 #include "soc/rtc.h"
 #include "soc/efuse_reg.h"
-#include "hal/efuse_hal.h"
 #include "hal/gpio_ll.h"
 #include "esp_image_format.h"
 #include "bootloader_sha.h"
@@ -64,13 +63,7 @@ esp_err_t bootloader_common_check_chip_validity(const esp_image_header_t* img_hd
    }

 #ifndef CONFIG_IDF_ENV_FPGA
-#if defined(CONFIG_IDF_TARGET_ESP32) || defined(CONFIG_IDF_TARGET_ESP32S2) || defined(CONFIG_IDF_TARGET_ESP32C2) || defined(CONFIG_IDF_TARGET_ESP32H2)
-    uint8_t revision = efuse_hal_get_major_chip_version();
-    // min_chip_rev keeps the MAJOR wafer version for these chips
-#else
-    uint8_t revision = efuse_hal_get_minor_chip_version();
-    // min_chip_rev keeps the MINOR wafer version for these chips
-#endif
+    uint8_t revision = bootloader_common_get_chip_revision();
    if (revision < img_hdr->min_chip_rev) {
        /* To fix this error, please update mininum supported chip revision from configuration,
         * located in TARGET (e.g. ESP32) specific options under "Component config" menu */
@@ -164,11 +157,7 @@ static void update_rtc_retain_mem_crc(void)

 void bootloader_common_reset_rtc_retain_mem(void)
 {
-    #pragma GCC diagnostic push
-    #pragma GCC diagnostic ignored "-Wstringop-overflow"
-    #pragma GCC diagnostic ignored "-Warray-bounds"
    memset(rtc_retain_mem, 0, sizeof(rtc_retain_mem_t));
-    #pragma GCC diagnostic pop
 }

 uint16_t bootloader_common_get_rtc_retain_mem_reboot_counter(void)
--- a/components/bootloader_support/src/bootloader_console.c
+++ b/components/bootloader_support/src/bootloader_console.c
@@ -90,7 +90,7 @@ void bootloader_console_init(void)
    // Set configured UART console baud rate
    uint32_t clock_hz = rtc_clk_apb_freq_get();
 #if ESP_ROM_UART_CLK_IS_XTAL
-    clock_hz = (uint32_t)rtc_clk_xtal_freq_get() * MHZ; // From esp32-s3 on, UART clk source is selected to XTAL in ROM
+    clock_hz = UART_CLK_FREQ_ROM; // From esp32-s3 on, UART clock source is selected to XTAL in ROM
 #endif
    esp_rom_uart_set_clock_baudrate(uart_num, clock_hz, CONFIG_ESP_CONSOLE_UART_BAUDRATE);
 }
--- a/components/bootloader_support/src/bootloader_efuse.c
+++ b/components/bootloader_support/src/bootloader_efuse.c
@@ -9,9 +9,13 @@
 #include "bootloader_common.h"
 #include "hal/efuse_ll.h"
 #include "hal/efuse_hal.h"
-#include "esp_attr.h"

-IRAM_ATTR uint32_t bootloader_common_get_chip_ver_pkg(void)
+uint8_t bootloader_common_get_chip_revision(void)
+{
+    return efuse_hal_get_chip_revision();
+}
+
+uint32_t bootloader_common_get_chip_ver_pkg(void)
 {
    return efuse_ll_get_chip_ver_pkg();
 }
--- a/components/bootloader_support/src/bootloader_init.c
+++ b/components/bootloader_support/src/bootloader_init.c
@@ -18,7 +18,6 @@
 #include "esp_cpu.h"
 #include "soc/rtc.h"
 #include "hal/wdt_hal.h"
-#include "hal/efuse_hal.h"

 static const char *TAG = "boot";

@@ -41,7 +40,9 @@ esp_err_t bootloader_read_bootloader_header(void)

 esp_err_t bootloader_check_bootloader_validity(void)
 {
-    ESP_LOGI(TAG, "chip revision: V%03d", efuse_hal_chip_revision());
+    /* read chip revision from efuse */
+    uint8_t revision = bootloader_common_get_chip_revision();
+    ESP_LOGI(TAG, "chip revision: %d", revision);
    /* compare with the one set in bootloader image header */
    if (bootloader_common_check_chip_validity(&bootloader_image_hdr, ESP_IMAGE_BOOTLOADER) != ESP_OK) {
        return ESP_FAIL;
--- a/components/bootloader_support/src/bootloader_mem.c
+++ b/components/bootloader_support/src/bootloader_mem.c
@@ -6,6 +6,7 @@

 #include <stdbool.h>

+#include "hal/cpu_hal.h"
 #include "hal/mpu_hal.h"
 #include "hal/mpu_types.h"
 #include "soc/soc_caps.h"
@@ -14,6 +15,8 @@

 void bootloader_init_mem(void)
 {
+    cpu_hal_init_hwloop();
+
 #ifdef CONFIG_BOOTLOADER_REGION_PROTECTION_ENABLE
    // protect memory region
    esp_cpu_configure_region_protection();
--- a/components/bootloader_support/src/bootloader_panic.c
+++ b/components/bootloader_support/src/bootloader_panic.c
@@ -4,9 +4,9 @@
 * SPDX-License-Identifier: Apache-2.0
 */

-#include "esp_cpu.h"
 #include "esp_log.h"
 #include "bootloader_common.h"
+#include "hal/cpu_hal.h"
 #include "esp_rom_sys.h"


@@ -22,8 +22,8 @@ void abort(void)
 #if !CONFIG_ESP_SYSTEM_PANIC_SILENT_REBOOT
    esp_rom_printf("abort() was called at PC 0x%08x\r\n", (intptr_t)__builtin_return_address(0) - 3);
 #endif
-    if (esp_cpu_dbgr_is_attached()) {
-        esp_cpu_dbgr_break();
+    if (cpu_hal_is_debugger_attached()) {
+        cpu_hal_break();
    }
    while (1) {
    }
--- a/components/bootloader_support/src/bootloader_random.c
+++ b/components/bootloader_support/src/bootloader_random.c
@@ -5,7 +5,7 @@
 */
 #include "sdkconfig.h"
 #include "bootloader_random.h"
-#include "esp_cpu.h"
+#include "hal/cpu_hal.h"
 #include "soc/wdev_reg.h"

 #ifndef BOOTLOADER_BUILD
@@ -43,10 +43,10 @@
               values.
            */
            random = REG_READ(WDEV_RND_REG);
-            start = esp_cpu_get_cycle_count();
+            start = cpu_hal_get_cycle_count();
            do {
                random ^= REG_READ(WDEV_RND_REG);
-                now = esp_cpu_get_cycle_count();
+                now = cpu_hal_get_cycle_count();
            } while (now - start < RNG_CPU_WAIT_CYCLE_NUM);
        }
        buffer_bytes[i] = random >> ((i % 4) * 8);
--- a/components/bootloader_support/src/bootloader_random_esp32c2.c
+++ b/components/bootloader_support/src/bootloader_random_esp32c2.c
@@ -11,57 +11,13 @@
 #include "soc/apb_saradc_reg.h"
 #include "soc/system_reg.h"
 #include "esp_private/regi2c_ctrl.h"
-#include "soc/regi2c_saradc.h"

 void bootloader_random_enable(void)
 {
-    /* RNG module is always clock enabled */
-    REG_SET_FIELD(RTC_CNTL_SENSOR_CTRL_REG, RTC_CNTL_FORCE_XPD_SAR, 0x3);
-    SET_PERI_REG_MASK(RTC_CNTL_ANA_CONF_REG, RTC_CNTL_SAR_I2C_PU_M);
-
-    // Bridging sar2 internal reference voltage
-    REGI2C_WRITE_MASK(I2C_SAR_ADC, ADC_SARADC2_ENCAL_REF_ADDR, 1);
-    REGI2C_WRITE_MASK(I2C_SAR_ADC, ADC_SARADC_DTEST_RTC_ADDR, 0);
-    REGI2C_WRITE_MASK(I2C_SAR_ADC, ADC_SARADC_ENT_RTC_ADDR, 0);
-    REGI2C_WRITE_MASK(I2C_SAR_ADC, ADC_SARADC_ENT_TSENS_ADDR, 0);
-
-    // Enable SAR ADC2 internal channel to read adc2 ref voltage for additional entropy
-    SET_PERI_REG_MASK(SYSTEM_PERIP_CLK_EN0_REG, SYSTEM_APB_SARADC_CLK_EN_M);
-    CLEAR_PERI_REG_MASK(SYSTEM_PERIP_RST_EN0_REG, SYSTEM_APB_SARADC_RST_M);
-    REG_SET_FIELD(APB_SARADC_APB_ADC_CLKM_CONF_REG, APB_SARADC_REG_CLK_SEL, 0x2);
-    SET_PERI_REG_MASK(APB_SARADC_APB_ADC_CLKM_CONF_REG, APB_SARADC_CLK_EN_M);
-    SET_PERI_REG_MASK(APB_SARADC_CTRL_REG, APB_SARADC_SAR_CLK_GATED_M);
-    REG_SET_FIELD(APB_SARADC_CTRL_REG, APB_SARADC_XPD_SAR_FORCE, 0x3);
-    REG_SET_FIELD(APB_SARADC_CTRL_REG, APB_SARADC_SAR_CLK_DIV, 1);
-
-    REG_SET_FIELD(APB_SARADC_FSM_WAIT_REG, APB_SARADC_RSTB_WAIT, 8);
-    REG_SET_FIELD(APB_SARADC_FSM_WAIT_REG, APB_SARADC_XPD_WAIT, 5);
-    REG_SET_FIELD(APB_SARADC_FSM_WAIT_REG, APB_SARADC_STANDBY_WAIT, 100);
-
-    SET_PERI_REG_MASK(APB_SARADC_CTRL_REG, APB_SARADC_SAR_PATT_P_CLEAR_M);
-    CLEAR_PERI_REG_MASK(APB_SARADC_CTRL_REG, APB_SARADC_SAR_PATT_P_CLEAR_M);
-    REG_SET_FIELD(APB_SARADC_CTRL_REG, APB_SARADC_SAR_PATT_LEN, 0);
-    REG_SET_FIELD(APB_SARADC_SAR_PATT_TAB1_REG, APB_SARADC_SAR_PATT_TAB1, 0x9cffff);// Set adc2 internal channel & atten
-    REG_SET_FIELD(APB_SARADC_SAR_PATT_TAB2_REG, APB_SARADC_SAR_PATT_TAB2, 0xffffff);
-    // Set ADC sampling frequency
-    REG_SET_FIELD(APB_SARADC_CTRL2_REG, APB_SARADC_TIMER_TARGET, 100);
-    REG_SET_FIELD(APB_SARADC_APB_ADC_CLKM_CONF_REG, APB_SARADC_REG_CLKM_DIV_NUM, 15);
-    CLEAR_PERI_REG_MASK(APB_SARADC_CTRL2_REG, APB_SARADC_MEAS_NUM_LIMIT);
-    SET_PERI_REG_MASK(APB_SARADC_DMA_CONF_REG, APB_SARADC_APB_ADC_TRANS_M);
-    SET_PERI_REG_MASK(APB_SARADC_CTRL2_REG, APB_SARADC_TIMER_EN);
+    // TODO: IDF-4021
 }

 void bootloader_random_disable(void)
 {
-    /* Restore internal I2C bus state */
-    REGI2C_WRITE_MASK(I2C_SAR_ADC, ADC_SARADC2_ENCAL_REF_ADDR, 0);
-
-    /* Restore SARADC to default mode */
-    CLEAR_PERI_REG_MASK(APB_SARADC_CTRL2_REG, APB_SARADC_TIMER_EN);
-    CLEAR_PERI_REG_MASK(APB_SARADC_DMA_CONF_REG, APB_SARADC_APB_ADC_TRANS_M);
-    REG_SET_FIELD(APB_SARADC_SAR_PATT_TAB1_REG, APB_SARADC_SAR_PATT_TAB1, 0xffffff);
-    REG_SET_FIELD(APB_SARADC_SAR_PATT_TAB2_REG, APB_SARADC_SAR_PATT_TAB2, 0xffffff);
-    CLEAR_PERI_REG_MASK(APB_SARADC_APB_ADC_CLKM_CONF_REG, APB_SARADC_CLK_EN_M);
-    REG_SET_FIELD(APB_SARADC_CTRL_REG, APB_SARADC_XPD_SAR_FORCE, 0);
-    REG_SET_FIELD(RTC_CNTL_SENSOR_CTRL_REG, RTC_CNTL_FORCE_XPD_SAR, 0);
+    // TODO: IDF-4021
 }
--- a/components/bootloader_support/src/bootloader_random_esp32c3.c
+++ b/components/bootloader_support/src/bootloader_random_esp32c3.c
@@ -11,7 +11,7 @@
 #include "soc/apb_saradc_reg.h"
 #include "soc/system_reg.h"
 #include "esp_private/regi2c_ctrl.h"
-#include "soc/regi2c_saradc.h"
+#include "regi2c_saradc.h"

 void bootloader_random_enable(void)
 {
--- a/components/bootloader_support/src/bootloader_random_esp32s3.c
+++ b/components/bootloader_support/src/bootloader_random_esp32s3.c
@@ -11,8 +11,8 @@
 #include "soc/apb_saradc_reg.h"
 #include "soc/rtc_cntl_reg.h"
 #include "soc/sens_reg.h"
-#include "hal/regi2c_ctrl.h"
-#include "soc/regi2c_saradc.h"
+#include "esp_private/regi2c_ctrl.h"
+#include "regi2c_saradc.h"

 void bootloader_random_enable(void)
 {
@@ -83,6 +83,8 @@ void bootloader_random_disable(void)
    REGI2C_WRITE_MASK(I2C_SAR_ADC, ADC_SARADC_ENT_RTC_ADDR, 0);
    REGI2C_WRITE_MASK(I2C_SAR_ADC, ADC_SARADC_DTEST_RTC_ADDR, 0);

+    //Stop SAR ADC clock
+    CLEAR_PERI_REG_MASK(SENS_SAR_PERI_CLK_GATE_CONF_REG, SENS_SARADC_CLK_EN);
    //Power off SAR ADC
    REG_SET_FIELD(SENS_SAR_POWER_XPD_SAR_REG, SENS_FORCE_XPD_SAR, 0);
    //return to ADC RTC controller
--- a/components/bootloader_support/src/bootloader_utility.c
+++ b/components/bootloader_support/src/bootloader_utility.c
@@ -478,7 +478,7 @@ void bootloader_utility_load_boot_image(const bootloader_state_t *bs, int start_
 {
    int index = start_index;
    esp_partition_pos_t part;
-    esp_image_metadata_t image_data = {0};
+    esp_image_metadata_t image_data;

    if (start_index == TEST_APP_INDEX) {
        if (check_anti_rollback(&bs->test) && try_load_partition(&bs->test, &image_data)) {
@@ -577,17 +577,6 @@ static void load_image(const esp_image_metadata_t *image_data)
    esp_err_t err;
 #endif

-#ifdef CONFIG_SECURE_BOOT_FLASH_ENC_KEYS_BURN_TOGETHER
-    if (esp_secure_boot_enabled() ^ esp_flash_encrypt_initialized_once()) {
-        ESP_LOGE(TAG, "Secure Boot and Flash Encryption cannot be enabled separately, only together (their keys go into one eFuse key block)");
-        return;
-    }
-
-    if (!esp_secure_boot_enabled() || !esp_flash_encryption_enabled()) {
-        esp_efuse_batch_write_begin();
-    }
-#endif // CONFIG_SECURE_BOOT_FLASH_ENC_KEYS_BURN_TOGETHER
-
 #ifdef CONFIG_SECURE_BOOT_V2_ENABLED
    err = esp_secure_boot_v2_permanently_enable(image_data);
    if (err != ESP_OK) {
@@ -615,50 +604,13 @@ static void load_image(const esp_image_metadata_t *image_data)
     *   5) Burn EFUSE to enable flash encryption
     */
    ESP_LOGI(TAG, "Checking flash encryption...");
-    bool flash_encryption_enabled = esp_flash_encrypt_state();
-    if (!flash_encryption_enabled) {
-#ifdef CONFIG_SECURE_FLASH_REQUIRE_ALREADY_ENABLED
-        ESP_LOGE(TAG, "flash encryption is not enabled, and SECURE_FLASH_REQUIRE_ALREADY_ENABLED is set, refusing to boot.");
+    bool flash_encryption_enabled = esp_flash_encryption_enabled();
+    err = esp_flash_encrypt_check_and_update();
+    if (err != ESP_OK) {
+        ESP_LOGE(TAG, "Flash encryption check failed (%d).", err);
        return;
-#endif // CONFIG_SECURE_FLASH_REQUIRE_ALREADY_ENABLED
-
-        if (esp_flash_encrypt_is_write_protected(true)) {
-            return;
-        }
-
-        err = esp_flash_encrypt_init();
-        if (err != ESP_OK) {
-            ESP_LOGE(TAG, "Initialization of Flash Encryption key failed (%d)", err);
-            return;
-        }
    }
-
-#ifdef CONFIG_SECURE_BOOT_FLASH_ENC_KEYS_BURN_TOGETHER
-    if (!esp_secure_boot_enabled() || !flash_encryption_enabled) {
-        err = esp_efuse_batch_write_commit();
-        if (err != ESP_OK) {
-            ESP_LOGE(TAG, "Error programming eFuses (err=0x%x).", err);
-            return;
-        }
-        assert(esp_secure_boot_enabled());
-        ESP_LOGI(TAG, "Secure boot permanently enabled");
-    }
-#endif // CONFIG_SECURE_BOOT_FLASH_ENC_KEYS_BURN_TOGETHER
-
-    if (!flash_encryption_enabled) {
-        err = esp_flash_encrypt_contents();
-        if (err != ESP_OK) {
-            ESP_LOGE(TAG, "Encryption flash contents failed (%d)", err);
-            return;
-        }
-
-        err = esp_flash_encrypt_enable();
-        if (err != ESP_OK) {
-            ESP_LOGE(TAG, "Enabling of Flash encryption failed (%d)", err);
-            return;
-        }
-    }
-#endif // CONFIG_SECURE_FLASH_ENC_ENABLED
+#endif

 #ifdef CONFIG_SECURE_BOOT_V1_ENABLED
    /* Step 6 (see above for full description):
@@ -772,12 +724,12 @@ static void set_cache_and_start_app(
    //The addr is aligned, so we add the mask off length to the size, to make sure the corresponding buses are enabled.
    drom_size = (drom_load_addr - drom_load_addr_aligned) + drom_size;
 #if CONFIG_IDF_TARGET_ESP32
-    uint32_t drom_page_count = (drom_size + SPI_FLASH_MMU_PAGE_SIZE - 1) / SPI_FLASH_MMU_PAGE_SIZE;
+    uint32_t drom_page_count = (drom_size + MMU_PAGE_SIZE - 1) / MMU_PAGE_SIZE;
    rc = cache_flash_mmu_set(0, 0, drom_load_addr_aligned, drom_addr_aligned, 64, drom_page_count);
    ESP_EARLY_LOGV(TAG, "rc=%d", rc);
    rc = cache_flash_mmu_set(1, 0, drom_load_addr_aligned, drom_addr_aligned, 64, drom_page_count);
    ESP_EARLY_LOGV(TAG, "rc=%d", rc);
-    ESP_EARLY_LOGV(TAG, "after mapping rodata, starting from paddr=0x%08x and vaddr=0x%08x, 0x%x bytes are mapped", drom_addr_aligned, drom_load_addr_aligned, drom_page_count * SPI_FLASH_MMU_PAGE_SIZE);
+    ESP_EARLY_LOGV(TAG, "after mapping rodata, starting from paddr=0x%08x and vaddr=0x%08x, 0x%x bytes are mapped", drom_addr_aligned, drom_load_addr_aligned, drom_page_count * MMU_PAGE_SIZE);
 #else
    uint32_t actual_mapped_len = 0;
    mmu_hal_map_region(0, MMU_TARGET_FLASH0, drom_load_addr_aligned, drom_addr_aligned, drom_size, &actual_mapped_len);
@@ -791,12 +743,12 @@ static void set_cache_and_start_app(
    //The addr is aligned, so we add the mask off length to the size, to make sure the corresponding buses are enabled.
    irom_size = (irom_load_addr - irom_load_addr_aligned) + irom_size;
 #if CONFIG_IDF_TARGET_ESP32
-    uint32_t irom_page_count = (irom_size + SPI_FLASH_MMU_PAGE_SIZE - 1) / SPI_FLASH_MMU_PAGE_SIZE;
+    uint32_t irom_page_count = (irom_size + MMU_PAGE_SIZE - 1) / MMU_PAGE_SIZE;
    rc = cache_flash_mmu_set(0, 0, irom_load_addr_aligned, irom_addr_aligned, 64, irom_page_count);
    ESP_EARLY_LOGV(TAG, "rc=%d", rc);
    rc = cache_flash_mmu_set(1, 0, irom_load_addr_aligned, irom_addr_aligned, 64, irom_page_count);
    ESP_LOGV(TAG, "rc=%d", rc);
-    ESP_EARLY_LOGV(TAG, "after mapping text, starting from paddr=0x%08x and vaddr=0x%08x, 0x%x bytes are mapped", irom_addr_aligned, irom_load_addr_aligned, irom_page_count * SPI_FLASH_MMU_PAGE_SIZE);
+    ESP_EARLY_LOGV(TAG, "after mapping text, starting from paddr=0x%08x and vaddr=0x%08x, 0x%x bytes are mapped", irom_addr_aligned, irom_load_addr_aligned, irom_page_count * MMU_PAGE_SIZE);
 #else
    mmu_hal_map_region(0, MMU_TARGET_FLASH0, irom_load_addr_aligned, irom_addr_aligned, irom_size, &actual_mapped_len);
    ESP_EARLY_LOGV(TAG, "after mapping text, starting from paddr=0x%08x and vaddr=0x%08x, 0x%x bytes are mapped", irom_addr_aligned, irom_load_addr_aligned, actual_mapped_len);
--- a/components/bootloader_support/src/esp32/bootloader_esp32.c
+++ b/components/bootloader_support/src/esp32/bootloader_esp32.c
@@ -27,8 +27,6 @@
 #include "soc/rtc.h"
 #include "soc/spi_periph.h"
 #include "hal/gpio_hal.h"
-#include "xtensa/config/core.h"
-#include "xt_instr_macros.h"

 #include "esp32/rom/cache.h"
 #include "esp_rom_gpio.h"
@@ -352,11 +350,6 @@ esp_err_t bootloader_init(void)
 {
    esp_err_t ret = ESP_OK;

-#if XCHAL_ERRATUM_572
-    uint32_t memctl = XCHAL_CACHE_MEMCTL_DEFAULT;
-    WSR(MEMCTL, memctl);
-#endif // XCHAL_ERRATUM_572
-
    bootloader_init_mem();

    // check that static RAM is after the stack
--- a/components/bootloader_support/src/esp32c2/bootloader_esp32c2.c
+++ b/components/bootloader_support/src/esp32c2/bootloader_esp32c2.c
@@ -263,8 +263,6 @@ esp_err_t bootloader_init(void)
    cache_hal_init();
    //reset mmu
    mmu_hal_init();
-    // config mmu page size
-    mmu_ll_set_page_size(0, SPI_FLASH_MMU_PAGE_SIZE);
    // config clock
    bootloader_clock_configure();
    // initialize console, from now on, we can use esp_log
--- a/components/bootloader_support/src/esp32c2/flash_encryption_secure_features.c
+++ b/components/bootloader_support/src/esp32c2/flash_encryption_secure_features.c
@@ -23,12 +23,7 @@ esp_err_t esp_flash_encryption_enable_secure_features(void)
    ESP_LOGW(TAG, "Not disabling UART bootloader encryption");
 #endif

-#ifndef CONFIG_SECURE_FLASH_UART_BOOTLOADER_ALLOW_CACHE
    ESP_LOGI(TAG, "Disable UART bootloader cache...");
-    esp_efuse_write_field_bit(ESP_EFUSE_DIS_DOWNLOAD_ICACHE);
-#else
-    ESP_LOGW(TAG, "Not disabling UART bootloader cache - SECURITY COMPROMISED");
-#endif

 #ifndef CONFIG_SECURE_BOOT_ALLOW_JTAG
    ESP_LOGI(TAG, "Disable JTAG...");
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Tobias Schramm	0446d77c3e	bootloader: ensure security eFuses are always burnt Previously security eFuses were only burnt if the flash was not encrypted yet. To enhance robustness of the security eFuse settings their correct setup should be verified on each bootup. Else it would be possible for an already encrypted ESP to be reflashed with firmware containing updated, more restrictive eFuse settings without them ever being applied. Additionally this change enables easy, secure use of ESPs with host sidee flash preencryption. Flash preencryption by the host computer performing the programming procedure can speed up the programming process by a great deal since the flash no longer needs to be read, erased and written again by the bootloader self-encryption routines. Additionally it avoids bricking of ESPs through interruption of the self-ecnryption procedure. Without this change the host would have to set up all fuses in the ESP correctly by itself, duplicating the fuse configuration code already present in the bootloader and creating additional maintenance burden for the host software if anything about the fuse setup logic changes. This commit changes the security eFuse configuration logic to always burn any configured security eFuses on bootup, regardless of current flash encryption status.	2022-07-19 12:42:55 +02:00
0xFEEDC0DE64	dc0b72f273	No more noreturn on asserts	2022-07-11 15:07:23 +02:00
0xFEEDC0DE64	fca498262f	Add esp_tls option to skip server verification	2022-07-11 15:07:16 +02:00
0xFEEDC0DE64	8c27cbfa13	Return EAGAIN in http client perform	2022-05-24 11:48:32 +02:00
0xFEEDC0DE64	b8147adbfb	log format improvements after rebase	2022-05-24 11:48:32 +02:00
0xFEEDC0DE64	c3ee0811cf	Fix random crash with Plex mdns queries on my network	2022-05-24 11:48:32 +02:00
0xFEEDC0DE64	23e9d15d19	Fixed crash in coredump handler, when using printf arguments	2022-05-24 11:48:32 +02:00
0xFEEDC0DE64	fdd9bcff7c	Reduced error log when calling esp_efuse_mac_get_custom()	2022-05-24 11:48:32 +02:00
0xFEEDC0DE64	38538babac	Fixed coredump logging	2022-05-24 11:48:32 +02:00
0xFEEDC0DE64	83f91ceea8	Add support for X-WWW-Authenticate header	2022-05-24 11:48:32 +02:00
0xFEEDC0DE64	07984ad576	Added filename, line number and method name to log output	2022-05-24 11:48:31 +02:00