Merge remote-tracking branch 'upstream/dev' into ci-uv-managed-python

# Conflicts: # .github/workflows/ci.yaml
Merge branch 'ci-cache-postgres-mariadb-deps' into ci-uv-managed-python
2026-05-27 19:25:18 +02:00 · 2026-05-26 18:24:40 -05:00 · 2026-05-21 14:33:44 -05:00 · 2026-05-21 14:32:58 -05:00 · 2026-05-21 13:37:03 -05:00 · 2026-05-21 13:16:39 -05:00
2 changed files with 87 additions and 47 deletions
@@ -0,0 +1,42 @@
 name: Set up uv and managed Python
 description: >-
  Pins uv (avoids the raw.githubusercontent.com manifest fetch on cache miss)
  and proactively installs the requested Python so cached venvs created with
  `uv venv` resolve their interpreter symlinks in jobs that only restore the
  venv. setup-uv alone only sets UV_PYTHON, it does not actually fetch the
  interpreter until uv first uses it, so jobs that just activate the venv
  blow up with broken symlinks on cache hit.
 inputs:
  python-version:
    description: The Python version uv should install and use.
    required: true
  uv-version:
    description: The uv version setup-uv should install.
    required: true
 outputs:
  python-version:
    description: The Python version uv reports as installed.
    value: ${{ steps.uv.outputs.python-version }}
 runs:
  using: composite
  steps:
    - name: Set up uv
      id: uv
      uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
      with:
        version: ${{ inputs.uv-version }}
        python-version: ${{ inputs.python-version }}
        # Persist astral's managed Python across jobs so 'uv venv' below is
        # fast on the second job onwards.
        cache-python: true
        # Lint-only and codegen jobs touch no Python deps, so the post-step
        # cache save would otherwise abort the job.
        ignore-nothing-to-cache: true
    - name: Install Python interpreter
      shell: bash
      env:
        PYTHON_VERSION: ${{ inputs.python-version }}
      run: uv python install "${PYTHON_VERSION}"
@@ -37,7 +37,7 @@ on:
        type: boolean
 env:
-  CACHE_VERSION: 3
+  CACHE_VERSION: 4
  MYPY_CACHE_VERSION: 1
  HA_SHORT_VERSION: "2026.6"
  ADDITIONAL_PYTHON_VERSIONS: "[]"
@@ -89,6 +89,8 @@ jobs:
      mariadb_groups: ${{ steps.info.outputs.mariadb_groups }}
      postgresql_groups: ${{ steps.info.outputs.postgresql_groups }}
      python_versions: ${{ steps.info.outputs.python_versions }}
      default_python: ${{ steps.info.outputs.default_python }}
      uv_version: ${{ steps.info.outputs.uv_version }}
      test_full_suite: ${{ steps.info.outputs.test_full_suite }}
      test_group_count: ${{ steps.info.outputs.test_group_count }}
      test_groups: ${{ steps.info.outputs.test_groups }}
@@ -235,6 +237,11 @@ jobs:
          echo "postgresql_groups=${postgresql_groups}" >> $GITHUB_OUTPUT
          echo "python_versions: ${all_python_versions}"
          echo "python_versions=${all_python_versions}" >> $GITHUB_OUTPUT
          echo "default_python: ${default_python}"
          echo "default_python=${default_python}" >> $GITHUB_OUTPUT
          uv_version=$(grep '^uv==' requirements.txt | cut -d'=' -f3)
          echo "uv_version: ${uv_version}"
          echo "uv_version=${uv_version}" >> $GITHUB_OUTPUT
          echo "test_full_suite: ${test_full_suite}"
          echo "test_full_suite=${test_full_suite}" >> $GITHUB_OUTPUT
          echo "integrations_glob: ${integrations_glob}"
@@ -344,12 +351,12 @@ jobs:
        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
        with:
          persist-credentials: false
-      - name: Set up Python ${{ matrix.python-version }}
+      - name: Set up uv and Python ${{ matrix.python-version }}
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
          uv-version: ${{ needs.info.outputs.uv_version }}
          python-version: ${{ matrix.python-version }}
          check-latest: true
      - name: Restore base Python virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -397,21 +404,13 @@ jobs:
            libudev-dev
          version: ${{ env.APT_CACHE_VERSION }}
          execute_install_scripts: true
      - name: Read uv version from requirements.txt
        if: steps.cache-venv.outputs.cache-hit != 'true'
        id: read-uv-version
        run: |
          echo "version=$(grep '^uv==' requirements.txt | cut -d'=' -f3)" >> "$GITHUB_OUTPUT"
      - name: Set up uv
        if: steps.cache-venv.outputs.cache-hit != 'true'
        uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
        with:
          version: ${{ steps.read-uv-version.outputs.version }}
      - name: Create Python virtual environment
        if: steps.cache-venv.outputs.cache-hit != 'true'
        id: create-venv
        env:
          PYTHON_VERSION: ${{ steps.python.outputs.python-version }}
        run: |
-          python -m venv venv
+          uv venv venv --python "${PYTHON_VERSION}"
          . venv/bin/activate
          python --version
          uv pip install -r requirements.txt
@@ -419,7 +418,6 @@ jobs:
          uv pip install -e . --config-settings editable_mode=compat
      - name: Dump pip freeze
        run: |
          python -m venv venv
          . venv/bin/activate
          python --version
          uv pip freeze >> pip_freeze.txt
@@ -480,10 +478,10 @@ jobs:
          version: ${{ env.APT_CACHE_VERSION }}
      - name: Set up Python
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
-          python-version-file: ".python-version"
+          uv-version: ${{ needs.info.outputs.uv_version }}
-          check-latest: true
+          python-version: ${{ needs.info.outputs.default_python }}
      - name: Restore full Python virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -517,10 +515,10 @@ jobs:
          persist-credentials: false
      - name: Set up Python
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
-          python-version-file: ".python-version"
+          uv-version: ${{ needs.info.outputs.uv_version }}
-          check-latest: true
+          python-version: ${{ needs.info.outputs.default_python }}
      - name: Restore full Python virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -553,10 +551,10 @@ jobs:
          persist-credentials: false
      - name: Set up Python
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
-          python-version-file: ".python-version"
+          uv-version: ${{ needs.info.outputs.uv_version }}
-          check-latest: true
+          python-version: ${{ needs.info.outputs.default_python }}
      - name: Run gen_copilot_instructions.py
        run: |
          python -m script.gen_copilot_instructions validate
@@ -608,10 +606,10 @@ jobs:
          persist-credentials: false
      - name: Set up Python ${{ matrix.python-version }}
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
          uv-version: ${{ needs.info.outputs.uv_version }}
          python-version: ${{ matrix.python-version }}
          check-latest: true
      - name: Restore full Python ${{ matrix.python-version }} virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -659,10 +657,10 @@ jobs:
          persist-credentials: false
      - name: Set up Python
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
-          python-version-file: ".python-version"
+          uv-version: ${{ needs.info.outputs.uv_version }}
-          check-latest: true
+          python-version: ${{ needs.info.outputs.default_python }}
      - name: Restore full Python virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -712,10 +710,10 @@ jobs:
          persist-credentials: false
      - name: Set up Python
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
-          python-version-file: ".python-version"
+          uv-version: ${{ needs.info.outputs.uv_version }}
-          check-latest: true
+          python-version: ${{ needs.info.outputs.default_python }}
      - name: Restore full Python virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -763,10 +761,10 @@ jobs:
          persist-credentials: false
      - name: Set up Python
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
-          python-version-file: ".python-version"
+          uv-version: ${{ needs.info.outputs.uv_version }}
-          check-latest: true
+          python-version: ${{ needs.info.outputs.default_python }}
      - name: Generate partial mypy restore key
        id: generate-mypy-key
        run: |
@@ -840,10 +838,10 @@ jobs:
          execute_install_scripts: true
      - name: Set up Python
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
-          python-version-file: ".python-version"
+          uv-version: ${{ needs.info.outputs.uv_version }}
-          check-latest: true
+          python-version: ${{ needs.info.outputs.default_python }}
      - name: Restore full Python virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -905,10 +903,10 @@ jobs:
          execute_install_scripts: true
      - name: Set up Python ${{ matrix.python-version }}
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
          uv-version: ${{ needs.info.outputs.uv_version }}
          python-version: ${{ matrix.python-version }}
          check-latest: true
      - name: Restore full Python ${{ matrix.python-version }} virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -1047,10 +1045,10 @@ jobs:
          execute_install_scripts: true
      - name: Set up Python ${{ matrix.python-version }}
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
          uv-version: ${{ needs.info.outputs.uv_version }}
          python-version: ${{ matrix.python-version }}
          check-latest: true
      - name: Restore full Python ${{ matrix.python-version }} virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -1203,10 +1201,10 @@ jobs:
          version: ${{ env.APT_CACHE_VERSION }}
      - name: Set up Python ${{ matrix.python-version }}
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
          uv-version: ${{ needs.info.outputs.uv_version }}
          python-version: ${{ matrix.python-version }}
          check-latest: true
      - name: Restore full Python ${{ matrix.python-version }} virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
@@ -1371,10 +1369,10 @@ jobs:
          execute_install_scripts: true
      - name: Set up Python ${{ matrix.python-version }}
        id: python
-        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        uses: ./.github/actions/setup-uv-python
        with:
          uv-version: ${{ needs.info.outputs.uv_version }}
          python-version: ${{ matrix.python-version }}
          check-latest: true
      - name: Restore full Python ${{ matrix.python-version }} virtual environment
        id: cache-venv
        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
Author	SHA1	Message	Date
J. Nick Koston	51cddb88f5	Merge remote-tracking branch 'upstream/dev' into ci-uv-managed-python # Conflicts: # .github/workflows/ci.yaml	2026-05-26 18:24:40 -05:00
J. Nick Koston	748a9842af	Merge branch 'ci-cache-postgres-mariadb-deps' into ci-uv-managed-python	2026-05-21 14:33:44 -05:00
J. Nick Koston	55786dbdfc	Use dpkg-architecture to derive multiarch lib path So the ldconfig workaround also works on non-x86_64 runners.	2026-05-21 14:32:58 -05:00
J. Nick Koston	e88c03a437	Merge branch 'dev' into ci-cache-postgres-mariadb-deps	2026-05-21 13:37:03 -05:00
J. Nick Koston	dbc0dc1ea6	Install Python eagerly via setup-uv-python composite setup-uv only sets UV_PYTHON, it does not actually fetch the interpreter; uv installs it lazily on the first 'uv venv' or 'uv pip install'. When the venv cache hits, no uv command runs, so the cached venv's bin/python3 symlink points at an interpreter that was never installed in this job and the next step that activates the venv aborts with 'broken symlink'. Extract setup-uv plus an explicit 'uv python install' into the .github/actions/setup-uv-python composite action so every job that restores the venv ends up with a real Python at the expected path. Enable cache-python in the wrapper so subsequent jobs reuse astral's download instead of refetching it.	2026-05-21 13:16:39 -05:00
J. Nick Koston	31271876bf	Pin uv version in every setup-uv call to skip manifest fetch Mirrors esphome/esphome#16534. Without an explicit version, setup-uv fetches uv.ndjson from raw.githubusercontent.com on every cache miss, which periodically times out and fails the job. Expose the uv version from requirements.txt via the info job and pass it to every setup-uv call. Also set ignore-nothing-to-cache: true so jobs that do not touch uv (e.g. gen-copilot-instructions) no longer fail on the post-step cache save.	2026-05-21 13:08:57 -05:00
J. Nick Koston	d5c31332b5	Switch CI to astral-managed Python via setup-uv Replace actions/setup-python with astral-sh/setup-uv so every job uses the python-build-standalone interpreter astral ships, which bakes in the PEP 744 tail call interpreter on 3.14. setup-uv handles both installing uv and provisioning the requested Python version, so the venv bootstrap uses 'uv venv' instead of 'python -m venv' and there is no longer a separate uv install step on cache miss. Bumps CACHE_VERSION so the old setup-python venv caches are invalidated; the venv symlinks would otherwise point at the absent hostedtoolcache interpreter.	2026-05-21 13:05:30 -05:00
J. Nick Koston	3f0c93c26c	Merge branch 'dev' into ci-cache-postgres-mariadb-deps	2026-05-21 12:48:19 -05:00
J. Nick Koston	07ed913ba2	Extract apt caching into composite action with alternatives workaround Wrap awalsh128/cache-apt-pkgs-action in .github/actions/cache-apt-packages so every job uses the same pattern, and route /usr/lib/x86_64-linux-gnu subdirectories through ldconfig. The upstream action does not run postinst on cache restore so update-alternatives symlinks (libblas, liblapack via ffmpeg) never appear; adding the subdirs to ld.so.conf.d lets the linker find the real libraries without those symlinks.	2026-05-21 10:45:13 -05:00
J. Nick Koston	b7905b163f	Run ldconfig after cache-apt-pkgs-action restore The action restores cached .deb files to disk but skips dpkg-trigger so /etc/ld.so.cache stays stale and ctypes-based loaders (eg opuslib) cannot find libopus.so.0. Add an explicit ldconfig step after each action call.	2026-05-21 10:02:39 -05:00
J. Nick Koston	c712b07da3	Switch CI apt caching to awalsh128/cache-apt-pkgs-action	2026-05-21 09:42:20 -05:00