Merge branch 'feature/idf_tools_github_mirror_v4.1' into 'release/v4.1'

Add GitHub mirror option for IDF tools installs (v4.1)

See merge request espressif/esp-idf!13766
This commit is contained in:
Angus Gratton
2021-06-01 09:48:12 +00:00
10 changed files with 124 additions and 19 deletions

View File

@ -192,6 +192,35 @@ Linux and macOS
cd ~/esp/esp-idf
./install.sh
Alternative File Downloads
~~~~~~~~~~~~~~~~~~~~~~~~~~
The tools installer downloads a number of files attached to GitHub Releases. If accessing GitHub is slow then it is possible to set an environment variable to prefer Espressif's download server for GitHub asset downloads.
.. note:: This setting only controls individual tools downloaded from GitHub releases, it doesn't change the URLs used to access any Git repositories.
Windows
-------
To prefer the Espressif download server when running the ESP-IDF Tools Installer or installing tools from the command line, open the System control panel, then click on Advanced Settings. Add a new Environment Variable (of type either User or System) with the name ``IDF_GITHUB_ASSETS`` and value ``dl.espressif.com/github_assets``. Click OK once done.
If the command line window or ESP-IDF Tools Installer window was already open before you added the new environment variable, you will need to close and reopen it.
While this environment variable is still set, the ESP-IDF Tools Installer and the command line installer will prefer the Espressif download server.
.. Once the ESP-IDF Tools Installer binary is updated to include the checkbox, the above can be rewritten to refer to it
Linux and macOS
---------------
To prefer the Espressif download server when installing tools, use the following sequence of commands when running ``install.sh``:
.. code-block:: bash
cd ~/esp/esp-idf
export IDF_GITHUB_ASSETS="dl.espressif.com/github_assets"
./install.sh
Customizing the tools installation path
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

View File

@ -26,7 +26,15 @@ The easiest way to install ESP-IDF's prerequisites is to download the ESP-IDF To
https://dl.espressif.com/dl/esp-idf-tools-setup-2.3.exe
The installer includes the cross-compilers, OpenOCD, cmake_ and Ninja_ build tool. The installer can also download and run installers for Python_ 3.7 and `Git For Windows`_ if they are not already installed on the computer.
.. IMPORTANT: Next time this link is updated, please go to get-started/index.rst and rewrite the section under "Alternative File Downloads ... Windows". Then delete this comment.
The installer includes the cross-compilers, OpenOCD, CMake_ and Ninja_ build tool. The installer can also download and run installers for Python_ 3.7 and `Git For Windows`_ if they are not already installed on the computer.
The installer also offers to download one of the ESP-IDF release versions. Please choose a directory for downloading ESP-IDF. The recommended directory is ``%userprofile%\esp`` where ``%userprofile%`` is your home directory. If you do not have it yet, please run the following command to create a new one:
.. code-block:: batch
mkdir %userprofile%\esp
The installer also offers to download one of the ESP-IDF release versions.
@ -59,7 +67,7 @@ For advanced users who want to customize the install process:
windows-setup-update
.. _MSYS2: https://www.msys2.org/
.. _cmake: https://cmake.org/download/
.. _CMake: https://cmake.org/download/
.. _ninja: https://ninja-build.org/
.. _Python: https://www.python.org/downloads/windows/
.. _Git for Windows: https://gitforwindows.org/

View File

@ -112,7 +112,7 @@ Bootloader Size
When secure boot is enabled the bootloader app binary ``bootloader.bin`` may exceed the default bootloader size limit. This is especially likely if flash encryption is enabled as well. The default size limit is 0x7000 (28672) bytes (partition table offset 0x8000 - bootloader offset 0x1000).
If the bootloader becomes too large, the {IDF_TARGET_NAME} will fail to boot - errors will be logged about either invalid partition table or invalid bootloader checksum.
If the bootloader becomes too large, the ESP32 will fail to boot - errors will be logged about either invalid partition table or invalid bootloader checksum.
When Secure Boot V2 is enabled, there is also an absolute binary size limit of {IDF_TARGET_MAX_BOOTLOADER_SIZE} (excluding the 4KB signature), because the bootloader is first loaded into a fixed size buffer for verification.

View File

@ -192,6 +192,37 @@ Linux 和 macOS 操作系统
cd ~/esp/esp-idf
./install.sh
下载工具备选方案
~~~~~~~~~~~~~~~~~~~~~~~~~~
ESP-IDF 工具安装器会下载 Github 发布版本中附带的一些工具,如果访问 Github 较为缓慢,则可以设置一个环境变量,实现优先选择 Espressif 的下载服务器进行 Github 资源下载。
.. 注解:: 该设置只影响从 Github 发布版本中下载的单个工具,它并不会改变访问任何 Git 仓库的 URL。
Windows 操作系统
-----------------
如果希望在运行 ESP-IDF 工具安装器或在使用命令行安装工具时优先选择 Espressif 下载服务器,可通过以下方式设置:打开系统控制面板,然后点击高级设置,添加一个新的环境变量(类型为用户或系统都可以,名称为 ``IDF_GITHUB_ASSETS``,值为 ``dl.espressif.com/github_assets``),最后点击确定。
如果在添加新的环境变量前命令行窗口或 ESP-IDF 工具安装器窗口已经打开,请关闭这些窗口后重新打开。
当设置好这个新的环境变量后ESP-IDF 工具安装器以及命令行安装程序将会优先选择 Espressif 下载服务器。
.. 在 ESP-IDF 工具安装器的二进制文件更新后(导入复选框),这段需要重新更新
Linux 和 macOS 操作系统
--------------------------
要在安装工具时优先选择 Espressif 下载服务器,请在运行 ``install.sh`` 时使用以下命令:
.. code-block:: bash
cd ~/esp/esp-idf
export IDF_GITHUB_ASSETS="dl.espressif.com/github_assets"
./install.sh
自定义工具安装路径
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

View File

@ -32,7 +32,7 @@ Linux 平台工具链的标准设置
权限问题 /dev/ttyUSB0
------------------------------------------------------------
使用某些 Linux 版本向 {IDF_TARGET_NAME} 烧录固件时,可能会出现 ``Failed to open port /dev/ttyUSB0`` 错误消息。此时可以将用户添加至 :ref:`Linux Dialout 组<linux-dialout-group>`
使用某些 Linux 版本向 ESP32 烧录固件时,可能会出现 ``Failed to open port /dev/ttyUSB0`` 错误消息。此时可以将用户添加至 :ref:`Linux Dialout 组<linux-dialout-group>`
修复 Ubuntu 16.04 损坏的 pip
=================================

View File

@ -1053,6 +1053,11 @@ def action_export(args):
raise SystemExit(1)
def apply_url_mirrors(args, tool_download_obj):
apply_mirror_prefix_map(args, tool_download_obj)
apply_github_assets_option(tool_download_obj)
def apply_mirror_prefix_map(args, tool_download_obj):
"""Rewrite URL for given tool_obj, given tool_version, and current platform,
if --mirror-prefix-map flag or IDF_MIRROR_PREFIX_MAP environment variable is given.
@ -1080,6 +1085,32 @@ def apply_mirror_prefix_map(args, tool_download_obj):
break
def apply_github_assets_option(tool_download_obj):
""" Rewrite URL for given tool_obj if the download URL is an https://github.com/ URL and the variable
IDF_GITHUB_ASSETS is set. The github.com part of the URL will be replaced.
"""
try:
github_assets = os.environ["IDF_GITHUB_ASSETS"].strip()
except KeyError:
return # no IDF_GITHUB_ASSETS
if not github_assets: # variable exists but is empty
return
# check no URL qualifier in the mirror URL
if '://' in github_assets:
fatal("IDF_GITHUB_ASSETS shouldn't include any URL qualifier, https:// is assumed")
raise SystemExit(1)
# Strip any trailing / from the mirror URL
github_assets = github_assets.rstrip('/')
old_url = tool_download_obj.url
new_url = re.sub(r'^https://github.com/', 'https://{}/'.format(github_assets), old_url)
if new_url != old_url:
info('Using GitHub assets mirror for URL: {} => {}'.format(old_url, new_url))
tool_download_obj.url = new_url
def action_download(args):
tools_info = load_tools_info()
tools_spec = args.tools
@ -1122,7 +1153,7 @@ def action_download(args):
tool_spec = '{}@{}'.format(tool_name, tool_version)
info('Downloading {}'.format(tool_spec))
apply_mirror_prefix_map(args, tool_obj.versions[tool_version].get_download_for_platform(platform))
apply_url_mirrors(args, tool_obj.versions[tool_version].get_download_for_platform(platform))
tool_obj.download(tool_version)
@ -1163,7 +1194,7 @@ def action_install(args):
continue
info('Installing {}'.format(tool_spec))
apply_mirror_prefix_map(args, tool_obj.versions[tool_version].get_download_for_platform(PYTHON_PLATFORM))
apply_url_mirrors(args, tool_obj.versions[tool_version].get_download_for_platform(PYTHON_PLATFORM))
tool_obj.download(tool_version)
tool_obj.install(tool_version)

View File

@ -24,34 +24,34 @@
"linux-amd64": {
"sha256": "674080a12f9c5ebe5a3a5ce51c6deaeffe6dfb06d6416233df86f25b574e9279",
"size": 85731226,
"url": "https://dl.espressif.com/dl/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-linux-amd64.tar.gz"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-linux-amd64.tar.gz"
},
"linux-armel": {
"sha256": "6771e011dffa2438ef84ff3474538b4a69df8f9d4cfae3b3707ca31c782ed7db",
"size": 83888892,
"url": "https://dl.espressif.com/dl/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-linux-armel.tar.gz"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-linux-armel.tar.gz"
},
"linux-i686": {
"sha256": "076b7e05304e26aa6ec105c9e0dc74addca079bc2cae6e42ee7575c5ded29877",
"size": 87715092,
"url": "https://dl.espressif.com/dl/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-linux-i686.tar.gz"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-linux-i686.tar.gz"
},
"macos": {
"sha256": "6845f786303b26c4a55ede57487ba65bd25737232fe6104be03f25bb62f4631c",
"size": 92424226,
"url": "https://dl.espressif.com/dl/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-macos.tar.gz"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-macos.tar.gz"
},
"name": "esp-2020r3-8.4.0",
"status": "recommended",
"win32": {
"sha256": "81cecd5493a3fcf2118977f3fd60bd0a13a4aeac8fe6760d912f96d2c34fab66",
"size": 104226379,
"url": "https://dl.espressif.com/dl/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-win32.zip"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-win32.zip"
},
"win64": {
"sha256": "58419852fefb7111fdec056ac2fd7c4bd09c1f4c17610a761a97b788413527cf",
"size": 106855139,
"url": "https://dl.espressif.com/dl/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-win64.zip"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32-elf-gcc8_4_0-esp-2020r3-win64.zip"
}
}
]
@ -80,34 +80,34 @@
"linux-amd64": {
"sha256": "40fafa47045167feda0cd07827db5207ebfeb4a3b6b24475957a921bc92805ed",
"size": 86069526,
"url": "https://dl.espressif.com/dl/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-linux-amd64.tar.gz"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-linux-amd64.tar.gz"
},
"linux-armel": {
"sha256": "6c1efec4c7829202279388ccb388e8a17a34464bc351d677c4f04d95ea4b4ce0",
"size": 84254468,
"url": "https://dl.espressif.com/dl/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-linux-armel.tar.gz"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-linux-armel.tar.gz"
},
"linux-i686": {
"sha256": "bd3a91166206a1a7ff7c572e15389e1938c3cdce588032a5e915be677a945638",
"size": 88053499,
"url": "https://dl.espressif.com/dl/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-linux-i686.tar.gz"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-linux-i686.tar.gz"
},
"macos": {
"sha256": "fe19b0c873879d8d89ec040f4db04a3ab27d769d3fd5f55fe59a28b6b111d09c",
"size": 92817351,
"url": "https://dl.espressif.com/dl/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-macos.tar.gz"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-macos.tar.gz"
},
"name": "esp-2020r3-8.4.0",
"status": "recommended",
"win32": {
"sha256": "d078d614ae864ae4a37fcb5b83323af0a5cfdbd8243607664becdd0f977a1e33",
"size": 104659541,
"url": "https://dl.espressif.com/dl/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-win32.zip"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-win32.zip"
},
"win64": {
"sha256": "6ea78b72ec52330b69af91dbe6c77c22bdc827817f044aa30306270453032bb4",
"size": 107228353,
"url": "https://dl.espressif.com/dl/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-win64.zip"
"url": "https://github.com/espressif/crosstool-NG/releases/download/esp-2020r3/xtensa-esp32s2-elf-gcc8_4_0-esp-2020r3-win64.zip"
}
}
]

View File

@ -54,7 +54,7 @@ begin
if IDFZIPFileVersion <> '' then
begin
Url := 'https://github.com/espressif/esp-idf/releases/download/' + IDFZIPFileVersion + '/esp-idf-' + IDFZIPFileVersion + '.zip';
MirrorUrl := 'https://dl.espressif.com/dl/esp-idf/releases/esp-idf-' + IDFZIPFileVersion + '.zip';
MirrorUrl := 'https://dl.espressif.com/github_assets/espressif/esp-idf/releases/download/' + IDFZIPFileVersion + '/esp-idf-' + IDFZIPFileVersion + '.zip';
IDFZIPFileName := ExpandConstant('{app}\releases\esp-idf-' + IDFZIPFileVersion + '.zip')
if not FileExists(IDFZIPFileName) then
begin

View File

@ -82,6 +82,7 @@ Name: createdsk; Description: "Create Desktop shortcut for the ESP-IDF Tools Com
; WD registration checkbox is identified by 'Windows Defender' substring anywhere in its caption, not by the position index in WizardForm.TasksList.Items
; Please, keep this in mind when making changes to the item's description - WD checkbox is to be disabled on systems without the Windows Defender installed
Name: wdexcl; Description: "Register the ESP-IDF Tools executables as Windows Defender exclusions (improves compilation speed, requires elevation)";
Name: idf_tools_use_mirror; Description: "Use Espressif download server instead of downloading tool packages from Github"; Flags: unchecked;
[Run]
Filename: "{app}\dist\{#PythonInstallerName}"; Parameters: "/passive PrependPath=1 InstallLauncherAllUsers=0 Include_dev=0 Include_tcltk=0 Include_launcher=0 Include_test=0 Include_doc=0"; Description: "Installing Python"; Check: PythonInstallRequired

View File

@ -117,6 +117,11 @@ begin
if not IDFUseExisting then
IDFDownload();
if WizardIsTaskSelected('idf_tools_use_mirror') then
begin
SetEnvironmentVariable('IDF_GITHUB_ASSETS', 'dl.espressif.com/github_assets')
end;
IDFToolsSetup();