Merge branch 'release/3.6.5'

HISTORY.rst

@@ -4,6 +4,17 @@ Release Notes
 PlatformIO 3.0
 --------------
 
+3.6.5 (2019-03-07)
+~~~~~~~~~~~~~~~~~~
+
+* Project Generator: added new targets for CLion IDE "BUILD_VERBOSE" and "MONITOR" (serial port monitor)
+  (`issue #359 <https://github.com/platformio/platformio-core/issues/359>`_)
+* Fixed an issue with slow updating of PlatformIO Core packages on Windows
+* Fixed an issue when `platformio ci <https://docs.platformio.org/en/latest/userguide/cmd_ci.html>`__ recompiles project if ``--keep-build-dir`` option is passed
+  (`issue #2109 <https://github.com/platformio/platformio-core/issues/2109>`_)
+* Fixed an issue when ``$PROJECT_HASH`` template was not expanded for the other directory ``***_dir`` options in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+  (`issue #2170 <https://github.com/platformio/platformio-core/issues/2170>`_)
+
 3.6.4 (2019-01-23)
 ~~~~~~~~~~~~~~~~~~
 
@@ -46,8 +57,8 @@ PlatformIO 3.0
 ~~~~~~~~~~~~~~~~~~
 
 * Generate an `include <http://docs.platformio.org/page/projectconf/section_platformio.html#include-dir>`__ and `test <http://docs.platformio.org/page/projectconf/section_platformio.html#test-dir>`__ directories with a README file when initializing a new project
-* Support in-line comments for multi-line value (``lib_deps``, ``build_flags``, etc) in `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+* Support in-line comments for multi-line value (``lib_deps``, ``build_flags``, etc) in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
-* Added ``$PROJECT_HASH`` template variable for `build_dir <http://docs.platformio.org/en/latest/projectconf/section_platformio.html#projectconf-pio-build-dir>`__. One of the use cases is setting a global storage for project artifacts using `PLATFORMIO_BUILD_DIR <http://docs.platformio.org/en/latest/envvars.html#envvar-PLATFORMIO_BUILD_DIR>`__ system environment variable. For example, ``/tmp/pio-build/$PROJECT_HASH`` (Unix) or ``%TEMP%/pio-build/$PROJECT_HASH`` (Windows)
+* Added ``$PROJECT_HASH`` template variable for `build_dir <http://docs.platformio.org/en/latest/projectconf/section_platformio.html#projectconf-pio-build-dir>`__. One of the use cases is setting a global storage for project artifacts using `PLATFORMIO_BUILD_DIR <http://docs.platformio.org/en/latest/envvars.html#envvar-PLATFORMIO_BUILD_DIR>`__ system environment variable. For example, ``/tmp/pio-build/$PROJECT_HASH`` (Unix) or ``$[sysenv.TEMP}/pio-build/$PROJECT_HASH`` (Windows)
 * Improved a loading speed of PIO Home "Recent News"
 * Improved `PIO Unified Debugger <https://docs.platformio.org/en/page/plus/debugging.html>`__ for "mbed" framework and fixed issue with missed local variables
 * Introduced `"Release" and "Debug" Build Configurations <http://docs.platformio.org/page/projectconf/build_configurations.html>`__
@@ -131,7 +142,7 @@ PlatformIO 3.0
 
 * Simplify configuration for `PIO Unit Testing <https://docs.platformio.org/page/plus/unit-testing.html>`__: separate main program from a test build process, drop
   requirement for ``#ifdef UNIT_TEST`` guard
-* Override any option from board manifest in `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf/section_env_board.html#more-options>`__
+* Override any option from board manifest in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf/section_env_board.html#more-options>`__
   (`issue #1612 <https://github.com/platformio/platformio-core/issues/1612>`_)
 * Configure a custom path to SVD file using `debug_svd_path <https://docs.platformio.org/page/projectconf/section_env_debug.html#debug-svd-path>`__
   option
@@ -194,7 +205,7 @@ PlatformIO 3.0
   (`issue #1274 <https://github.com/platformio/platformio-core/issues/1274>`_)
 * Configure a custom firmware/program name in build directory (`example <https://docs.platformio.org/page/projectconf/advanced_scripting.html#custom-firmware-program-name>`__)
 * Renamed ``envs_dir`` option to ``build_dir``
-  in `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf/section_platformio.html#build-dir>`__
+  in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf/section_platformio.html#build-dir>`__
 * Refactored code without "arrow" dependency (resolve issue with "ImportError:
   No module named backports.functools_lru_cache")
 * Improved support of PIO Unified Debugger for Eclipse Oxygen
@@ -229,7 +240,7 @@ PlatformIO 3.0
   folder for project's header files
   (`issue #1107 <https://github.com/platformio/platformio-core/issues/1107>`_)
 * Depend on development platform using VCS URL (Git, Mercurial and Subversion)
-  instead of a name in `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf/section_env_general.html#platform>`__.
+  instead of a name in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf/section_env_general.html#platform>`__.
   Drop support for ``*_stage`` dev/platform names (use VCS URL instead).
 * Reinstall/redownload package with a new ``-f, --force`` option for
   `platformio lib install <https://docs.platformio.org/page/userguide/lib/cmd_install.html>`__
@@ -250,7 +261,7 @@ PlatformIO 3.0
   - Parse library source file in pair with a header when they have the same name
     (`issue #1175 <https://github.com/platformio/platformio-core/issues/1175>`_)
   - Handle library dependencies defined as VCS or SemVer in
-    `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf/section_env_general.html#platform>`__
+    `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf/section_env_general.html#platform>`__
     (`issue #1155 <https://github.com/platformio/platformio-core/issues/1155>`_)
   - Added option to configure library `Compatible Mode <https://docs.platformio.org/page/librarymanager/ldf.html#compatibility-mode>`__
     using `library.json <https://docs.platformio.org/page/librarymanager/config.html>`__
@@ -286,7 +297,7 @@ PlatformIO 3.0
 * Use a root of library when filtering source code using
   `library.json <https://docs.platformio.org/page/librarymanager/config.html>`__
   and ``srcFilter`` field
-* Added ``monitor_*`` options to white-list for `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+* Added ``monitor_*`` options to white-list for `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
   (`issue #982 <https://github.com/platformio/platformio-core/issues/982>`_)
 * Do not ask for board ID when initialize project for desktop platform
 * Handle broken PIO Core state and create new one
@@ -307,18 +318,18 @@ PlatformIO 3.0
   - Integration with `Eclipse <https://docs.platformio.org/page/ide/eclipse.html>`__ and `Sublime Text <https://docs.platformio.org/page/ide/sublimetext.html>`__
 
 * Filter `PIO Unit Testing <https://docs.platformio.org/page/plus/unit-testing.html>`__
-  tests using a new ``test_filter`` option in `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+  tests using a new ``test_filter`` option in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
   or `platformio test --filter <https://docs.platformio.org/page/userguide/cmd_test.html#cmdoption-platformio-test-f>`__ command
   (`issue #934 <https://github.com/platformio/platformio-core/issues/934>`_)
 * Custom ``test_transport`` for `PIO Unit Testing <https://docs.platformio.org/page/plus/unit-testing.html>`__ Engine
-* Configure Serial Port Monitor in `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+* Configure Serial Port Monitor in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
   (`issue #787 <https://github.com/platformio/platformio-core/issues/787>`_)
 * New `monitor <https://docs.platformio.org/page/userguide/cmd_run.html#cmdoption-platformio-run-t>`__
   target which allows to launch Serial Monitor automatically after successful
   "build" or "upload" operations
   (`issue #788 <https://github.com/platformio/platformio-core/issues/788>`_)
 * Project generator for `VIM <https://docs.platformio.org/page/ide/vim.html>`__
-* Multi-line support for the different options in `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__,
+* Multi-line support for the different options in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__,
   such as: ``build_flags``, ``build_unflags``, etc.
   (`issue #889 <https://github.com/platformio/platformio-core/issues/889>`_)
 * Handle dynamic ``SRC_FILTER`` environment variable from
@@ -330,7 +341,7 @@ PlatformIO 3.0
   that were installed from repository
 * Add support for ``.*cc`` extension
   (`issue #939 <https://github.com/platformio/platformio-core/issues/939>`_)
-* Handle ``env_default`` in `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+* Handle ``env_default`` in `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
   when re-initializing a project
   (`issue #950 <https://github.com/platformio/platformio-core/issues/950>`_)
 * Use root directory for PIO Home when path contains non-ascii characters
@@ -428,10 +439,10 @@ PlatformIO 3.0
   (`issue #808 <https://github.com/platformio/platformio-core/issues/808>`_,
   `issue #467 <https://github.com/platformio/platformio-core/issues/467>`_)
 * Inject system environment variables to configuration settings in
-  `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+  `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
   (`issue #792 <https://github.com/platformio/platformio-core/issues/792>`_)
 * Custom boards per project with ``boards_dir`` option in
-  `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+  `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
   (`issue #515 <https://github.com/platformio/platformio-core/issues/515>`_)
 * Unix shell-style wildcards for `upload_port <https://docs.platformio.org/page/projectconf/section_env_upload.html#upload-port>`_
   (`issue #839 <https://github.com/platformio/platformio-core/issues/839>`_)
@@ -442,7 +453,7 @@ PlatformIO 3.0
 * Added new `LDF Modes <https://docs.platformio.org/page/librarymanager/ldf.html#ldf-mode>`__:
   ``chain+`` and ``deep+`` and set ``chain+`` as default
 * Added global ``lib_extra_dirs`` option to ``[platformio]`` section for
-  `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+  `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
   (`issue #842 <https://github.com/platformio/platformio-core/issues/842>`_)
 * Enabled caching by default for API requests and Library Manager (see `enable_cache <https://docs.platformio.org/page/userguide/cmd_settings.html#enable-cache>`__ setting)
 * Native integration with VIM/Neovim using `neomake-platformio <https://github.com/coddingtonbear/neomake-platformio>`__ plugin
@@ -465,7 +476,7 @@ PlatformIO 3.0
 3.1.0 (2016-09-19)
 ~~~~~~~~~~~~~~~~~~
 
-* New! Dynamic variables/templates for `“platformio.ini” (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
+* New! Dynamic variables/templates for `"platformio.ini" (Project Configuration File) <https://docs.platformio.org/page/projectconf.html>`__
   (`issue #705 <https://github.com/platformio/platformio-core/issues/705>`_)
 * Summary about processed environments
   (`issue #777 <https://github.com/platformio/platformio-core/issues/777>`_)

platformio/__init__.py

@@ -14,7 +14,7 @@
 
 import sys
 
-VERSION = (3, 6, 4)
+VERSION = (3, 6, 5)
 __version__ = ".".join([str(s) for s in VERSION])
 
 __title__ = "platformio"

platformio/app.py

@@ -352,8 +352,8 @@ def get_cid():
                 pass
         cid = str(
             uuid.UUID(
-                bytes=hashlib.md5(str(_uid if _uid else uuid.getnode())).
+                bytes=hashlib.md5(str(
-                digest()))
+                    _uid if _uid else uuid.getnode())).digest()))
         if "windows" in util.get_systype() or os.getuid() > 0:
             set_state_item("cid", cid)
     return cid

platformio/builder/tools/pioplatform.py

@@ -95,8 +95,10 @@ def LoadPioPlatform(env, variables):
         for key, value in variables.UnknownVariables().items():
             if not key.startswith("BOARD_"):
                 continue
-            env.Replace(
+            env.Replace(**{
-                **{key.upper().replace("BUILD.", ""): base64.b64decode(value)})
+                key.upper().replace("BUILD.", ""):
+                base64.b64decode(value)
+            })
         return
 
     # update board manifest with a custom data

platformio/commands/ci.py

@@ -73,7 +73,7 @@ def validate_path(ctx, param, value):  # pylint: disable=unused-argument
 @click.option("-O", "--project-option", multiple=True)
 @click.option("-v", "--verbose", is_flag=True)
 @click.pass_context
-def cli(  # pylint: disable=too-many-arguments
+def cli(  # pylint: disable=too-many-arguments, too-many-branches
         ctx, src, lib, exclude, board, build_dir, keep_build_dir, project_conf,
         project_option, verbose):
 
@@ -84,7 +84,11 @@ def cli(  # pylint: disable=too-many-arguments
 
     try:
         app.set_session_var("force_option", True)
-        _clean_dir(build_dir)
+
+        if not keep_build_dir and isdir(build_dir):
+            util.rmtree_(build_dir)
+        if not isdir(build_dir):
+            makedirs(build_dir)
 
         for dir_name, patterns in dict(lib=lib, src=src).items():
             if not patterns:
@@ -116,11 +120,6 @@ def cli(  # pylint: disable=too-many-arguments
             util.rmtree_(build_dir)
 
 
-def _clean_dir(dirpath):
-    util.rmtree_(dirpath)
-    makedirs(dirpath)
-
-
 def _copy_contents(dst_dir, contents):
     items = {"dirs": set(), "files": set()}
 

platformio/commands/platform.py

@@ -365,8 +365,8 @@ def platform_update(platforms, only_packages, only_check, json_output):
             if not pkg_dir:
                 continue
             latest = pm.outdated(pkg_dir, requirements)
-            if (not latest and not PlatformFactory.newPlatform(pkg_dir).
+            if (not latest and not PlatformFactory.newPlatform(
-                    are_outdated_packages()):
+                    pkg_dir).are_outdated_packages()):
                 continue
             data = _get_installed_platform_data(
                 pkg_dir, with_boards=False, expose_packages=False)

platformio/ide/tpls/clion/CMakeLists.txt.tpl

@@ -9,6 +9,12 @@ add_custom_target(
     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
 )
 
+add_custom_target(
+    PLATFORMIO_BUILD_VERBOSE ALL
+    COMMAND ${PLATFORMIO_CMD} -f -c clion run --verbose
+    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+)
+
 add_custom_target(
     PLATFORMIO_UPLOAD ALL
     COMMAND ${PLATFORMIO_CMD} -f -c clion run --target upload
@@ -21,6 +27,12 @@ add_custom_target(
     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
 )
 
+add_custom_target(
+    PLATFORMIO_MONITOR ALL
+    COMMAND ${PLATFORMIO_CMD} -f -c clion device monitor
+    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+)
+
 add_custom_target(
     PLATFORMIO_TEST ALL
     COMMAND ${PLATFORMIO_CMD} -f -c clion test

platformio/ide/tpls/clion/CMakeListsPrivate.txt.tpl

@@ -1,3 +1,7 @@
+# !!! WARNING !!!
+# PLEASE DO NOT MODIFY THIS FILE!
+# USE https://docs.platformio.org/page/projectconf/section_env_build.html#build-flags
+
 % def _normalize_path(path):
 %   if project_dir in path:
 %     path = path.replace(project_dir, "${CMAKE_CURRENT_LIST_DIR}")

platformio/managers/core.py

@@ -106,9 +106,10 @@ def update_core_packages(only_check=False, silent=False):
 
 def shutdown_piohome_servers():
     port = 8010
-    while port < 8100:
+    while port < 8050:
         try:
-            requests.get("http://127.0.0.1:%d?__shutdown__=1" % port)
+            requests.get(
+                "http://127.0.0.1:%d?__shutdown__=1" % port, timeout=0.01)
         except:  # pylint: disable=bare-except
             pass
         port += 1

platformio/managers/platform.py

@@ -633,10 +633,8 @@ class PlatformBase(  # pylint: disable=too-many-public-methods
                 if not isdir(libcore_dir):
                     continue
                 storages.append({
-                    "name":
+                    "name": "%s-core-%s" % (opts['package'], item),
-                    "%s-core-%s" % (opts['package'], item),
+                    "path": libcore_dir
-                    "path":
-                    libcore_dir
                 })
 
         return storages

platformio/util.py

@@ -203,28 +203,34 @@ def pioversion_to_intstr():
 
 
 def get_project_optional_dir(name, default=None):
-    data = None
+    paths = None
     var_name = "PLATFORMIO_%s" % name.upper()
     if var_name in os.environ:
-        data = os.getenv(var_name)
+        paths = os.getenv(var_name)
     else:
         try:
             config = load_project_config()
             if (config.has_section("platformio")
                     and config.has_option("platformio", name)):
-                data = config.get("platformio", name)
+                paths = config.get("platformio", name)
         except exception.NotPlatformIOProject:
             pass
 
-    if not data:
+    if not paths:
         return default
 
     items = []
-    for item in data.split(", "):
+    for item in paths.split(", "):
         if item.startswith("~"):
             item = expanduser(item)
         items.append(abspath(item))
-    return ", ".join(items)
+    paths = ", ".join(items)
+
+    while "$PROJECT_HASH" in paths:
+        paths = paths.replace("$PROJECT_HASH",
+                              sha1(get_project_dir()).hexdigest()[:10])
+
+    return paths
 
 
 def get_home_dir():
@@ -314,9 +320,6 @@ def get_projectboards_dir():
 def get_projectbuild_dir(force=False):
     path = get_project_optional_dir("build_dir",
                                     join(get_project_dir(), ".pioenvs"))
-    if "$PROJECT_HASH" in path:
-        path = path.replace("$PROJECT_HASH",
-                            sha1(get_project_dir()).hexdigest()[:10])
     try:
         if not isdir(path):
             os.makedirs(path)
@@ -565,16 +568,11 @@ def get_mdns_services():
                 pass
 
             items.append({
-                "type":
+                "type": service.type,
-                service.type,
+                "name": service.name,
-                "name":
+                "ip": ".".join([str(ord(c)) for c in service.address]),
-                service.name,
+                "port": service.port,
-                "ip":
+                "properties": properties
-                ".".join([str(ord(c)) for c in service.address]),
-                "port":
-                service.port,
-                "properties":
-                properties
             })
     return items
 
@@ -682,7 +680,7 @@ def get_api_result(url, params=None, data=None, auth=None, cache_valid=None):
 
 PING_INTERNET_IPS = [
     "192.30.253.113",  # github.com
-    "159.122.18.156",  # dl.bintray.com
+    "18.195.111.75",  # dl.bintray.com
     "193.222.52.25"  # dl.platformio.org
 ]