pio pkg outdated - check for project outdated packages // Issue #3373

2022-02-12 23:06:10 +02:00
parent e549a07901
commit 0bdef36e2a
6 changed files with 265 additions and 8 deletions
--- a/HISTORY.rst
+++ b/HISTORY.rst
@ -11,8 +11,15 @@ PlatformIO Core 5
 5.3.0 (2022-02-??)
 ~~~~~~~~~~~~~~~~~~

- Run command from a PlatformIO package with a new `pio exec <https://docs.platformio.org/en/latest/core/userguide/cmd_exec.html>`__ (`issue #4163 <https://github.com/platformio/platformio-core/issues/4163>`_)
- Improved PIO Remote setup on credit-card sized computers (Raspberry Pi, BeagleBon, etc) (`issue #3865 <https://github.com/platformio/platformio-core/issues/3865>`_)
+* **Package Management**
+
+  - New unified Package Management CLI (``pio pkg``):
+
+    * `pio pkg outdated <https://docs.platformio.org/en/latest/core/userguide/pkg/cmd_pack.html>`__ - check for project outdated packages
+
+  - Run command from a PlatformIO package with a new `pio exec <https://docs.platformio.org/en/latest/core/userguide/cmd_exec.html>`__ (`issue #4163 <https://github.com/platformio/platformio-core/issues/4163>`_)
+
+* Improved PIO Remote setup on credit-card sized computers (Raspberry Pi, BeagleBon, etc) (`issue #3865 <https://github.com/platformio/platformio-core/issues/3865>`_)

 5.2.5 (2022-02-10)
 ~~~~~~~~~~~~~~~~~~
@ -93,7 +100,7 @@ PlatformIO Core 5
    * Check for duplicates and used version
    * Validate package manifest

-  - Added a new option ``--non-interactive`` to `pio package publish <https://docs.platformio.org/en/latest/core/userguide/package/cmd_publish.html>`__ command
+  - Added a new option ``--non-interactive`` to `pio package publish <https://docs.platformio.org/en/latest/core/userguide/pkg/cmd_publish.html>`__ command

 * **Build System**

@ -187,7 +194,7 @@ PlatformIO Core 5
 - Force VSCode's intelliSenseMode to "gcc-x64" when GCC toolchain is used
 - Print ignored test suites and environments in the test summary report only in verbose mode (`issue #3726 <https://github.com/platformio/platformio-core/issues/3726>`_)
 - Fixed an issue when the package manager tries to install a built-in library from the registry (`issue #3662 <https://github.com/platformio/platformio-core/issues/3662>`_)
- Fixed an issue when `pio package pack <https://docs.platformio.org/en/latest/core/userguide/package/cmd_pack.html>`__ ignores some folders (`issue #3730 <https://github.com/platformio/platformio-core/issues/3730>`_)
+- Fixed an issue when `pio package pack <https://docs.platformio.org/en/latest/core/userguide/pkg/cmd_pack.html>`__ ignores some folders (`issue #3730 <https://github.com/platformio/platformio-core/issues/3730>`_)

 5.0.2 (2020-10-30)
 ~~~~~~~~~~~~~~~~~~
@ -199,7 +206,7 @@ PlatformIO Core 5
 - Fixed a "KeyError: 'versions'" when dependency does not exist in the registry (`issue #3666 <https://github.com/platformio/platformio-core/issues/3666>`_)
 - Fixed an issue with GCC linker when "native" dev-platform is used in pair with library dependencies (`issue #3669 <https://github.com/platformio/platformio-core/issues/3669>`_)
 - Fixed an "AssertionError: ensure_dir_exists" when checking library updates from simultaneous subprocesses (`issue #3677 <https://github.com/platformio/platformio-core/issues/3677>`_)
- Fixed an issue when `pio package publish <https://docs.platformio.org/en/latest/core/userguide/package/cmd_publish.html>`__ command removes original archive after submitting to the registry (`issue #3716 <https://github.com/platformio/platformio-core/issues/3716>`_)
+- Fixed an issue when `pio package publish <https://docs.platformio.org/en/latest/core/userguide/pkg/cmd_publish.html>`__ command removes original archive after submitting to the registry (`issue #3716 <https://github.com/platformio/platformio-core/issues/3716>`_)
 - Fixed an issue when multiple `pio lib install <https://docs.platformio.org/en/latest/core/userguide/lib/cmd_install.html>`__ command with the same local library results in duplicates in ``lib_deps`` (`issue #3715 <https://github.com/platformio/platformio-core/issues/3715>`_)
 - Fixed an issue with a "wrong" timestamp in device monitor output using `"time" filter <https://docs.platformio.org/en/latest/core/userguide/device/cmd_monitor.html#filters>`__ (`issue #3712 <https://github.com/platformio/platformio-core/issues/3712>`_)

@ -209,7 +216,7 @@ PlatformIO Core 5
 - Added support for "owner" requirement when declaring ``dependencies`` using `library.json <https://docs.platformio.org/en/latest/librarymanager/config.html#dependencies>`__
 - Fixed an issue when using a custom git/ssh package with `platform_packages <https://docs.platformio.org/en/latest/projectconf/section_env_platform.html#platform-packages>`__ option (`issue #3624 <https://github.com/platformio/platformio-core/issues/3624>`_)
 - Fixed an issue with "ImportError: cannot import name '_get_backend' from 'cryptography.hazmat.backends'" when using `Remote Development <https://docs.platformio.org/en/latest/plus/pio-remote.html>`__ on RaspberryPi device (`issue #3652 <https://github.com/platformio/platformio-core/issues/3652>`_)
- Fixed an issue when `pio package unpublish <https://docs.platformio.org/en/latest/core/userguide/package/cmd_unpublish.html>`__ command crashes (`issue #3660 <https://github.com/platformio/platformio-core/issues/3660>`_)
+- Fixed an issue when `pio package unpublish <https://docs.platformio.org/en/latest/core/userguide/pkg/cmd_unpublish.html>`__ command crashes (`issue #3660 <https://github.com/platformio/platformio-core/issues/3660>`_)
 - Fixed an issue when the package manager tries to install a built-in library from the registry (`issue #3662 <https://github.com/platformio/platformio-core/issues/3662>`_)
 - Fixed an issue with incorrect value for C++ language standard in IDE projects when an in-progress language standard is used (`issue #3653 <https://github.com/platformio/platformio-core/issues/3653>`_)
 - Fixed an issue with "Invalid simple block (semantic_version)" from library dependency that refs to an external source (repository, ZIP/Tar archives) (`issue #3658 <https://github.com/platformio/platformio-core/issues/3658>`_)
@ -233,7 +240,7 @@ Please check `Migration guide from 4.x to 5.0 <https://docs.platformio.org/en/la
  - Built-in fine-grained access control (role-based, teams, organizations)
  - New CLI commands:

-    * `pio package <https://docs.platformio.org/en/latest/core/userguide/package/index.html>`__ – manage packages in the registry
+    * `pio package <https://docs.platformio.org/en/latest/core/userguide/pkg/index.html>`__ – manage packages in the registry
    * `pio access <https://docs.platformio.org/en/latest/core/userguide/access/index.html>`__ – manage package access for users, teams, and maintainers

 * Integration with the new **Account Management System**
--- a/2
+++ b/2
--- a/platformio/commands/pkg.py
+++ b/platformio/commands/pkg.py
@ -14,6 +14,7 @@

 import click

+from platformio.package.commands.outdated import package_outdated_cmd
 from platformio.package.commands.pack import package_pack_cmd
 from platformio.package.commands.publish import package_publish_cmd
 from platformio.package.commands.unpublish import package_unpublish_cmd
@ -22,6 +23,7 @@ from platformio.package.commands.unpublish import package_unpublish_cmd
@click.group(
    "pkg",
    commands=[
+        package_outdated_cmd,
        package_pack_cmd,
        package_publish_cmd,
        package_unpublish_cmd,
--- a/platformio/package/commands/outdated.py
+++ b/platformio/package/commands/outdated.py
@ -0,0 +1,220 @@
+# Copyright (c) 2014-present PlatformIO <contact@platformio.org>
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+import click
+from tabulate import tabulate
+
+from platformio import fs
+from platformio.package.manager.library import LibraryPackageManager
+from platformio.package.manager.platform import PlatformPackageManager
+from platformio.package.meta import PackageSpec
+from platformio.platform.factory import PlatformFactory
+from platformio.project.config import ProjectConfig
+
+
+class OutdatedCandidate:
+    def __init__(self, pm, pkg, spec, envs=None):
+        self.pm = pm
+        self.pkg = pkg
+        self.spec = spec
+        self.envs = envs or []
+        self.outdated = None
+        if not isinstance(self.envs, list):
+            self.envs = [self.envs]
+
+    def __eq__(self, other):
+        return all(
+            [
+                self.pm.package_dir == other.pm.package_dir,
+                self.pkg == other.pkg,
+                self.spec == other.spec,
+            ]
+        )
+
+    def check(self):
+        self.outdated = self.pm.outdated(self.pkg, self.spec)
+
+    def is_outdated(self):
+        if not self.outdated:
+            self.check()
+        return self.outdated.is_outdated(allow_incompatible=self.pm.pkg_type != "tool")
+
+
+@click.command("outdated", short_help="Check for outdated packages")
+@click.option(
+    "-d",
+    "--project-dir",
+    default=os.getcwd,
+    type=click.Path(exists=True, file_okay=False, dir_okay=True, resolve_path=True),
+)
+@click.option("-e", "--environment", "environments", multiple=True)
+def package_outdated_cmd(project_dir, environments):
+    candidates = fetch_outdated_candidates(
+        project_dir, environments, with_progress=True
+    )
+    print_outdated_candidates(candidates)
+
+
+def print_outdated_candidates(candidates):
+    if not candidates:
+        click.secho("Everything is up-to-date!", fg="green")
+        return
+    tabulate_data = [
+        (
+            click.style(
+                candidate.pkg.metadata.name,
+                fg=get_candidate_update_color(candidate.outdated),
+            ),
+            candidate.outdated.current,
+            candidate.outdated.wanted,
+            click.style(candidate.outdated.latest, fg="cyan"),
+            candidate.pm.pkg_type.capitalize(),
+            ", ".join(set(candidate.envs)),
+        )
+        for candidate in candidates
+    ]
+    click.echo()
+    click.secho("Semantic Versioning color legend:", bold=True)
+    click.echo(
+        tabulate(
+            [
+                (
+                    click.style("<Major Update>", fg="red"),
+                    "backward-incompatible updates",
+                ),
+                (
+                    click.style("<Minor Update>", fg="yellow"),
+                    "backward-compatible features",
+                ),
+                (
+                    click.style("<Patch Update>", fg="green"),
+                    "backward-compatible bug fixes",
+                ),
+            ],
+            tablefmt="plain",
+        )
+    )
+    click.echo()
+    click.echo(
+        tabulate(
+            tabulate_data,
+            headers=["Package", "Current", "Wanted", "Latest", "Type", "Environments"],
+        )
+    )
+
+
+def get_candidate_update_color(outdated):
+    if outdated.update_increment_type == outdated.UPDATE_INCREMENT_MAJOR:
+        return "red"
+    if outdated.update_increment_type == outdated.UPDATE_INCREMENT_MINOR:
+        return "yellow"
+    if outdated.update_increment_type == outdated.UPDATE_INCREMENT_PATCH:
+        return "green"
+    return None
+
+
+def fetch_outdated_candidates(project_dir, environments, with_progress=False):
+    candidates = []
+
+    def _add_candidate(data):
+        new_candidate = OutdatedCandidate(
+            data["pm"], data["pkg"], data["spec"], data["env"]
+        )
+        for candidate in candidates:
+            if candidate == new_candidate:
+                candidate.envs.append(data["env"])
+                return
+        candidates.append(new_candidate)
+
+    with fs.cd(project_dir):
+        config = ProjectConfig.get_instance()
+        config.validate(environments)
+
+        # platforms
+        for item in find_platform_candidates(config, environments):
+            _add_candidate(item)
+            # platform package dependencies
+            for dep_item in find_platform_dependency_candidates(item):
+                _add_candidate(dep_item)
+
+        # libraries
+        for item in find_library_candidates(config, environments):
+            _add_candidate(item)
+
+    result = []
+    if not with_progress:
+        for candidate in candidates:
+            if candidate.is_outdated():
+                result.append(candidate)
+        return result
+
+    with click.progressbar(candidates, label="Checking") as pb:
+        for candidate in pb:
+            if candidate.is_outdated():
+                result.append(candidate)
+    return result
+
+
+def find_platform_candidates(config, environments):
+    result = []
+    pm = PlatformPackageManager()
+    for env in config.envs():
+        platform = config.get(f"env:{env}", "platform")
+        if not platform or (environments and env not in environments):
+            continue
+        spec = PackageSpec(platform)
+        pkg = pm.get_package(spec)
+        if not pkg:
+            continue
+        result.append(dict(env=env, pm=pm, pkg=pkg, spec=spec))
+    return result
+
+
+def find_platform_dependency_candidates(platform_candidate):
+    result = []
+    p = PlatformFactory.new(platform_candidate["spec"])
+    p.configure_project_packages(platform_candidate["env"])
+    for pkg in p.get_installed_packages():
+        result.append(
+            dict(
+                env=platform_candidate["env"],
+                pm=p.pm,
+                pkg=pkg,
+                spec=p.get_package_spec(pkg.metadata.name),
+            )
+        )
+    return sorted(result, key=lambda item: item["pkg"].metadata.name)
+
+
+def find_library_candidates(config, environments):
+    result = []
+    for env in config.envs():
+        if environments and env not in environments:
+            continue
+        package_dir = os.path.join(config.get("platformio", "libdeps_dir") or "", env)
+        lib_deps = [
+            item for item in config.get(f"env:{env}", "lib_deps", []) if "/" in item
+        ]
+        if not os.path.isdir(package_dir) or not lib_deps:
+            continue
+        pm = LibraryPackageManager(package_dir)
+        for lib in lib_deps:
+            spec = PackageSpec(lib)
+            pkg = pm.get_package(spec)
+            if not pkg:
+                continue
+            result.append(dict(env=env, pm=pm, pkg=pkg, spec=spec))
+    return sorted(result, key=lambda item: item["pkg"].metadata.name)
--- a/platformio/package/manager/base.py
+++ b/platformio/package/manager/base.py
@ -59,6 +59,12 @@ class BasePackageManager(  # pylint: disable=too-many-public-methods
        self._tmp_dir = None
        self._registry_client = None

+    def __repr__(self):
+        return (
+            f"{self.__class__.__name__} <type={self.pkg_type} "
+            f"package_dir={self.package_dir}>"
+        )
+
    def lock(self):
        if self._lockfile:
            return
--- a/platformio/package/meta.py
+++ b/platformio/package/meta.py
@ -67,6 +67,10 @@ class PackageType(object):


 class PackageOutdatedResult(object):
+    UPDATE_INCREMENT_MAJOR = "major"
+    UPDATE_INCREMENT_MINOR = "minor"
+    UPDATE_INCREMENT_PATCH = "patch"
+
    def __init__(self, current, latest=None, wanted=None, detached=False):
        self.current = current
        self.latest = latest
@ -93,6 +97,24 @@ class PackageOutdatedResult(object):
            value = cast_version_to_semver(str(value))
        return super(PackageOutdatedResult, self).__setattr__(name, value)

+    @property
+    def update_increment_type(self):
+        if not self.current or not self.latest:
+            return None
+        patch_conds = [
+            self.current.major == self.latest.major,
+            self.current.minor == self.latest.minor,
+        ]
+        if all(patch_conds):
+            return self.UPDATE_INCREMENT_PATCH
+        minor_conds = [
+            self.current.major == self.latest.major,
+            self.current.major > 0,
+        ]
+        if all(minor_conds):
+            return self.UPDATE_INCREMENT_MINOR
+        return self.UPDATE_INCREMENT_MAJOR
+
    def is_outdated(self, allow_incompatible=False):
        if self.detached or not self.latest or self.current == self.latest:
            return False