Introduce "lib_compat_level" option for project configuration file

HISTORY.rst

@@ -14,8 +14,11 @@ PlatformIO 3.0
 * Unit Testing for Embedded (`docs <http://docs.platformio.org/en/latest/platforms/unit_testing.html>`__)
   (`issue #408 <https://github.com/platformio/platformio/issues/408>`_)
 * New Library Build System: intelligent dependency finder that interprets
-  C Preprocessor conditional macros, `library deep search <http://docs.platformio.org/en/latest/projectconf.html#lib-deep-search>`__, support for the 3rd party
+  C Preprocessor conditional macros,
-  manifests (Arduino IDE ``library.properties``, ARM mbed ``module.json``)
+  `library deep search <http://docs.platformio.org/en/latest/projectconf.html#lib-deep-search>`__,
+  `library compatibility level <http://docs.platformio.org/en/latest/projectconf.html#lib-compat-level>`__,
+  support for the 3rd party manifests (Arduino IDE ``library.properties``,
+  ARM mbed ``module.json``)
   (`issue #432 <https://github.com/platformio/platformio/issues/432>`_)
 * New `lib_extra_dirs <http://docs.platformio.org/en/latest/projectconf.html#lib-extra-dirs>`__ option for project environment.
   Multiple custom library locations!

docs/faq.rst

@@ -33,6 +33,33 @@ What is ``.pioenvs`` directory
 
 Please refer to :ref:`projectconf_pio_envs_dir`.
 
+.. _faq_ldf:
+
+How works Library Dependency Finder (LDF)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Library Dependency Finder is a part of PlatformIO Library Build System. It
+operates with the header files (``*.h, *.hpp``) and looks for
+``#include <...>`` directives. What is more, LDF interprets C Preprocessor
+conditional macros (``#ifdef ...``, etc.). Library Dependency Finder starts
+work from analyzing source files from :ref:`projectconf_pio_src_dir`. It
+understands "nested includes/chain" by default if they depend on each other.
+
+There are different library storages where Library Dependency Finder looks for
+dependencies. These storages/folders have priority. LDF operates in the next
+order:
+
+1. :ref:`projectconf_lib_extra_dirs`
+2. :ref:`projectconf_pio_lib_dir`
+3. :ref:`projectconf_pio_home_dir`/lib
+
+Library Dependency Finder has a few key factors from :ref:`projectconf`:
+
+* :ref:`projectconf_lib_ignore`
+* :ref:`projectconf_lib_deep_search`
+* :ref:`projectconf_lib_extra_dirs`
+* :ref:`projectconf_lib_compat_level`
+
 Command completion in Terminal
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

docs/projectconf.rst

@@ -606,10 +606,15 @@ Example:
 ``lib_force``
 ^^^^^^^^^^^^^
 
-Force Library Build System to build specified libraries if even they are not
+Force Library Build System to build specified libraries if they even are not
 included in the project source code. Also, these libraries will be processed
 in the first order.
 
+The correct value for this option is library name (not
+folder name). In the most cases, library name is pre-defined in manifest file
+(:ref:`library_config`, ``library.properties``, ``module.json``). The multiple
+library names are allowed, split them with comma ``,`` separator.
+
 Example:
 
 .. code-block:: ini
@@ -617,10 +622,19 @@ Example:
     [env:myenv]
     lib_force = OneWire, SPI
 
+.. _projectconf_lib_ignore:
+
 ``lib_ignore``
 ^^^^^^^^^^^^^^
 
-Specify libraries which should be ignored by ``Library Dependency Finder (LDF)``
+Please make sure to read :ref:`faq_ldf` guides first.
+
+Specify libraries which should be ignored by Library Dependency Finder.
+
+The correct value for this option is library name (not
+folder name). In the most cases, library name is pre-defined in manifest file
+(:ref:`library_config`, ``library.properties``, ``module.json``). The multiple
+library names are allowed, split them with comma ``,`` separator.
 
 Example:
 
@@ -629,18 +643,25 @@ Example:
     [env:ignore_some_libs]
     lib_ignore = SPI, Ethernet
 
+.. _projectconf_lib_deep_search:
+
 ``lib_deep_search``
 ^^^^^^^^^^^^^^^^^^^
 
+Please make sure to read :ref:`faq_ldf` guides first.
+
 By default, this option is turned OFF (``lib_deep_search = false``) and means
-that ``Library Dependency Finder (LDF)`` will look only for the libraries
+that Library Dependency Finder will analyzes only "nested includes/chain".
-that are mentioned (using ``#include <...>``) in the source files from the
+
-project :ref:`projectconf_pio_src_dir`. Also, ``LDF`` analyzes nested
+Nevertheless, some libraries depend on other libraries and the
-``#include <...>`` by default.
+``#include <...>`` directives for these libraries are not declared in the
+"main" header file that is used by upper library. In this case, LDF will not
+handle these libraries automatically because it doesn't analyze "each source
+file" of the nested libraries.
 
 If you want to enable deep search, please set this option to ``true``.
-Found library will be treated like the new source files and
+Found library will be treated like the new source files and LDF will
-``LDF`` will search dependencies for it.
+search dependencies for it.
 
 For example, there are 2 libraries:
 
@@ -688,17 +709,19 @@ For example, there are 2 libraries:
 ``lib_extra_dirs``
 ^^^^^^^^^^^^^^^^^^
 
-A list with extra directories where ``Library Dependency Finder (LDF)`` will
+Please make sure to read :ref:`faq_ldf` guides first.
-look for dependencies. Multiple paths are allowed. Please separate them using
+
-comma ``,`` symbol.
+A list with extra directories/storages where Library Dependency Finder will
+look for dependencies. Multiple paths are allowed. Please separate them
+using comma ``,`` symbol.
 
 This option can be set by global environment variable
 :envvar:`PLATFORMIO_LIB_EXTRA_DIRS`.
 
 .. warning::
   This is a not direct path to library with source code. It should be the path
-  to directory that contains libraries grouped by folders. For example,
+  to storage that contains libraries grouped by folders. For example,
-  ``/extra/lib/path/`` but not ``/extra/lib/path/MyLibrary``.
+  ``/extra/lib/storage/`` but not ``/extra/lib/storage/MyLibrary``.
 
 Example:
 
@@ -707,6 +730,27 @@ Example:
     [env:custom_lib_dirs]
     lib_extra_dirs = /path/to/private/dir1,/path/to/private/dir2
 
+.. _projectconf_lib_compat_level:
+
+``lib_compat_level``
+^^^^^^^^^^^^^^^^^^^^
+
+Please make sure to read :ref:`faq_ldf` guides first.
+
+Library compatibility level that allows to control Library Dependency Finder
+strictness. If library contains manifest file (:ref:`library_config`,
+``library.properties``, ``module.json``), then LDF check compatibility of this
+library with real build environment. Available compatibility levels:
+
+* ``0`` - don't check for compatibility (disable)
+* ``1`` - check for the compatibility with :ref:`projectconf_env_framework`
+  from build environment
+* ``2`` - check for the compatibility with :ref:`projectconf_env_framework`
+  and :ref:`projectconf_env_platform` from build environment.
+
+By default, this value is set to ``lib_compat_level = 1`` and means that LDF
+will check only for framework compatibility.
+
 -----------
 
 .. _projectconf_examples:

platformio/__init__.py

@@ -14,7 +14,7 @@
 
 import sys
 
-VERSION = (3, 0, "0.dev9")
+VERSION = (3, 0, "0.dev10")
 __version__ = ".".join([str(s) for s in VERSION])
 
 __title__ = "platformio"

platformio/builder/main.py

@@ -45,6 +45,7 @@ commonvars.AddVariables(
 
     # library options
     ("LIB_DEEP_SEARCH",),
+    ("LIB_COMPAT_LEVEL",),
     ("LIB_IGNORE",),
     ("LIB_FORCE",),
     ("LIB_EXTRA_DIRS",),
@@ -121,6 +122,7 @@ for opt in ("LIB_IGNORE", "LIB_FORCE", "LIB_EXTRA_DIRS"):
         continue
     env[opt] = [l.strip() for l in env[opt].split(",") if l.strip()]
 
+env.Prepend(LIBSOURCE_DIRS=env.get("LIB_EXTRA_DIRS", []))
 env.LoadDevPlatform(commonvars)
 
 env.SConscriptChdir(0)

platformio/builder/tools/piolib.py

@@ -17,8 +17,8 @@
 from __future__ import absolute_import
 
 import os
+import sys
 from os.path import basename, commonprefix, isdir, isfile, join, realpath
-from sys import modules
 
 import SCons.Scanner
 
@@ -44,7 +44,7 @@ class LibBuilderFactory(object):
             elif used_frameworks:
                 clsname = "%sLibBuilder" % used_frameworks[0].title()
 
-        obj = getattr(modules[__name__], clsname)(env, path)
+        obj = getattr(sys.modules[__name__], clsname)(env, path)
         assert isinstance(obj, LibBuilderBase)
         return obj
 
@@ -327,27 +327,34 @@ def find_and_build_deps(env, lib_builders, scanner,
 
 def GetLibBuilders(env):
     items = []
-    libs_dirs = []
     env_frameworks = [
-        f.lower().strip() for f in env.get("PIOFRAMEWORK", "").split(",")]
+        f.lower().strip() for f in env.get("PIOFRAMEWORK", "").split(",")
+    ]
+    compat_level = int(env.get("LIB_COMPAT_LEVEL", 1))
 
-    for key in ("LIB_EXTRA_DIRS", "LIBSOURCE_DIRS"):
+    for libs_dir in env['LIBSOURCE_DIRS']:
-        for d in env.get(key, []):
+        libs_dir = env.subst(libs_dir)
-            d = env.subst(d)
+        if not isdir(libs_dir):
-            if isdir(d):
+            continue
-                libs_dirs.append(d)
-
-    for libs_dir in libs_dirs:
         for item in sorted(os.listdir(libs_dir)):
             if item == "__cores__" or not isdir(join(libs_dir, item)):
                 continue
             lb = LibBuilderFactory.new(env, join(libs_dir, item))
             if lb.name in env.get("LIB_IGNORE", []):
+                if not env.GetOption("silent"):
+                    print "Ignored library " + lb.path
                 continue
-            if not lb.is_platform_compatible(env['PIOPLATFORM']):
+            if compat_level > 1 and not lb.is_platform_compatible(env[
+                    'PIOPLATFORM']):
+                if not env.GetOption("silent"):
+                    sys.stderr.write("Platform incompatible library %s\n" %
+                                     lb.path)
                 continue
-            if not any([lb.is_framework_compatible(f)
+            if compat_level > 0 and not any([lb.is_framework_compatible(f)
-                        for f in env_frameworks]):
+                                             for f in env_frameworks]):
+                if not env.GetOption("silent"):
+                    sys.stderr.write("Framework incompatible library %s\n" %
+                                     lb.path)
                 continue
             items.append(lb)
     return items
@@ -358,8 +365,8 @@ def BuildDependentLibraries(env, src_dir):
     scanner = SCons.Scanner.C.CScanner()
     lib_builders = env.GetLibBuilders()
 
-    print "Looking for dependencies..."
     print "Collecting %d compatible libraries" % len(lib_builders)
+    print "Looking for dependencies..."
 
     built_lib_names = []
     for lib_name in env.get("LIB_FORCE", []):