Merge branch 'release/v0.9.0'

Resolve issue #17

.gitignore

@@ -6,3 +6,4 @@
 examples/ide-eclipse/.metadata
 examples/ide-eclipse/RemoteSystemsTempFiles
 docs/_build
+dist

.pylintrc

@@ -38,7 +38,7 @@ load-plugins=
 # --enable=similarities". If you want to run only the classes checker, but have
 # no Warning level messages displayed, use"--disable=all --enable=classes
 # --disable=W"
-disable=C0103,C0111,E0611,F0401,I0011,R0801,R0903
+disable=C0103,C0111,E0611,F0401,I0011,R0801,R0903,R0922
 
 
 [REPORTS]

HISTORY.rst

@@ -1,17 +1,61 @@
 Release History
 ===============
 
+0.9.0 (2014-12-01)
+------------------
+
+* Implemented `platformio settings <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_settings.html>`_ command
+* Improved `platformio init <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_init.html>`_ command.
+  Added new option ``--project-dir`` where you can specify another path to
+  directory where new project will be initialized (`issue #31 <https://github.com/ivankravets/platformio/issues/31>`_)
+* Added *Migration Manager* which simplifies process with upgrading to a
+  major release
+* Added *Telemetry Service* which should help us make *PlatformIO* better
+* Implemented *PlatformIO AppState Manager* which allow to have multiple
+  ``.platformio`` states.
+* Refactored *Package Manager*
+* Download Manager: fixed SHA1 verification within *Cygwin Environment*
+  (`issue #26 <https://github.com/ivankravets/platformio/issues/26>`_)
+* Fixed bug with code builder and built-in Arduino libraries
+  (`issue #28 <https://github.com/ivankravets/platformio/issues/28>`_)
+
+0.8.0 (2014-10-19)
+------------------
+
+* Avoided trademark issues in `library.json <http://docs.platformio.ikravets.com/en/latest/librarymanager/config.html>`_
+  with the new fields: `frameworks <http://docs.platformio.ikravets.com/en/latest/librarymanager/config.html#frameworks>`_,
+  `platforms <http://docs.platformio.ikravets.com/en/latest/librarymanager/config.html#platforms>`_
+  and `dependencies <http://docs.platformio.ikravets.com/en/latest/librarymanager/config.html#dependencies>`_
+  (`issue #17 <https://github.com/ivankravets/platformio/issues/17>`_)
+* Switched logic from "Library Name" to "Library Registry ID" for all
+  `platformio lib <http://docs.platformio.ikravets.com/en/latest/userguide/lib/index.html>`_
+  commands (install, uninstall, update and etc.)
+* Renamed ``author`` field to `authors <http://docs.platformio.ikravets.com/en/latest/librarymanager/config.html#authors>`_
+  and allowed to setup multiple authors per library in `library.json <http://docs.platformio.ikravets.com/en/latest/librarymanager/config.html>`_
+* Added option to specify "maintainer" status in `authors <http://docs.platformio.ikravets.com/en/latest/librarymanager/config.html#authors>`_ field
+* New filters/options for `platformio lib search <http://docs.platformio.ikravets.com/en/latest/userguide/lib/cmd_search.html>`_
+  command: ``--framework`` and ``--platform``
+
+0.7.1 (2014-10-06)
+------------------
+
+* Fixed bug with order for includes in conversation from INO/PDE to CPP
+* Automatic detection of port on upload (`issue #15 <https://github.com/ivankravets/platformio/issues/15>`_)
+* Fixed lib update crashing when no libs are installed (`issue #19 <https://github.com/ivankravets/platformio/issues/19>`_)
+
+
 0.7.0 (2014-09-24)
 ------------------
 
-* Implemented new ``[platformio]`` section for Configuration File with ``home_dir``
+* Implemented new `[platformio] <http://docs.platformio.ikravets.com/en/latest/projectconf.html#platformio>`_
+  section for Configuration File with `home_dir <http://docs.platformio.ikravets.com/en/latest/projectconf.html#home-dir>`_
   option (`issue #14 <https://github.com/ivankravets/platformio/issues/14>`_)
 * Implemented *Library Manager* (`issue #6 <https://github.com/ivankravets/platformio/issues/6>`_)
 
 0.6.0 (2014-08-09)
 ------------------
 
-* Implemented ``serialports monitor`` (`issue #10 <https://github.com/ivankravets/platformio/issues/10>`_)
+* Implemented `platformio serialports monitor <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_serialports.html#platformio-serialports-monitor>`_ (`issue #10 <https://github.com/ivankravets/platformio/issues/10>`_)
 * Fixed an issue ``ImportError: No module named platformio.util`` (`issue #9 <https://github.com/ivankravets/platformio/issues/9>`_)
 * Fixed bug with auto-conversation from Arduino \*.ino to \*.cpp
 
@@ -23,27 +67,31 @@ Release History
 * Added auto-conversation from \*.ino to valid \*.cpp for Arduino/Energia
   frameworks (`issue #7 <https://github.com/ivankravets/platformio/issues/7>`_)
 * Added `Arduino example <https://github.com/ivankravets/platformio/tree/develop/examples/arduino-adafruit-library>`_
-  with external library (Adafruit CC3000)
-* Implemented ``platformio upgrade`` command and "auto-check" for the latest
+  with external library (*Adafruit CC3000*)
+* Implemented `platformio upgrade <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_upgrade.html>`_
+  command and "auto-check" for the latest
   version (`issue #8 <https://github.com/ivankravets/platformio/issues/8>`_)
-* Fixed an issue with "auto-reset" for Raspduino board
+* Fixed an issue with "auto-reset" for *Raspduino* board
 * Fixed a bug with nested libs building
 
 0.4.0 (2014-07-31)
 ------------------
 
-* Implemented ``serialports`` command
+* Implemented `platformio serialports <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_serialports.html>`_ command
 * Allowed to put special build flags only for ``src`` files via
-  ``srcbuild_flags`` environment option
+  `srcbuild_flags <http://docs.platformio.ikravets.com/en/latest/projectconf.html#srcbuild-flags>`_
+  environment option
 * Allowed to override some of settings via system environment variables
   such as: ``$PIOSRCBUILD_FLAGS`` and ``$PIOENVS_DIR``
-* Added ``--upload-port`` option for ``platformio run`` command
+* Added ``--upload-port`` option for `platformio run <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_run.html#cmdoption--upload-port>`_ command
 * Implemented (especially for `SmartAnthill <http://smartanthill.ikravets.com/>`_)
-  ``platformio run -t uploadlazy`` target (no dependencies to framework libs,
-  ELF and etc.)
-* Allowed to skip default packages via ``platformio install --skip-default-package`` flag
-* Added tools for Raspberry Pi platform
-* Added support for Microduino and Raspduino boards in ``atmelavr`` platform
+  `platformio run -t uploadlazy <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_run.html>`_
+  target (no dependencies to framework libs, ELF and etc.)
+* Allowed to skip default packages via `platformio install --skip-default-package <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_install.html#cmdoption--skip-default>`_
+  option
+* Added tools for *Raspberry Pi* platform
+* Added support for *Microduino* and *Raspduino* boards in
+  `atmelavr <http://docs.platformio.ikravets.com/en/latest/platforms/atmelavr.html>`_ platform
 
 
 0.3.1 (2014-06-21)

README.rst

@@ -17,60 +17,107 @@ PlatformIO
     :target: https://pypi.python.org/pypi/platformio/
     :alt:  License
 
-`Website <http://platformio.ikravets.com>`_ |
+`Website + Library Search <http://platformio.ikravets.com>`_ |
 `Documentation <http://docs.platformio.ikravets.com>`_ |
 `Project Examples <https://github.com/ivankravets/platformio/tree/develop/examples>`_ |
-`Bugs/Questions <https://github.com/ivankravets/platformio/issues>`_ |
 `Blog <http://www.ikravets.com/category/computer-life/platformio>`_ |
-`Twitter <https://twitter.com/smartanthill>`_
+`Twitter <https://twitter.com/PlatformIOTool>`_
 
-**PlatformIO** is a cross-platform code builder and library manager.
+.. image:: https://raw.githubusercontent.com/ivankravets/platformio/develop/docs/_static/platformio-logo.png
+    :target: http://platformio.ikravets.com
 
-* `Website + Library Search <http://platformio.ikravets.com>`_
-* `Quickstart <http://docs.platformio.ikravets.com/en/latest/quickstart.html>`_
-* `Installation <http://docs.platformio.ikravets.com/en/latest/installation.html>`_
-* `Project Configuration File <http://docs.platformio.ikravets.com/en/latest/projectconf.html>`_
-* `Platforms & Embedded Boards <http://docs.platformio.ikravets.com/en/latest/platforms/index.html>`_
+`PlatformIO <http://platformio.ikravets.com>`_ is a cross-platform code builder
+and the missing library manager.
+
+* `Get Started <http://platformio.ikravets.com/#!/get-started>`_
+* `Web 2.0 Library Search <http://platformio.ikravets.com/#!/lib>`_
+* `Development Platforms <http://platformio.ikravets.com/#!/platforms>`_
+* `Embedded Boards <http://platformio.ikravets.com/#!/boards>`_
+* `Library Manager <http://docs.platformio.ikravets.com/en/latest/librarymanager/index.html>`_
 * `User Guide <http://docs.platformio.ikravets.com/en/latest/userguide/index.html>`_
 * `IDE Integration <http://docs.platformio.ikravets.com/en/latest/ide.html>`_
 * `Release History <http://docs.platformio.ikravets.com/en/latest/history.html>`_
 
-You have no need to install any *IDE* or compile any tool chains. *PlatformIO*
+You have **no need** to install any *IDE* or compile any tool chains. *PlatformIO*
 has pre-built different development platforms including: compiler, debugger,
 uploader (for embedded boards) and many other useful tools.
 
-**PlatformIO** allows developer to compile the same code with different
-platforms using only one command
+Use whenever. *Run everywhere.*
+-------------------------------
+*PlatformIO* is written in pure *Python* and **doesn't depend** on any
+additional libraries/tools from an operation system. It allows you to use
+*PlatformIO* beginning from *PC (Mac, Linux, Win)* and ending with credit-card
+sized computers (like *Raspberry Pi*).
+
+Embedded Development. *Easier Than Ever.*
+-----------------------------------------
+*PlatformIO* is well suited for embedded development and has pre-configured
+settings for most popular `Embedded Boards <http://platformio.ikravets.com/#!/boards>`_.
+
+* Colourful `command-line output <https://raw.githubusercontent.com/ivankravets/platformio/develop/examples/platformio-examples.png>`_
+* Built-in `Serial Port Monitor <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_serialports.html#platformio-serialports-monitor>`_
+* Configurable `build -flags/-options <http://docs.platformio.ikravets.com/en/latest/projectconf.html#build-flags>`_
+* Automatic **firmware uploading**
+* Integration with `development environments (IDE) <http://docs.platformio.ikravets.com/en/latest/ide.html>`_
+* Ready for **cloud compilers**
+* Pre-built tool chains, frameworks for the popular `Hardware Platforms <http://platformio.ikravets.com/#!/platforms>`_
+
+.. image:: https://raw.githubusercontent.com/ivankravets/platformio-web/develop/app/images/platformio-embedded-development.png
+    :target: http://platformio.ikravets.com
+    :alt:  PlatformIO Embedded Development Process
+
+The Missing Library Manager. *It's here!*
+-----------------------------------------
+*PlatformIO Library Manager* is the missing library manager for development
+platforms which allows you to organize and have up-to-date external libraries.
+
+* Friendly `Command-Line Interface <http://docs.platformio.ikravets.com/en/latest/librarymanager/index.html>`_
+* Modern `Web 2.0 Library Search <http://platformio.ikravets.com/#!/lib>`_
+* Open Source `Library Registry API <https://github.com/ivankravets/platformio-api>`_
+* Library Crawler based on `library.json <http://docs.platformio.ikravets.com/en/latest/librarymanager/config.html>`_
+  specification
+* Library **dependency management**
+* Automatic library updating
+
+.. image:: https://raw.githubusercontent.com/ivankravets/platformio-web/develop/app/images/platformio-library-manager.png
+    :target: http://platformio.ikravets.com
+    :alt:  PlatformIO Library Manager Architecture
+
+Smart Code Builder. *Fast and Reliable.*
+----------------------------------------
+*PlatformIO Code Builder* is built-on a next-generation software construction
+tool named `SCons <http://www.scons.org/>`_. Think of *SCons* as an improved,
+cross-platform substitute for the classic *Make* utility.
+
+* Reliable, automatic *dependency analysis*
+* Reliable detection of *build changes*
+* Improved support for *parallel builds*
+* Ability to share *built files in a cache*
+* Lookup for external libraries which are installed via `Library Manager <http://docs.platformio.ikravets.com/en/latest/librarymanager/index.html>`_
+
+.. image:: https://raw.githubusercontent.com/ivankravets/platformio-web/develop/app/images/platformio-scons-builder.png
+    :target: http://platformio.ikravets.com
+    :alt:  PlatformIO Code Builder Architecture
+
+Single source code. *Multiple platforms.*
+-----------------------------------------
+*PlatformIO* allows developer to compile the same code with different
+development platforms using the *Only One Command*
 `platformio run <http://docs.platformio.ikravets.com/en/latest/userguide/cmd_run.html>`_.
 This happens due to
-`Project Configuration File <http://docs.platformio.ikravets.com/en/latest/projectconf.html>`_
-where you can setup different environments with specific
-options: platform type, firmware uploading settings, pre-built framework
-and many more.
-
-.. image:: examples/platformio-examples.png
-    :target: https://github.com/ivankravets/platformio/raw/develop/examples/platformio-examples.png
-    :alt:  Examples
-    :width: 730px
-
-**PlatformIO** is well suited for **embedded development**. It can:
-
-* Automatically analyse dependency
-* Reliably detect build changes
-* Build framework or library source code to static library
-* Build *ELF* (executable and linkable firmware)
-* Convert *ELF* to *HEX* or *BIN* file
-* Extract *EEPROM* data
-* Upload firmware to your device
+`Project Configuration File (platformio.ini) <http://docs.platformio.ikravets.com/en/latest/projectconf.html>`_
+where you can setup different environments with specific options (platform
+type, firmware uploading settings, pre-built framework, build flags and many
+more).
 
 It has support for many popular embedded platforms like these:
 
-* ``atmelavr`` `Atmel AVR <http://en.wikipedia.org/wiki/Atmel_AVR>`_
-  (including `Arduino <http://www.arduino.cc>`_ based boards)
-* ``timsp430`` `TI MSP430 <http://www.ti.com/lsds/ti/microcontroller/16-bit_msp430/overview.page>`_
-  (including `MSP430 LaunchPads <http://www.ti.com/ww/en/launchpad/launchpads-msp430.html>`_)
-* ``titiva`` `TI TIVA C <http://www.ti.com/lsds/ti/microcontroller/tiva_arm_cortex/c_series/overview.page>`_
-  (including `TIVA C Series LaunchPads <http://www.ti.com/ww/en/launchpad/launchpads-connected.html>`_)
+* ``atmelavr`` `Atmel AVR <http://platformio.ikravets.com/#!/platforms/atmelavr>`_
+  (including *Arduino*-based boards, *Microduino, Raspduino, Teensy*)
+* ``timsp430`` `TI MSP430 <http://platformio.ikravets.com/#!/platforms/timsp430>`_
+  (including *MSP430* LaunchPads)
+* ``titiva`` `TI TIVA C <http://platformio.ikravets.com/#!/platforms/titiva>`_
+  (including *TIVA C* Series LaunchPads)
 
 
 Licence
@@ -79,4 +126,3 @@ Licence
 Copyright (C) 2014 Ivan Kravets
 
 Licenced under the MIT Licence.
-

docs/conf.py

@@ -274,4 +274,5 @@ if not on_rtd:  # only import and set the theme if we're building docs locally
 
 # A timeout value, in seconds, for the linkcheck builder
 # http://sphinx-doc.org/config.html#confval-linkcheck_timeout
-linkcheck_timeout = 30
+linkcheck_timeout = 10
+linkcheck_anchors = False

docs/ide.rst

@@ -3,14 +3,36 @@
 IDE Integration
 ===============
 
+Arduino IDE
+-----------
+
+.. image:: _static/ide-platformio-arduino.png
+    :target: http://www.ikravets.com/computer-life/platformio/2014/10/07/integration-of-platformio-library-manager-to-arduino-and-energia-ides
+
+* `Integration of PlatformIO library manager to Arduino IDE <http://www.ikravets.com/computer-life/platformio/2014/10/07/integration-of-platformio-library-manager-to-arduino-and-energia-ides>`_
+
 Eclipse
 -------
-`Building and debugging Atmel AVR (Arduino-based) project using Eclipse IDE+PlatformIO <http://www.ikravets.com/computer-life/programming/2014/06/20/building-and-debugging-atmel-avr-arduino-based-project-using-eclipse-ideplatformio>`_
 
 
+.. image:: _static/ide-platformio-eclipse.png
+    :target: http://www.ikravets.com/computer-life/programming/2014/06/20/building-and-debugging-atmel-avr-arduino-based-project-using-eclipse-ideplatformio
+
+* `Building and debugging Atmel AVR (Arduino-based) project using Eclipse IDE+PlatformIO <http://www.ikravets.com/computer-life/programming/2014/06/20/building-and-debugging-atmel-avr-arduino-based-project-using-eclipse-ideplatformio>`_
+
+Energia IDE
+-----------
+
+.. image:: _static/ide-platformio-energia.png
+    :target: http://www.ikravets.com/computer-life/platformio/2014/10/07/integration-of-platformio-library-manager-to-arduino-and-energia-ides
+
+* `Integration of PlatformIO library manager to Energia IDE <http://www.ikravets.com/computer-life/platformio/2014/10/07/integration-of-platformio-library-manager-to-arduino-and-energia-ides>`_
+
 VIM
 ---
 
+.. image:: _static/ide-platformio-vim.png
+
 Recommended bundles:
 
 * Syntax highlight - `Arduino-syntax-file <https://github.com/vim-scripts/Arduino-syntax-file>`_

docs/index.rst

@@ -1,12 +1,15 @@
-PlatformIO: A cross-platform code builder and library manager
-=============================================================
+PlatformIO: A cross-platform code builder and the missing library manager
+=========================================================================
+
+.. image:: _static/platformio-logo.png
+    :target: http://platformio.ikravets.com
 
 `Website + Library Search <http://platformio.ikravets.com>`_ |
 `Project Examples <https://github.com/ivankravets/platformio/tree/develop/examples>`_ |
 `Source Code <https://github.com/ivankravets/platformio>`_ |
-`Bugs/Questions <https://github.com/ivankravets/platformio/issues>`_ |
+`Issues <https://github.com/ivankravets/platformio/issues>`_ |
 `Blog <http://www.ikravets.com/category/computer-life/platformio>`_ |
-`Twitter <https://twitter.com/smartanthill>`_
+`Twitter <https://twitter.com/PlatformIOTool>`_
 
 You have no need to install any *IDE* or compile any tool chains. *PlatformIO*
 has pre-built different development platforms including: compiler, debugger,
@@ -27,9 +30,7 @@ instruments.
 * Automatically analyse dependency
 * Reliably detect build changes
 * Build framework or library source code to static library
-* Build *ELF* (executable and linkable firmware)
-* Convert *ELF* to *HEX* or *BIN* file
-* Extract *EEPROM* data
+* Lookup for external libraries which are installed via :ref:`librarymanager`
 * Upload firmware to your device
 
 

docs/installation.rst

@@ -3,31 +3,51 @@
 Installation
 ============
 
-*PlatformIO* is written in `Python <https://www.python.org>`_ and works with
-versions 2.6 and 2.7 on Unix/Linux, OS X, Windows and Credit-card ARM-based
-computers (Raspberry Pi).
+*PlatformIO* is written in `Python <http://python.org/download/>`_ and works
+on *Mac OS X*, *Linux*, *Windows OS* and Credit-card *ARM*-based
+computers (*Raspberry Pi*).
+
+System requirements
+-------------------
+
+* **Operating systems:**
+    * Mac OS X
+    * Linux
+    * Windows
+* `Python 2.6 or Python 2.7 <http://python.org/download/>`_
 
 All commands below should be executed in
 `Command-line <http://en.wikipedia.org/wiki/Command-line_interface>`_
-application in your *OS*:
+application:
 
-* *Unix/Linux/OS X* this is *Terminal* application.
+* *Mac OS X / Linux* this is *Terminal* application.
 * *Windows* this is
   `Command Prompt <http://en.wikipedia.org/wiki/Command_Prompt>`_ (``cmd.exe``)
   application.
 
+Please *choose one of* the following:
 
-Super-Quick
------------
+Super-Quick (Mac / Linux)
+-------------------------
 
-To install or upgrade *PlatformIO*, download
-`get-platformio.py <https://raw.githubusercontent.com/ivankravets/platformio/develop/scripts/get-platformio.py>`_ script.
-
-Then run the following (which may require administrator access):
+To install or upgrade *PlatformIO* paste that at a *Terminal* prompt
+(you might need to run ``sudo`` first):
 
 .. code-block:: bash
 
-    $ [sudo] python get-platformio.py
+    python -c "$(curl -fsSL https://raw.githubusercontent.com/ivankravets/platformio/master/scripts/get-platformio.py)"
+
+
+Installer Script (Mac / Linux / Windows)
+----------------------------------------
+
+To install or upgrade *PlatformIO*, download
+`get-platformio.py <https://raw.githubusercontent.com/ivankravets/platformio/develop/scripts/get-platformio.py>`_
+script. Then run the following (you might need to run ``sudo`` first):
+
+.. code-block:: bash
+
+    python get-platformio.py
 
 
 On *Windows OS* it may look like:

docs/librarymanager/config.rst

@@ -1,18 +1,20 @@
+.. |PIOAPICR| replace:: *PlatformIO Library Registry Crawler*
 .. _library_config:
 
 library.json
 ============
 
-*PlatformIO*-suitable library should be defined by a manifest file
-``library.json`` in a `JSON-style <http://en.wikipedia.org/wiki/JSON>`_.
-A data in ``library.json`` should be represented via
-`associative array <http://en.wikipedia.org/wiki/Associative_array>`_
-(name/value pairs). An order doesn't matter.
+``library.json`` is a manifest file of development library.
 
-The allowable fields (names from pairs) are described below. The fields
-(:ref:`libjson_name`, :ref:`libjson_description` and :ref:`libjson_keywords`)
-will be displayed in the search results at the :ref:`cmd_lib_search` (*CLI*)
-and at the *WebSite*. Also, they can be used for searching for libraries.
+Initially it was
+developed for :ref:`librarymanager`, but later was accepted by worldwide embedded
+community like a **standard library specification**.
+
+A data in ``library.json`` should be represented
+in `JSON-style <http://en.wikipedia.org/wiki/JSON>`_ via
+`associative array <http://en.wikipedia.org/wiki/Associative_array>`_
+(name/value pairs). An order doesn't matter. The allowable fields
+(names from pairs) are described below.
 
 .. contents::
 
@@ -60,6 +62,89 @@ start/end with them). A list from the keywords can be specified with
 separator ``,``
 
 
+.. _libjson_authors:
+
+``authors``
+-----------
+
+*Required* if :ref:`libjson_repository` field is not defined | Type: ``Object``
+or ``Array``
+
+An author contact information
+
+* ``name`` Full name (**Required**)
+* ``email``
+* ``url`` An author's contact page
+* ``maintainer`` Specify "maintainer" status
+
+Examples:
+
+.. code-block:: javascript
+
+    "authors":
+    {
+        "name": "John Smith",
+        "email": "me@john-smith.com",
+        "url": "http://www.john-smith/contact"
+    }
+
+    ...
+
+    "authors":
+    [
+        {
+            "name": "John Smith",
+            "email": "me@john-smith.com",
+            "url": "http://www.john-smith/contact"
+        },
+        {
+            "name": "Andrew Smith",
+            "email": "me@andrew-smith.com",
+            "url": "http://www.andrew-smith/contact",
+            "maintainer": true
+        }
+    ]
+
+
+.. note::
+    You can omit :ref:`libjson_authors` field and define
+    :ref:`libjson_repository` field. Only *GitHub-based* repository is
+    supported now. In this case
+    |PIOAPICR| will use information from
+    `GitHub API Users <https://developer.github.com/v3/users/>`_.
+
+
+.. _libjson_repository:
+
+``repository``
+--------------
+
+*Required* if :ref:`libjson_downloadurl` field is not defined | Type: ``Object``
+
+The repository in which the source code can be found.
+
+Example:
+
+.. code-block:: javascript
+
+    "repository":
+    {
+        "type": "git",
+        "url": "https://github.com/foo/bar.git"
+    }
+
+
+.. _libjson_downloadurl:
+
+``downloadUrl``
+---------------
+
+*Required* if :ref:`libjson_repository` field is not defined | Type: ``String``
+
+It is the *HTTP URL* to the archived source code of library. It should end
+with the type of archive (``.zip`` or ``.tar.gz``).
+
+
 .. _libjson_version:
 
 ``version``
@@ -79,70 +164,7 @@ A version of the current library source code.
 .. note::
     You can omit :ref:`libjson_version` field and define
     :ref:`libjson_repository` field. In this case
-    *PlatformIO-API Crawler* will use the *CVS*-revision from the latest commit.
-
-
-.. _libjson_author:
-
-``author``
-----------
-
-*Required* if :ref:`libjson_repository` field is not defined | Type: ``Object``
-
-An author contact information
-
-* ``name`` Full name (**Required**)
-* ``email``
-* ``url`` An author's contact page
-
-Example:
-
-.. code-block:: javascript
-
-    "author":
-    {
-        "name": "John Smith",
-        "email": "me@john-smith.com",
-        "url": "http://www.john-smith/contact"
-    }
-
-.. note::
-    You can omit :ref:`libjson_author` field and define
-    :ref:`libjson_repository` field. Only *GitHub-based* repository is
-    supported now. In this case
-    *PlatformIO-API Crawler* will use information from
-    `GitHub API Users <https://developer.github.com/v3/users/>`_.
-
-
-.. _libjson_repository:
-
-``repository``
---------------
-
-*Required* if :ref:`libjson_downloadurl` field is not defined | Type: ``Object``
-
-The repository in which the source code can be found.
-
-Example:
-
-.. code-block:: javascript
-
-    "repository":
-    {
-        "name": "git",
-        "url": "https://github.com/foo/bar.git"
-    }
-
-
-.. _libjson_downloadurl:
-
-``downloadUrl``
----------------
-
-*Required* if :ref:`libjson_repository` field is not defined | Type: ``String``
-
-It is the *HTTP URL* to the archived source code of library. It should end
-with the type of archive (``.zip`` or ``.tar.gz``).
+    |PIOAPICR| will use the *CVS*-revision from the latest commit.
 
 
 .. _libjson_include:
@@ -154,7 +176,7 @@ with the type of archive (``.zip`` or ``.tar.gz``).
 `Glob Pattern <http://en.wikipedia.org/wiki/Glob_(programming)>`_
 
 If :ref:`libjson_include` field is a type of ``String``, then
-*PlatformIO-API Crawler* will recognize it like a "relative path inside
+|PIOAPICR| will recognize it like a "relative path inside
 repository/archive to library source code". See example below where the only
 source code from the relative directory ``LibrarySourceCodeHere`` will be
 included.
@@ -164,7 +186,7 @@ included.
     "include": "some/child/dir/LibrarySourceCodeHere"
 
 If :ref:`libjson_include` field is a type of ``Array``, then
-*PlatformIO-API Crawler* firstly will apply :ref:`libjson_exclude` filter and
+|PIOAPICR| firstly will apply :ref:`libjson_exclude` filter and
 then include only directories/files which match with :ref:`libjson_include`
 patterns.
 
@@ -207,25 +229,63 @@ Pattern	Meaning
 Exclude the directories and files which match with :ref:`libjson_exclude`
 patterns.
 
+.. _libjson_frameworks:
+
+``frameworks``
+--------------
+
+*Optional* | Type: ``String`` or ``Array``
+
+A list with compatible frameworks. The available framework types are defined in
+the :ref:`platforms` section.
+
+
+.. _libjson_platforms:
+
+``platforms``
+-------------
+
+*Optional* | Type: ``String`` or ``Array``
+
+A list with compatible platforms. The available platform types are
+defined in :ref:`platforms` section.
+
 
 .. _libjson_dependencies:
 
 ``dependencies``
 ----------------
 
-*Optional* | Type: ``Array``
+*Optional* | Type: ``Array`` or ``Object``
 
 A list of dependent libraries. They will be installed automatically with
 :ref:`cmd_lib_install` command.
 
+Allowed requirements for dependent library:
+
+* ``name`` | Type: ``String``
+* ``authors`` | Type: ``String`` or ``Array``
+* ``frameworks`` | Type: ``String`` or ``Array``
+* ``platforms`` | Type: ``String`` or ``Array``
+
 Example:
 
 .. code-block:: javascript
 
     "dependencies":
     [
-        "Library-Foo",
-        "Library-Bar"
+        {
+            "name": "Library-Foo",
+            "authors":
+            [
+                "Jhon Smith",
+                "Andrew Smith"
+            ]
+        },
+        {
+            "name": "Library-Bar",
+            "frameworks": "FrameworkFoo, FrameworkBar"
+        }
     ]
 
 

docs/librarymanager/creating.rst

@@ -1,5 +1,5 @@
 .. _library_creating:
-.. |PIOAPICR| replace:: *PlatformIO-API Crawler*
+.. |PIOAPICR| replace:: *PlatformIO Library Registry Crawler*
 
 Creating Library
 ================
@@ -36,14 +36,14 @@ you **need to specify** only these fields in the :ref:`library_config`:
 * :ref:`libjson_repository`
 
 |PIOAPICR| will populate the rest fields, like :ref:`libjson_version` or
-:ref:`libjson_author` with an actual information from *GitHub*.
+:ref:`libjson_authors` with an actual information from *GitHub*.
 
 Example:
 
 .. code-block:: javascript
 
     {
-        "name": "Arduino-IRremote",
+        "name": "IRremote",
         "keywords": "infrared, ir, remote",
         "description": "Send and receive infrared signals with multiple protocols",
         "repository":
@@ -62,7 +62,7 @@ The list of **required** fields in the :ref:`library_config` will look like:
 * :ref:`libjson_name`
 * :ref:`libjson_keywords`
 * :ref:`libjson_description`
-* :ref:`libjson_author`
+* :ref:`libjson_authors`
 * :ref:`libjson_repository`
 
 Example:
@@ -70,10 +70,10 @@ Example:
 .. code-block:: javascript
 
     {
-        "name": "Arduino-XBee",
+        "name": "XBee",
         "keywords": "xbee, protocol, radio",
         "description": "Arduino library for communicating with XBees in API mode",
-        "author":
+        "authors":
         {
             "name": "Andrew Rapp",
             "email": "andrew.rapp@gmail.com",
@@ -97,17 +97,17 @@ of **required** fields in the :ref:`library_config` will look like:
 * :ref:`libjson_name`
 * :ref:`libjson_keywords`
 * :ref:`libjson_description`
-* :ref:`libjson_author`
+* :ref:`libjson_authors`
 * :ref:`libjson_version`
 * :ref:`libjson_downloadurl`
 
 .. code-block:: javascript
 
     {
-        "name": "Arduino-OneWire",
+        "name": "OneWire",
         "keywords": "onewire, 1-wire, bus, sensor, temperature, ibutton",
         "description": "Control devices (from Dallas Semiconductor) that use the One Wire protocol (DS18S20, DS18B20, DS2408 and etc)",
-        "author":
+        "authors":
         {
             "name": "Paul Stoffregen",
             "url": "http://www.pjrc.com/teensy/td_libs_OneWire.html"

docs/librarymanager/index.rst

@@ -5,11 +5,11 @@ Library Manager
 
 ..
 
-    *"The missing library manager for embedded platforms"* [#]_
+    *"The missing library manager for development platforms"* [#]_
 
 *PlatformIO Library Manager* allows you to organize external embedded libraries.
-You can search for new libraries via :ref:`Command Line <cmd_lib_search>`
-or `WebSite <http://platformio.ikravets.com>`_ interfaces.
+You can search for new libraries via :ref:`Command Line interface <cmd_lib_search>`
+or `Web 2.0 Library Search <http://platformio.ikravets.com/#!/lib>`_.
 
 You don't need to bother for finding the latest version of library. Due to
 :ref:`cmd_lib_update` command you will have up-to-date external libraries.
@@ -17,8 +17,9 @@ You don't need to bother for finding the latest version of library. Due to
 .. toctree::
     :maxdepth: 2
 
-    creating
     config
+    creating
+    User Guide <../userguide/lib/index.rst>
 
 .. [#] Inspired by `npm <https://www.npmjs.org>`_ and `bower
     <http://bower.io>`_ package managers for web.

docs/projectconf.rst

@@ -246,10 +246,6 @@ Examples
     framework = arduino
     board = uno
 
-    upload_port = /dev/ttyUSB0
-    # for Windows OS
-    # upload_port = COM3
-
     # enable auto-uploading
     targets = upload
 
@@ -265,10 +261,6 @@ Examples
     framework = arduino
     board = 168pa8m
 
-    upload_port = /dev/ttyUSB0
-    # for Windows OS
-    # upload_port = COM3
-
     # enable auto-uploading
     targets = upload
 

docs/quickstart.rst

@@ -3,6 +3,10 @@
 Quickstart
 ==========
 
+.. note::
+    Please read `Get Started <http://platformio.ikravets.com/#!/get-started>`_
+    article from the official WebSite.
+
 First, :ref:`Install PlatformIO <installation>`.
 
 Print all available development platforms for installing

docs/userguide/cmd_init.rst

@@ -10,7 +10,7 @@ Usage
 
 .. code-block:: bash
 
-    platformio init
+    platformio init [OPTIONS]
 
 
 Description
@@ -21,22 +21,50 @@ Initialize new PlatformIO based project.
 
 This command will create:
 
-* ``.pioenvs`` - a temporary working directory.
-* ``lib`` - a directory for project specific libraries. PlatformIO will
-  compile them to static libraries and link to executable file
-* ``src`` - a source directory. Put your source code here.
 * :ref:`projectconf`
+* ``src`` - a source directory. Put your source code here
+* ``lib`` - a directory for the project specific libraries. PlatformIO will
+  compile them to static libraries and link to executable file
+
+Options
+-------
+
+.. option::
+    --project-dir, -d
+
+Specified path to the directory where *PlatformIO* will initialize new project.
 
 
 Examples
 --------
 
+1. Create new project in the current working directory
+
 .. code-block:: bash
 
-    # Change directory to the future project
-    $ cd /path/to/empty/directory
     $ platformio init
-    Project has been initialized!
-    Please put your source code to `src` directory, external libraries to `lib`
-    and setup environments in `platformio.ini` file.
-    Then process project with `platformio run` command.
+    The current working directory *** will be used for the new project.
+    You can specify another project directory via
+    `platformio init -d %PATH_TO_PROJECT_DIR%` command.
+
+    The next files/directories will be created in ***
+    platformio.ini - Project Configuration File
+    src - a source directory. Put your source code here
+    lib - a directory for the project specific libraries
+    Do you want to continue? [y/N]: y
+    Project has been successfully initialized!
+    Now you can process it with `platformio run` command.
+
+
+2. Create new project in the specified directory
+
+.. code-block:: bash
+
+    $ platformio init -d %PATH_TO_DIR%
+    The next files/directories will be created in ***
+    platformio.ini - Project Configuration File
+    src - a source directory. Put your source code here
+    lib - a directory for the project specific libraries
+    Do you want to continue? [y/N]: y
+    Project has been successfully initialized!
+    Now you can process it with `platformio run` command.

docs/userguide/cmd_search.rst

@@ -11,7 +11,7 @@ Usage
 .. code-block:: bash
 
     # Print all available development platforms
-    platformio search all
+    platformio search
 
     # Filter platforms by "Query"
     platformio search QUERY

docs/userguide/cmd_settings.rst

@@ -0,0 +1,114 @@
+.. _cmd_settings:
+
+platformio settings
+===================
+
+Manage PlatformIO settings
+
+.. contents::
+
+platformio settings get
+-----------------------
+
+Usage
+~~~~~
+
+.. code-block:: bash
+
+    platformio settings get [NAME]
+
+
+Description
+~~~~~~~~~~~
+
+Get/List existing settings
+
+
+Examples
+~~~~~~~~
+
+1. List all settings and current their values
+
+.. code-block:: bash
+
+    $ platformio settings get
+    Name                            Value [Default]   Description
+    ------------------------------------------------------------------------------------------
+    auto_update_libraries           Yes               Automatically update libraries (Yes/No)
+    auto_update_platforms           Yes               Automatically update platforms (Yes/No)
+    check_libraries_interval        7                 Check for the library updates interval (days)
+    check_platformio_interval       3                 Check for the new PlatformIO interval (days)
+    check_platforms_interval        7                 Check for the platforms updates interval (days)
+
+
+2. Show specified setting
+
+.. code-block:: bash
+
+    $ platformio settings get auto_update_platforms
+    Name                            Value [Default]   Description
+    ------------------------------------------------------------------------------------------
+    auto_update_platforms           Yes               Automatically update platforms (Yes/No)
+
+
+platformio settings set
+-----------------------
+
+Usage
+~~~~~
+
+.. code-block:: bash
+
+    platformio settings set NAME VALUE
+
+
+Description
+~~~~~~~~~~~
+
+Set new value for the setting
+
+Examples
+~~~~~~~~
+
+Change to check for the new PlatformIO each day
+
+.. code-block:: bash
+
+    $ platformio settings set check_platformio_interval 1
+    The new value for the setting has been set!
+    Name                            Value [Default]   Description
+    ------------------------------------------------------------------------------------------
+    check_platformio_interval       1 [3]             Check for the new PlatformIO interval (days)
+
+
+platformio settings reset
+-------------------------
+
+Usage
+~~~~~
+
+.. code-block:: bash
+
+    platformio settings reset
+
+
+Description
+~~~~~~~~~~~
+
+Reset settings to default
+
+Examples
+~~~~~~~~
+
+.. code-block:: bash
+
+    $ platformio settings reset
+    The settings have been reseted!
+
+    Name                            Value [Default]   Description
+    ------------------------------------------------------------------------------------------
+    auto_update_libraries           Yes               Automatically update libraries (Yes/No)
+    auto_update_platforms           Yes               Automatically update platforms (Yes/No)
+    check_libraries_interval        7                 Check for the library updates interval (days)
+    check_platformio_interval       3                 Check for the new PlatformIO interval (days)
+    check_platforms_interval        7                 Check for the platform updates interval (days)

docs/userguide/index.rst

@@ -21,6 +21,7 @@ To print all available commands and options use:
     cmd_run
     cmd_search
     cmd_serialports
+    cmd_settings
     cmd_show
     cmd_uninstall
     cmd_update

docs/userguide/lib/cmd_install.rst

@@ -10,13 +10,14 @@ Usage
 
 .. code-block:: bash
 
-    platformio lib install [OPTIONS] [NAMES]
+    platformio lib install [OPTIONS] [LIBRARY_ID]
 
 
 Description
 -----------
 
-Install new library
+Install new library  by specified
+`PlatformIO Library Registry ID <http://platformio.ikravets.com/#!/lib>`_.
 
 Options
 -------
@@ -34,35 +35,38 @@ Examples
 
 .. code-block:: bash
 
-    $ platformio lib install Arduino-IRremote
-    Installing Arduino-IRremote library:
-    Downloading  [####################################]  100%
-    Unpacking  [####################################]  100%
-    The library 'Arduino-IRremote' has been successfully installed!
+    # IRremote: http://platformio.ikravets.com/#!/lib/show/4/IRremote
+    $ platformio lib install 4
+    # Installing library [ 4 ]:
+    # Downloading  [####################################]  100%
+    # Unpacking  [####################################]  100%
+    # The library #4 'IRremote' has been successfully installed!
 
 
 2. Install specified version of library
 
 .. code-block:: bash
 
-    $ platformio lib install Arduino-XBee --version=0.5
-    Installing Arduino-XBee library:
-    Downloading  [####################################]  100%
-    Unpacking  [####################################]  100%
-    The library 'Arduino-XBee' has been successfully installed!
+    # XBee: http://platformio.ikravets.com/#!/lib/show/6/XBee
+    $ platformio lib install 6 --version=0.5
+    # Installing library [ 6 ]:
+    # Downloading  [####################################]  100%
+    # Unpacking  [####################################]  100%
+    # The library #6 'XBee' has been successfully installed!
 
 
 3. Install library with dependencies
 
 .. code-block:: bash
 
-    $ platformio lib install Adafruit-Arduino-ST7735
-    Installing Adafruit-Arduino-ST7735 library:
-    Downloading  [####################################]  100%
-    Unpacking  [####################################]  100%
-    The library 'Adafruit-Arduino-ST7735' has been successfully installed!
-    Installing dependencies:
-    Installing Adafruit-Arduino-GFX library:
-    Downloading  [####################################]  100%
-    Unpacking  [####################################]  100%
-    The library 'Adafruit-Arduino-GFX' has been successfully installed!
+    # Adafruit-ST7735: http://platformio.ikravets.com/#!/lib/show/12/Adafruit-ST7735
+    $ platformio lib install 12
+    # Installing library [ 12 ]:
+    # Downloading  [####################################]  100%
+    # Unpacking  [####################################]  100%
+    # The library #12 'Adafruit-ST7735' has been successfully installed!
+    # Installing dependencies:
+    # Installing library [ 13 ]:
+    # Downloading  [####################################]  100%
+    # Unpacking  [####################################]  100%
+    # The library #13 'Adafruit-GFX' has been successfully installed!

docs/userguide/lib/cmd_list.rst

@@ -25,7 +25,15 @@ Examples
 .. code-block:: bash
 
     $ platformio lib list
-    Arduino-IRremote      Send and receive infrared signals with multiple protocols
-    ...
-    Arduino-Webduino      An extensible web server library (for use with the Arduino Ethernet Shield)
-    Arduino-XBee          Arduino library for communicating with XBees in API mode
+    #
+    # [ ID  ] Name             Compatibility         "Authors": Description
+    # -------------------------------------------------------------------------------------
+    # [ 23  ] Adafruit-L3GD20-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for the L3GD20 Gyroscope
+    # [ 12  ] Adafruit-ST7735  arduino, atmelavr     "Adafruit Industries": A library for the Adafruit 1.8" SPI display
+    # [ 31  ] Adafruit-Unified-Sensor arduino, atmelavr     "Adafruit Industries": Adafruit Unified Sensor Driver
+    # [ 26  ] Adafruit-LSM303DLHC-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for Adafruit's LSM303 Breakout (Accelerometer + Magnetometer)
+    # [  6  ] XBee             arduino, atmelavr     "Andrew Rapp": Arduino library for communicating with XBees in API mode
+    # [ 13  ] Adafruit-GFX     arduino, atmelavr     "Adafruit Industries": A core graphics library for all our displays, providing a common set of graphics primitives (points, lines, circles, etc.)
+    # [  4  ] IRremote         arduino, atmelavr     "Ken Shirriff": Send and receive infrared signals with multiple protocols
+    # [ 14  ] Adafruit-9DOF-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for the Adafruit 9DOF Breakout (L3GD20 / LSM303)
+    # ...

docs/userguide/lib/cmd_search.rst

@@ -16,8 +16,8 @@ Usage
 Description
 -----------
 
-Search for library over ``name``, ``description`` and ``keywords`` fields from
-the :ref:`library_config` file in the boolean mode.
+Search for library in `PlatformIO Library Registry <http://platformio.ikravets.com/#!/lib>`_
+by :ref:`library_config` fields in the boolean mode.
 
 The boolean search capability supports the following operators:
 
@@ -71,66 +71,121 @@ Filter libraries by specified author
 
 Filter libraries by specified keyword
 
+
+.. option::
+    -f, --framework
+
+Filter libraries by specified framework
+
+
+.. option::
+    -p, --platform
+
+Filter libraries by specified keyword
+
 Examples
 --------
 
-1. Search for "1-Wire" library
+1. List all libraries
 
 .. code-block:: bash
 
-    $ platformio lib search 1-wire
-    Found N libraries:
-    Arduino-OneWire       Control devices (from Dallas Semiconductor) that use the One Wire protocol
-    ...
+    $ platformio lib search
+    # Found N libraries:
+    #
+    # [ ID  ] Name             Compatibility         "Authors": Description
+    # -------------------------------------------------------------------------------------
+    # [ 14  ] Adafruit-9DOF-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for the Adafruit 9DOF Breakout (L3GD20 / LSM303)
+    # [ 13  ] Adafruit-GFX     arduino, atmelavr     "Adafruit Industries": A core graphics library for all our displays, providing a common set of graphics primitives (points, lines, circles, etc.)
+    # [ 23  ] Adafruit-L3GD20-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for the L3GD20 Gyroscope
+    # [ 26  ] Adafruit-LSM303DLHC-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for Adafruit's LSM303 Breakout (Accelerometer + Magnetometer)
+    # [ 12  ] Adafruit-ST7735  arduino, atmelavr     "Adafruit Industries": A library for the Adafruit 1.8" SPI display
+    # [ 31  ] Adafruit-Unified-Sensor arduino, atmelavr     "Adafruit Industries": Adafruit Unified Sensor Driver
+    # [  4  ] IRremote         arduino, atmelavr     "Ken Shirriff": Send and receive infrared signals with multiple protocols
+    # [  1  ] OneWire          arduino, atmelavr     "Paul Stoffregen": Control devices (from Dallas Semiconductor) that use the One Wire protocol (DS18S20, DS18B20, DS2408 and etc)
+    # [  6  ] XBee             arduino, atmelavr     "Andrew Rapp": Arduino library for communicating with XBees in API mode
+    # [ 15  ] Adafruit-ADXL345-Unified arduino, atmelavr     "Adafruit Industries": Unified driver for the ADXL345 Accelerometer
+    # Show next libraries? [y/N]:
+    # ...
 
-2. Search for Arduino-based "I2C" libraries. The ``+`` sign is here like ``AND``
-   operator.
+2. Search for `1-Wire libraries <http://platformio.ikravets.com/#!/lib/search?query=%25221-wire%2522>`_
 
 .. code-block:: bash
 
-    $ platformio lib search "+i2c +arduino"
-    Found N libraries:
-    i2cdevlib-Arduino-i2cdev The I2C Device Library (i2cdevlib) is a collection of uniform and well-documented classes to provide simple and intuitive interfaces to I2C devices.
-    i2cdevlib-Arduino-AK8975 AK8975 is 3-axis electronic compass IC with high sensitive Hall sensor technology
-    ...
+    $ platformio lib search "1-wire"
+    # Found N libraries:
+    #
+    # [ ID  ] Name             Compatibility         "Authors": Description
+    # -------------------------------------------------------------------------------------
+    # [  1  ] OneWire          arduino, atmelavr     "Paul Stoffregen": Control devices (from Dallas Semiconductor) that use the One Wire protocol (DS18S20, DS18B20, DS2408 and etc)
+    # ...
 
-3. Search for libraries by "web" and "http" keywords. The ``""`` here is for
-   "empty" query argument.
+3. Search for `Arduino-based "I2C" libraries <http://platformio.ikravets.com/#!/lib/search?query=framework%253Aarduino%2520i2c&page=1>`_
 
 .. code-block:: bash
 
-    $ platformio lib search "" --keyword web --keyword http
-    Found N libraries:
-    Arduino-Webduino      An extensible web server library (for use with the Arduino Ethernet Shield)
-    Arduino-aJson         An Arduino library to enable JSON processing with Arduino
-    ...
+    $ platformio lib search "i2c" --framework="arduino"
+    # Found N libraries:
+    #
+    # [ ID  ] Name             Compatibility         "Authors": Description
+    # -------------------------------------------------------------------------------------
+    # [ 11  ] I2Cdevlib-Core   arduino, atmelavr     "Jeff Rowberg": The I2C Device Library (I2Cdevlib) is a collection of uniform and well-documented classes to provide simple and intuitive interfaces to I2C devices.
+    # [ 24  ] Adafruit-L3GD20  arduino, atmelavr     "Adafruit Industries": Driver for Adafruit's L3GD20 I2C Gyroscope Breakout
+    # [ 10  ] I2Cdevlib-AK8975 arduino, atmelavr     "Jeff Rowberg": AK8975 is 3-axis electronic compass IC with high sensitive Hall sensor technology
+    # [ 14  ] Adafruit-9DOF-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for the Adafruit 9DOF Breakout (L3GD20 / LSM303)
+    # ...
 
-4. Search for libraries from "Adafruit Industries" author.
+4. Search for `libraries by "web" and "http" keywords <http://platformio.ikravets.com/#!/lib/search?query=keyword%253A%2522web%2522%2520keyword%253A%2522http%2522&page=1>`_.
 
 .. code-block:: bash
 
-    $ platformio lib search "" --author "Adafruit Industries"
-    Found N libraries:
-    Adafruit-Arduino-ST7735 A library for the Adafruit 1.8" SPI display
-    Adafruit-Arduino-GFX  A core graphics library for all our displays, providing a common set of graphics primitives (points, lines, circles, etc.)
-    ...
+    $ platformio lib search --keyword="web" --keyword="http"
+    # Found N libraries:
+    #
+    # [ ID  ] Name             Compatibility         "Authors": Description
+    # -------------------------------------------------------------------------------------
+    # [  5  ] Webduino         arduino, atmelavr     "Ben Combee": An extensible web server library (for use with the Arduino WizNet Ethernet Shield)
+    # [ 17  ] Adafruit-CC3000  arduino, atmelavr     "Adafruit Industries": Library code for Adafruit's CC3000 Wi-Fi/WiFi breakouts
+    # ...
 
-5. Search for libraries that are compatible with Dallas temperature sensors
+5. Search for `libraries from "Adafruit Industries" author <http://platformio.ikravets.com/#!/lib/search?query=author%253A%2522Adafruit%20Industries%2522>`_
+
+.. code-block:: bash
+
+    $ platformio lib search --author="Adafruit Industries"
+    # Found N libraries:
+    #
+    # [ ID  ] Name             Compatibility         "Authors": Description
+    # -------------------------------------------------------------------------------------
+    # [ 14  ] Adafruit-9DOF-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for the Adafruit 9DOF Breakout (L3GD20 / LSM303)
+    # [ 13  ] Adafruit-GFX     arduino, atmelavr     "Adafruit Industries": A core graphics library for all our displays, providing a common set of graphics primitives (points, lines, circles, etc.)
+    # [ 23  ] Adafruit-L3GD20-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for the L3GD20 Gyroscope
+    # [ 26  ] Adafruit-LSM303DLHC-Unified arduino, atmelavr     "Adafruit Industries": Unified sensor driver for Adafruit's LSM303 Breakout (Accelerometer + Magnetometer)
+    # ...
+
+6. Search for `libraries that are compatible with Dallas temperature sensors <http://platformio.ikravets.com/#!/lib/search?query=DS*>`_
    like DS18B20, DS18S20 and etc.
 
 .. code-block:: bash
 
     $ platformio lib search "DS*"
-    Found N libraries:
-    Arduino-OneWire       Control devices (from Dallas Semiconductor) that use the One Wire protocol
-    ...
+    # Found N libraries:
+    #
+    # [ ID  ] Name             Compatibility         "Authors": Description
+    # -------------------------------------------------------------------------------------
+    # [  1  ] OneWire          arduino, atmelavr     "Paul Stoffregen": Control devices (from Dallas Semiconductor) that use the One Wire protocol (DS18S20, DS18B20, DS2408 and etc)
+    # ...
 
-6. Search for Arduino-based *X10* or *XBee* libraries. The search query that is
-   described below can be interpreted like ``arduino x10 OR arduino xbee``.
+7. Search for `Arduino-based *X10* or *XBee* libraries <http://platformio.ikravets.com/#!/lib/search?query=framework%253Aarduino%2520%252B(x10%2520xbee)&page=1>`_.
+   The search query that is described below can be interpreted like
+   ``arduino x10 OR arduino xbee``
 
 .. code-block:: bash
 
-    $ platformio lib search "+arduino +(x10 xbee)"
-    Found 2 libraries:
-    Arduino-X10           Sending X10 signals over AC power lines
-    Arduino-XBee          Arduino library for communicating with XBees in API mode
+    $ platformio lib search "+(x10 xbee)" --framework="arduino"
+    # Found 2 libraries:
+    #
+    # [ ID  ] Name             Compatibility         "Authors": Description
+    # -------------------------------------------------------------------------------------
+    # [ 36  ] X10              arduino, atmelavr     "Doug Clinton": Sending X10 signals over AC power lines (PL513, TW523 and etc)
+    # [  6  ] XBee             arduino, atmelavr     "Andrew Rapp": Arduino library for communicating with XBees in API mode

docs/userguide/lib/cmd_show.rst

@@ -10,7 +10,7 @@ Usage
 
 .. code-block:: bash
 
-    platformio lib show NAME
+    platformio lib show ID
 
 
 Description
@@ -24,11 +24,14 @@ Examples
 
 .. code-block:: bash
 
-    $ platformio lib show Arduino-XBee
-    Arduino-XBee
-    ------------
-    Author: Andrew Rapp <andrew.rapp@gmail.com>
-    Keywords: xbee, protocol, radio
-    Version: 0.5
-
-    Arduino library for communicating with XBees in API mode
+    # OneWire: http://platformio.ikravets.com/#!/lib/show/1/OneWire
+    $ platformio lib show 1
+    # OneWire
+    # -------
+    # Authors: Paul Stoffregen <paul@pjrc.com> http://www.pjrc.com/teensy/td_libs_OneWire.html (maintainer), Jim Studt, Jason Dangel <dangel.jason@gmail.com>, Derek Yerger, Tom Pollard <pollard@alum.mit.edu>, Robin James
+    # Keywords: onewire, 1-wire, bus, sensor, temperature, ibutton
+    # Frameworks: arduino
+    # Platforms: atmelavr
+    # Version: 2.2
+    #
+    # Control devices (from Dallas Semiconductor) that use the One Wire protocol (DS18S20, DS18B20, DS2408 and etc)

docs/userguide/lib/cmd_uninstall.rst

@@ -10,7 +10,7 @@ Usage
 
 .. code-block:: bash
 
-    platformio lib uninstall NAME
+    platformio lib uninstall ID
 
 
 Description
@@ -24,8 +24,6 @@ Examples
 
 .. code-block:: bash
 
-    $ platformio lib install Arduino-IRremote
-    Installing Arduino-IRremote library:
-    Downloading  [####################################]  100%
-    Unpacking  [####################################]  100%
-    The library 'Arduino-IRremote' has been successfully installed!
+    # XBee: http://platformio.ikravets.com/#!/lib/show/6/XBee
+    $ platformio lib uninstall 6
+    # The library #6 'XBee' has been successfully uninstalled!

docs/userguide/lib/cmd_update.rst

@@ -25,9 +25,19 @@ Examples
 .. code-block:: bash
 
     $ platformio lib update
-    Updating Arduino-IRremote library:
-    Versions: Current=24ba950f5c, Latest=24ba950f5c     [Up-to-date]
-    Updating Arduino-Webduino library:
-    Versions: Current=3631af8e02, Latest=3631af8e02     [Up-to-date]
-    Updating Arduino-XBee library:
-    Versions: Current=0.5, Latest=0.5                   [Up-to-date]
+    # Updating  [ 23 ] Adafruit-L3GD20-Unified library:
+    # Versions: Current=63de2eb9ea, Latest=63de2eb9ea 	 [Up-to-date]
+    # Updating  [ 12 ] Adafruit-ST7735 library:
+    # Versions: Current=e880eb1687, Latest=e880eb1687 	 [Up-to-date]
+    # Updating  [ 31 ] Adafruit-Unified-Sensor library:
+    # Versions: Current=88ae805bce, Latest=88ae805bce 	 [Up-to-date]
+    # Updating  [ 26 ] Adafruit-LSM303DLHC-Unified library:
+    # Versions: Current=59767208a8, Latest=59767208a8 	 [Up-to-date]
+    # Updating  [ 13 ] Adafruit-GFX library:
+    # Versions: Current=a9e5bc4707, Latest=a9e5bc4707 	 [Up-to-date]
+    # Updating  [ 1 ] OneWire library:
+    # Versions: Current=2.2, Latest=2.2 	 [Up-to-date]
+    # Updating  [ 4 ] IRremote library:
+    # Versions: Current=f2dafe5030, Latest=f2dafe5030 	 [Up-to-date]
+    # Updating  [ 14 ] Adafruit-9DOF-Unified library:
+    # Versions: Current=b2f07242ac, Latest=b2f07242ac 	 [Up-to-date]

platformio/__init__.py

@@ -1,11 +1,12 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-VERSION = (0, 7, 0)
+VERSION = (0, 9, 0)
 __version__ = ".".join([str(s) for s in VERSION])
 
 __title__ = "platformio"
-__description__ = ("A cross-platform code builder and library manager")
+__description__ = ("A cross-platform code builder and "
+                   "the missing library manager")
 __url__ = "http://platformio.ikravets.com"
 
 __author__ = "Ivan Kravets"
@@ -15,4 +16,3 @@ __license__ = "MIT License"
 __copyright__ = "Copyright (C) 2014 Ivan Kravets"
 
 __apiurl__ = "http://api.platformio.ikravets.com"
-__pkgmanifesturl__ = "http://dl.platformio.ikravets.com/packages/manifest.json"

platformio/__main__.py

@@ -1,21 +1,19 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-from os import listdir, makedirs
-from os.path import getmtime, isdir, isfile, join
+from os import listdir
+from os.path import join
 from sys import exit as sys_exit
-from time import time
 from traceback import format_exc
 
-from click import command, MultiCommand, secho, version_option
+import click
 
-from platformio import __version__
-from platformio.commands.upgrade import get_latest_version
+from platformio import __version__, maintenance
 from platformio.exception import PlatformioException, UnknownCLICommand
-from platformio.util import get_home_dir, get_source_dir
+from platformio.util import get_source_dir
 
 
-class PlatformioCLI(MultiCommand):  # pylint: disable=R0904
+class PlatformioCLI(click.MultiCommand):  # pylint: disable=R0904
 
     def list_commands(self, ctx):
         cmds = []
@@ -28,6 +26,7 @@ class PlatformioCLI(MultiCommand):  # pylint: disable=R0904
         return cmds
 
     def get_command(self, ctx, name):
+        mod = None
         try:
             mod = __import__("platformio.commands." + name,
                              None, None, ["cli"])
@@ -36,35 +35,27 @@ class PlatformioCLI(MultiCommand):  # pylint: disable=R0904
         return mod.cli
 
 
-@command(cls=PlatformioCLI)
-@version_option(__version__, prog_name="PlatformIO")
-def cli():
-    pass
+@click.command(cls=PlatformioCLI)
+@click.version_option(__version__, prog_name="PlatformIO")
+@click.pass_context
+def cli(ctx):
+    maintenance.on_platformio_start(ctx)
 
 
-def autocheck_latest_version():
-    check_interval = 3600 * 24 * 7  # 1 week
-    checkfile = join(get_home_dir(), ".pioupgrade")
-    if isfile(checkfile) and getmtime(checkfile) > (time() - check_interval):
-        return False
-    if not isdir(get_home_dir()):
-        makedirs(get_home_dir())
-    with open(checkfile, "w") as f:
-        f.write(str(time()))
-    return get_latest_version() != __version__
+@cli.resultcallback()
+@click.pass_context
+def process_result(ctx, result):
+    maintenance.on_platformio_end(ctx, result)
 
 
 def main():
     try:
-        if autocheck_latest_version():
-            secho("\nThere is a new version of PlatformIO available.\n"
-                  "Please upgrade it via `platformio upgrade` command.\n",
-                  fg="yellow")
-
-        cli()
+        cli(None)
     except Exception as e:  # pylint: disable=W0703
+        maintenance.on_platformio_exception(e)
         if isinstance(e, PlatformioException):
-            sys_exit("Error: " + str(e))
+            click.echo("Error: " + str(e))
+            sys_exit(1)
         else:
             print format_exc()
 

platformio/app.py

@@ -0,0 +1,111 @@
+# Copyright (C) Ivan Kravets <me@ikravets.com>
+# See LICENSE for details.
+
+import json
+from os.path import isfile, join
+
+from platformio import __version__
+from platformio.exception import InvalidSettingName, InvalidSettingValue
+from platformio.util import get_home_dir
+
+DEFAULT_SETTINGS = {
+    "check_platformio_interval": {
+        "description": "Check for the new PlatformIO interval (days)",
+        "value": 3
+    },
+    "check_platforms_interval": {
+        "description": "Check for the platform updates interval (days)",
+        "value": 7
+    },
+    "check_libraries_interval": {
+        "description": "Check for the library updates interval (days)",
+        "value": 7
+    },
+    "auto_update_platforms": {
+        "description": "Automatically update platforms (Yes/No)",
+        "value": True
+    },
+    "auto_update_libraries": {
+        "description": "Automatically update libraries (Yes/No)",
+        "value": True
+    },
+    "enable_telemetry": {
+        "description": ("Shares commands, platforms and libraries usage"
+                        " to help us make PlatformIO better (Yes/No)"),
+        "value": True
+    }
+}
+
+
+class State(object):
+
+    def __init__(self, path=None):
+        self.path = path
+        if not self.path:
+            self.path = join(get_home_dir(), "appstate.json")
+        self._state = {}
+
+    def __enter__(self):
+        try:
+            if isfile(self.path):
+                with open(self.path, "r") as fp:
+                    self._state = json.load(fp)
+        except ValueError:
+            self._state = {}
+        return self._state
+
+    def __exit__(self, type_, value, traceback):
+        with open(self.path, "w") as fp:
+            if "dev" in __version__:
+                json.dump(self._state, fp, indent=4)
+            else:
+                json.dump(self._state, fp)
+
+
+def get_state_item(name, default=None):
+    with State() as data:
+        return data.get(name, default)
+
+
+def set_state_item(name, value):
+    with State() as data:
+        data[name] = value
+
+
+def get_setting(name):
+    if name not in DEFAULT_SETTINGS:
+        raise InvalidSettingName(name)
+
+    with State() as data:
+        if "settings" in data and name in data['settings']:
+            return data['settings'][name]
+
+    return DEFAULT_SETTINGS[name]['value']
+
+
+def set_setting(name, value):
+    if name not in DEFAULT_SETTINGS:
+        raise InvalidSettingName(name)
+
+    defdata = DEFAULT_SETTINGS[name]
+    try:
+        if "validator" in defdata:
+            value = defdata['validator']()
+        elif (isinstance(defdata['value'], bool)
+              and not isinstance(value, bool)):
+            value = str(value).lower() in ("yes", "y", "1")
+        elif isinstance(defdata['value'], int):
+            value = int(value)
+    except Exception:
+        raise InvalidSettingValue(value, name)
+
+    with State() as data:
+        if "settings" not in data:
+            data['settings'] = {}
+        data['settings'][name] = value
+
+
+def reset_settings():
+    with State() as data:
+        if "settings" in data:
+            del data['settings']

platformio/builder/main.py

@@ -11,10 +11,10 @@ except ImportError:
             break
     from platformio.util import get_home_dir
 
-from os.path import isdir, join
+from os.path import join
 
-from SCons.Script import (DefaultEnvironment, Exit, SConscript,
-                          SConscriptChdir, Variables)
+from SCons.Script import (DefaultEnvironment, SConscript, SConscriptChdir,
+                          Variables)
 
 from platformio.util import (get_lib_dir, get_pioenvs_dir, get_project_dir,
                              get_source_dir)
@@ -24,8 +24,16 @@ from platformio.util import (get_lib_dir, get_pioenvs_dir, get_project_dir,
 # allow common variables from INI file
 commonvars = Variables(None)
 commonvars.AddVariables(
+    ("BUILD_SCRIPT",),
     ("PIOENV",),
     ("PLATFORM",),
+
+    # package aliases
+    ("PIOPACKAGE_TOOLCHAIN",),
+    ("PIOPACKAGE_UPLOADER",),
+    ("PIOPACKAGE_FRAMEWORK",),
+
+    # options
     ("FRAMEWORK",),
     ("BUILD_FLAGS",),
     ("SRCBUILD_FLAGS",),
@@ -46,14 +54,13 @@ DefaultEnvironment(
     toolpath=[join("$PIOBUILDER_DIR", "tools")],
     variables=commonvars,
 
-    PIOBUILDER_DIR=join(get_source_dir(), "builder"),
+    PIOHOME_DIR=get_home_dir(),
     PROJECT_DIR=get_project_dir(),
     PIOENVS_DIR=get_pioenvs_dir(),
 
-    PLATFORMIOHOME_DIR=get_home_dir(),
-    PLATFORM_DIR=join("$PLATFORMIOHOME_DIR", "$PLATFORM"),
-    PLATFORMFW_DIR=join("$PLATFORM_DIR", "frameworks", "$FRAMEWORK"),
-    PLATFORMTOOLS_DIR=join("$PLATFORM_DIR", "tools"),
+    PIOBUILDER_DIR=join(get_source_dir(), "builder"),
+    PIOPACKAGES_DIR=join("$PIOHOME_DIR", "packages"),
+    PLATFORMFW_DIR=join("$PIOPACKAGES_DIR", "$PIOPACKAGE_FRAMEWORK"),
 
     BUILD_DIR=join("$PIOENVS_DIR", "$PIOENV"),
     LIBSOURCE_DIRS=[
@@ -64,14 +71,10 @@ DefaultEnvironment(
 )
 
 env = DefaultEnvironment()
-
-if not isdir(env['PLATFORMIOHOME_DIR']):
-    Exit("You haven't installed any platforms yet. Please use "
-         "`platformio install` command")
-elif not isdir(env.subst("$PLATFORM_DIR")):
-    Exit("An '%s' platform hasn't been installed yet. Please use "
-         "`platformio install %s` command" % (env['PLATFORM'],
-                                              env['PLATFORM']))
+env.PrependENVPath(
+    "PATH",
+    env.subst(join("$PIOPACKAGES_DIR", "$PIOPACKAGE_TOOLCHAIN", "bin"))
+)
 
 SConscriptChdir(0)
-SConscript(env.subst(join("$PIOBUILDER_DIR", "scripts", "${PLATFORM}.py")))
+SConscript(env.subst("$BUILD_SCRIPT"))

platformio/builder/scripts/atmelavr.py

@@ -11,7 +11,7 @@ from time import sleep
 from SCons.Script import (AlwaysBuild, Builder, COMMAND_LINE_TARGETS, Default,
                           DefaultEnvironment, Exit)
 
-from platformio.util import reset_serialport
+from platformio.util import get_serialports, reset_serialport
 
 env = DefaultEnvironment()
 
@@ -53,13 +53,13 @@ env.Replace(
         "-mmcu=$BOARD_MCU"
     ],
 
-    UPLOADER=join("$PLATFORMTOOLS_DIR", "avrdude", "avrdude"),
+    UPLOADER=join("$PIOPACKAGES_DIR", "tool-avrdude", "avrdude"),
     UPLOADERFLAGS=[
         "-V",  # do not verify
         "-q",  # suppress progress output
         "-D",  # disable auto erase for flash memory
         "-p", "$BOARD_MCU",
-        "-C", join("$PLATFORMTOOLS_DIR", "avrdude", "avrdude.conf"),
+        "-C", join("$PIOPACKAGES_DIR", "tool-avrdude", "avrdude.conf"),
         "-c", "$UPLOAD_PROTOCOL",
         "-b", "$UPLOAD_SPEED",
         "-P", "$UPLOAD_PORT"
@@ -164,9 +164,19 @@ AlwaysBuild(uploadeep)
 
 is_uptarget = (set(["upload", "uploadlazy", "uploadeep"]) &
                set(COMMAND_LINE_TARGETS))
-if is_uptarget and not env.subst("$UPLOAD_PORT"):
-    Exit("Please specify environment 'upload_port' or use global "
-         "--upload-port option.")
+
+if is_uptarget:
+    # try autodetect upload port
+    if "UPLOAD_PORT" not in env:
+        for item in get_serialports():
+            if "VID:PID" in item['hwid']:
+                print "Auto-detected UPLOAD_PORT: %s" % item['port']
+                env['UPLOAD_PORT'] = item['port']
+                break
+
+    if "UPLOAD_PORT" not in env:
+        Exit("Please specify environment 'upload_port' or use global "
+             "--upload-port option.")
 
 #
 # Setup default targets

platformio/builder/scripts/frameworks/arduino.py

@@ -31,18 +31,19 @@ if "build.usb_product" in BOARD_OPTIONS:
 
 # include board variant
 env.VariantDir(
-    join("$BUILD_DIR", "variant"),
+    join("$BUILD_DIR", "FrameworkArduinoVariant"),
     join("$PLATFORMFW_DIR", "variants", BOARD_OPTIONS['build.variant'])
 )
 
 env.Append(
     CPPDEFINES=[
+        "ARDUINO_ARCH_AVR",  # @TODO Should be dynamic
         "ARDUINO=%d" % ARDUINO_VERSION,
         "ARDUINO_%s" % BOARD_OPTIONS['build.board']
     ] + ARDUINO_USBDEFINES,
     CPPPATH=[
-        join("$BUILD_DIR", "core"),
-        join("$BUILD_DIR", "variant")
+        join("$BUILD_DIR", "FrameworkArduino"),
+        join("$BUILD_DIR", "FrameworkArduinoVariant")
     ]
 )
 
@@ -63,7 +64,7 @@ libs = []
 #
 
 libs.append(env.BuildLibrary(
-    join("$BUILD_DIR", "core"),
+    join("$BUILD_DIR", "FrameworkArduino"),
     join("$PLATFORMFW_DIR", "cores", BOARD_OPTIONS['build.core'])
 ))
 

platformio/builder/scripts/frameworks/energia.py

@@ -22,7 +22,7 @@ ENERGIA_VERSION = int(
 
 # include board variant
 env.VariantDir(
-    join("$BUILD_DIR", "variant"),
+    join("$BUILD_DIR", "FrameworkEnergiaVariant"),
     join("$PLATFORMFW_DIR", "variants", BOARD_OPTIONS['build.variant'])
 )
 
@@ -32,8 +32,8 @@ env.Append(
         "ENERGIA=%d" % ENERGIA_VERSION
     ],
     CPPPATH=[
-        join("$BUILD_DIR", "core"),
-        join("$BUILD_DIR", "variant")
+        join("$BUILD_DIR", "FrameworkEnergia"),
+        join("$BUILD_DIR", "FrameworkEnergiaVariant")
     ]
 )
 
@@ -60,7 +60,7 @@ libs = []
 #
 
 libs.append(env.BuildLibrary(
-    join("$BUILD_DIR", "core"),
+    join("$BUILD_DIR", "FrameworkEnergia"),
     join("$PLATFORMFW_DIR", "cores", BOARD_OPTIONS['build.core'])
 ))
 

platformio/builder/scripts/timsp430.py

@@ -51,7 +51,7 @@ env.Replace(
         "-Wl,-gc-sections,-u,main"
     ],
 
-    UPLOADER=join("$PLATFORMTOOLS_DIR", "mspdebug", "mspdebug"),
+    UPLOADER=join("$PIOPACKAGES_DIR", "tool-mspdebug", "mspdebug"),
     UPLOADERFLAGS=[
         "$UPLOAD_PROTOCOL" if system() != "Windows" else "tilib",
         "--force-reset"

platformio/builder/scripts/titiva.py

@@ -71,7 +71,7 @@ env.Replace(
         "-fsingle-precision-constant"
     ],
 
-    UPLOADER=join("$PLATFORMTOOLS_DIR", "lm4flash", "lm4flash"),
+    UPLOADER=join("$PIOPACKAGES_DIR", "tool-lm4flash", "lm4flash"),
     UPLOADCMD="$UPLOADER $SOURCES"
 )
 

platformio/builder/tools/platformio.py

@@ -14,11 +14,6 @@ def ProcessGeneral(env):
     if "BUILD_FLAGS" in env:
         env.MergeFlags(env['BUILD_FLAGS'])
 
-    env.PrependENVPath(
-        "PATH",
-        join(env.subst("$PLATFORMTOOLS_DIR"), "toolchain", "bin")
-    )
-
     if "FRAMEWORK" in env:
         if env['FRAMEWORK'] in ("arduino", "energia"):
             env.ConvertInotoCpp()
@@ -72,9 +67,9 @@ def BuildDependentLibraries(env, src_dir):
     deplibs = env.GetDependentLibraries(src_dir)
     env.Append(CPPPATH=[join("$BUILD_DIR", l) for (l, _) in deplibs])
 
-    for (libname, lsd_dir) in deplibs:
+    for (libname, inc_dir) in deplibs:
         lib = env.BuildLibrary(
-            join("$BUILD_DIR", libname), join(lsd_dir, libname))
+            join("$BUILD_DIR", libname), inc_dir)
         env.Clean(libname, lib)
         libs.append(lib)
     return libs
@@ -98,19 +93,24 @@ def GetDependentLibraries(env, src_dir):
 
 def ParseIncludesRecurive(env, regexp, source_file, includes):
     matches = regexp.findall(source_file.get_text_contents())
-    for inc_name in matches:
-        if inc_name in includes:
+    for inc_fname in matches:
+        if inc_fname in includes:
             continue
         for lsd_dir in env['LIBSOURCE_DIRS']:
             lsd_dir = env.subst(lsd_dir)
             if not isdir(lsd_dir):
                 continue
             for libname in listdir(lsd_dir):
-                inc_path = join(lsd_dir, libname, inc_name)
-                if not isfile(inc_path):
+                inc_dir = join(lsd_dir, libname)
+                inc_file = join(inc_dir, inc_fname)
+                if not isfile(inc_file):
+                    # if source code is in "src" dir
+                    inc_dir = join(lsd_dir, libname, "src")
+                    inc_file = join(inc_dir, inc_fname)
+                if not isfile(inc_file):
                     continue
-                includes[inc_name] = (len(includes) + 1, libname, lsd_dir)
-                env.ParseIncludesRecurive(regexp, env.File(inc_path), includes)
+                includes[inc_fname] = (len(includes) + 1, libname, inc_dir)
+                env.ParseIncludesRecurive(regexp, env.File(inc_file), includes)
 
 
 def VariantDirRecursive(env, variant_dir, src_dir, duplicate=True):
@@ -173,22 +173,29 @@ def ConvertInotoCpp(env):
             continue
         ino_contents = item.get_text_contents()
 
-        # fetch prototypes
-        regexp = re.compile(
+        re_includes = re.compile(r"^(#include\s+(?:\<|\")[^\r\n]+)",
+                                 re.M | re.I)
+        includes = re_includes.findall(ino_contents)
+        prototypes = re.findall(
             r"""^(
             (?:\s*[a-z_\d]+){1,2}       # return type
             \s+[a-z_\d]+\s*             # name of prototype
             \([a-z_,\.\*\&\[\]\s\d]*\)  # args
             )\s*\{                      # must end with {
             """,
+            ino_contents,
             re.X | re.M | re.I
         )
-        prototypes = regexp.findall(ino_contents)
-        # print prototypes
+        # print includes, prototypes
+
+        # disable previous includes
+        ino_contents = re_includes.sub(r"//\1", ino_contents)
 
         # create new temporary C++ valid file
         with open(cppfile, "w") as f:
             f.write("#include <Arduino.h>\n")
+            if includes:
+                f.write("%s\n" % "\n".join(includes))
             if prototypes:
                 f.write("%s;\n" % ";\n".join(prototypes))
             f.write("#line 1 \"%s\"\n" % basename(item.path))

platformio/commands/init.py

@@ -1,30 +1,58 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-from os import makedirs
+from os import getcwd, makedirs
 from os.path import isdir, isfile, join
 from shutil import copyfile
 
-from click import command, secho
+import click
 
 from platformio.exception import ProjectInitialized
 from platformio.util import get_source_dir
 
 
-@command("init", short_help="Initialize new PlatformIO based project")
-def cli():
+@click.command("init", short_help="Initialize new PlatformIO based project")
+@click.option("--project-dir", "-d", default=getcwd(),
+              type=click.Path(exists=True, file_okay=False, dir_okay=True,
+                              writable=True, resolve_path=True))
+def cli(project_dir):
 
-    if isfile("platformio.ini") and isdir("src"):
+    project_file = join(project_dir, "platformio.ini")
+    src_dir = join(project_dir, "src")
+    lib_dir = join(project_dir, "lib")
+    if all([isfile(project_file), isdir(src_dir), isdir(lib_dir)]):
         raise ProjectInitialized()
-    for d in ("lib", "src"):
-        if not isdir(d):
-            makedirs(d)
-    if not isfile("platformio.ini"):
-        copyfile(join(get_source_dir(), "projectconftpl.ini"),
-                 "platformio.ini")
-    secho("Project has been initialized!\n"
-          "Please put your source code to `src` directory, "
-          "external libraries to `lib` and "
-          "setup environments in `platformio.ini` file.\n"
-          "Then process project with `platformio run` command.",
-          fg="green")
+
+    if project_dir == getcwd():
+        click.secho("The current working directory", fg="yellow", nl=False)
+        click.secho(" %s " % project_dir, fg="blue", nl=False)
+        click.secho(
+            "will be used for the new project.\n"
+            "You can specify another project directory via\n"
+            "`platformio init -d %PATH_TO_PROJECT_DIR%` command.\n",
+            fg="yellow"
+        )
+
+    click.echo("The next files/directories will be created in %s" %
+               click.style(project_dir, fg="blue"))
+    click.echo("%s - Project Configuration File" %
+               click.style("platformio.ini", fg="cyan"))
+    click.echo("%s - a source directory. Put your source code here" %
+               click.style("src", fg="cyan"))
+    click.echo("%s - a directory for the project specific libraries" %
+               click.style("lib", fg="cyan"))
+
+    if click.confirm("Do you want to continue?"):
+        for d in (src_dir, lib_dir):
+            if not isdir(d):
+                makedirs(d)
+        if not isfile(project_file):
+            copyfile(join(get_source_dir(), "projectconftpl.ini"),
+                     project_file)
+        click.secho(
+            "Project has been successfully initialized!\n"
+            "Now you can process it with `platformio run` command.",
+            fg="green"
+        )
+    else:
+        click.secho("Aborted by user", fg="red")

platformio/commands/lib.py

@@ -3,10 +3,38 @@
 
 import click
 
-from platformio.exception import LibAlreadyInstalledError
+from platformio.exception import (LibAlreadyInstalledError,
+                                  LibInstallDependencyError)
 from platformio.libmanager import LibraryManager
 from platformio.util import get_api_result, get_lib_dir
 
+LIBLIST_TPL = ("[{id:^14}] {name:<25} {compatibility:<30} "
+               "\"{authornames}\": {description}")
+
+
+def echo_liblist_header():
+    click.echo(LIBLIST_TPL.format(
+        id=click.style("ID", fg="green"),
+        name=click.style("Name", fg="cyan"),
+        compatibility=click.style("Compatibility", fg="yellow"),
+        authornames="Authors",
+        description="Description"
+    ))
+    click.echo("-" * 85)
+
+
+def echo_liblist_item(item):
+    click.echo(LIBLIST_TPL.format(
+        id=click.style(str(item['id']), fg="green"),
+        name=click.style(item['name'], fg="cyan"),
+        compatibility=click.style(
+            ", ".join(item['frameworks'] + item['platforms']),
+            fg="yellow"
+        ),
+        authornames=", ".join(item['authornames']),
+        description=item['description']
+    ))
+
 
 @click.group(short_help="Library Manager")
 def cli():
@@ -16,22 +44,27 @@ def cli():
 @cli.command("search", short_help="Search for library")
 @click.option("-a", "--author", multiple=True)
 @click.option("-k", "--keyword", multiple=True)
-@click.argument("query")
-def lib_search(query, author, keyword):
-    for key, values in dict(author=author, keyword=keyword).iteritems():
+@click.option("-f", "--framework", multiple=True)
+@click.option("-p", "--platform", multiple=True)
+@click.argument("query", required=False)
+def lib_search(query, **filters):
+    if not query:
+        query = ""
+
+    for key, values in filters.iteritems():
         for value in values:
             query += ' %s:"%s"' % (key, value)
 
     result = get_api_result("/lib/search", dict(query=query))
-    click.secho("Found %d libraries:" % result['total'],
+    click.secho("Found %d libraries:\n" % result['total'],
                 fg="green" if result['total'] else "yellow")
 
+    if result['total']:
+        echo_liblist_header()
+
     while True:
         for item in result['items']:
-            click.echo("{name:<30} {description}".format(
-                name=click.style(item['name'], fg="cyan"),
-                description=item['description']
-            ))
+            echo_liblist_item(item)
 
         if int(result['page'])*int(result['perpage']) >= int(result['total']):
             break
@@ -46,91 +79,108 @@ def lib_search(query, author, keyword):
 
 
 @cli.command("install", short_help="Install library")
-@click.argument("names", nargs=-1)
+@click.argument("libid", type=click.INT, nargs=-1, metavar="[LIBRARY_ID]")
 @click.option("-v", "--version")
-def lib_install_cli(names, version):
-    lib_install(names, version)
-
-
-def lib_install(names, version=None):
+@click.pass_context
+def lib_install(ctx, libid, version):
     lm = LibraryManager(get_lib_dir())
-    for name in names:
-        click.echo("Installing %s library:" % click.style(name, fg="cyan"))
+    for id_ in libid:
+        click.echo(
+            "Installing library [ %s ]:" % click.style(str(id_), fg="green"))
         try:
-            if lm.install(name, version):
-                click.secho(
-                    "The library '%s' has been successfully installed!" %
-                    name, fg="green")
-                info = lm.get_info(name)
-                if "dependencies" in info:
-                    click.secho("Installing dependencies:", fg="yellow")
-                    lib_install(info['dependencies'])
+            if not lm.install(id_, version):
+                continue
+
+            info = lm.get_info(id_)
+            click.secho(
+                "The library #%s '%s' has been successfully installed!"
+                % (str(id_), info['name']), fg="green")
+
+            if "dependencies" in info:
+                click.secho("Installing dependencies:", fg="yellow")
+                _dependencies = info['dependencies']
+                if not isinstance(_dependencies, list):
+                    _dependencies = [_dependencies]
+                for item in _dependencies:
+                    try:
+                        lib_install_dependency(ctx, item)
+                    except AssertionError:
+                        raise LibInstallDependencyError(str(item))
 
         except LibAlreadyInstalledError:
             click.secho("Already installed", fg="yellow")
 
 
+def lib_install_dependency(ctx, data):
+    assert isinstance(data, dict)
+    query = []
+    for key in data.keys():
+        if key in ("authors", "frameworks", "platforms", "keywords"):
+            values = data[key]
+            if not isinstance(values, list):
+                values = [v.strip() for v in values.split(",") if v]
+            for value in values:
+                query.append('%s:"%s"' % (key[:-1], value))
+        elif isinstance(data[key], basestring):
+            query.append('+"%s"' % data[key])
+
+    result = get_api_result("/lib/search", dict(query=" ".join(query)))
+    assert result['total'] == 1
+    ctx.invoke(lib_install, libid=[result['items'][0]['id']])
+
+
 @cli.command("uninstall", short_help="Uninstall libraries")
-@click.argument("names", nargs=-1)
-def lib_uninstall_cli(names):
-    lib_uninstall(names)
-
-
-def lib_uninstall_dependency(dependency):
+@click.argument("libid", type=click.INT, nargs=-1)
+def lib_uninstall(libid):
     lm = LibraryManager(get_lib_dir())
-
-    for name in lm.get_installed():
-        info = lm.get_info(name)
-        if dependency in info.get("dependencies", []):
-            return
-
-    lib_uninstall([dependency])
-
-
-def lib_uninstall(names):
-    lm = LibraryManager(get_lib_dir())
-    for name in names:
-        info = lm.get_info(name)
-        if lm.uninstall(name):
-            # find dependencies
-            if "dependencies" in info:
-                for d in info['dependencies']:
-                    lib_uninstall_dependency(d)
-
-            click.secho("The library '%s' has been successfully "
-                        "uninstalled!" % name, fg="green")
+    for id_ in libid:
+        info = lm.get_info(id_)
+        if lm.uninstall(id_):
+            click.secho("The library #%s '%s' has been successfully "
+                        "uninstalled!" % (str(id_), info['name']), fg="green")
 
 
 @cli.command("list", short_help="List installed libraries")
 def lib_list():
     lm = LibraryManager(get_lib_dir())
-    for name in lm.get_installed():
-        info = lm.get_info(name)
-        click.echo("{name:<30} {description}".format(
-            name=click.style(info['name'], fg="cyan"),
-            description=info['description']
-        ))
+    items = lm.get_installed().values()
+    if not items:
+        return
+
+    echo_liblist_header()
+    for item in items:
+        item['authornames'] = [i['name'] for i in item['authors']]
+        echo_liblist_item(item)
 
 
 @cli.command("show", short_help="Show details about installed libraries")
-@click.argument("name")
-def lib_show(name):
+@click.argument("libid", type=click.INT)
+def lib_show(libid):
     lm = LibraryManager(get_lib_dir())
-    info = lm.get_info(name)
+    info = lm.get_info(libid)
     click.secho(info['name'], fg="cyan")
     click.echo("-" * len(info['name']))
 
-    if "author" in info:
+    _authors = []
+    for author in info['authors']:
         _data = []
-        for k in ("name", "email"):
-            if k in info['author'] and info['author'][k] is not None:
-                _value = info['author'][k]
-                if k == "email":
-                    _value = "<%s>" % _value
-                _data.append(_value)
-        click.echo("Author: %s" % " ".join(_data))
+        for key in ("name", "email", "url", "maintainer"):
+            if not author[key]:
+                continue
+            if key == "email":
+                _data.append("<%s>" % author[key])
+            elif key == "maintainer":
+                _data.append("(maintainer)")
+            else:
+                _data.append(author[key])
+        _authors.append(" ".join(_data))
+    click.echo("Authors: %s" % ", ".join(_authors))
 
-    click.echo("Keywords: %s" % info['keywords'])
+    click.echo("Keywords: %s" % ", ".join(info['keywords']))
+    if "frameworks" in info:
+        click.echo("Frameworks: %s" % ", ".join(info['frameworks']))
+    if "platforms" in info:
+        click.echo("Platforms: %s" % ", ".join(info['platforms']))
     click.echo("Version: %s" % info['version'])
     click.echo()
     click.echo(info['description'])
@@ -138,18 +188,20 @@ def lib_show(name):
 
 
 @cli.command("update", short_help="Update installed libraries")
-def lib_update():
+@click.pass_context
+def lib_update(ctx):
     lm = LibraryManager(get_lib_dir())
-    lib_names = lm.get_installed()
-    versions = get_api_result("/lib/version/" + ",".join(lib_names))
+    for id_, latest_version in (lm.get_latest_versions() or {}).items():
+        info = lm.get_info(int(id_))
 
-    for name in lib_names:
-        info = lm.get_info(name)
-
-        click.echo("Updating %s library:" % click.style(name, fg="yellow"))
+        click.echo("Updating  [ %s ] %s library:" % (
+            click.style(id_, fg="yellow"),
+            click.style(info['name'], fg="cyan")))
 
         current_version = info['version']
-        latest_version = versions[name]
+        if latest_version is None:
+            click.secho("Unknown library", fg="red")
+            continue
 
         click.echo("Versions: Current=%s, Latest=%s \t " % (
             current_version, latest_version), nl=False)
@@ -160,8 +212,8 @@ def lib_update():
         else:
             click.echo("[%s]" % (click.style("Out-of-date", fg="red")))
 
-        lib_uninstall([name])
-        lib_install([name])
+        ctx.invoke(lib_uninstall, libid=[int(id_)])
+        ctx.invoke(lib_install, libid=[int(id_)])
 
 
 @cli.command("register", short_help="Register new library")

platformio/commands/list.py

@@ -3,14 +3,19 @@
 
 from click import command, echo, style
 
-from platformio.pkgmanager import PackageManager
+from platformio.platforms.base import PlatformFactory
 
 
 @command("list", short_help="List installed platforms")
 def cli():
 
-    for name, pkgs in PackageManager.get_installed().items():
+    installed_platforms = PlatformFactory.get_platforms(
+        installed=True).keys()
+    installed_platforms.sort()
+
+    for platform in installed_platforms:
+        p = PlatformFactory().newPlatform(platform)
         echo("{name:<20} with packages: {pkgs}".format(
-            name=style(name, fg="cyan"),
-            pkgs=", ".join(pkgs.keys())
+            name=style(p.get_name(), fg="cyan"),
+            pkgs=", ".join(p.get_installed_packages())
         ))

platformio/commands/run.py

@@ -3,6 +3,7 @@
 
 from click import command, echo, option, secho, style
 
+from platformio import telemetry
 from platformio.exception import (InvalidEnvName, ProjectEnvsNotAvaialable,
                                   UndefinedEnvPlatform, UnknownEnvNames)
 from platformio.platforms.base import PlatformFactory
@@ -25,7 +26,10 @@ def cli(environment, target, upload_port):
         raise UnknownEnvNames(", ".join(unknown))
 
     for section in config.sections():
-        if section[:4] != "env:":
+        # skip main configuration section
+        if section == "platformio":
+            continue
+        elif section[:4] != "env:":
             raise InvalidEnvName(section)
 
         envname = section[4:]
@@ -53,6 +57,8 @@ def cli(environment, target, upload_port):
         if not config.has_option(section, "platform"):
             raise UndefinedEnvPlatform(envname)
 
+        telemetry.on_run_environment(config.items(section), envtargets)
+
         p = PlatformFactory().newPlatform(config.get(section, "platform"))
         result = p.run(variables, envtargets)
         secho(result['out'], fg="green")

platformio/commands/search.py

@@ -4,19 +4,21 @@
 from click import argument, command, echo, style
 
 from platformio.platforms.base import PlatformFactory
-from platformio.util import get_platforms
 
 
 @command("search", short_help="Search for development platforms")
-@argument("query")
+@argument("query", required=False)
 def cli(query):
-    for platform in get_platforms():
+    for platform in PlatformFactory.get_platforms().keys():
         p = PlatformFactory().newPlatform(platform)
         name = p.get_name()
         shinfo = p.get_short_info()
 
+        if query == "all":
+            query = ""
+
         search_data = "%s %s" % (name, shinfo)
-        if query != "all" and query.lower() not in search_data.lower():
+        if query and query.lower() not in search_data.lower():
             continue
 
         echo("{name:<20} - {info}".format(name=style(name, fg="cyan"),

platformio/commands/settings.py

@@ -0,0 +1,68 @@
+# Copyright (C) Ivan Kravets <me@ikravets.com>
+# See LICENSE for details.
+
+import click
+
+from platformio import app
+
+
+@click.group(short_help="Manage PlatformIO settings")
+def cli():
+    pass
+
+
+@cli.command("get", short_help="Get existing setting/-s")
+@click.argument("name", required=False)
+def settings_get(name):
+
+    list_tpl = "{name:<40} {value:<35} {description}"
+
+    click.echo(list_tpl.format(
+        name=click.style("Name", fg="cyan"),
+        value=(click.style("Value", fg="green") +
+               click.style(" [Default]", fg="yellow")),
+        description="Description"
+    ))
+    click.echo("-" * 90)
+
+    for _name, _data in sorted(app.DEFAULT_SETTINGS.items()):
+        if name and name != _name:
+            continue
+        _value = app.get_setting(_name)
+
+        _value_str = str(_value)
+        if isinstance(_value, bool):
+            _value_str = "Yes" if _value else "No"
+        _value_str = click.style(_value_str, fg="green")
+
+        if _value != _data['value']:
+            _defvalue_str = str(_data['value'])
+            if isinstance(_data['value'], bool):
+                _defvalue_str = "Yes" if _data['value'] else "No"
+            _value_str += click.style(" [%s]" % _defvalue_str, fg="yellow")
+        else:
+            _value_str += click.style(" ", fg="yellow")
+
+        click.echo(list_tpl.format(
+            name=click.style(_name, fg="cyan"),
+            value=_value_str,
+            description=_data['description']
+        ))
+
+
+@cli.command("set", short_help="Set new value for the setting")
+@click.argument("name")
+@click.argument("value")
+@click.pass_context
+def settings_set(ctx, name, value):
+    app.set_setting(name, value)
+    click.secho("The new value for the setting has been set!", fg="green")
+    ctx.invoke(settings_get, name=name)
+
+
+@cli.command("reset", short_help="Reset settings to default")
+@click.pass_context
+def settings_reset(ctx):
+    app.reset_settings()
+    click.secho("The settings have been reseted!", fg="green")
+    ctx.invoke(settings_get)

platformio/commands/show.py

@@ -1,7 +1,7 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-from os.path import join
+from datetime import datetime
 
 from click import argument, command, echo, style
 
@@ -13,20 +13,25 @@ from platformio.platforms.base import PlatformFactory
 @command("show", short_help="Show details about installed platforms")
 @argument("platform")
 def cli(platform):
-    p = PlatformFactory().newPlatform(platform)
-    if platform not in PackageManager.get_installed():
+
+    installed_platforms = PlatformFactory.get_platforms(
+        installed=True).keys()
+
+    if platform not in installed_platforms:
         raise PlatformNotInstalledYet(platform)
 
-    # print info about platform
+    p = PlatformFactory().newPlatform(platform)
     echo("{name:<20} - {info}".format(name=style(p.get_name(), fg="cyan"),
                                       info=p.get_short_info()))
 
-    pm = PackageManager(platform)
-    for name, data in pm.get_installed(platform).items():
+    installed_packages = PackageManager.get_installed()
+    for name in p.get_installed_packages():
+        data = installed_packages[name]
         pkgalias = p.get_pkg_alias(name)
         echo("----------")
         echo("Package: %s" % style(name, fg="yellow"))
         if pkgalias:
             echo("Alias: %s" % pkgalias)
-        echo("Location: %s" % join(pm.get_platform_dir(), data['path']))
         echo("Version: %d" % int(data['version']))
+        echo("Installed: %s" % datetime.fromtimestamp(
+            data['time']).strftime("%Y-%m-%d %H:%M:%S"))

platformio/commands/uninstall.py

@@ -3,8 +3,6 @@
 
 from click import argument, command, secho
 
-from platformio.exception import PlatformNotInstalledYet
-from platformio.pkgmanager import PackageManager
 from platformio.platforms.base import PlatformFactory
 
 
@@ -13,10 +11,6 @@ from platformio.platforms.base import PlatformFactory
 def cli(platforms):
 
     for platform in platforms:
-
-        if platform not in PackageManager.get_installed():
-            raise PlatformNotInstalledYet(platform)
-
         p = PlatformFactory().newPlatform(platform)
         if p.uninstall():
             secho("The platform '%s' has been successfully "

platformio/commands/update.py

@@ -3,14 +3,17 @@
 
 from click import command, echo, style
 
-from platformio.pkgmanager import PackageManager
 from platformio.platforms.base import PlatformFactory
 
 
 @command("update", short_help="Update installed platforms")
 def cli():
 
-    for platform in PackageManager.get_installed().keys():
+    installed_platforms = PlatformFactory.get_platforms(
+        installed=True).keys()
+    installed_platforms.sort()
+
+    for platform in installed_platforms:
         echo("\nPlatform %s" % style(platform, fg="cyan"))
         echo("--------")
         p = PlatformFactory().newPlatform(platform)

platformio/downloader.py

@@ -67,17 +67,17 @@ class FileDownloader(object):
 
         dlsha1 = None
         try:
-            res = check_output(["sha1sum", self._destination])
-            dlsha1 = res[:40]
+            dlsha1 = check_output(["sha1sum", self._destination])
         except OSError:
             try:
-                res = check_output(["shasum", "-a", "1", self._destination])
-                dlsha1 = res[:40]
+                dlsha1 = check_output(["shasum", "-a", "1", self._destination])
             except OSError:
                 pass
 
-        if dlsha1 and sha1 != dlsha1:
-            raise FDSHASumMismatch(dlsha1, self._fname, sha1)
+        if dlsha1:
+            dlsha1 = dlsha1[1:41] if dlsha1.startswith("\\") else dlsha1[:40]
+            if sha1 != dlsha1:
+                raise FDSHASumMismatch(dlsha1, self._fname, sha1)
 
     def _preserve_filemtime(self, lmdate):
         timedata = parsedate_tz(lmdate)

platformio/exception.py

@@ -118,4 +118,29 @@ class LibAlreadyInstalledError(PlatformioException):
 
 class LibNotInstalledError(PlatformioException):
 
-    MESSAGE = "Library '%s' has not been installed yet"
+    MESSAGE = "Library #%d has not been installed yet"
+
+
+class LibInstallDependencyError(PlatformioException):
+
+    MESSAGE = "Error has been occurred for library dependency '%s'"
+
+
+class BuildScriptNotFound(PlatformioException):
+
+    MESSAGE = "Invalid path '%s' to build script"
+
+
+class InvalidSettingName(PlatformioException):
+
+    MESSAGE = "Invalid setting with the name '%s'"
+
+
+class InvalidSettingValue(PlatformioException):
+
+    MESSAGE = "Invalid value '%s' for the setting '%s'"
+
+
+class UpgraderFailed(PlatformioException):
+
+    MESSAGE = "An error occurred while upgrading PlatformIO"

platformio/libmanager.py

@@ -2,11 +2,13 @@
 # See LICENSE for details.
 
 import json
-from os import listdir, makedirs, remove
+import re
+from os import listdir, makedirs, remove, rename
 from os.path import isdir, isfile, join
 from shutil import rmtree
 from tempfile import gettempdir
 
+from platformio import telemetry
 from platformio.downloader import FileDownloader
 from platformio.exception import LibAlreadyInstalledError, LibNotInstalledError
 from platformio.unpacker import FileUnpacker
@@ -15,7 +17,7 @@ from platformio.util import get_api_result
 
 class LibraryManager(object):
 
-    CONFIG_NAME = "library.json"
+    CONFIG_NAME = ".library.json"
 
     def __init__(self, lib_dir):
         self.lib_dir = lib_dir
@@ -32,44 +34,78 @@ class LibraryManager(object):
         return fu.start()
 
     def get_installed(self):
-        items = []
-        for item in listdir(self.lib_dir):
-            conf_path = join(self.lib_dir, item, self.CONFIG_NAME)
-            if isfile(conf_path):
-                items.append(item)
+        items = {}
+        if not isdir(self.lib_dir):
+            return items
+        for dirname in listdir(self.lib_dir):
+            conf_path = join(self.lib_dir, dirname, self.CONFIG_NAME)
+            if not isfile(conf_path):
+                continue
+            with open(conf_path, "r") as f:
+                items[dirname] = json.load(f)
         return items
 
-    def get_info(self, name):
-        conf_path = join(self.lib_dir, name, self.CONFIG_NAME)
-        if not isfile(conf_path):
-            raise LibNotInstalledError(name)
-        with open(conf_path, "r") as f:
-            return json.load(f)
+    def get_latest_versions(self):
+        lib_ids = [str(item['id']) for item in self.get_installed().values()]
+        if not lib_ids:
+            return None
+        return get_api_result("/lib/version/" + str(",".join(lib_ids)))
 
-    def is_installed(self, name):
-        return isfile(join(self.lib_dir, name, self.CONFIG_NAME))
+    def get_outdated(self):
+        outdated = []
+        for id_, latest_version in (self.get_latest_versions() or {}).items():
+            info = self.get_info(int(id_))
+            if latest_version != info['version']:
+                outdated.append(info['name'])
+        return outdated
 
-    def install(self, name, version=None):
-        if self.is_installed(name):
+    def get_info(self, id_):
+        for item in self.get_installed().values():
+            if "id" in item and item['id'] == id_:
+                return item
+        raise LibNotInstalledError(id_)
+
+    def is_installed(self, id_):
+        try:
+            return int(self.get_info(id_)['id']) == id_
+        except LibNotInstalledError:
+            return False
+
+    def install(self, id_, version=None):
+        if self.is_installed(id_):
             raise LibAlreadyInstalledError()
 
-        _lib_dir = join(self.lib_dir, name)
-        if not isdir(_lib_dir):
-            makedirs(_lib_dir)
-
-        dlinfo = get_api_result("/lib/download/" + name, dict(version=version)
-                                if version else None)
+        dlinfo = get_api_result("/lib/download/" + str(id_),
+                                dict(version=version) if version else None)
+        dlpath = None
+        tmplib_dir = join(self.lib_dir, str(id_))
         try:
             dlpath = self.download(dlinfo['url'], gettempdir())
-            self.unpack(dlpath, _lib_dir)
+            if not isdir(tmplib_dir):
+                makedirs(tmplib_dir)
+            self.unpack(dlpath, tmplib_dir)
         finally:
-            remove(dlpath)
+            if dlpath:
+                remove(dlpath)
 
-        return self.is_installed(name)
+        info = self.get_info(id_)
+        rename(tmplib_dir, join(self.lib_dir, "%s_ID%d" % (
+            re.sub(r"[^\da-z]+", "_", info['name'], flags=re.I), id_)))
 
-    def uninstall(self, name):
-        if self.is_installed(name):
-            rmtree(join(self.lib_dir, name))
-            return True
-        else:
-            raise LibNotInstalledError(name)
+        telemetry.on_event(
+            category="LibraryManager", action="Install",
+            label="#%d %s" % (id_, info['name'])
+        )
+
+        return True
+
+    def uninstall(self, id_):
+        for libdir, item in self.get_installed().iteritems():
+            if "id" in item and item['id'] == id_:
+                rmtree(join(self.lib_dir, libdir))
+                telemetry.on_event(
+                    category="LibraryManager", action="Uninstall",
+                    label="#%d %s" % (id_, item['name'])
+                )
+                return True
+        raise LibNotInstalledError(id_)

platformio/maintenance.py

@@ -0,0 +1,187 @@
+# Copyright (C) Ivan Kravets <me@ikravets.com>
+# See LICENSE for details.
+
+import re
+from os import remove
+from os.path import isdir, isfile, join
+from shutil import rmtree
+from time import time
+
+import click
+
+from platformio import __version__, app, telemetry
+from platformio.commands.install import cli as cli_install
+from platformio.commands.lib import lib_update as cli_libraries_update
+from platformio.commands.update import cli as cli_platforms_update
+from platformio.commands.upgrade import get_latest_version
+from platformio.exception import UpgraderFailed
+from platformio.libmanager import LibraryManager
+from platformio.platforms.base import PlatformFactory
+from platformio.util import get_home_dir, get_lib_dir
+
+
+def on_platformio_start(ctx):
+    telemetry.on_command(ctx)
+    after_upgrade(ctx)
+    check_platformio_upgrade()
+    check_internal_updates(ctx, "platforms")
+    check_internal_updates(ctx, "libraries")
+
+
+def on_platformio_end(ctx, result):  # pylint: disable=W0613
+    pass
+
+
+def on_platformio_exception(e):
+    telemetry.on_exception(e)
+
+
+class Upgrader(object):
+
+    def __init__(self, from_version, to_version):
+        self.from_version = self.version_to_int(from_version)
+        self.to_version = self.version_to_int(to_version)
+
+    @staticmethod
+    def version_to_int(version):
+        return int(re.sub(r"[^\d]+", "", version))
+
+    def run(self, ctx):
+        if self.from_version > self.to_version:
+            return True
+
+        result = [True]
+        for v in (90, ):
+            if self.from_version >= v:
+                continue
+            result.append(getattr(self, "_upgrade_to_%d" % v)(ctx))
+
+        return all(result)
+
+    def _upgrade_to_90(self, ctx):  # pylint: disable=R0201
+        prev_platforms = []
+
+        # remove platform's folder (obsoleted package structure)
+        for name in PlatformFactory.get_platforms().keys():
+            pdir = join(get_home_dir(), name)
+            if not isdir(pdir):
+                continue
+            prev_platforms.append(name)
+            rmtree(pdir)
+
+        # remove unused files
+        for fname in (".pioupgrade", "installed.json"):
+            if isfile(join(get_home_dir(), fname)):
+                remove(join(get_home_dir(), fname))
+
+        if prev_platforms:
+            ctx.invoke(cli_install, platforms=prev_platforms)
+
+        return True
+
+
+def after_upgrade(ctx):
+    if app.get_state_item("last_version", None) == __version__:
+        return
+
+    # promotion
+    click.echo("\nIf you like %s, please:" % (
+        click.style("PlatformIO", fg="cyan")
+    ))
+    click.echo(
+        "- %s us on Twitter to stay up-to-date "
+        "on the latest project news > %s" %
+        (click.style("follow", fg="cyan"),
+         click.style("https://twitter.com/platformiotool", fg="blue"))
+    )
+    click.echo("- %s us a star on GitHub > %s" % (
+        click.style("give", fg="cyan"),
+        click.style("https://github.com/ivankravets/platformio", fg="blue")
+    ))
+    click.secho("Thanks a lot!\n", fg="green", blink=True)
+
+    if not isdir(get_home_dir()):
+        return
+
+    click.secho("Please wait while upgrading PlatformIO ...",
+                fg="yellow")
+
+    last_version = app.get_state_item("last_version", "0.0.0")
+    u = Upgrader(last_version, __version__)
+    if u.run(ctx):
+        app.set_state_item("last_version", __version__)
+        click.secho("PlatformIO has been successfully upgraded to %s!\n" %
+                    __version__, fg="green")
+
+        telemetry.on_event(category="Auto", action="Upgrade",
+                           label="%s > %s" % (last_version, __version__))
+    else:
+        raise UpgraderFailed()
+    click.echo("")
+
+
+def check_platformio_upgrade():
+    last_check = app.get_state_item("last_check", {})
+    interval = int(app.get_setting("check_platformio_interval")) * 3600 * 24
+    if (time() - interval) < last_check.get("platformio_upgrade", 0):
+        return
+
+    last_check['platformio_upgrade'] = int(time())
+    app.set_state_item("last_check", last_check)
+
+    latest_version = get_latest_version()
+    if (latest_version == __version__ or
+            Upgrader.version_to_int(latest_version) <
+            Upgrader.version_to_int(__version__)):
+        return
+
+    click.secho("There is a new version %s of PlatformIO available.\n"
+                "Please upgrade it via " % latest_version,
+                fg="yellow", nl=False)
+    click.secho("`platformio upgrade`", fg="cyan", nl=False)
+    click.secho(" command.\nChanges: ", fg="yellow", nl=False)
+    click.secho("http://docs.platformio.ikravets.com/en/latest/history.html\n",
+                fg="blue")
+
+
+def check_internal_updates(ctx, what):
+    last_check = app.get_state_item("last_check", {})
+    interval = int(app.get_setting("check_%s_interval" % what)) * 3600 * 24
+    if (time() - interval) < last_check.get(what + "_update", 0):
+        return
+
+    last_check[what + '_update'] = int(time())
+    app.set_state_item("last_check", last_check)
+
+    outdated_items = []
+    if what == "platforms":
+        for platform in PlatformFactory.get_platforms(installed=True).keys():
+            p = PlatformFactory().newPlatform(platform)
+            if p.is_outdated():
+                outdated_items.append(platform)
+    elif what == "libraries":
+        lm = LibraryManager(get_lib_dir())
+        outdated_items = lm.get_outdated()
+
+    if not outdated_items:
+        return
+
+    click.secho("There are the new updates for %s (%s)" %
+                (what, ", ".join(outdated_items)), fg="yellow")
+
+    if not app.get_setting("auto_update_" + what):
+        click.secho("Please update them via ", fg="yellow", nl=False)
+        click.secho("`platformio %supdate`" %
+                    ("lib " if what == "libraries" else ""),
+                    fg="cyan", nl=False)
+        click.secho(" command.\n", fg="yellow")
+    else:
+        click.secho("Please wait while updating %s ..." % what, fg="yellow")
+        if what == "platforms":
+            ctx.invoke(cli_platforms_update)
+        elif what == "libraries":
+            ctx.invoke(cli_libraries_update)
+        click.echo()
+
+        telemetry.on_event(category="Auto", action="Update",
+                           label=what.title())

platformio/pkgmanager.py

@@ -1,40 +1,37 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-import json
 from os import makedirs, remove
-from os.path import isdir, isfile, join
+from os.path import isdir, join
 from shutil import rmtree
+from time import time
 
 from click import echo, secho, style
-from requests import get
-from requests.utils import default_user_agent
 
-from platformio import __pkgmanifesturl__, __version__
+from platformio import telemetry
+from platformio.app import get_state_item, set_state_item
 from platformio.downloader import FileDownloader
 from platformio.exception import (InvalidPackageVersion, NonSystemPackage,
                                   UnknownPackage)
 from platformio.unpacker import FileUnpacker
-from platformio.util import get_home_dir, get_systype
+from platformio.util import get_api_result, get_home_dir, get_systype
 
 
 class PackageManager(object):
 
-    DBFILE_PATH = join(get_home_dir(), "installed.json")
+    def __init__(self):
+        self._package_dir = join(get_home_dir(), "packages")
+        if not isdir(self._package_dir):
+            makedirs(self._package_dir)
+        assert isdir(self._package_dir)
 
-    def __init__(self, platform_name):
-        self._platform_name = platform_name
-
-    @staticmethod
-    def get_manifest():
+    @classmethod
+    def get_manifest(cls):
         try:
-            return PackageManager._cached_manifest
+            return cls._cached_manifest
         except AttributeError:
-            headers = {"User-Agent": "PlatformIO/%s %s" % (
-                __version__, default_user_agent())}
-            PackageManager._cached_manifest = get(__pkgmanifesturl__,
-                                                  headers=headers).json()
-        return PackageManager._cached_manifest
+            cls._cached_manifest = get_api_result("/packages")
+        return cls._cached_manifest
 
     @staticmethod
     def download(url, dest_dir, sha1=None):
@@ -49,20 +46,18 @@ class PackageManager(object):
         return fu.start()
 
     @staticmethod
-    def get_installed(platform=None):
-        data = {}
-        if isfile(PackageManager.DBFILE_PATH):
-            with open(PackageManager.DBFILE_PATH) as fp:
-                data = json.load(fp)
-        return data.get(platform, None) if platform else data
+    def get_installed():
+        return get_state_item("installed_packages", {})
 
-    def get_platform_dir(self):
-        return join(get_home_dir(), self._platform_name)
+    def get_outdated(self):
+        outdated = []
+        for name, data in self.get_installed().items():
+            if data['version'] != self.get_info(name)['version']:
+                outdated.append(name)
+        return outdated
 
     def is_installed(self, name):
-        installed = self.get_installed()
-        return (self._platform_name in installed and name in
-                installed[self._platform_name])
+        return name in self.get_installed()
 
     def get_info(self, name, version=None):
         manifest = self.get_manifest()
@@ -84,35 +79,47 @@ class PackageManager(object):
         else:
             return sorted(builds, key=lambda s: s['version'])[-1]
 
-    def install(self, name, path):
+    def install(self, name):
         echo("Installing %s package:" % style(name, fg="cyan"))
 
         if self.is_installed(name):
             secho("Already installed", fg="yellow")
-            return
+            return False
 
         info = self.get_info(name)
-        pkg_dir = join(self.get_platform_dir(), path)
+        pkg_dir = join(self._package_dir, name)
         if not isdir(pkg_dir):
             makedirs(pkg_dir)
 
         dlpath = self.download(info['url'], pkg_dir, info['sha1'])
         if self.unpack(dlpath, pkg_dir):
-            self._register(name, info['version'], path)
+            self._register(name, info['version'])
         # remove archive
         remove(dlpath)
 
-    def uninstall(self, name, path):
+        telemetry.on_event(
+            category="PackageManager", action="Install", label=name)
+
+    def uninstall(self, name):
         echo("Uninstalling %s package: \t" % style(name, fg="cyan"),
              nl=False)
-        rmtree(join(self.get_platform_dir(), path))
+
+        if not self.is_installed(name):
+            secho("Not installed", fg="yellow")
+            return False
+
+        rmtree(join(self._package_dir, name))
         self._unregister(name)
         echo("[%s]" % style("OK", fg="green"))
 
+        # report usage
+        telemetry.on_event(
+            category="PackageManager", action="Uninstall", label=name)
+
     def update(self, name):
         echo("Updating %s package:" % style(name, fg="yellow"))
 
-        installed = self.get_installed(self._platform_name)
+        installed = self.get_installed()
         current_version = installed[name]['version']
         latest_version = self.get_info(name)['version']
 
@@ -125,36 +132,21 @@ class PackageManager(object):
         else:
             echo("[%s]" % (style("Out-of-date", fg="red")))
 
-        self.uninstall(name, installed[name]['path'])
-        self.install(name, installed[name]['path'])
+        self.uninstall(name)
+        self.install(name)
 
-    def register_platform(self, name):
-        data = self.get_installed()
-        if name not in data:
-            data[name] = {}
-        self._update_db(data)
-        return data
+        telemetry.on_event(
+            category="PackageManager", action="Update", label=name)
 
-    def unregister_platform(self, name):
+    def _register(self, name, version):
         data = self.get_installed()
-        del data[name]
-        self._update_db(data)
-
-    def _register(self, name, version, path):
-        data = self.get_installed()
-        if self._platform_name not in data:
-            data = self.register_platform(self._platform_name)
-        data[self._platform_name][name] = {
+        data[name] = {
             "version": version,
-            "path": path
+            "time": int(time())
         }
-        self._update_db(data)
+        set_state_item("installed_packages", data)
 
     def _unregister(self, name):
         data = self.get_installed()
-        del data[self._platform_name][name]
-        self._update_db(data)
-
-    def _update_db(self, data):
-        with open(self.DBFILE_PATH, "w") as fp:
-            json.dump(data, fp)
+        del data[name]
+        set_state_item("installed_packages", data)

platformio/platforms/atmelavr.py

@@ -1,8 +1,6 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-from os.path import join
-
 from platformio.platforms.base import BasePlatform
 
 
@@ -15,26 +13,21 @@ class AtmelavrPlatform(BasePlatform):
     PACKAGES = {
 
         "toolchain-atmelavr": {
-            "path": join("tools", "toolchain"),
             "alias": "toolchain",
             "default": True
         },
 
         "tool-avrdude": {
-            "path": join("tools", "avrdude"),
             "alias": "uploader",
             "default": True
         },
 
         "framework-arduinoavr": {
-            "path": join("frameworks", "arduino"),
+            "alias": "framework",
             "default": True
         }
     }
 
-    def get_name(self):
-        return "atmelavr"
-
     def after_run(self, result):
         # fix STDERR "flash written" for avrdude
         if "flash written" in result['err']:

platformio/platforms/base.py

@@ -1,29 +1,76 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-from os.path import join
-from shutil import rmtree
+from imp import load_source
+from os import listdir
+from os.path import isdir, isfile, join
 
-from platformio.exception import UnknownPackage, UnknownPlatform
+from platformio.app import get_state_item, set_state_item
+from platformio.exception import (BuildScriptNotFound, PlatformNotInstalledYet,
+                                  UnknownPackage, UnknownPlatform)
 from platformio.pkgmanager import PackageManager
-from platformio.util import exec_command, get_platforms, get_source_dir
+from platformio.util import exec_command, get_home_dir, get_source_dir
 
 
 class PlatformFactory(object):
 
     @staticmethod
-    def newPlatform(name):
-        clsname = "%sPlatform" % name.title()
+    def get_clsname(name):
+        return "%sPlatform" % name.title()
+
+    @staticmethod
+    def load_module(name, path):
+        module = None
         try:
-            assert name in get_platforms()
-            mod = __import__("platformio.platforms." + name.lower(),
-                             None, None, [clsname])
-        except (AssertionError, ImportError):
+            module = load_source(
+                "platformio.platforms.%s" % name, path)
+        except ImportError:
+            raise UnknownPlatform(name)
+        return module
+
+    @classmethod
+    def get_platforms(cls, installed=False):
+        platforms = {}
+        for d in (get_home_dir(), get_source_dir()):
+            pdir = join(d, "platforms")
+            if not isdir(pdir):
+                continue
+            for p in listdir(pdir):
+                if p in ("__init__.py", "base.py") or not p.endswith(".py"):
+                    continue
+                name = p[:-3]
+                path = join(pdir, p)
+                try:
+                    isplatform = hasattr(
+                        cls.load_module(name, path),
+                        cls.get_clsname(name)
+                    )
+                    if isplatform:
+                        platforms[name] = path
+                except UnknownPlatform:
+                    pass
+
+        if not installed:
+            return platforms
+
+        installed_platforms = {}
+        for name in get_state_item("installed_platforms", []):
+            if name in platforms:
+                installed_platforms[name] = platforms[name]
+        return installed_platforms
+
+    @classmethod
+    def newPlatform(cls, name):
+        platforms = cls.get_platforms()
+        if name not in platforms:
             raise UnknownPlatform(name)
 
-        obj = getattr(mod, clsname)()
-        assert isinstance(obj, BasePlatform)
-        return obj
+        _instance = getattr(
+            cls.load_module(name, platforms[name]),
+            cls.get_clsname(name)
+        )()
+        assert isinstance(_instance, BasePlatform)
+        return _instance
 
 
 class BasePlatform(object):
@@ -31,6 +78,13 @@ class BasePlatform(object):
     PACKAGES = {}
 
     def get_name(self):
+        return self.__class__.__name__[:-8].lower()
+
+    def get_build_script(self):
+        builtin = join(get_source_dir(), "builder", "scripts", "%s.py" %
+                       self.get_name())
+        if isfile(builtin):
+            return builtin
         raise NotImplementedError()
 
     def get_short_info(self):
@@ -40,6 +94,9 @@ class BasePlatform(object):
         else:
             raise NotImplementedError()
 
+    def get_packages(self):
+        return self.PACKAGES
+
     def get_pkg_alias(self, pkgname):
         return self.PACKAGES[pkgname].get("alias", None)
 
@@ -47,62 +104,116 @@ class BasePlatform(object):
         names = []
         for alias in aliases:
             name = alias
-            # lookup by packages alias
-            if name not in self.PACKAGES:
-                for _name, _opts in self.PACKAGES.items():
-                    if _opts.get("alias", None) == alias:
-                        name = _name
-                        break
+            # lookup by package aliases
+            for _name, _opts in self.get_packages().items():
+                if _opts.get("alias", None) == alias:
+                    name = _name
+                    break
             names.append(name)
         return names
 
+    def get_installed_packages(self):
+        pm = PackageManager()
+        return [n for n in self.get_packages().keys() if pm.is_installed(n)]
+
     def install(self, with_packages, without_packages, skip_default_packages):
         with_packages = set(self.pkg_aliases_to_names(with_packages))
         without_packages = set(self.pkg_aliases_to_names(without_packages))
 
         upkgs = with_packages | without_packages
-        ppkgs = set(self.PACKAGES.keys())
+        ppkgs = set(self.get_packages().keys())
         if not upkgs.issubset(ppkgs):
             raise UnknownPackage(", ".join(upkgs - ppkgs))
 
         requirements = []
-        for name, opts in self.PACKAGES.items():
+        for name, opts in self.get_packages().items():
             if name in without_packages:
                 continue
             elif (name in with_packages or (not skip_default_packages and
                                             opts['default'])):
-                requirements.append((name, opts['path']))
+                requirements.append(name)
+
+        pm = PackageManager()
+        for name in requirements:
+            pm.install(name)
+
+        # register installed platform
+        data = get_state_item("installed_platforms", [])
+        if self.get_name() not in data:
+            data.append(self.get_name())
+            set_state_item("installed_platforms", data)
 
-        pm = PackageManager(self.get_name())
-        for (package, path) in requirements:
-            pm.install(package, path)
         return len(requirements)
 
     def uninstall(self):
         platform = self.get_name()
-        pm = PackageManager(platform)
+        installed_platforms = PlatformFactory.get_platforms(
+            installed=True).keys()
 
-        for package, data in pm.get_installed(platform).items():
-            pm.uninstall(package, data['path'])
+        if platform not in installed_platforms:
+            raise PlatformNotInstalledYet(platform)
+
+        deppkgs = set()
+        for item in installed_platforms:
+            if item == platform:
+                continue
+            p = PlatformFactory().newPlatform(item)
+            deppkgs = deppkgs.union(set(p.get_packages().keys()))
+
+        pm = PackageManager()
+        for name in self.get_packages().keys():
+            if not pm.is_installed(name) or name in deppkgs:
+                continue
+            pm.uninstall(name)
+
+        # unregister installed platform
+        installed_platforms.remove(platform)
+        set_state_item("installed_platforms", installed_platforms)
 
-        pm.unregister_platform(platform)
-        rmtree(pm.get_platform_dir())
         return True
 
     def update(self):
-        platform = self.get_name()
-        pm = PackageManager(platform)
-        for package in pm.get_installed(platform).keys():
-            pm.update(package)
+        pm = PackageManager()
+        for name in self.get_installed_packages():
+            pm.update(name)
+
+    def is_outdated(self):
+        pm = PackageManager()
+        obsolated = pm.get_outdated()
+        return not set(self.get_packages().keys()).isdisjoint(set(obsolated))
 
     def run(self, variables, targets):
         assert isinstance(variables, list)
         assert isinstance(targets, list)
 
+        installed_platforms = PlatformFactory.get_platforms(
+            installed=True).keys()
+        installed_packages = PackageManager.get_installed()
+
+        if self.get_name() not in installed_platforms:
+            raise PlatformNotInstalledYet(self.get_name())
+
         if "clean" in targets:
             targets.remove("clean")
             targets.append("-c")
 
+        if not any([v.startswith("BUILD_SCRIPT=") for v in variables]):
+            variables.append("BUILD_SCRIPT=%s" % self.get_build_script())
+
+        for v in variables:
+            if not v.startswith("BUILD_SCRIPT="):
+                continue
+            _, path = v.split("=", 2)
+            if not isfile(path):
+                raise BuildScriptNotFound(path)
+
+        # append aliases of installed packages
+        for name, options in self.get_packages().items():
+            if name not in installed_packages:
+                continue
+            variables.append(
+                "PIOPACKAGE_%s=%s" % (options['alias'].upper(), name))
+
         result = exec_command([
             "scons",
             "-Q",

platformio/platforms/timsp430.py

@@ -1,8 +1,6 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-from os.path import join
-
 from platformio.platforms.base import BasePlatform
 
 
@@ -15,22 +13,17 @@ class Timsp430Platform(BasePlatform):
     PACKAGES = {
 
         "toolchain-timsp430": {
-            "path": join("tools", "toolchain"),
             "alias": "toolchain",
             "default": True
         },
 
         "tool-mspdebug": {
-            "path": join("tools", "mspdebug"),
             "alias": "uploader",
             "default": True
         },
 
         "framework-energiamsp430": {
-            "path": join("frameworks", "energia"),
+            "alias": "framework",
             "default": True
         }
     }
-
-    def get_name(self):
-        return "timsp430"

platformio/platforms/titiva.py

@@ -1,8 +1,6 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-from os.path import join
-
 from platformio.platforms.base import BasePlatform
 
 
@@ -15,22 +13,17 @@ class TitivaPlatform(BasePlatform):
     PACKAGES = {
 
         "toolchain-gccarmnoneeabi": {
-            "path": join("tools", "toolchain"),
             "alias": "toolchain",
             "default": True
         },
 
         "tool-lm4flash": {
-            "path": join("tools", "lm4flash"),
             "alias": "uploader",
             "default": True
         },
 
         "framework-energiativa": {
-            "path": join("frameworks", "energia"),
+            "alias": "framework",
             "default": True
         }
     }
-
-    def get_name(self):
-        return "titiva"

platformio/projectconftpl.ini

@@ -1,6 +1,3 @@
-# Copyright (C) Ivan Kravets <me@ikravets.com>
-# See LICENSE for details.
-
 #
 # Project Configuration File
 #

platformio/telemetry.py

@@ -0,0 +1,202 @@
+# Copyright (C) Ivan Kravets <me@ikravets.com>
+# See LICENSE for details.
+
+import platform
+import re
+import uuid
+from sys import argv as sys_argv
+from time import time
+
+import click
+import requests
+
+from platformio import __version__, app
+from platformio.util import exec_command, get_systype
+
+
+class TelemetryBase(object):
+
+    MACHINE_ID = str(uuid.uuid5(uuid.NAMESPACE_OID, str(uuid.getnode())))
+
+    def __init__(self):
+        self._params = {}
+
+    def __getitem__(self, name):
+        return self._params.get(name, None)
+
+    def __setitem__(self, name, value):
+        self._params[name] = value
+
+    def __delitem__(self, name):
+        if name in self._params:
+            del self._params[name]
+
+    def send(self, hittype):
+        raise NotImplementedError()
+
+
+class MeasurementProtocol(TelemetryBase):
+
+    TRACKING_ID = "UA-1768265-9"
+    PARAMS_MAP = {
+        "screen_name": "cd",
+        "event_category": "ec",
+        "event_action": "ea",
+        "event_label": "el",
+        "event_value": "ev"
+    }
+
+    def __init__(self):
+        TelemetryBase.__init__(self)
+        self['v'] = 1
+        self['tid'] = self.TRACKING_ID
+        self['cid'] = self.MACHINE_ID
+
+        self['sr'] = "%dx%d" % click.get_terminal_size()
+        self._prefill_screen_name()
+        self._prefill_appinfo()
+        self._prefill_custom_data()
+
+    @classmethod
+    def session_instance(cls):
+        try:
+            return cls._session_instance
+        except AttributeError:
+            cls._session_instance = requests.Session()
+        return cls._session_instance
+
+    def __getitem__(self, name):
+        if name in self.PARAMS_MAP:
+            name = self.PARAMS_MAP[name]
+        return TelemetryBase.__getitem__(self, name)
+
+    def __setitem__(self, name, value):
+        if name in self.PARAMS_MAP:
+            name = self.PARAMS_MAP[name]
+        TelemetryBase.__setitem__(self, name, value)
+
+    def _prefill_appinfo(self):
+        self['av'] = __version__
+
+        # gather dependent packages
+        dpdata = []
+        dpdata.append("Click/%s" % click.__version__)
+        # dpdata.append("Requests/%s" % requests.__version__)
+        try:
+            result = exec_command(["scons", "--version"])
+            match = re.search(r"engine: v([\d\.]+)", result['out'])
+            if match:
+                dpdata.append("SCons/%s" % match.group(1))
+        except:  # pylint: disable=W0702
+            pass
+        self['an'] = " ".join(dpdata)
+
+    def _prefill_custom_data(self):
+        self['cd1'] = get_systype()
+        self['cd2'] = "Python/%s %s" % (platform.python_version(),
+                                        platform.platform())
+
+    def _prefill_screen_name(self):
+        args = [str(s).lower() for s in sys_argv[1:]]
+        if not args:
+            return
+
+        if args[0] in ("lib", "settings"):
+            cmd_path = args[:2]
+        else:
+            cmd_path = args[:1]
+
+        self['screen_name'] = " ".join([p.title() for p in cmd_path])
+        self['cd3'] = " ".join(args)
+
+    def send(self, hittype):
+        self['t'] = hittype
+
+        # correct queue time
+        if "qt" in self._params and isinstance(self['qt'], float):
+            self['qt'] = int((time() - self['qt']) * 1000)
+
+        try:
+            r = self.session_instance().post(
+                "https://ssl.google-analytics.com/collect",
+                data=self._params
+            )
+            r.raise_for_status()
+        except:  # pylint: disable=W0702
+            backup_report(self._params)
+            return False
+        return True
+
+
+def on_command(ctx):  # pylint: disable=W0613
+    mp = MeasurementProtocol()
+    if mp.send("screenview"):
+        resend_backuped_reports()
+
+
+def on_run_environment(options, targets):
+    opts = ["%s=%s" % (opt, value) for opt, value in sorted(options)]
+    targets = [t.title() for t in targets or ["run"]]
+    on_event("Env", " ".join(targets), " ".join(opts))
+
+
+def on_event(category, action, label=None, value=None, screen_name=None):
+    mp = MeasurementProtocol()
+    mp['event_category'] = category[:150]
+    mp['event_action'] = action[:500]
+    if label:
+        mp['event_label'] = label[:500]
+    if value:
+        mp['event_value'] = int(value)
+    if screen_name:
+        mp['screen_name'] = screen_name[:2048]
+    return mp.send("event")
+
+
+def on_exception(e):
+    mp = MeasurementProtocol()
+    mp['exd'] = "%s: %s" % (type(e).__name__, e)
+    mp['exf'] = 1
+    return mp.send("exception")
+
+
+def backup_report(params):
+    KEEP_MAX_REPORTS = 1000
+    tm = app.get_state_item("telemetry", {})
+    if "backup" not in tm:
+        tm['backup'] = []
+
+    # skip static options
+    for key in params.keys():
+        if key in ("v", "tid", "cid", "cd1", "cd2", "sr", "an"):
+            del params[key]
+
+    # store time in UNIX format
+    if "qt" not in params:
+        params['qt'] = time()
+    elif not isinstance(params['qt'], float):
+        params['qt'] = time() - (params['qt'] / 1000)
+
+    tm['backup'].append(params)
+    tm['backup'] = tm['backup'][KEEP_MAX_REPORTS*-1:]
+    app.set_state_item("telemetry", tm)
+
+
+def resend_backuped_reports():
+    MAX_RESEND_REPORTS = 10
+
+    resent_nums = 0
+    while resent_nums < MAX_RESEND_REPORTS:
+        tm = app.get_state_item("telemetry", {})
+        if "backup" not in tm or not tm['backup']:
+            break
+
+        report = tm['backup'].pop()
+        app.set_state_item("telemetry", tm)
+        resent_nums += 1
+
+        mp = MeasurementProtocol()
+        for key, value in report.items():
+            mp[key] = value
+        if not mp.send(report['t']):
+            break

platformio/util.py

@@ -2,15 +2,13 @@
 # See LICENSE for details.
 
 from os import name as os_name
-from os import getcwd, getenv, listdir, utime
-from os.path import dirname, expanduser, isfile, join, realpath
+from os import getcwd, getenv, makedirs, utime
+from os.path import dirname, expanduser, isdir, isfile, join, realpath
 from platform import system, uname
 from subprocess import PIPE, Popen
 from time import sleep
 
-from requests import get, post
-from requests.exceptions import ConnectionError, HTTPError
-from requests.utils import default_user_agent
+import requests
 from serial import Serial
 
 from platformio import __apiurl__, __version__
@@ -31,14 +29,24 @@ def get_systype():
 
 
 def get_home_dir():
+    home_dir = None
+
     try:
         config = get_project_config()
         if (config.has_section("platformio") and
                 config.has_option("platformio", "home_dir")):
-            return config.get("platformio", "home_dir")
+            home_dir = config.get("platformio", "home_dir")
     except NotPlatformProject:
         pass
-    return expanduser("~/.platformio")
+
+    if not home_dir:
+        home_dir = expanduser("~/.platformio")
+
+    if not isdir(home_dir):
+        makedirs(home_dir)
+
+    assert isdir(home_dir)
+    return home_dir
 
 
 def get_lib_dir():
@@ -46,7 +54,11 @@ def get_lib_dir():
         config = get_project_config()
         if (config.has_section("platformio") and
                 config.has_option("platformio", "lib_dir")):
-            return config.get("platformio", "lib_dir")
+            lib_dir = config.get("platformio", "lib_dir")
+            if lib_dir.startswith("~"):
+                return expanduser(lib_dir)
+            else:
+                return lib_dir
     except NotPlatformProject:
         pass
     return join(get_home_dir(), "lib")
@@ -73,15 +85,6 @@ def get_project_config():
     return cp
 
 
-def get_platforms():
-    platforms = []
-    for p in listdir(join(get_source_dir(), "platforms")):
-        if p in ("__init__.py", "base.py") or not p.endswith(".py"):
-            continue
-        platforms.append(p[:-3])
-    return platforms
-
-
 def change_filemtime(path, time):
     utime(path, (time, time))
 
@@ -117,23 +120,31 @@ def get_serialports():
 def get_api_result(path, params=None, data=None):
     result = None
     r = None
+
     try:
         headers = {"User-Agent": "PlatformIO/%s %s" % (
-            __version__, default_user_agent())}
-        if data:
-            r = post(__apiurl__ + path, params=params, data=data,
-                     headers=headers)
+            __version__, requests.utils.default_user_agent())}
+        # if packages - redirect to SF
+        if path == "/packages":
+            r = requests.get(
+                "https://sourceforge.net/projects/platformio-storage/files/"
+                "packages/manifest.json/download",
+                params=params, headers=headers)
+        elif data:
+            r = requests.post(__apiurl__ + path, params=params, data=data,
+                              headers=headers)
         else:
-            r = get(__apiurl__ + path, params=params, headers=headers)
+            r = requests.get(__apiurl__ + path, params=params, headers=headers)
         result = r.json()
         r.raise_for_status()
-    except HTTPError as e:
+    except requests.exceptions.HTTPError as e:
         if result and "errors" in result:
             raise APIRequestError(result['errors'][0]['title'])
         else:
             raise APIRequestError(e)
-    except ConnectionError:
-        raise APIRequestError("Could not connect to PlatformIO API Service")
+    except requests.exceptions.ConnectionError:
+        raise APIRequestError(
+            "Could not connect to PlatformIO Registry Service")
     except ValueError:
         raise APIRequestError("Invalid response: %s" % r.text)
     finally:

requirements.txt

@@ -1,5 +1,5 @@
 click==3.3
-colorama==0.3.1
+colorama==0.3.2
 pyserial==2.7
-requests==2.4.1
+requests==2.4.3
 scons==2.3.0

scripts/get-platformio.py

@@ -73,16 +73,20 @@ def install_pip():
 
 
 def install_pypi_packages(packages):
-    for p in packages:
-        print (exec_python_cmd(["-m", "pip", "install", "-U"] + p.split()))
+    for pipargs in packages:
+        print (exec_python_cmd(["-m", "pip", "install", "-U"] + pipargs))
 
 
 def main():
     steps = [
         ("Fixing Windows %PATH% Environment", fix_winpython_pathenv, []),
         ("Installing Python Package Manager", install_pip, []),
-        ("Installing PlatformIO and dependencies", install_pypi_packages,
-         (["platformio", "--egg scons"],)),
+        ("Installing PlatformIO and dependencies", install_pypi_packages, [
+            [["platformio"], [
+                "--egg",
+                "http://sourceforge.net/projects/scons/files/latest/download"
+            ]]
+        ])
     ]
 
     if not IS_WINDOWS:

tox.ini

@@ -15,7 +15,7 @@ deps =
     isort
     flake8
 commands =
-    pip install --egg scons
+    pip install --egg http://sourceforge.net/projects/scons/files/latest/download
 
 [testenv:docs]
 deps =
@@ -23,8 +23,8 @@ deps =
     sphinx_rtd_theme
 commands =
     sphinx-build -W -b html -d {envtmpdir}/doctrees docs docs/_build/html
-    sphinx-build -W -b latex -d {envtmpdir}/doctrees docs docs/_build/latex    
-    /bin/bash -c "if [[ $CI != \\"true\\" ]]; then sphinx-build -W -b linkcheck docs docs/_build/html; fi"    
+    sphinx-build -W -b latex -d {envtmpdir}/doctrees docs docs/_build/latex
+    /bin/bash -c "if [[ $CI != \\"true\\" ]]; then sphinx-build -W -b linkcheck docs docs/_build/html; fi"
 
 [testenv:lint]
 deps =