Significantly improve "Project Generator" // Resolve #280

HISTORY.rst

@@ -20,14 +20,12 @@ PlatformIO 2.0
 * Simplified `installation <http://docs.platformio.org/en/latest/installation.html>`__
   process of PlatformIO
   (`issue #274 <https://github.com/platformio/platformio/issues/274>`_)
-* Added support for Adafruit Gemma board to
+* Significantly improved `Project Generator <http://docs.platformio.org/en/latest/userguide/cmd_init.html#cmdoption-platformio-init--ide>`__ which allows to integrate with `the most popular
-  `atmelavr <http://docs.platformio.org/en/latest/platforms/atmelavr.html#boards>`__
+  IDE <http://docs.platformio.org/en/latest/ide.html>`__
-  platform
-  (`pull #256 <https://github.com/platformio/platformio/pull/256>`_)
 * Added short ``-h`` help option for PlatformIO and sub-commands
 * Updated `mbed <http://docs.platformio.org/en/latest/frameworks/mbed.html>`__
   framework
-* Updated ``tool-teensy`` for `Teensy <http://docs.platformio.org/en/latest/platforms/teensy.html>`__
+* Updated ``tool-teensy`` package for `Teensy <http://docs.platformio.org/en/latest/platforms/teensy.html>`__
   platform
   (`issue #268 <https://github.com/platformio/platformio/issues/268>`_)
 * Added FAQ answer when `Program "platformio" not found in PATH <http://docs.platformio.org/en/latest/faq.html#faq-troubleshooting-pionotfoundinpath>`_
@@ -36,6 +34,10 @@ PlatformIO 2.0
   (`issue #273 <https://github.com/platformio/platformio/issues/273>`_)
 * Use toolchain's includes pattern ``include*`` for Project Generator
   (`issue #277 <https://github.com/platformio/platformio/issues/277>`_)
+* Added support for Adafruit Gemma board to
+  `atmelavr <http://docs.platformio.org/en/latest/platforms/atmelavr.html#boards>`__
+  platform
+  (`pull #256 <https://github.com/platformio/platformio/pull/256>`_)
 * Fixed includes list for Windows OS when generating project for `Eclipse IDE <http://docs.platformio.org/en/latest/ide/eclipse.html>`__
   (`issue #270 <https://github.com/platformio/platformio/issues/270>`_)
 * Fixed ``AttributeError: 'module' object has no attribute 'packages'``

docs/ide/atom.rst

@@ -20,11 +20,6 @@ page for more detailed information.
 Integration
 -----------
 
-.. note::
-    Please verify that folder where is located ``platformio`` program is added
-    to `PATH (wiki) <https://en.wikipedia.org/wiki/PATH_(variable)>`_ environment
-    variable. See FAQ: :ref:`faq_troubleshooting_pionotfoundinpath`.
-
 Using `Atom Packages <https://atom.io/docs/v1.0.2/using-atom-atom-packages>`_
 please install `platomformio <https://atom.io/packages/platomformio>`_ package.
 

docs/ide/clion.rst

@@ -22,11 +22,6 @@ page for more detailed information.
 Integration
 -----------
 
-.. note::
-    Please verify that folder where is located ``platformio`` program is added
-    to `PATH (wiki) <https://en.wikipedia.org/wiki/PATH_(variable)>`_ environment
-    variable. See FAQ: :ref:`faq_troubleshooting_pionotfoundinpath`.
-
 Project Generator
 ^^^^^^^^^^^^^^^^^
 
@@ -41,6 +36,12 @@ Since PlatformIO 2.0 you can generate CLion compatible project using
 Then import this project from start menu or via ``File > Import Project>`` and
 specify root directory where is located :ref:`projectconf`.
 
+There are 3 predefined targets for building:
+
+* ``PLATFORMIO_BUILD`` - build project without auto-uploading
+* ``PLATFORMIO_UPLOAD`` - build and upload (if no errors)
+* ``PLATFORMIO_CLEAN`` - clean compiled objects and etc.
+
 .. warning::
     CLion is still in the development stage, so some of the features (like,
     auto-complete) probably will not work with PlatformIO. See

docs/ide/eclipse.rst

@@ -26,17 +26,6 @@ page for more detailed information.
 Integration
 -----------
 
-.. note::
-    Please verify that folder where is located ``platformio`` program is added
-    to `PATH (wiki) <https://en.wikipedia.org/wiki/PATH_(variable)>`_ environment
-    variable. See FAQ: :ref:`faq_troubleshooting_pionotfoundinpath`.
-
-    What is more, you can update Eclipse ``$PATH`` variable. In Eclipse IDE make
-    right click on the created project ("Project Explorer"):
-    ``Properties > C/C++ Build > Environment > PATH > Edit``
-    and set to ``/usr/local/bin:/bin:/usr/bin:/usr/sbin:/sbin``. If ``PATH``
-    variable doesn't exists, please ``Add`` it.
-
 Project Generator
 ^^^^^^^^^^^^^^^^^
 

docs/ide/sublimetext.rst

@@ -18,11 +18,6 @@ page for more detailed information.
 Integration
 -----------
 
-.. note::
-    Please verify that folder where is located ``platformio`` program is added
-    to `PATH (wiki) <https://en.wikipedia.org/wiki/PATH_(variable)>`_ environment
-    variable. See FAQ: :ref:`faq_troubleshooting_pionotfoundinpath`.
-
 Project Generator
 ^^^^^^^^^^^^^^^^^
 
@@ -40,6 +35,11 @@ directory where is located :ref:`projectconf`.
 Manual Integration
 ^^^^^^^^^^^^^^^^^^
 
+.. note::
+    Please verify that folder where is located ``platformio`` program is added
+    to `PATH (wiki) <https://en.wikipedia.org/wiki/PATH_(variable)>`_ environment
+    variable. See FAQ: :ref:`faq_troubleshooting_pionotfoundinpath`.
+
 Initial configuration
 ~~~~~~~~~~~~~~~~~~~~~
 
@@ -50,17 +50,17 @@ described below:
 .. code-block:: bash
 
     {
-        "cmd": ["platformio", "--force", "run"],
+        "cmd": ["platformio", "-f", "-c", "sublimetext", "run"],
         "working_dir": "${project_path:${folder}}",
         "variants":
         [
             {
                 "name": "Clean",
-                "cmd": ["platformio", "--force", "run", "--target", "clean"]
+                "cmd": ["platformio", "-f", "-c", "sublimetext", "run", "--target", "clean"]
             },
             {
                 "name": "Upload",
-                "cmd": ["platformio", "--force", "run", "--target", "upload"]
+                "cmd": ["platformio", "-f", "-c", "sublimetext", "run", "--target", "upload"]
             }
         ]
     }

docs/ide/vim.rst

@@ -35,10 +35,10 @@ Put to the project directory ``Makefile`` wrapper with contents:
     #PATH := /usr/local/bin:$(PATH)
 
     all:
-        platformio --force run --target upload
+        platformio -f -c vim run --target upload
 
     clean:
-        platformio --force run --target clean
+        platformio -f -c vim run --target clean
 
 
 Now, in VIM ``cd /path/to/this/project`` and press ``Ctrl+B`` or ``Cmd+B``

platformio/__init__.py

@@ -1,7 +1,7 @@
 # Copyright (C) Ivan Kravets <me@ikravets.com>
 # See LICENSE for details.
 
-VERSION = (2, 3, "0b1")
+VERSION = (2, 3, "0b2")
 __version__ = ".".join([str(s) for s in VERSION])
 
 __title__ = "platformio"

platformio/builder/tools/piomisc.py

@@ -7,7 +7,7 @@ from glob import glob
 from os import environ, remove
 from os.path import basename, join
 
-from platformio.util import exec_command
+from platformio.util import exec_command, where_is_program
 
 
 class InoToCPPConverter(object):
@@ -122,7 +122,9 @@ def ConvertInoToCpp(env):
 def DumpIDEData(env):
     data = {
         "defines": [],
-        "includes": []
+        "includes": [],
+        "cxx_path": where_is_program(
+            env.subst("$CXX"), env.subst("${ENV['PATH']}"))
     }
 
     # includes from framework and libs

platformio/ide/projectgenerator.py

@@ -2,7 +2,7 @@
 # See LICENSE for details.
 
 import json
-from os import listdir, makedirs, walk
+import os
 from os.path import abspath, basename, expanduser, isdir, join, relpath
 
 import bottle
@@ -23,7 +23,7 @@ class ProjectGenerator(object):
     @staticmethod
     def get_supported_ides():
         tpls_dir = join(util.get_source_dir(), "ide", "tpls")
-        return sorted([d for d in listdir(tpls_dir)
+        return sorted([d for d in os.listdir(tpls_dir)
                        if isdir(join(tpls_dir, d))])
 
     @util.memoized
@@ -43,25 +43,30 @@ class ProjectGenerator(object):
 
     @util.memoized
     def get_project_build_data(self):
+        data = {
+            "defines": [],
+            "includes": [],
+            "cxx_path": None
+        }
         envdata = self.get_project_env()
         if "env_name" not in envdata:
-            return None
+            return data
         result = util.exec_command(
             ["platformio", "run", "-t", "idedata", "-e", envdata['env_name'],
              "--project-dir", self.project_dir]
         )
-        if result['returncode'] != 0 or '{"includes":' not in result['out']:
+        if result['returncode'] != 0 or '"includes":' not in result['out']:
-            return None
+            return data
 
         output = result['out']
         try:
-            start_index = output.index('\n{"includes":')
+            start_index = output.index('\n{"')
             stop_index = output.rindex('}')
-            return json.loads(output[start_index + 1:stop_index + 1])
+            data = json.loads(output[start_index + 1:stop_index + 1])
         except ValueError:
             pass
 
-        return None
+        return data
 
     def get_project_name(self):
         return basename(self.project_dir)
@@ -69,7 +74,7 @@ class ProjectGenerator(object):
     def get_srcfiles(self):
         result = []
         with util.cd(self.project_dir):
-            for root, _, files in walk(util.get_projectsrc_dir()):
+            for root, _, files in os.walk(util.get_projectsrc_dir()):
                 for f in files:
                     result.append(relpath(join(root, f)))
         return result
@@ -77,20 +82,23 @@ class ProjectGenerator(object):
     def get_tpls(self):
         tpls = []
         tpls_dir = join(util.get_source_dir(), "ide", "tpls", self.ide)
-        for root, _, files in walk(tpls_dir):
+        for root, _, files in os.walk(tpls_dir):
             for f in files:
-                if f.endswith(".tpl"):
+                if not f.endswith(".tpl"):
-                    tpls.append((
+                    continue
-                        root.replace(tpls_dir, ""), join(root, f)))
+                _relpath = root.replace(tpls_dir, "")
+                if _relpath.startswith(os.sep):
+                    _relpath = _relpath[len(os.sep):]
+                tpls.append((_relpath, join(root, f)))
         return tpls
 
     def generate(self):
         for _relpath, _path in self.get_tpls():
             tpl_dir = self.project_dir
             if _relpath:
-                tpl_dir = join(self.project_dir, _relpath)[1:]
+                tpl_dir = join(self.project_dir, _relpath)
                 if not isdir(tpl_dir):
-                    makedirs(tpl_dir)
+                    os.makedirs(tpl_dir)
 
             file_name = basename(_path)[:-4]
             with open(join(tpl_dir, file_name), "w") as f:
@@ -104,17 +112,14 @@ class ProjectGenerator(object):
 
     def _gather_tplvars(self):
         self._tplvars.update(self.get_project_env())
-
+        self._tplvars.update(self.get_project_build_data())
-        build_data = self.get_project_build_data()
-
         self._tplvars.update({
             "project_name": self.get_project_name(),
-            "includes": (build_data['includes']
-                         if build_data and "includes" in build_data else []),
-            "defines": (build_data['defines']
-                        if build_data and "defines" in build_data else []),
             "srcfiles": self.get_srcfiles(),
             "user_home_dir": abspath(expanduser("~")),
             "project_dir": self.project_dir,
-            "systype": util.get_systype()
+            "systype": util.get_systype(),
+            "platformio_path": util.where_is_program("platformio"),
+            "env_pathsep": os.pathsep,
+            "env_path": os.getenv("PATH")
         })

platformio/ide/tpls/clion/CMakeLists.txt.tpl

@@ -1,5 +1,9 @@
 cmake_minimum_required(VERSION 3.2)
 project({{project_name}})
+
+set(ENV{PATH} "{{env_path}}")
+set(PLATFORMIO_CMD "{{platformio_path}}")
+
 % for include in includes:
 % if include.startswith(user_home_dir):
 include_directories("$ENV{HOME}{{include.replace(user_home_dir, '').replace("\\", "/")}}")
@@ -14,18 +18,18 @@ add_definitions(-D{{define}})
 
 add_custom_target(
     PLATFORMIO_BUILD ALL
-    COMMAND platformio -f run
+    COMMAND ${PLATFORMIO_CMD} -f -c clion run
     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
 )
 
 add_custom_target(
     PLATFORMIO_UPLOAD ALL
-    COMMAND platformio -f run --target upload
+    COMMAND ${PLATFORMIO_CMD} -f -c clion run --target upload
     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
 )
 
 add_custom_target(
     PLATFORMIO_CLEAN ALL
-    COMMAND platformio -f run --target clean
+    COMMAND ${PLATFORMIO_CMD} -f -c clion run --target clean
     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
 )

platformio/ide/tpls/eclipse/.cproject.tpl

@@ -19,7 +19,7 @@
 					<folderInfo id="0.910961921." name="/" resourcePath="">
 						<toolChain id="org.eclipse.cdt.build.core.prefbase.toolchain.952979152" name="No ToolChain" resourceTypeBasedDiscovery="false" superClass="org.eclipse.cdt.build.core.prefbase.toolchain">
 							<targetPlatform binaryParser="org.eclipse.cdt.core.ELF" id="org.eclipse.cdt.build.core.prefbase.toolchain.952979152.52310970" name=""/>
-							<builder cleanBuildTarget="--force run --target clean" command="platformio" id="org.eclipse.cdt.build.core.settings.default.builder.1519453406" incrementalBuildTarget="--force run" keepEnvironmentInBuildfile="false" managedBuildOn="false" name="Gnu Make Builder" superClass="org.eclipse.cdt.build.core.settings.default.builder"/>
+							<builder cleanBuildTarget="-f -c eclipse run --target clean" command="platformio" id="org.eclipse.cdt.build.core.settings.default.builder.1519453406" incrementalBuildTarget="-f -c eclipse run" keepEnvironmentInBuildfile="false" managedBuildOn="false" name="Gnu Make Builder" superClass="org.eclipse.cdt.build.core.settings.default.builder"/>
 							<tool id="org.eclipse.cdt.build.core.settings.holder.libs.1409095472" name="holder for library settings" superClass="org.eclipse.cdt.build.core.settings.holder.libs"/>
 							<tool id="org.eclipse.cdt.build.core.settings.holder.1624502120" name="Assembly" superClass="org.eclipse.cdt.build.core.settings.holder">
 								<option id="org.eclipse.cdt.build.core.settings.holder.incpaths.239157887" name="Include Paths" superClass="org.eclipse.cdt.build.core.settings.holder.incpaths" valueType="includePath">

platformio/ide/tpls/eclipse/.settings/language.settings.xml.tpl

@@ -0,0 +1,18 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<project>
+	<configuration id="0.910961921" name="Default">
+		<extension point="org.eclipse.cdt.core.LanguageSettingsProvider">
+			<provider copy-of="extension" id="org.eclipse.cdt.ui.UserLanguageSettingsProvider"/>
+			<provider-reference id="org.eclipse.cdt.core.ReferencedProjectsLanguageSettingsProvider" ref="shared-provider"/>
+			<provider-reference id="org.eclipse.cdt.managedbuilder.core.MBSLanguageSettingsProvider" ref="shared-provider"/>
+			% if "windows" in systype:
+			<provider class="org.eclipse.cdt.internal.build.crossgcc.CrossGCCBuiltinSpecsDetector" console="false" env-hash="-869785120007741010" id="org.eclipse.cdt.build.crossgcc.CrossGCCBuiltinSpecsDetector" keep-relative-paths="false" name="CDT Cross GCC Built-in Compiler Settings" parameter="${USERPROFILE}{{cxx_path.replace(user_home_dir, '')}} ${FLAGS} -E -P -v -dD &quot;${INPUTS}&quot;" prefer-non-shared="true" store-entries-with-project="false">
+			% else:
+			<provider class="org.eclipse.cdt.internal.build.crossgcc.CrossGCCBuiltinSpecsDetector" console="false" env-hash="-869785120007741010" id="org.eclipse.cdt.build.crossgcc.CrossGCCBuiltinSpecsDetector" keep-relative-paths="false" name="CDT Cross GCC Built-in Compiler Settings" parameter="${HOME}{{cxx_path.replace(user_home_dir, '')}} ${FLAGS} -E -P -v -dD &quot;${INPUTS}&quot;" prefer-non-shared="true" store-entries-with-project="false">
+			% end
+				<language-scope id="org.eclipse.cdt.core.gcc"/>
+				<language-scope id="org.eclipse.cdt.core.g++"/>
+			</provider>
+		</extension>
+	</configuration>
+</project>

platformio/ide/tpls/eclipse/.settings/org.eclipse.cdt.core.prefs.tpl

@@ -0,0 +1,6 @@
+eclipse.preferences.version=1
+environment/project/0.910961921/PATH/delimiter=\{{env_pathsep}}
+environment/project/0.910961921/PATH/operation=replace
+environment/project/0.910961921/PATH/value={{env_path.replace(env_pathsep, "\\%s" % env_pathsep)}}
+environment/project/0.910961921/append=true
+environment/project/0.910961921/appendContributed=false

platformio/ide/tpls/sublimetext/platformio.sublime-project.tpl

@@ -5,9 +5,10 @@
 			"cmd":
 			[
 				"platformio",
-				"--force",
+				"-f", "-c", "sublimetext",
 				"run"
 			],
+			"path": "{{env_path}}"
 			"name": "PlatformIO",
 			"variants":
 			[
@@ -15,7 +16,7 @@
 					"cmd":
 					[
 						"platformio",
-						"--force",
+						"-f", "-c", "sublimetext",
 						"run",
 						"--target",
 						"clean"
@@ -26,7 +27,7 @@
 					"cmd":
 					[
 						"platformio",
-						"--force",
+						"-f", "-c", "sublimetext",
 						"run",
 						"--target",
 						"upload"

platformio/ide/tpls/visualstudio/platformio.vcxproj.tpl

@@ -1,5 +1,8 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <PropertyGroup>
+    <Path>{{env_path}}</Path>
+  </PropertyGroup>
   <ItemGroup Label="ProjectConfigurations">
     <ProjectConfiguration Include="Debug|Win32">
       <Configuration>Debug</Configuration>
@@ -36,8 +39,8 @@
   </ImportGroup>
   <PropertyGroup Label="UserMacros" />
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
-    <NMakeBuildCommandLine>platformio --force run</NMakeBuildCommandLine>
+    <NMakeBuildCommandLine>platformio -f -c visualstudio run</NMakeBuildCommandLine>
-    <NMakeCleanCommandLine>platformio --force run --target clean</NMakeCleanCommandLine>
+    <NMakeCleanCommandLine>platformio -f -c visualstudio run --target clean</NMakeCleanCommandLine>
     <NMakePreprocessorDefinitions>{{";".join(defines)}}</NMakePreprocessorDefinitions>
     <NMakeIncludeSearchPath>{{";".join(["$(HOMEDRIVE)$(HOMEPATH)%s" % i.replace(user_home_dir, "") if i.startswith(user_home_dir) else i for i in includes])}}</NMakeIncludeSearchPath>
   </PropertyGroup>

platformio/util.py

@@ -376,3 +376,29 @@ def get_frameworks(type_=None):
         return frameworks[type_]
 
     return frameworks
+
+
+def where_is_program(program, envpath=None):
+    env = os.environ
+    if envpath:
+        env['PATH'] = envpath
+
+    # try OS's built-in commands
+    try:
+        result = exec_command(
+            ["where" if "windows" in get_systype() else "which", program],
+            env=env
+        )
+        if result['returncode'] == 0 and isfile(result['out'].strip()):
+            return result['out'].strip()
+    except OSError:
+        pass
+
+    # look up in $PATH
+    for bin_dir in env.get("PATH", "").split(os.pathsep):
+        if isfile(join(bin_dir, program)):
+            return join(bin_dir, program)
+        elif isfile(join(bin_dir, "%s.exe" % program)):
+            return join(bin_dir, "%s.exe" % program)
+
+    return program