espressif/esp-idf
1
0
Fork 1
mirror of https://github.com/espressif/esp-idf.git synced 2025-08-07 06:34:34 +02:00
Code Issues Packages Projects Releases Wiki Activity

examples: create example for both Python and CLI parttool interfaces

Browse Source
This commit is contained in:
Renz Christian Bagaporo
2019-05-27 11:09:35 +08:00
parent 1de627e68a
commit 5d41322412
4 changed files with 141 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
examples/storage/parttool/README.md
View File

@@ -4,10 +4,11 @@ This example demonstrates common operations the partitions tool [parttool.py](..
- reading, writing and erasing partitions, - reading, writing and erasing partitions,
- retrieving info on a certain partition, - retrieving info on a certain partition,
- dumping the entire partition table, and - dumping the entire partition table
- generating a blank partition file.
Users taking a look at this example should focus on the contents of the python script [parttool_example.py](parttool_example.py). The script contains programmatic invocations of [parttool.py](../../../components/partition_table/parttool.py) in Python for the operations mentioned above; and can serve as a guide for users wanting to do the same in their applications. Users taking a look at this example should focus on the contents of the Python script [parttool_example.py](parttool_example.py) or shell script [parttool_example.sh](parttool_example.sh). The scripts contain
programmatic invocation of the tool's functions via the Python API and command-line interface, respectively. Note
that on Windows, the shell script example requires a POSIX-compatible environment via MSYS2/Git Bash/WSL etc.
The example performs the operations mentioned above in a straightforward manner: it performs writes to partitions and then verifies correct content The example performs the operations mentioned above in a straightforward manner: it performs writes to partitions and then verifies correct content
by reading it back. For partitions, contents are compared to the originally written file. For the partition table, contents are verified against the partition table CSV by reading it back. For partitions, contents are compared to the originally written file. For the partition table, contents are verified against the partition table CSV
@@ -17,50 +18,54 @@ file. An erased partition's contents is compared to a generated blank file.
### Build and Flash ### Build and Flash
Before running the example script [parttool_example.py](parttool_example.py), it is necessary to build and flash the firmware using the usual means: Before running either of the example scripts, it is necessary to build and flash the firmware using the usual means:
Make:
```bash ```bash
# If using Make
make build flash make build flash
```
# If using CMake CMake:
```bash
idf.py build flash idf.py build flash
``` ```
### Running [parttool_example.py](parttool_example.py) ### Running [parttool_example.py](parttool_example.py)
The example can be executed by running the script [parttool_example.py](parttool_example.py). Either run it directly using The example can be executed by running the script [parttool_example.py](parttool_example.py) or [parttool_example.sh](parttool_example.sh).
```bash
./parttool_example.py
```
or run it using
Python script:
```bash ```bash
python parttool_example.py python parttool_example.py
``` ```
Shell script:
```
./parttool_example.sh
```
The script searches for valid target devices connected to the host and performs the operations on the first one it finds. To perform the operations on a specific device, specify the port it is attached to during script invocation: The script searches for valid target devices connected to the host and performs the operations on the first one it finds. To perform the operations on a specific device, specify the port it is attached to during script invocation:
Python script:
```bash ```bash
# The target device is attached to /dev/ttyUSB2, for example
python parttool_example.py --port /dev/ttyUSB2 python parttool_example.py --port /dev/ttyUSB2
``` ```
Shell script:
```
./parttool_example.sh /dev/ttyUSB2
```
## Example output ## Example output
Running the script produces the following output: Running the script produces the following output:
``` ```
Checking if device app binary matches built binary Checking if device app binary matches built binary
Checking if device partition table matches partition table csv
Retrieving data partition offset and size
Found data partition at offset 0x110000 with size 0x10000 Found data partition at offset 0x110000 with size 0x10000
Writing to data partition Writing to data partition
Reading data partition Reading data partition
Erasing data partition Erasing data partition
Generating blank data partition file
Reading data partition Reading data partition
Partition tool operations performed successfully! Partition tool operations performed successfully!

191
examples/storage/parttool/parttool_example.py
View File

@@ -18,19 +18,12 @@
# limitations under the License. # limitations under the License.
import os import os
import sys import sys
import subprocess
import argparse import argparse
IDF_PATH = os.path.expandvars("$IDF_PATH") PARTITION_TABLE_DIR = os.path.join("components", "partition_table", "")
PARTTOOL_PY = os.path.join(IDF_PATH, "components", "partition_table", "parttool.py")
PARTITION_TABLE_OFFSET = 0x8000
INVOKE_ARGS = [sys.executable, PARTTOOL_PY, "-q", "--partition-table-offset", str(PARTITION_TABLE_OFFSET)]
def sized_file_compare(file1, file2): def assert_file_same(file1, file2, err):
with open(file1, "rb") as f1: with open(file1, "rb") as f1:
with open(file2, "rb") as f2: with open(file2, "rb") as f2:
f1 = f1.read() f1 = f1.read()
@@ -41,121 +34,17 @@ def sized_file_compare(file1, file2):
else: else:
f1 = f1[:len(f2)] f1 = f1[:len(f2)]
return f1 == f2 if not f1 == f2:
raise Exception(err)
def check(condition, message):
if not condition:
print("Error: " + message)
sys.exit(1)
def write_data_partition(size):
print("Writing to data partition")
with open("write.bin", "wb") as f:
# Create a file to write to the data partition with randomly generated content
f.write(os.urandom(int(size, 16)))
# Invokes the command
#
# parttool.py --partition-table-offset 0x8000 -q --partition-name storage write_partition --input write.bin
#
# to write the contents of a file to a partition in the device.
invoke_args = INVOKE_ARGS + ["--partition-name", "storage", "write_partition", "--input", f.name]
subprocess.check_call(invoke_args)
return f.name
def read_data_partition():
print("Reading data partition")
# Invokes the command
#
# parttool.py --partition-table-offset 0x8000 -q --partition-name storage read_partition --output read.bin
#
# to read the contents of a partition in the device, which is then written to a file.
f = "read.bin"
invoke_args = INVOKE_ARGS + ["--partition-name", "storage", "read_partition", "--output", f]
subprocess.check_call(invoke_args)
return f
def get_data_partition_info():
print("Retrieving data partition offset and size")
# Invokes the command
#
# parttool.py --partition-table-offset 0x8000 -q --partition-name storage get_partition_info --info offset size
#
# to get the offset and size of a partition named 'storage'.
invoke_args = INVOKE_ARGS + ["--partition-name", "storage", "get_partition_info", "--info", "offset", "size"]
(offset, size) = subprocess.check_output(invoke_args).strip().split(b" ")
return (offset, size)
def check_app(args):
print("Checking if device app binary matches built binary")
# Invokes the command
#
# parttool.py --partition-table-offset 0x8000 --partition-type app --partition-subtype factory read_partition --output app.bin"
#
# to read the app binary and write it to a file. The read app binary is compared to the built binary in the build folder.
invoke_args = INVOKE_ARGS + ["--partition-type", "app", "--partition-subtype", "factory", "read_partition", "--output", "app.bin"]
subprocess.check_call(invoke_args)
app_same = sized_file_compare("app.bin", args.binary)
check(app_same, "Device app binary does not match built binary")
def check_partition_table():
sys.path.append(os.path.join(IDF_PATH, "components", "partition_table"))
import gen_esp32part as gen
print("Checking if device partition table matches partition table csv")
# Invokes the command
#
# parttool.py --partition-table-offset 0x8000 get_partition_info --table table.bin
#
# to read the device partition table and write it to a file. The read partition table is compared to
# the partition table csv.
invoke_args = INVOKE_ARGS + ["get_partition_info", "--table", "table.bin"]
subprocess.check_call(invoke_args)
with open("table.bin", "rb") as read:
partition_table_csv = os.path.join(IDF_PATH, "examples", "storage", "parttool", "partitions_example.csv")
with open(partition_table_csv, "r") as csv:
read = gen.PartitionTable.from_binary(read.read())
csv = gen.PartitionTable.from_csv(csv.read())
check(read == csv, "Device partition table does not match csv partition table")
def erase_data_partition():
print("Erasing data partition")
# Invokes the command
#
# parttool.py --partition-table-offset 0x8000 --partition-name storage erase_partition
#
# to erase the 'storage' partition.
invoke_args = INVOKE_ARGS + ["--partition-name", "storage", "erase_partition"]
subprocess.check_call(invoke_args)
def generate_blank_data_file():
print("Generating blank data partition file")
# Invokes the command
#
# parttool.py --partition-table-offset 0x8000 --partition-name storage generate_blank_partition_file --output blank.bin
#
# to generate a blank partition file and write it to a file. The blank partition file has the same size as the
# 'storage' partition.
f = "blank.bin"
invoke_args = INVOKE_ARGS + ["--partition-name", "storage", "generate_blank_partition_file", "--output", f]
subprocess.check_call(invoke_args)
return f
def main(): def main():
global INVOKE_ARGS COMPONENTS_PATH = os.path.expandvars(os.path.join("$IDF_PATH", "components"))
PARTTOOL_DIR = os.path.join(COMPONENTS_PATH, "partition_table")
sys.path.append(PARTTOOL_DIR)
from parttool import PartitionName, PartitionType, ParttoolTarget
from gen_empty_partition import generate_blanked_file
parser = argparse.ArgumentParser("ESP-IDF Partitions Tool Example") parser = argparse.ArgumentParser("ESP-IDF Partitions Tool Example")
@@ -164,43 +53,53 @@ def main():
args = parser.parse_args() args = parser.parse_args()
if args.port: target = ParttoolTarget(args.port)
INVOKE_ARGS += ["--port", args.port]
# Before proceeding, do checks to verify whether the app and partition table in the device matches the built binary and # Read app partition and save the contents to a file. The app partition is identified
# the generated partition table during build # using type-subtype combination
check_app(args) print("Checking if device app binary matches built binary")
check_partition_table() factory = PartitionType("app", "factory")
target.read_partition(factory, "app.bin")
assert_file_same(args.binary, "app.bin", "Device app binary does not match built binary")
# Get the offset and size of the data partition # Retrieve info on data storage partition, this time identifying it by name.
(offset, size) = get_data_partition_info() storage = PartitionName("storage")
storage_info = target.get_partition_info(storage)
print("Found data partition at offset 0x{:x} with size 0x{:x}".format(storage_info.offset, storage_info.size))
print("Found data partition at offset %s with size %s" % (offset, size)) # Create a file whose contents will be written to the storage partition
with open("write.bin", "wb") as f:
# Create a file to write to the data partition with randomly generated content
f.write(os.urandom(storage_info.size))
# Write a generated file of random bytes to the found data partition # Write the contents of the created file to storage partition
written = write_data_partition(size) print("Writing to data partition")
target.write_partition(storage, "write.bin")
# Read back the contents of the data partition # Read back the contents of the storage partition
read = read_data_partition() print("Reading data partition")
target.read_partition(storage, "read.bin")
# Compare the written and read back data assert_file_same("write.bin", "read.bin", "Read contents of storage partition does not match source file contents")
data_same = sized_file_compare(read, written)
check(data_same, "Read contents of the data partition does not match written data")
# Erase the data partition # Erase contents of the storage partition
erase_data_partition() print("Erasing data partition")
target.erase_partition(storage)
# Read back the erase data partition, which should be all 0xFF's after erasure # Read back the erased data partition
read = read_data_partition() print("Reading data partition")
target.read_partition(storage, "read.bin")
# Generate blank partition file (all 0xFF's) # Generate a file of all 0xFF
blank = generate_blank_data_file() generate_blanked_file(storage_info.size, "blank.bin")
# Verify that the partition has been erased by comparing the contents to the generated blank file assert_file_same("blank.bin", "read.bin", "Contents of storage partition not fully erased")
data_same = sized_file_compare(read, blank)
check(data_same, "Erased data partition contents does not match blank partition file")
# Example end and cleanup
print("\nPartition tool operations performed successfully!") print("\nPartition tool operations performed successfully!")
clean_files = ["app.bin", "read.bin", "blank.bin", "write.bin"]
for clean_file in clean_files:
os.unlink(clean_file)
if __name__ == '__main__': if __name__ == '__main__':

73
examples/storage/parttool/parttool_example.sh Normal file
View File

@@ -0,0 +1,73 @@
#!/bin/bash
#
# Demonstrates command-line interface of Partition Tool, parttool.py
#
#
# $1 - serial port where target device to operate on is connnected to, by default the first found valid serial port
# $2 - path to this example's built binary file (parttool.bin), by default $PWD/build/parttool.bin
PORT=$1
PARTTOOL_PY="python $IDF_PATH/components/partition_table/parttool.py -q"
if [[ "$PORT" != "" ]]; then
PARTTOOL_PY="$PARTTOOL_PY --port $PORT"
fi
GEN_EMPTY_PARTITION_PY="python $IDF_PATH/components/partition_table/gen_empty_partition.py"
BINARY=$2
if [[ "$BINARY" == "" ]]; then
BINARY=build/parttool.bin
fi
function assert_file_same()
{
sz_a=$(stat -c %s $1)
sz_b=$(stat -c %s $2)
sz=$((sz_a < sz_b ? sz_a : sz_b))
res=$(cmp -s -n $sz $1 $2) ||
(echo "!!!!!!!!!!!!!!!!!!!"
echo "FAILURE: $3"
echo "!!!!!!!!!!!!!!!!!!!")
}
# Read app partition and save the contents to a file. The app partition is identified
# using type-subtype combination
echo "Checking if device app binary matches built binary"
$PARTTOOL_PY read_partition --partition-type=app --partition-subtype=factory --output=app.bin
assert_file_same app.bin $BINARY "Device app binary does not match built binary"
# Retrieve info on data storage partition, this time identifying it by name.
offset=$($PARTTOOL_PY get_partition_info --partition-name=storage --info offset)
size=$($PARTTOOL_PY get_partition_info --partition-name=storage --info size)
echo "Found data partition at offset $offset with size $size"
# Create a file whose contents will be written to the storage partition
head -c $(($size)) /dev/urandom > write.bin
# Write the contents of the created file to storage partition
echo "Writing to data partition"
$PARTTOOL_PY write_partition --partition-name=storage --input write.bin
# Read back the contents of the storage partition
echo "Reading data partition"
$PARTTOOL_PY read_partition --partition-name=storage --output read.bin
assert_file_same write.bin read.bin "Read contents of storage partition does not match source file contents"
# Erase contents of the storage partition
echo "Erasing data partition"
$PARTTOOL_PY erase_partition --partition-name=storage
# Read back the erased data partition
echo "Reading data partition"
$PARTTOOL_PY read_partition --partition-name=storage --output read.bin
# Generate a file of all 0xFF
$GEN_EMPTY_PARTITION_PY $(($size)) blank.bin
assert_file_same read.bin blank.bin "Contents of storage partition not fully erased"
# Example end and cleanup
printf "\nPartition tool operations performed successfully\n"
rm -rf app.bin read.bin blank.bin write.bin

1
tools/ci/executable-list.txt
View File

@@ -22,6 +22,7 @@ examples/build_system/cmake/idf_as_lib/build.sh
examples/build_system/cmake/idf_as_lib/run-esp32.sh examples/build_system/cmake/idf_as_lib/run-esp32.sh
examples/build_system/cmake/idf_as_lib/run.sh examples/build_system/cmake/idf_as_lib/run.sh
examples/storage/parttool/parttool_example.py examples/storage/parttool/parttool_example.py
examples/storage/parttool/parttool_example.sh
examples/system/ota/otatool/otatool_example.py examples/system/ota/otatool/otatool_example.py
tools/check_kconfigs.py tools/check_kconfigs.py
tools/check_python_dependencies.py tools/check_python_dependencies.py