feedc0de/esp-idf
1
0
Fork 0
forked from espressif/esp-idf
Code Pull Requests Activity

refactor(usb/host): Update UVC example for new UVC driver v2.0

Browse Source
This commit is contained in:
Tomas Rezucha
2025-04-10 13:37:04 +02:00
parent dc678decf7
commit 1816b95261
14 changed files with 243 additions and 841 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/en/api-reference/peripherals/usb_host.rst
View File

@@ -403,7 +403,7 @@ UVC
"""
* A host class driver for the USB Video Device Class is distributed as a managed component via the `ESP Component Registry <https://components.espressif.com/component/espressif/usb_host_uvc>`__.
* :example:`peripherals/usb/host/uvc` demonstrates how to capture video from a USB camera using the `libuvc` library and stream the video over Wi-Fi by hosting a TCP server, with the option to visualize the captured video on a PC using the provided `player.py` script.
* :example:`peripherals/usb/host/uvc` demonstrates how to capture video frames from a USB camera using the UVC driver.
.. ---------------------------------------------- USB Host Menuconfig --------------------------------------------------

204
examples/peripherals/usb/host/uvc/README.md
View File

@@ -5,16 +5,9 @@
## Overview
This example demonstrates how to:
This example shows basic usage of [USB UVC host driver](https://components.espressif.com/components/espressif/usb_host_uvc). The example waits for USB camera connection, it then configures the camera and starts streaming video frames until the device is removed.
- Capture video from a USB camera using the `libuvc` library.
- Stream the video over WiFi by hosting a TCP server.
The example enumerates a connected USB camera, negotiates a selected resolution along with an associated `FPS`, then starts capturing video. The `frame_callback` function is then invoked after receiving each frame. Users can process the received frames according to their needs.
Optionally, the captured video can be visualized on a PC with help of the `player.py` script provided in this example. After setting the `Example Configuration->Enable streaming` option in menuconfig, the example will create a TCP server upon startup, and waits until `player.py` connects to the server. Once a connection is established, the example streams each received frame to the PC for visualization. The network connection can be configured in menuconfig via `Example Connection Configuration`.
**Notice** that `libuvc` selects highest possible `dwMaxPayloadTransferSize` by default. As a result, this example will manually overwrite this value to 512 bytes (the maximum packet size supported by ESP32-S2/S3).
More advanced examples can be found in [UVC host driver repository](https://github.com/espressif/esp-usb/tree/master/host/class/uvc/usb_host_uvc/examples).
(See the README.md file in the upper level 'examples' directory for more information about examples.)
@@ -26,52 +19,18 @@ Optionally, the captured video can be visualized on a PC with help of the `playe
* Exposed USB host connector
* USB camera
Running this example on an **ESP module without external PSRAM will fail on initialization**. Please select your PSRAM configuration in menuconfig `Component config->ESP PSRAM`. If you manually disable PSRAM, the required framebuffers might not fit into DRAM (especially on ESP32-S2).
Running this example on an **ESP module without external PSRAM will fail on initialization**. Please select your PSRAM configuration in menuconfig `Component config->ESP PSRAM`. If you manually disable PSRAM, the required frame buffers might not fit into DRAM (especially on ESP32-S2). If you absolutely must not use PSRAM, consider the following for minimizing RAM usage:
* Select smaller resolution (eg. 320x240)
* Select fewer frame buffers (eg. 2)
* Select smaller and fewer URBs (eg. 2x 2kB)
Follow instructions in [examples/usb/README.md](../../README.md) for specific hardware setup.
### Configure the project
### Frame format
Following configuration is needed for streaming video:
You can select frame format in [main.c](./main/main.c) file in `stream_config` variable. Default is set to very common 640x480@15FPS, MJPEG encoded.
Open the project configuration menu (`idf.py menuconfig`).
In the `Example Connection Configuration` menu:
* Set the Wi-Fi configuration.
* Set `WiFi SSID`.
* Set `WiFi Password`.
In the `Example Configuration` menu:
* Set the Example configuration
* `Enable streaming`
* Select one of UVC Protocol Mode
* `Auto`
* `Custom`
Optional: If you need, change the other options according to your requirements.
Additionally, the `player.py` python script makes use of the `opencv-python` and `numpy` packages which are not included in the `idf-env` environment by default. Run following command to install those packages:
```bash
pip install opencv-python numpy
```
#### UVC Protocol Mode: Auto
When the protocol mode is set to Auto, the example will make three attempts to negotiate the protocol with following parameters:
1. Attempt: 640x480, 15 FPS, MJPEG
2. Attempt: 320x240, 30 FPS, MJPEG
3. Attempt: 320x240, first available FPS, MJPEG
If all three attempts result in an error, the example displays the error message and suggests to try another USB UVC Device.
#### UVC Protocol Mode: Custom
When the protocol mode set to Custom, the example tries to negotiate the protocol using user provided values for the following parameters: Attempts, Width, Height, FPS, and Frame Coding format. After all attempts result in an error, the example displays the error message and suggests to try another USB UVC device.
This example will print supported formats on device connection - this can be disabled in menuconfig ` Component config → USB HOST UVC`. The frame format can also be changed in runtime by calling `uvc_host_stream_format_select()` function.
### Build and Flash
@@ -87,128 +46,51 @@ See the Getting Started Guide for all the steps to configure and use the ESP-IDF
## Known limitations
Having only a Full Speed USB peripheral and hardware limited MPS (maximum packet size) to 512 bytes, the ESP32-S2/S3 is capable of reading data at approximately 0.5 MB/s. When connected to Full Speed USB host, cameras normally provide resolution no larger than 640x480 pixels. The following two formats are the most commonly supported at Full Speed (both encoded in MJPEG):
* 320x240 30 FPS
* 640x480 15 FPS
On targets with Full Speed USB peripheral the MPS (maximum packet size) is limited to 512 bytes, the ESP32-S2/S3 is capable of reading data at approximately 0.5 MB/s. When connected to Full Speed USB host, cameras usually provide resolution no larger than 640x480 pixels. The following two formats are the most commonly supported at Full Speed :
* 320x240 30 FPS MJPEG
* 640x480 15 FPS MJPEG
## Tested cameras
* Logitech C980
* Logitech C270
* Logitech C170
* Logitech Brio 100
* CANYON CNE-CWC2
* Trust WebCam
* Anker Powerconf C200
* Microsoft LifeCam HD-6000
* Asus Webcam C3
## Example Output
Output with USB camera already connected at program start:
```
...
I (1186) example: Waiting for USB UVC device connection ...
I (1606) example: Device found
DEVICE CONFIGURATION (0c45:6340/ S) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 129
Formats:
MJPEGFormat(1)
bits per pixel: 0
GUID: 4d4a5047000000000000000000000000 (MJPG)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 640x480
bit rate: 24576000-147456000
max frame size: 614400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(2)
capabilities: 00
size: 352x288
bit rate: 8110080-48660480
max frame size: 202752
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(3)
capabilities: 00
size: 320x240
bit rate: 6144000-36864000
max frame size: 153600
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(4)
capabilities: 00
size: 176x144
bit rate: 2027520-12165120
max frame size: 50688
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(5)
capabilities: 00
size: 160x120
bit rate: 1536000-9216000
max frame size: 38400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
StillFrameDescriptor
bEndPointAddress: 00
wWidth(1) = 640
wHeight(1) = 480
wWidth(2) = 352
wHeight(2) = 288
wWidth(3) = 320
wHeight(3) = 240
wWidth(4) = 176
wHeight(4) = 144
wWidth(5) = 160
wHeight(5) = 120
END DEVICE CONFIGURATION
I (1796) example: Negotiate streaming profile 640x480, fps 15 ...
I (1816) example: Negotiation complete.
bmHint: 0001
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 23469
dwMaxVideoFrameSize: 614400
dwMaxPayloadTransferSize: 512
bInterfaceNumber: 1
I (1836) example: Streaming...
I (4016) example: fps: 10, bytes per second: 69920
I (1460) main_task: Calling app_main()
I (1460) UVC example: Installing USB Host
I (1500) UVC example: Installing UVC driver
I (1500) UVC example: Opening UVC device 0x0000:0x0000 640x480@15.0FPS...
I (1500) main_task: Returned from app_main()
*** Configuration descriptor ***
...
Here is rest of Configuration descriptor
...
I (44916) example: fps: 9, bytes per second: 62928
E (45626) USBH: Device 1 gone
I (45636) example: Done streaming.
I (3140) UVC example: UVC Device OPENED!
I (3240) UVC example: Stream start. Iteration 0
I (4410) UVC example: New frame! Len: 13392
I (4470) UVC example: New frame! Len: 13392
I (4530) UVC example: New frame! Len: 13392
...
More frames received
...
I (8310) UVC example: New frame! Len: 18512
I (8310) UVC example: Stream stop
I (10440) UVC example: Stream start. Iteration 1
...
Streaming continues here
```

8
examples/peripherals/usb/host/uvc/main/CMakeLists.txt
View File

@@ -1,11 +1,5 @@
idf_component_register(SRCS "main.c" "tcp_server.c"
INCLUDE_DIRS ""
idf_component_register(SRCS "main.c"
PRIV_REQUIRES
nvs_flash
usb
esp_ringbuf
esp_psram # Required for CONFIG_SPIRAM
esp_timer
esp_wifi
esp_driver_gpio
)

106
examples/peripherals/usb/host/uvc/main/Kconfig.projbuild
View File

@@ -1,106 +0,0 @@
menu "Example Configuration"
config EXAMPLE_ENABLE_STREAMING
bool "Enable streaming"
default n
help
Enables streaming of captured video.
choice EXAMPLE_UVC_PROTOCOL_MODE
prompt "UVC Protocol mode"
default EXAMPLE_UVC_PROTOCOL_MODE_AUTO
config EXAMPLE_UVC_PROTOCOL_MODE_AUTO
bool "Auto"
help
When protocol mode set to Auto, the example tries to make three attempts to negotiate
the protocol with following parameters:
1 Attempt: 640x480, 15 FPS, MJPEG
2 Attempt: 320x240, 30 FPS, MJPEG
3 Attempt: 320x240, first available FPS, MJPEG
If all three attempts result in an error, the example displays the error message and
suggests to try another USB UVC Device.
config EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM
bool "Custom"
help
When protocol mode set to Custom, the example tries to negotiate protocol with
configured parameters: Attempts, Width, Height, FPS, Frame Coding format.
After all attempts result in an error, the example displays the error message and
suggests to try another USB UVC Device.
endchoice
config EXAMPLE_NEGOTIATION_ATTEMPTS
int "Negotiation attempts"
default 3
help
Number of attempts to negotiate custom protocol parameters.
menu "UVC Protocol parameters"
depends on EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM
config EXAMPLE_WIDTH_PARAM
int "Width resolution in pixels"
default 320
help
Configure the negotiation width parameter during UVC device stream getting.
config EXAMPLE_HEIGHT_PARAM
int "Height resolution in pixels"
default 240
help
Configure the negotiation height parameter during UVC device stream getting.
config EXAMPLE_FPS_PARAM
int "FPS"
default 30
help
Configure the negotiation FPS parameter during UVC device stream getting.
Can be any value, available from the Frame Descriptor of the UVC device.
When 0 - negotiation accept the first rate available.
choice EXAMPLE_NEGOTIATION_FORMAT
prompt "Frame coding format of the stream, transport-independent"
default UVC_FRAME_FORMAT_MJPEG
help
Configure the negotiation frame coding of stream.
config UVC_FRAME_FORMAT_UNKNOWN
bool "Unknown"
config UVC_FRAME_FORMAT_UNCOMPRESSED
bool "Uncompressed"
config UVC_FRAME_FORMAT_COMPRESSED
bool "Compressed"
config UVC_FRAME_FORMAT_YUYV
bool "YUYV"
config UVC_FRAME_FORMAT_UYVY
bool "UYVY"
config UVC_FRAME_FORMAT_RGB
bool "RGB"
config UVC_FRAME_FORMAT_BGR
bool "BGR"
config UVC_FRAME_FORMAT_MJPEG
bool "MJPEG"
config UVC_FRAME_FORMAT_GRAY8
bool "GRAY8"
config UVC_FRAME_FORMAT_GRAY16
bool "GRAY16"
config UVC_FRAME_FORMAT_NV12
bool "NV12"
endchoice
config EXAMPLE_FORMAT_PARAM
int
default 0 if UVC_FRAME_FORMAT_UNKNOWN
default 1 if UVC_FRAME_FORMAT_UNCOMPRESSED
default 2 if UVC_FRAME_FORMAT_COMPRESSED
default 3 if UVC_FRAME_FORMAT_YUYV
default 4 if UVC_FRAME_FORMAT_UYVY
default 5 if UVC_FRAME_FORMAT_RGB
default 6 if UVC_FRAME_FORMAT_BGR
default 7 if UVC_FRAME_FORMAT_MJPEG
default 9 if UVC_FRAME_FORMAT_GRAY8
default 10 if UVC_FRAME_FORMAT_GRAY16
default 17 if UVC_FRAME_FORMAT_NV12
endmenu
endmenu

5
examples/peripherals/usb/host/uvc/main/idf_component.yml
View File

@@ -1,7 +1,4 @@
## IDF Component Manager Manifest File
dependencies:
idf: ">=5.0"
usb_host_uvc: "^1.0.3"
mdns: "^1.2.5"
protocol_examples_common:
path: ${IDF_PATH}/examples/common_components/protocol_examples_common
usb_host_uvc: "^2.2.0"

455
examples/peripherals/usb/host/uvc/main/main.c
View File

@@ -1,300 +1,233 @@
/*
* SPDX-FileCopyrightText: 2022-2024 Espressif Systems (Shanghai) CO LTD
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO LTD
*
* SPDX-License-Identifier: Apache-2.0
*/
#include <stdio.h>
#include <unistd.h>
#include <inttypes.h>
#include "sdkconfig.h"
#include "esp_system.h"
#include "esp_log.h"
#include "tcp_server.h"
#include "libuvc/libuvc.h"
#include "libuvc_helper.h"
#include "libuvc_adapter.h"
#include "esp_err.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/semphr.h"
#include "freertos/event_groups.h"
#include "driver/gpio.h"
#include "freertos/queue.h"
#include "usb/usb_host.h"
#include "esp_err.h"
#include "esp_log.h"
#include "esp_timer.h"
#include "usb/uvc_host.h"
static const char *TAG = "example";
#define EXAMPLE_USB_HOST_PRIORITY (15)
#define EXAMPLE_RECORDING_LENGTH_S (5) // This example will enable the stream, run it for EXAMPLE_RECORDING_LENGTH_S and then stop it
#define USB_DISCONNECT_PIN GPIO_NUM_0
// Private function prototypes
static bool frame_callback(const uvc_host_frame_t *frame, void *user_ctx);
static void stream_callback(const uvc_host_stream_event_data_t *event, void *user_ctx);
#if (CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_AUTO)
#define EXAMPLE_UVC_PROTOCOL_AUTO_COUNT 3
typedef struct {
enum uvc_frame_format format;
int width;
int height;
int fps;
const char* name;
} uvc_stream_profile_t;
// Private variables
static QueueHandle_t rx_frames_queue;
static bool dev_connected = false;
static const char *TAG = "UVC example";
uvc_stream_profile_t uvc_stream_profiles[EXAMPLE_UVC_PROTOCOL_AUTO_COUNT] = {
{UVC_FRAME_FORMAT_MJPEG, 640, 480, 15, "640x480, fps 15"},
{UVC_FRAME_FORMAT_MJPEG, 320, 240, 30, "320x240, fps 30"},
{UVC_FRAME_FORMAT_MJPEG, 320, 240, 0, "320x240, any fps"}
/**
* @brief Configuration of the UVC stream
*
* Update the vs_format to match your camera's capabilities.
* The example will try to open the camera with the specified resolution and FPS.
*/
static const uvc_host_stream_config_t stream_config = {
.event_cb = stream_callback,
.frame_cb = frame_callback,
.user_ctx = &rx_frames_queue,
.usb = {
.vid = UVC_HOST_ANY_VID, // Set to 0 to match any VID
.pid = UVC_HOST_ANY_PID, // Set to 0 to match any PID
.uvc_stream_index = 0, /* Index of UVC function you want to use. Set to 0 to use first available UVC function.
Setting this to >= 1 will only work if the camera has multiple UVC functions (eg. multiple image sensors in one USB device) */
},
.vs_format = {
// Note: For Full-Speed targets, only limited set of formats might be supported by the camera.
// Check the camera's Configuration descriptor for supported formats and resolutions.
#if CONFIG_SPIRAM
.h_res = 640,
.v_res = 480,
#else
.h_res = 320,
.v_res = 240,
#endif
.fps = 15,
.format = UVC_VS_FORMAT_MJPEG,
},
.advanced = {
.frame_size = 0, /* == 0: Use dwMaxVideoFrameSize from format negotiation result (might be too large)
>= 0: Use user provide frame size -> can save RAM, but occasionally a frame can overflow */
#if CONFIG_SPIRAM
.number_of_frame_buffers = 3, // Use triple buffering scheme if SPIRAM is available
.number_of_urbs = 3, // 3x 10kB URBs is usually enough, even for higher resolutions
.urb_size = 10 * 1024, // Larger values result in less frequent interrupts at the cost of memory consumption
.frame_heap_caps = MALLOC_CAP_SPIRAM, // Use SPIRAM for frame buffers
#else
.number_of_frame_buffers = 2, // Use double buffering scheme if SPIRAM is not available
.number_of_urbs = 2, // 2x 2kB URBs for memory constrained applications
.urb_size = 2 * 1024, // Larger values result in less frequent interrupts at the cost of memory consumption
#endif
},
};
#endif // CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_AUTO
#if (CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM)
#define FPS CONFIG_EXAMPLE_FPS_PARAM
#define WIDTH CONFIG_EXAMPLE_WIDTH_PARAM
#define HEIGHT CONFIG_EXAMPLE_HEIGHT_PARAM
#define FORMAT CONFIG_EXAMPLE_FORMAT_PARAM
#endif // CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM
static bool frame_callback(const uvc_host_frame_t *frame, void *user_ctx)
{
assert(frame);
assert(user_ctx);
QueueHandle_t frame_q = *((QueueHandle_t *)user_ctx);
// Attached camera can be filtered out based on (non-zero value of) PID, VID, SERIAL_NUMBER
#define PID 0
#define VID 0
#define SERIAL_NUMBER NULL
// Send the received frame to queue for further processing
ESP_LOGD(TAG, "Frame callback! data len: %d", frame->data_len);
BaseType_t result = xQueueSendToBack(frame_q, &frame, 0);
if (pdPASS != result) {
ESP_LOGW(TAG, "Queue full, losing frame"); // This should never happen
return true; // We will not process this frame, return it immediately
}
return false; // We only passed the frame to Queue, so we must return false and call uvc_host_frame_return() later
}
#define UVC_CHECK(exp) do { \
uvc_error_t _err_ = (exp); \
if(_err_ < 0) { \
ESP_LOGE(TAG, "UVC error: %s", \
uvc_error_string(_err_)); \
assert(0); \
} \
} while(0)
static void stream_callback(const uvc_host_stream_event_data_t *event, void *user_ctx)
{
switch (event->type) {
case UVC_HOST_TRANSFER_ERROR:
ESP_LOGE(TAG, "USB error has occurred, err_no = %i", event->transfer_error.error);
break;
case UVC_HOST_DEVICE_DISCONNECTED:
ESP_LOGI(TAG, "Device suddenly disconnected");
dev_connected = false;
ESP_ERROR_CHECK(uvc_host_stream_close(event->device_disconnected.stream_hdl));
break;
case UVC_HOST_FRAME_BUFFER_OVERFLOW:
// The Frame was discarded because it exceeded the available frame buffer size.
// To resolve this, increase the `frame_size` parameter in `uvc_host_stream_config_t.advanced` to allocate a larger buffer.
ESP_LOGW(TAG, "Frame buffer overflow");
break;
case UVC_HOST_FRAME_BUFFER_UNDERFLOW:
// The Frame was discarded because no available buffer was free for storage.
// To resolve this, either optimize your processing speed or increase the `number_of_frame_buffers` parameter in
// `uvc_host_stream_config_t.advanced` to allocate additional buffers.
ESP_LOGW(TAG, "Frame buffer underflow");
break;
default:
abort();
break;
}
}
static SemaphoreHandle_t ready_to_uninstall_usb;
static EventGroupHandle_t app_flags;
// Handles common USB host library events
static void usb_lib_handler_task(void *args)
static void usb_lib_task(void *arg)
{
while (1) {
// Start handling system events
uint32_t event_flags;
usb_host_lib_handle_events(portMAX_DELAY, &event_flags);
// Release devices once all clients has deregistered
if (event_flags & USB_HOST_LIB_EVENT_FLAGS_NO_CLIENTS) {
usb_host_device_free_all();
}
// Give ready_to_uninstall_usb semaphore to indicate that USB Host library
// can be deinitialized, and terminate this task.
if (event_flags & USB_HOST_LIB_EVENT_FLAGS_ALL_FREE) {
xSemaphoreGive(ready_to_uninstall_usb);
ESP_LOGI(TAG, "USB: All devices freed");
// Continue handling USB events to allow device reconnection
}
}
vTaskDelete(NULL);
}
static esp_err_t initialize_usb_host_lib(void)
static void frame_handling_task(void *arg)
{
TaskHandle_t task_handle = NULL;
const uvc_host_stream_config_t *stream_config = (const uvc_host_stream_config_t *)arg;
QueueHandle_t frame_q = *((QueueHandle_t *)(stream_config->user_ctx));
// This is the main program loop, it tries to open the UVC device and start streaming
while (true) {
uvc_host_stream_hdl_t uvc_stream = NULL;
ESP_LOGI(TAG, "Opening UVC device 0x%04X:0x%04X %dx%d@%2.1fFPS...",
stream_config->usb.vid, stream_config->usb.pid, stream_config->vs_format.h_res, stream_config->vs_format.v_res, stream_config->vs_format.fps);
esp_err_t err = uvc_host_stream_open(stream_config, pdMS_TO_TICKS(5000), &uvc_stream);
if (ESP_OK != err) {
ESP_LOGI(TAG, "Failed to open device");
vTaskDelay(pdMS_TO_TICKS(5000));
continue;
}
dev_connected = true;
ESP_LOGI(TAG, "UVC Device OPENED!");
vTaskDelay(pdMS_TO_TICKS(100));
unsigned count = 0;
// This is the main processing loop. It enables the stream for EXAMPLE_RECORDING_LENGTH_S seconds and then closes it
while (true) {
ESP_LOGI(TAG, "Stream start. Iteration %u", count);
count++;
uvc_host_stream_start(uvc_stream);
TickType_t timeout_ticks = pdMS_TO_TICKS(EXAMPLE_RECORDING_LENGTH_S * 1000);
TimeOut_t stream_timeout;
vTaskSetTimeOutState(&stream_timeout);
do {
uvc_host_frame_t *frame;
if (xQueueReceive(frame_q, &frame, pdMS_TO_TICKS(5000)) == pdPASS) {
ESP_LOGI(TAG, "New frame! Len: %d", frame->data_len);
// Process the frame data here
// For example, you can:
// - Save the frame to a file
// - Send the frame over the network
// - Decode the frame (typically MJPEG encoded)
// - Display the frame on a screen
// - etc.
uvc_host_frame_return(uvc_stream, frame);
} else {
ESP_LOGW(TAG, "Stream: Frame not received on time");
break;
}
} while (xTaskCheckForTimeOut(&stream_timeout, &timeout_ticks) == pdFALSE);
if (dev_connected) {
// Stop and wait 2 seconds
ESP_LOGI(TAG, "Stream stop");
uvc_host_stream_stop(uvc_stream);
vTaskDelay(pdMS_TO_TICKS(2000));
} else {
ESP_LOGI(TAG, "device disconnected, breaking");
break;
}
}
}
}
/**
* @brief Main application
*/
void app_main(void)
{
rx_frames_queue = xQueueCreate(3, sizeof(uvc_host_frame_t *));
assert(rx_frames_queue);
// Install USB Host driver. Should only be called once in entire application
ESP_LOGI(TAG, "Installing USB Host");
const usb_host_config_t host_config = {
.intr_flags = ESP_INTR_FLAG_LEVEL1
.skip_phy_setup = false,
.intr_flags = ESP_INTR_FLAG_LEVEL1,
};
ESP_ERROR_CHECK(usb_host_install(&host_config));
esp_err_t err = usb_host_install(&host_config);
if (err != ESP_OK) {
return err;
}
// Create a task that will handle USB library events
BaseType_t task_created = xTaskCreatePinnedToCore(usb_lib_task, "usb_lib", 4096, NULL, EXAMPLE_USB_HOST_PRIORITY, NULL, tskNO_AFFINITY);
assert(task_created == pdTRUE);
ready_to_uninstall_usb = xSemaphoreCreateBinary();
if (ready_to_uninstall_usb == NULL) {
usb_host_uninstall();
return ESP_ERR_NO_MEM;
}
if (xTaskCreate(usb_lib_handler_task, "usb_events", 4096, NULL, 2, &task_handle) != pdPASS) {
vSemaphoreDelete(ready_to_uninstall_usb);
usb_host_uninstall();
return ESP_ERR_NO_MEM;
}
return ESP_OK;
}
static void uninitialize_usb_host_lib(void)
{
xSemaphoreTake(ready_to_uninstall_usb, portMAX_DELAY);
vSemaphoreDelete(ready_to_uninstall_usb);
if (usb_host_uninstall() != ESP_OK) {
ESP_LOGE(TAG, "Failed to uninstall usb_host");
}
}
/* This callback function runs once per frame. Use it to perform any
* quick processing you need, or have it put the frame into your application's
* input queue. If this function takes too long, you'll start losing frames. */
void frame_callback(uvc_frame_t *frame, void *ptr)
{
static size_t fps;
static size_t bytes_per_second;
static int64_t start_time;
int64_t current_time = esp_timer_get_time();
bytes_per_second += frame->data_bytes;
fps++;
if (!start_time) {
start_time = current_time;
}
if (current_time > start_time + 1000000) {
ESP_LOGI(TAG, "fps: %u, bytes per second: %u", fps, bytes_per_second);
start_time = current_time;
bytes_per_second = 0;
fps = 0;
}
// Stream received frame to client, if enabled
tcp_server_send(frame->data, frame->data_bytes);
}
void button_callback(int button, int state, void *user_ptr)
{
printf("button %d state %d\n", button, state);
}
static void libuvc_adapter_cb(libuvc_adapter_event_t event)
{
xEventGroupSetBits(app_flags, event);
}
static EventBits_t wait_for_event(EventBits_t event)
{
return xEventGroupWaitBits(app_flags, event, pdTRUE, pdFALSE, portMAX_DELAY) & event;
}
static uvc_error_t uvc_negotiate_stream_profile(uvc_device_handle_t *devh,
uvc_stream_ctrl_t *ctrl)
{
uvc_error_t res;
#if (CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_AUTO)
for (int idx = 0; idx < EXAMPLE_UVC_PROTOCOL_AUTO_COUNT; idx++) {
int attempt = CONFIG_EXAMPLE_NEGOTIATION_ATTEMPTS;
do {
/*
The uvc_get_stream_ctrl_format_size() function will attempt to set the desired format size.
On first attempt, some cameras would reject the format, even if they support it.
So we ask 3x by default. The second attempt is usually successful.
*/
ESP_LOGI(TAG, "Negotiate streaming profile %s ...", uvc_stream_profiles[idx].name);
res = uvc_get_stream_ctrl_format_size(devh,
ctrl,
uvc_stream_profiles[idx].format,
uvc_stream_profiles[idx].width,
uvc_stream_profiles[idx].height,
uvc_stream_profiles[idx].fps);
} while (--attempt && !(UVC_SUCCESS == res));
if (UVC_SUCCESS == res) {
break;
}
}
#elif (CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM)
int attempt = CONFIG_EXAMPLE_NEGOTIATION_ATTEMPTS;
while (attempt--) {
ESP_LOGI(TAG, "Negotiate streaming profile %dx%d, %d fps ...", WIDTH, HEIGHT, FPS);
res = uvc_get_stream_ctrl_format_size(devh, ctrl, FORMAT, WIDTH, HEIGHT, FPS);
if (UVC_SUCCESS == res) {
break;
}
ESP_LOGE(TAG, "Negotiation failed. Try again (%d) ...", attempt);
}
#endif // CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM
if (UVC_SUCCESS == res) {
ESP_LOGI(TAG, "Negotiation complete.");
} else {
ESP_LOGE(TAG, "Try another UVC USB device of change negotiation parameters.");
}
return res;
}
int app_main(int argc, char **argv)
{
uvc_context_t *ctx;
uvc_device_t *dev;
uvc_device_handle_t *devh;
uvc_stream_ctrl_t ctrl;
app_flags = xEventGroupCreate();
assert(app_flags);
const gpio_config_t input_pin = {
.pin_bit_mask = BIT64(USB_DISCONNECT_PIN),
.mode = GPIO_MODE_INPUT,
.pull_up_en = GPIO_PULLUP_ENABLE,
};
ESP_ERROR_CHECK(gpio_config(&input_pin));
ESP_ERROR_CHECK(initialize_usb_host_lib());
libuvc_adapter_config_t config = {
ESP_LOGI(TAG, "Installing UVC driver");
const uvc_host_driver_config_t uvc_driver_config = {
.driver_task_stack_size = 4 * 1024,
.driver_task_priority = EXAMPLE_USB_HOST_PRIORITY + 1,
.xCoreID = tskNO_AFFINITY,
.create_background_task = true,
.task_priority = 5,
.stack_size = 4096,
.callback = libuvc_adapter_cb
};
ESP_ERROR_CHECK(uvc_host_install(&uvc_driver_config));
libuvc_adapter_set_config(&config);
UVC_CHECK(uvc_init(&ctx, NULL));
// Streaming takes place only when enabled in menuconfig
ESP_ERROR_CHECK(tcp_server_wait_for_connection());
do {
ESP_LOGI(TAG, "Waiting for USB UVC device connection ...");
wait_for_event(UVC_DEVICE_CONNECTED);
if (uvc_find_device(ctx, &dev, PID, VID, SERIAL_NUMBER) != UVC_SUCCESS) {
ESP_LOGW(TAG, "UVC device not found");
continue; // Continue waiting for UVC device
}
ESP_LOGI(TAG, "UVC device found");
// UVC Device open
UVC_CHECK(uvc_open(dev, &devh));
// Uncomment to print configuration descriptor
// libuvc_adapter_print_descriptors(devh);
uvc_set_button_callback(devh, button_callback, NULL);
// Print known device information
uvc_print_diag(devh, stderr);
// Negotiate stream profile
if (UVC_SUCCESS == uvc_negotiate_stream_profile(devh, &ctrl)) {
// dwMaxPayloadTransferSize has to be overwritten to MPS (maximum packet size)
// supported by ESP32-S2(S3), as libuvc selects the highest possible MPS by default.
ctrl.dwMaxPayloadTransferSize = 512;
uvc_print_stream_ctrl(&ctrl, stderr);
UVC_CHECK(uvc_start_streaming(devh, &ctrl, frame_callback, NULL, 0));
ESP_LOGI(TAG, "Streaming...");
wait_for_event(UVC_DEVICE_DISCONNECTED);
uvc_stop_streaming(devh);
ESP_LOGI(TAG, "Done streaming.");
} else {
wait_for_event(UVC_DEVICE_DISCONNECTED);
}
// UVC Device close
uvc_close(devh);
} while (gpio_get_level(USB_DISCONNECT_PIN) != 0);
tcp_server_close_when_done();
uvc_exit(ctx);
ESP_LOGI(TAG, "UVC exited");
uninitialize_usb_host_lib();
return 0;
task_created = xTaskCreatePinnedToCore(frame_handling_task, "frame_hdl", 4096, (void *)&stream_config, EXAMPLE_USB_HOST_PRIORITY - 2, NULL, tskNO_AFFINITY);
assert(task_created == pdTRUE);
}

226
examples/peripherals/usb/host/uvc/main/tcp_server.c
View File

@@ -1,226 +0,0 @@
/*
* SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
*
* SPDX-License-Identifier: Unlicense OR CC0-1.0
*/
#include <string.h>
#include <sys/param.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/ringbuf.h"
#include "freertos/event_groups.h"
#include "esp_system.h"
#include "esp_wifi.h"
#include "esp_event.h"
#include "esp_log.h"
#include "nvs_flash.h"
#include "esp_netif.h"
#include "protocol_examples_common.h"
#include "addr_from_stdin.h"
#include "lwip/err.h"
#include "lwip/sockets.h"
#include "tcp_server.h"
#include "mdns.h"
#define TAG "tcp_server"
#define PORT 2222
typedef struct {
int sock;
int listen_sock;
RingbufHandle_t buffer;
volatile bool close_request;
bool is_active;
} tcp_server_t;
#ifdef CONFIG_EXAMPLE_ENABLE_STREAMING
static tcp_server_t *s_server;
void socket_close(tcp_server_t *server)
{
ESP_LOGI(TAG, "Closing socket");
shutdown(server->sock, 0);
close(server->sock);
close(server->listen_sock);
}
static void sender_task(void *arg)
{
tcp_server_t *server = (tcp_server_t *)arg;
server->is_active = true;
while (1) {
size_t bytes_received = 0;
char *payload = (char *)xRingbufferReceiveUpTo(
server->buffer, &bytes_received, pdMS_TO_TICKS(2500), 20000);
if (payload != NULL && server->is_active) {
int sent = send(server->sock, payload, bytes_received, 0);
if (sent < 0) {
ESP_LOGE(TAG, "Error occurred during sending: errno %d, \
Shutting down tcp server...", errno);
server->is_active = false;
}
vRingbufferReturnItem(server->buffer, (void *)payload);
}
if (server->close_request) {
socket_close(server);
vRingbufferDelete(server->buffer);
vTaskDelete(NULL);
s_server = NULL;
free(server);
return;
}
}
}
esp_err_t tcp_server_send(uint8_t *payload, size_t size)
{
if (!s_server || !s_server->is_active) {
return ESP_OK;
}
if (xRingbufferSend(s_server->buffer, payload, size, pdMS_TO_TICKS(1)) != pdTRUE) {
ESP_LOGW(TAG, "Failed to send frame to ring buffer.");
return ESP_FAIL;
}
return ESP_OK;
}
static esp_err_t start_mdns_service(void)
{
esp_err_t err = mdns_init();
if (err) {
printf("MDNS Init failed: %d\n", err);
return ESP_FAIL;
}
mdns_hostname_set("esp-cam");
return ESP_OK;
}
static esp_err_t create_server(tcp_server_t *server)
{
char addr_str[128];
int ip_protocol = 0;
int addr_family = AF_INET;
struct sockaddr_storage dest_addr;
struct sockaddr_in *dest_addr_ip4 = (struct sockaddr_in *)&dest_addr;
dest_addr_ip4->sin_addr.s_addr = htonl(INADDR_ANY);
dest_addr_ip4->sin_family = addr_family;
dest_addr_ip4->sin_port = htons(PORT);
ip_protocol = IPPROTO_IP;
server->listen_sock = socket(addr_family, SOCK_STREAM, ip_protocol);
if (server->listen_sock < 0) {
ESP_LOGE(TAG, "Unable to create socket: errno %d", errno);
return ESP_FAIL;
}
int opt = 1;
setsockopt(server->listen_sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
ESP_LOGI(TAG, "Socket created");
int err = bind(server->listen_sock, (struct sockaddr *)&dest_addr, sizeof(dest_addr));
if (err != 0) {
ESP_LOGE(TAG, "Socket unable to bind: errno %d", errno);
ESP_LOGE(TAG, "IPPROTO: %d", addr_family);
close(server->listen_sock);
return ESP_FAIL;
}
ESP_LOGI(TAG, "Socket bound, port %d", PORT);
err = listen(server->listen_sock, 1);
if (err != 0) {
ESP_LOGE(TAG, "Error occurred during listen: errno %d", errno);
close(server->listen_sock);
return ESP_FAIL;
}
ESP_LOGI(TAG, "Socket listening...");
ESP_LOGI(TAG, "Execute player.py script");
struct sockaddr_storage source_addr;
socklen_t addr_len = sizeof(source_addr);
server->sock = accept(server->listen_sock, (struct sockaddr *)&source_addr, &addr_len);
if (server->sock < 0) {
ESP_LOGE(TAG, "Unable to accept connection: errno %d", errno);
close(server->listen_sock);
return ESP_FAIL;
}
// Convert ip address to string
if (source_addr.ss_family == PF_INET) {
inet_ntoa_r(((struct sockaddr_in *)&source_addr)->sin_addr, addr_str, sizeof(addr_str) - 1);
}
ESP_LOGI(TAG, "Socket accepted ip address: %s", addr_str);
return ESP_OK;
}
esp_err_t tcp_server_wait_for_connection(void)
{
TaskHandle_t task_handle = NULL;
ESP_ERROR_CHECK(nvs_flash_init());
ESP_ERROR_CHECK(esp_netif_init());
ESP_ERROR_CHECK(esp_event_loop_create_default());
ESP_ERROR_CHECK(start_mdns_service());
ESP_ERROR_CHECK(example_connect());
tcp_server_t *server = calloc(1, sizeof(tcp_server_t));
if (server == NULL) {
return ESP_ERR_NO_MEM;
}
server->buffer = xRingbufferCreate(100000, RINGBUF_TYPE_BYTEBUF);
if (server->buffer == NULL) {
free(server);
return ESP_ERR_NO_MEM;;
}
if (create_server(server) != ESP_OK) {
vRingbufferDelete(server->buffer);
free(server);
return ESP_FAIL;
}
BaseType_t task_created = xTaskCreate(sender_task, "sender_task", 4096, server, 10, &task_handle);
if (!task_created) {
socket_close(server);
vRingbufferDelete(server->buffer);
free(server);
return ESP_ERR_NO_MEM;
}
s_server = server;
return ESP_OK;
}
void tcp_server_close_when_done(void)
{
if (s_server) {
s_server->close_request = true;
}
}
#else
esp_err_t tcp_server_wait_for_connection(void)
{
return ESP_OK;
}
esp_err_t tcp_server_send(uint8_t *payload, size_t size)
{
return ESP_OK;
}
void tcp_server_close_when_done(void) { }
#endif

22
examples/peripherals/usb/host/uvc/main/tcp_server.h
View File

@@ -1,22 +0,0 @@
/*
* SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD
*
* SPDX-License-Identifier: Unlicense OR CC0-1.0
*/
#pragma once
#include "esp_err.h"
#ifdef __cplusplus
extern "C" {
#endif
esp_err_t tcp_server_wait_for_connection(void);
esp_err_t tcp_server_send(uint8_t *payload, size_t size);
void tcp_server_close_when_done(void);
#ifdef __cplusplus
}
#endif

37
examples/peripherals/usb/host/uvc/player.py
View File

@@ -1,37 +0,0 @@
# SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Unlicense OR CC0-1.0
import socket
import cv2
import numpy as np
frame_count = 0
stream = bytearray()
print('Connecting to server...')
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.connect(('esp-cam.local', 2222))
print('Receiving data ')
while True:
data = sock.recv(4096)
if not data:
break
stream += data
print('.', end='', flush=True)
a = stream.find(b'\xff\xd8')
b = stream.find(b'\xff\xd9', a)
if a != -1 and b != -1:
jpg = stream[a:b + 2]
stream = stream[b + 2:]
buffer = np.frombuffer(jpg, dtype=np.uint8)
image = cv2.imdecode(buffer, cv2.IMREAD_COLOR)
cv2.imshow('Stream', image)
if cv2.waitKey(10) == 27:
exit(0)
frame_count += 1
print('\nFrames received ', frame_count)

0
examples/peripherals/usb/host/uvc/sdkconfig.ci
View File

1
examples/peripherals/usb/host/uvc/sdkconfig.ci.custom_format
View File

@@ -1 +0,0 @@
CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM=y

1
examples/peripherals/usb/host/uvc/sdkconfig.ci.stream
View File

@@ -1 +0,0 @@
CONFIG_EXAMPLE_ENABLE_STREAMING=y

15
examples/peripherals/usb/host/uvc/sdkconfig.defaults
View File

@@ -10,17 +10,4 @@ CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y
#
CONFIG_USB_HOST_CONTROL_TRANSFER_MAX_SIZE=3000
CONFIG_USB_HOST_HW_BUFFER_BIAS_IN=y
#
# WIFI
#
CONFIG_ESP_WIFI_STATIC_RX_BUFFER_NUM=8
CONFIG_ESP_WIFI_DYNAMIC_RX_BUFFER_NUM=8
CONFIG_ESP_WIFI_STATIC_TX_BUFFER_NUM=8
CONFIG_ESP_WIFI_CACHE_TX_BUFFER_NUM=16
CONFIG_ESP_WIFI_RX_BA_WIN=8
#
# LWIP
#
CONFIG_LWIP_TCP_SND_BUF_DEFAULT=15000
CONFIG_PRINTF_UVC_CONFIGURATION_DESCRIPTOR=y

2
examples/peripherals/usb/host/uvc/sdkconfig.defaults.esp32p4 Normal file
View File

@@ -0,0 +1,2 @@
CONFIG_IDF_EXPERIMENTAL_FEATURES=y
CONFIG_SPIRAM_SPEED_200M=y