{IDF_TARGET_NAME} includes an Image Signal Processor (ISP), which is a feature pipeline that consists of many image processing algorithms. ISP receives image data from the DVP camera or MIPI-CSI camera, or system memory, and writes the processed image data to the system memory through DMA. The ISP is designed to work with other camera controller modules and can not operate independently.
- MIPI-CSI: Camera serial interface, a high-speed serial interface for cameras compliant with MIPI specifications
- DVP: Digital video parallel interface, generally composed of vsync, hsync, de, and data signals
- RAW: Unprocessed data directly output from an image sensor, typically divided into R, Gr, Gb, and B four channels classified into RAW8, RAW10, RAW12, etc., based on bit width
- RGB: Colored image format composed of red, green, and blue colors classified into RGB888, RGB565, etc., based on the bit width of each color
- YUV: Colored image format composed of luminance and chrominance classified into YUV444, YUV422, YUV420, etc., based on the data arrangement
-:ref:`isp-resource-allocation` - covers how to allocate ISP resources with properly set of configurations. It also covers how to recycle the resources when they finished working.
-:ref:`isp-enable-disable` - covers how to enable and disable an ISP processor.
-:ref:`isp-af-statistics` - covers how to get AF statistics one-shot or continuously.
-:ref:`isp-awb-statistics` - covers how to get AWB white patches statistics one-shot or continuously.
-:ref:`isp-ae-statistics` - covers how to get AE statistics one-shot or continuously.
-:ref:`isp-hist-statistics` - covers how to get histogram statistics one-shot or continuously.
-:ref:`isp-bf` - covers how to enable and configure BF function.
If the configurations in :cpp:type:`esp_isp_processor_cfg_t` is specified, users can call :cpp:func:`esp_isp_new_processor` to allocate and initialize an ISP processor. This function will return an ISP processor handle if it runs correctly. You can take following code as reference:
If the configurations in :cpp:type:`esp_isp_af_config_t` is specified, users can call :cpp:func:`esp_isp_new_af_controller` to allocate and initialize an ISP AF controller. This function will return an ISP AF controller handle if it runs correctly. You can take following code as reference:
If an :cpp:type:`esp_isp_awb_config_t` configuration is specified, you can call :cpp:func:`esp_isp_new_awb_controller` to allocate and initialize an ISP AWB controller. This function will return an ISP AWB controller handle on success. You can take following code as reference:
If the configurations in :cpp:type:`esp_isp_ae_config_t` is specified, call :cpp:func:`esp_isp_new_ae_controller` to allocate and initialize an ISP AE controller. This function will return an ISP AE controller handle if it runs correctly. You can take following code as reference.
If the configurations in :cpp:type:`esp_isp_hist_config_t` is specified, users can call :cpp:func:`esp_isp_new_hist_controller` to allocate and initialize an ISP Histogram controller. This function will return an ISP HIST controller handle if it runs correctly. You can take following code as reference.
If previously installed ISP drivers are no longer needed, it's recommended to recycle the resource by following APIs to release the underlying hardware:
Aside from the above one-shot API, the ISP AF driver also provides a way to start AF statistics continuously. Calling :cpp:func:`esp_isp_af_controller_start_continuous_statistics` to start the continuous statistics and :cpp:func:`esp_isp_af_controller_stop_continuous_statistics` to stop it.
Note that if you want to use the continuous statistics, you need to register the :cpp:member:`esp_isp_af_env_detector_evt_cbs_t::on_env_statistics_done` or :cpp:member:`esp_isp_af_env_detector_evt_cbs_t::on_env_change` callbacks to get the statistics result. See how to register in :ref:`isp-callback`.
Before doing ISP AWB, you need to enable the ISP AWB controller first, by calling :cpp:func:`esp_isp_awb_controller_enable`. This function:
* Switches the driver state from **init** to **enable**.
Calling :cpp:func:`esp_isp_awb_controller_disable` does the opposite, that is, put the driver back to the **init** state.
.._isp-awb-statistics:
AWB One-shot and Continuous Statistics
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Calling :cpp:func:`esp_isp_awb_controller_get_oneshot_statistics` to get oneshot AWB statistics result of white patches. You can take following code as reference.
Aside from the above one-shot API, the ISP AWB driver also provides a way to start AWB statistics continuously. Calling :cpp:func:`esp_isp_awb_controller_start_continuous_statistics` starts the continuous statistics and :cpp:func:`esp_isp_awb_controller_stop_continuous_statistics` stops it.
Note that if you want to use the continuous statistics, you need to register the :cpp:member:`esp_isp_awb_cbs_t::on_statistics_done` callback to get the statistics result. See how to register it in :ref:`isp-callback`.
When using AE oneshot statistics, the AE continuous mode need to be disabled otherwise the result may be overwritten by the environment detector. After oneshot operation finishes, you need to restart continuous mode again.
Aside from the above oneshot API, the ISP AE driver also provides a way to start AE statistics continuously. Calling :cpp:func:`esp_isp_ae_controller_start_continuous_statistics` to start the continuous statistics and :cpp:func:`esp_isp_ae_controller_stop_continuous_statistics` to stop it.
Note that if you want to use the continuous statistics, you need to register the :cpp:member:`esp_isp_ae_env_detector_evt_cbs_t::on_env_statistics_done` or :cpp:member:`esp_isp_ae_env_detector_evt_cbs_t::on_env_change` callback to get the statistics result. See how to register in :ref:`isp-callback`.
Before doing ISP histogram statistics, you need to enable the ISP histogram controller first, by calling :cpp:func:`esp_isp_hist_controller_enable`. This function:
* Switches the driver state from **init** to **enable**.
Calling :cpp:func:`esp_isp_hist_controller_disable` does the opposite, that is, put the driver back to the **init** state.
.._isp-hist-statistics:
Histogram One-shot and Continuous Statistics
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Calling :cpp:func:`esp_isp_hist_controller_get_oneshot_statistics` to get oneshot histogram statistics result. You can take following code as reference.
Aside from the above oneshot API, the ISP histogram driver also provides a way to start histogram statistics continuously. Calling :cpp:func:`esp_isp_hist_controller_start_continuous_statistics` starts the continuous statistics and :cpp:func:`esp_isp_hist_controller_stop_continuous_statistics` stops it.
Note that if you want to use the continuous statistics, you need to register the :cpp:member:`esp_isp_hist_cbs_t::on_statistics_done` callback to get the statistics result. See how to register it in :ref:`isp-callback`.
:cpp:member:`esp_isp_bf_config_t::bf_template` is used for bayer denoise. You can set the :cpp:member:`esp_isp_bf_config_t::bf_template` with a Gaussian filter template or an average filter template.
Lens Shading Correction (LSC) aims for the issues caused by the uneven refraction of light through the camera lens.
Calling :cpp:func:`esp_isp_lsc_configure` to configure the LSC module to do the correction. The :cpp:type:`esp_isp_lsc_gain_array_t` is necessary for the hardware to do the correction related calculation. :cpp:func:`esp_isp_lsc_allocate_gain_array` is a helper function to help allocate proper size of memory for the gains.
After calling :cpp:func:`esp_isp_lsc_configure`, you need to enable the ISP LSC controller by calling :cpp:func:`esp_isp_lsc_enable`. The LSC can be disabled by calling :cpp:func:`esp_isp_lsc_disable`. It is allowed to call :cpp:func:`esp_isp_lsc_configure` when the LSC is not enabled, but the LSC function will only take effect when it is enabled.
- Brightness value should be {IDF_TARGET_SOC_ISP_COLOR_BRIGHTNESS_MIN} ~ {IDF_TARGET_SOC_ISP_COLOR_BRIGHTNESS_MAX}, default {IDF_TARGET_SOC_ISP_COLOR_BRIGHTNESS_DEFAULT}
After calling :cpp:func:`esp_isp_color_configure`, you need to enable the ISP color controller, by calling :cpp:func:`esp_isp_color_enable`. This function:
When the ISP DVP peripheral is used with the output color format set to the RGB color space, :ref:`isp-color` is automatically enabled in the camera driver to ensure correct data output. The function :cpp:func:`esp_isp_color_disable` should never be called in this case, otherwise it may result in disarrayed camera data.
Color correction matrix can scale the color ratio of RGB888 pixels. It can be used for adjusting the image color via some algorithms, for example, used for white balance by inputting the AWB computed result, or used as a filter with some filter algorithms.
After calling :cpp:func:`esp_isp_demosaic_configure`, you need to enable the ISP Demosaic controller, by calling :cpp:func:`esp_isp_demosaic_enable`. This function:
* Switches the driver state from **init** to **enable**.
Calling :cpp:func:`esp_isp_demosaic_disable` does the opposite, that is, put the driver back to the **init** state.
:cpp:func:`esp_isp_demosaic_configure` is allowed to be called even if the driver is in **init** state, but the demosaic configurations will only be taken into effect when in **enable** state.
The human visual system is non-linearly sensitive to the physical luminance. Adding gamma correction to the ISP pipeline to transform RGB coordinates into a space in which coordinates are proportional to subjective brightness.
The driver provides a helper API :cpp:func:`esp_isp_gamma_fill_curve_points` to fill :cpp:type:`isp_gamma_curve_points_t`, which is a group of points used to describe the gamma correction curve. Or you can manually declare the points as your desired gamma correction curve. Each R/G/B component can have its own gamma correction curve, you can set the configuration by calling :cpp:func:`esp_isp_gamma_configure`.
:cpp:member:`esp_isp_sharpen_config_t::sharpen_template` is used for sharpening. You can set the :cpp:member:`esp_isp_sharpen_config_t::sharpen_template` with a Gaussian filter template or an average filter template.
After calling :cpp:func:`esp_isp_sharpen_configure`, you need to enable the ISP Sharpen controller, by calling :cpp:func:`esp_isp_sharpen_enable`. This function:
* Switches the driver state from **init** to **enable**.
Calling :cpp:func:`esp_isp_sharpen_disable` does the opposite, that is, put the driver back to the **init** state.
:cpp:func:`esp_isp_sharpen_configure` is allowed to be called even if the driver is in **init** state, but the sharpen configurations will only be taken into effect when in **enable** state.
You can save your own context to callback function as well, via the parameter ``user_data``. The user data will be directly passed to the callback function.
The below-mentioned callback functions are called within an ISR context. You must ensure that the functions do not attempt to block (e.g., by making sure that only FreeRTOS APIs with ``ISR`` suffix are called from within the function).
After the ISP processor is enabled, it can generate multiple events of multiple ISP submodules dynamically. You can hook your functions to the interrupt service routine by calling :cpp:func:`esp_isp_register_event_callbacks`. All supported event callbacks are listed in :cpp:type:`esp_isp_evt_cbs_t`:
-:cpp:member:`esp_isp_evt_cbs_t::on_sharpen_frame_done` sets a callback function for sharpen frame done. It will be called after the ISP sharpen submodule finishes its operation for one frame. The function prototype is declared in :cpp:type:`esp_isp_sharpen_callback_t`.
After the ISP AF environment detector starts up, it can generate a specific event dynamically. If you have some functions that should be called when the event happens, please hook your function to the interrupt service routine by calling :cpp:func:`esp_isp_af_env_detector_register_event_callbacks`. All supported event callbacks are listed in :cpp:type:`esp_isp_af_env_detector_evt_cbs_t`:
-:cpp:member:`esp_isp_af_env_detector_evt_cbs_t::on_env_statistics_done` sets a callback function for environment statistics done. The function prototype is declared in :cpp:type:`esp_isp_af_env_detector_callback_t`.
-:cpp:member:`esp_isp_af_env_detector_evt_cbs_t::on_env_change` sets a callback function for environment change. The function prototype is declared in :cpp:type:`esp_isp_af_env_detector_callback_t`.
After the ISP AWB controller finished statistics of white patches, it can generate a specific event dynamically. If you want to be informed when the statistics done event takes place, please hook your function to the interrupt service routine by calling :cpp:func:`esp_isp_awb_register_event_callbacks`. All supported event callbacks are listed in :cpp:type:`esp_isp_awb_cbs_t`:
-:cpp:member:`esp_isp_awb_cbs_t::on_statistics_done` sets a callback function when finishing statistics of the white patches. The function prototype is declared in :cpp:type:`esp_isp_awb_callback_t`.
After the ISP AE environment detector starts up, it can generate a specific event dynamically. If you have some functions that should be called when the event happens, please hook your function to the interrupt service routine by calling :cpp:func:`esp_isp_ae_env_detector_register_event_callbacks`. All supported event callbacks are listed in :cpp:type:`esp_isp_ae_env_detector_evt_cbs_t`:
-:cpp:member:`esp_isp_ae_env_detector_evt_cbs_t::on_env_statistics_done` sets a callback function for environment statistics done. The function prototype is declared in :cpp:type:`esp_isp_ae_env_detector_callback_t`.
-:cpp:member:`esp_isp_ae_env_detector_evt_cbs_t::on_env_change` sets a callback function for environment change. The function prototype is declared in :cpp:type:`esp_isp_ae_env_detector_callback_t`.
After the ISP HIST controller finished statistics of brightness, it can generate a specific event dynamically. If you want to be informed when the statistics done event takes place, please hook your function to the interrupt service routine by calling :cpp:func:`esp_isp_hist_register_event_callbacks`. All supported event callbacks are listed in :cpp:type:`esp_isp_hist_cbs_t`:
-:cpp:member:`esp_isp_hist_cbs_t::on_statistics_done` sets a callback function when finishing statistics of the brightness. The function prototype is declared in :cpp:type:`esp_isp_hist_callback_t`.
- Enable the interrupt being serviced even when the cache is disabled
- Place all functions that used by the ISR into IRAM
- Place driver object into DRAM (in case it is mapped to PSRAM by accident)
This allows the interrupt to run while the cache is disabled, but comes at the cost of increased IRAM consumption. With this option enabled, the ISR callbacks will be running when cache is disabled. Therefore you should make sure the callbacks and its involved context are IRAM-safe as well.
*:example:`peripherals/isp/multi_pipelines` demonstrates how to use the ISP pipelines to process the image signals from camera sensors and display the video on LCD screen via DSI peripheral.
