update docu
31
README.md
|
@ -15,9 +15,9 @@ In general, the ESP-WHO features will be supported as shown below:
|
|||
In ESP-WHO, Detection, Recognition and Image Utility are at the core of the platform.
|
||||
|
||||
|
||||
* **Image Utility** offers fundamental image processing APIs for ESP32.
|
||||
* **Image Utility** offers fundamental image processing APIs.
|
||||
|
||||
* **Detection** takes images as input and give the position of face if there is a face. It is implemented with MTMN model, which refers to [MTCNN](http://cn.arxiv.org/abs/1604.02878) and [MobileNets](https://arxiv.org/abs/1801.04381). However, we replace CNN with MobileNets block and the model size is quite smaller than other face detection models.
|
||||
* **Detection** takes images as input and give the position of face if there is a face. It is implemented with MTMN model, which refers to [MTCNN](http://cn.arxiv.org/abs/1604.02878) and [MobileNets](https://arxiv.org/abs/1801.04381).
|
||||
|
||||
* **Recognition** is to identify the particular person, and it needs the results of detection. It is implemented with MobileFace model.
|
||||
|
||||
|
@ -34,7 +34,7 @@ Both input and output are flexible.
|
|||
|
||||
## Hardware Preparation
|
||||
|
||||
To run ESP-WHO, you need to have a development board which integrates a **ESP32 module** that has sufficient GPIO pins and more than **4 MB** external SPI RAM. In this case, we take **ESP-WROVER-KIT** as the test board, which is highly recommended for getting started with ESP-WHO.
|
||||
To run ESP-WHO, you need to have a development board which integrates a **ESP32 module** that has sufficient GPIO pins and more than **4 MB** external SPI RAM. Either [ESP-WROVER-KIT](https://www.espressif.com/en/products/hardware/esp-wrover-kit/overview) or [ESP-EYE](https://www.espressif.com/en/products/hardware/esp-eye/overview) can be a good choice as the test board.
|
||||
|
||||
On how to configure ESP32 module for your applications, please refer to the README.md of each example.
|
||||
|
||||
|
@ -44,9 +44,9 @@ On how to configure ESP32 module for your applications, please refer to the READ
|
|||
|
||||
The recommended resolution of input image is **QVGA (320x240)**.
|
||||
|
||||
As for choosing camera as an image offer, make sure that the ESP32 module you choose offers specific pins that your camera needs. If there are pin conflicts, please refer to the pin connection in README.md of each example.
|
||||
As for choosing camera as an image offer, make sure that the ESP32 module you choose offers specific pins that your camera needs.
|
||||
|
||||
By now, we have provided the driver of **OV2640**, which is highly recommended to get started with.
|
||||
By now, we have provided the driver of **OV2640** and **OV3660**, which are highly recommended to get started with.
|
||||
|
||||
### ESP-WHO
|
||||
Make sure you clone the project recursive:
|
||||
|
@ -60,21 +60,20 @@ git clone --recursive https://github.com/espressif/esp-who.git
|
|||
### ESP-IDF
|
||||
See setup guides for detailed instructions to set up the ESP-IDF:
|
||||
|
||||
* [Getting Started Guide for the stable ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/stable/get-started/)
|
||||
* [Getting Started Guide for the latest (master branch) ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/latest/get-started/)
|
||||
* [Getting Started Guide for the stable ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/stable/get-started-cmake/index.html)
|
||||
|
||||
In this case, we take [ESP-IDF v3.1](https://github.com/espressif/esp-idf/releases/v3.1) as the test version.
|
||||
In this case, we take [ESP-IDF v3.2](https://github.com/espressif/esp-idf/releases/v3.2) as the test version.
|
||||
|
||||
If you have already configured ESP-IDF before and don't want to change the existing one, you can just point `IDF_PATH` to the ESP-IDF that is in ESP-WHO.
|
||||
|
||||
|
||||
# Components
|
||||
|
||||
Components is the main framework of the SDK, with camera driver and algorithm inside.
|
||||
|
||||
See the following sections for more details about components.
|
||||
Components is the main framework of the SDK, with some drivers and algorithm inside.
|
||||
|
||||
## Camera
|
||||
|
||||
The [camera](https://github.com/espressif/esp32-camera/tree/master) component contains drivers for camera devices of ESP32. Now OV2640 is supported.
|
||||
The [camera](https://github.com/espressif/esp32-camera/tree/master) component contains drivers for camera devices of ESP32.
|
||||
|
||||
|
||||
## esp-face
|
||||
|
@ -92,12 +91,12 @@ Take one Face Detection as an example.
|
|||
cd esp-who/examples/single_chip/detection_with_command_line
|
||||
```
|
||||
|
||||
2. Compile the project. You may run `make defconfig` before `make flash` to get the default configurations.
|
||||
2. Compile and flash the project.
|
||||
```
|
||||
make defconfig
|
||||
make flash
|
||||
idf.py build
|
||||
idf.py flash -p [PORT]
|
||||
```
|
||||
3. Advanced users can modify some options by using the command `make menuconfig`.
|
||||
3. Advanced users can modify some options by using the command `idf.py menuconfig`.
|
||||
|
||||
|
||||
Check README.md of each example for more details.
|
||||
|
|
|
@ -0,0 +1,28 @@
|
|||
|
||||
# Connecting
|
||||
|
||||
The table below lists the specific pins used in this example for connecting the ESP32 module and the camera module.
|
||||
|
||||
| Interface | Camera Pin | ESP32-WROVER | ESP-EYE |
|
||||
| :--- | :---: | :---: |
|
||||
| SCCB Clock | SIOC | IO27 | IO23 |
|
||||
| SCCB Data | SIOD | IO26 | IO18 |
|
||||
| System Clock | XCLK | IO21 | IO4 |
|
||||
| Vertical Sync | VSYNC | IO25 | IO5 |
|
||||
| Horizontal Reference | HREF | IO23 | IO27 |
|
||||
| Pixel Clock | PCLK | IO22 | IO25 |
|
||||
| Pixel Data Bit 0 | D2 | IO4 | IO34 |
|
||||
| Pixel Data Bit 1 | D3 | IO5 | IO13 |
|
||||
| Pixel Data Bit 2 | D4 | IO18 | IO14 |
|
||||
| Pixel Data Bit 3 | D5 | IO19 | IO35 |
|
||||
| Pixel Data Bit 4 | D6 | IO36 | IO39 |
|
||||
| Pixel Data Bit 5 | D7 | IO39 | IO38 |
|
||||
| Pixel Data Bit 6 | D8 | IO34 | IO37 |
|
||||
| Pixel Data Bit 7 | D9 | IO35 | IO36 |
|
||||
| Camera Reset | RESET | -1 | -1 |
|
||||
| Camera Power Down | PWDN | -1 | -1 |
|
||||
| Power Supply 3.3V | 3V3 | 3V3 | 3v3 |
|
||||
| Ground | GND | GND | GND |
|
||||
|
||||
> The pin mapping will be slightly different if you use other ESP32 modules.
|
||||
|
|
@ -79,116 +79,3 @@ The list and figure below describe the key components, interfaces and functions
|
|||
A reserved port for data transmission.
|
||||
|
||||
|
||||
## Software Development
|
||||
|
||||
ESP-EYE supports firmware downloading with a Linux, MacOS or Windows PC. At present, users must set up a toolchain in the development environment before starting software development.
|
||||
|
||||
### Preparation
|
||||
|
||||
- Go through [Get Started](https://docs.espressif.com/projects/esp-idf/en/v3.1.1/get-started/index.html) to set up the toolchain in your PC.
|
||||
- Connect the ESP-EYE to your PC with a Micro USB cable.
|
||||
- Launch your development-environment tool, such as Terminal (Linux/MacOS) or MinGW (Windows).
|
||||
|
||||
### Getting ESP-WHO
|
||||
|
||||
To obtain a local copy, open your Terminal (such as Terminal in a Linux environment), go to the directory in which you want to store ESP-WHO, and clone the repository by using the `git clone` command:
|
||||
|
||||
```
|
||||
git clone --recursive https://github.com/espressif/esp-who.git
|
||||
```
|
||||
|
||||
ESP-WHO will be downloaded into an automatically-generated folder named `esp-who`.
|
||||
|
||||
> Do not miss the `--recursive` option. If you have already cloned ESP-WHO without this option, please run another command to get all the submodules: `git submodule update --init --recursive`
|
||||
|
||||
### Setting up a Path to ESP-WHO
|
||||
|
||||
Please follow the instruction described in [Setup Path to ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/v3.1.1/get-started/index.html#get-started-setup-path) to configure the `IDF_PATH` variable to `esp-who/esp-idf`.
|
||||
|
||||
### Downloading Firmware
|
||||
|
||||
This section describes the steps for downloading firmware to ESP-EYE (taking the Linux operating system as an example):
|
||||
|
||||
- Power up ESP-EYE by connecting it to your PC with a USB cable.
|
||||
- Run `ls /dev/ttyUSB*` in your Terminal to check if the board has connected itself to your PC successfully. If the connection is successful, you will see a newly generated item named e.g, `/dev/ttyUSB0` on your list, after running the command.
|
||||
- Go to an example directory, such as `cd esp-who/examples/single_chip/recognition_solution`.
|
||||
- Run `make defconfig` to complete the default configuration.
|
||||
- Run `make menuconfig` and go to `Serial flasher config` -> `Default serial port` to configure the device name, according to the name of the item you saw in the second step, e.g. `/dev/ttyUSB0`. Then, save it and exit.
|
||||
- Run `make flash` to download the firmware.
|
||||
|
||||
### Checking Logs
|
||||
|
||||
This section describes the steps for checking the logs on your Terminal (taking the Linux operating system as an example).
|
||||
|
||||
- Launch your Terminal.
|
||||
- Run `make monitor`.
|
||||
|
||||
> Note: The ESP-EYE development board will reboot after this operation.
|
||||
|
||||
### Key Process
|
||||
|
||||
The figure below describes the workflow of ESP-EYE:
|
||||
|
||||
![esp-eye-workflow](../../_static/get-started/work_flow_en.png)
|
||||
|
||||
#### 1. Voice Wake-up
|
||||
|
||||
ESP-EYE awaits to be woken up after powering up (Red LED on and white LED off). The board wakes up after recognizing the wake-up command "Hi Lexin" ([Ləsɪ:n]), and then awaits for networking (Red LED flashing and white LED off). Subsequently, users can initiate the networking.
|
||||
|
||||
>Note: If you want to download an audio clip of our wake-up command "Hi Lexin", please click [here](https://dl.espressif.com/dl/Hi_Lexin_wake-up_commend.wav).
|
||||
|
||||
|
||||
#### 2. Networking
|
||||
|
||||
Users can connect their PCs or mobile phones to ESP-EYE's Wi-Fi (by default), with the following information:
|
||||
|
||||
- Username: esp-eye-xxxx (xxxx should be the board's MAC address)
|
||||
- Password: not needed
|
||||
|
||||
Alternatively, users can also follow the steps below to configure the username and password of the board's Wi-Fi connection:
|
||||
|
||||
- Launch your Terminal.
|
||||
- Run `make menuconfig` and complete the configuration, as instructed in the figure below:
|
||||
|
||||
![wifi connection](../../_static/get-started/wifi_connection.jpeg)
|
||||
|
||||
> Note: After reconfiguring the Wi-Fi username and password, you will have to restart from the point of downloading firmware.
|
||||
|
||||
#### 3. Face Detection
|
||||
|
||||
ESP-EYE starts the face detection after networking. Users can see the real-time image captured by the board, through their browser (address: `192.168.4.1/face_stream`). During this step, the red LED is off and the white LED is on.
|
||||
|
||||
#### 4. Face Recognition
|
||||
|
||||
After detecting a face, ESP-EYE will start the face
|
||||
recognition if there are any enrolled Face IDs stored in the board:
|
||||
|
||||
- When there is a match, the red LED on the board flashes once and the browser displays **HELLO ID XXX**.
|
||||
- When there is no match, the board shows no signs and the browser displays **WHO?**.
|
||||
|
||||
If there is no enrolled Face ID, the board continues the face-detecting process. You should enroll at least one Face ID if you want to start face
|
||||
recognition.
|
||||
|
||||
#### 5. Add/delete a Face ID
|
||||
|
||||
The users can add/delete a Face ID after the network is successfully established.
|
||||
|
||||
##### 5.1 Add a Face ID
|
||||
|
||||
![Enroll a Face ID](../../_static/get-started/face_id_enrollment_en.png)
|
||||
|
||||
- Single-click the Side Tactile Button to enroll a new Face ID. At this point, the red LED is on and the browser displays **START ENROLLING**;
|
||||
- Once you put a face in front of the camera, the face-sampling starts automatically. The red LED flashes whenever the board gets a face sample and the browser displays the ordinal number of the current face sample, i.e. **THE 1st SAMPLE** etc. By default, the board has to take three samples to add one Face ID. Users can configure the number of samples needed for one Face ID. (Please adjust your position/distance from the camera and try again if you cannot see the red LED flashing for some time).
|
||||
- After the Face ID enrollment, the red LED on the board is off and the browser displays **ENROLLED FACE ID XXX**;
|
||||
- The board enters Face Detection after the Face ID enrollment.
|
||||
|
||||
Currently, ESP-EYE can enroll up to 10 Face IDs. Please note that the maximum number of enrolled Face IDs can be configured according to how users allocate the flash memory. However, we recommend a number that is no greater than 30.
|
||||
|
||||
##### 5.2 Delete a Face ID
|
||||
|
||||
- Double-click the Side Tactile Button to delete an existing Face ID.
|
||||
- After that, the board deletes the earliest record of all the existing enrolled Face IDs. The white LED on the board flashes, and the browser displays: **XXX ID(S) LEFT**.
|
||||
|
||||
#### Troubleshooting
|
||||
|
||||
The board returns to the "awaiting to be woken up" status when there are network anomalies, such as "network disconnection" and "network timeout".
|
|
@ -4,62 +4,18 @@
|
|||
|
||||
To run this example, you need the following components:
|
||||
|
||||
* An ESP32 Module: **ESP32-WROVER**, which we highly recommend for beginners, is used in this example.
|
||||
* A Camera Module: the **OV2640** image sensor, which we highly recommend for beginners, is used in this example.
|
||||
* SDKs:
|
||||
* [ESP-IDF](https://github.com/espressif/esp-idf)
|
||||
* [ESP-WHO](https://github.com/espressif/esp-who)
|
||||
|
||||
For the detailed introduction about preparation, please see [here](https://github.com/espressif/esp-who).
|
||||
* An ESP32 Module: Either **ESP32-WROVER-KIT** or **ESP-EYE**, which we highly recommend for beginners, is used in this example.
|
||||
* A Camera Module: Either **OV2640** or **OV3660** image sensor, which we highly recommend for beginners, is used in this example.
|
||||
|
||||
# Quick Start
|
||||
|
||||
After you've completed the hardware settings, please follow the steps below:
|
||||
|
||||
1. **Connect** the camera to ESP32 module;
|
||||
2. **Configure** the example through `make menuconfig`;
|
||||
1. **Connect** the camera to ESP32 module. For connection pins, please see [here](../../../docs/en/Camera_connections.md)
|
||||
2. **Configure** the example through `idf.py menuconfig`;
|
||||
3. **Build And Flash** the application to ESP32;
|
||||
4. **Open Your Browser** and point it to `http://[ip-of-esp32]/`;
|
||||
5. **To Get Image** press `Get Still` or `Start Stream`;
|
||||
6. **Use The Options** to enable/disable Face Detection, Face Recognition and more;
|
||||
|
||||
## Connecting
|
||||
|
||||
The table below lists the specific pins used in this example for connecting the ESP32 module and the camera module.
|
||||
|
||||
| Interface | Camera Pin | Pin Mapping for ESP32-WROVER |
|
||||
| :--- | :---: | :---: |
|
||||
| SCCB Clock | SIOC | IO27 |
|
||||
| SCCB Data | SIOD | IO26 |
|
||||
| System Clock | XCLK | IO21 |
|
||||
| Vertical Sync | VSYNC | IO25 |
|
||||
| Horizontal Reference | HREF | IO23 |
|
||||
| Pixel Clock | PCLK | IO22 |
|
||||
| Pixel Data Bit 0 | D2 | IO4 |
|
||||
| Pixel Data Bit 1 | D3 | IO5 |
|
||||
| Pixel Data Bit 2 | D4 | IO18 |
|
||||
| Pixel Data Bit 3 | D5 | IO19 |
|
||||
| Pixel Data Bit 4 | D6 | IO36 |
|
||||
| Pixel Data Bit 5 | D7 | IO39 |
|
||||
| Pixel Data Bit 6 | D8 | IO34 |
|
||||
| Pixel Data Bit 7 | D9 | IO35 |
|
||||
| Camera Reset | RESET | IO2 |
|
||||
| Camera Power Down | PWDN | IO0 |
|
||||
| Power Supply 3.3V | 3V3 | 3V3 |
|
||||
| Ground | GND | GND |
|
||||
|
||||
> The pin mapping will be slightly different if you use other ESP32 modules.
|
||||
|
||||
In particular, if you are using a **ESP-WROVER-KIT** for your development, whose camera connector is already broken out (the one labeled Camera / JP4), please follow the steps below:
|
||||
|
||||
1. Plug your camera module, i.e. the OV2640 module in this example, on the board;
|
||||
2. Connect the 3V3 and GND pins on the camera module to those counterparts on the board.
|
||||
|
||||
The image below shows a **ESP-WROVER-KIT** development board with a **OV2640** camera module installed on it.
|
||||
|
||||
![esp_wrover_kit_with_ov2640](../../../img/esp_wrover_kit_with_ov2640.png)
|
||||
|
||||
## Flashing to ESP32
|
||||
|
||||
Please see [here](https://github.com/espressif/esp-who).
|
||||
|
||||
For more details of the http handler, please refer to [esp32-camera](https://github.com/espressif/esp32-camera).
|
||||
|
|
|
@ -1,71 +1,26 @@
|
|||
# Camera with Command Line in Single Chip
|
||||
|
||||
This example demonstrates **Human Face Detection** with a single ESP32 chip (without using any LCD module). ESP32 firstly gets images that are captured by the camera module, then determines if there are any human faces as well as displays its **Detection Results** in the **Serial Terminal**.
|
||||
This example demonstrates **Face Detection** with a single ESP32 chip (without using any LCD module). ESP32 firstly gets images that are captured by the camera module, then determines if there are any human faces as well as displays its **Detection Results** in the **Serial Terminal**.
|
||||
|
||||
# Preparation
|
||||
|
||||
To run this example, you need the following components:
|
||||
|
||||
* An ESP32 Module: **ESP32-WROVER**, which we highly recommend for beginners, is used in this example.
|
||||
* A Camera Module: the **OV2640** image sensor, which we highly recommend for beginners, is used in this example.
|
||||
* SDKs:
|
||||
* [ESP-IDF](https://github.com/espressif/esp-idf)
|
||||
* [ESP-WHO](https://github.com/espressif/esp-who)
|
||||
|
||||
For the detailed introduction about preparation, please see [here](https://github.com/espressif/esp-who).
|
||||
* An ESP32 Module: Either **ESP32-WROVER-KIT** or **ESP-EYE**, which we highly recommend for beginners, is used in this example.
|
||||
* A Camera Module: Either **OV2640** or **OV3660** image sensor, which we highly recommend for beginners, is used in this example.
|
||||
|
||||
# Quick Start
|
||||
|
||||
After you've completed the hardware settings, please follow the steps below:
|
||||
|
||||
1. **Connect** the camera to ESP32 module;
|
||||
1. **Connect** the camera to ESP32 module. For connection pins, please see [here](../../../docs/en/Camera_connections.md)
|
||||
2. **Flash Applications** to ESP32;
|
||||
3. **Start Human Face Detection** and **Check Detection Results**.
|
||||
|
||||
## Connecting
|
||||
|
||||
The table below lists the specific pins used in this example for connecting the ESP32 module and the camera module.
|
||||
|
||||
| Interface | Camera Pin | Pin Mapping for ESP32-WROVER |
|
||||
| :--- | :---: | :---: |
|
||||
| SCCB Clock | SIOC | IO27 |
|
||||
| SCCB Data | SIOD | IO26 |
|
||||
| System Clock | XCLK | IO21 |
|
||||
| Vertical Sync | VSYNC | IO25 |
|
||||
| Horizontal Reference | HREF | IO23 |
|
||||
| Pixel Clock | PCLK | IO22 |
|
||||
| Pixel Data Bit 0 | D2 | IO4 |
|
||||
| Pixel Data Bit 1 | D3 | IO5 |
|
||||
| Pixel Data Bit 2 | D4 | IO18 |
|
||||
| Pixel Data Bit 3 | D5 | IO19 |
|
||||
| Pixel Data Bit 4 | D6 | IO36 |
|
||||
| Pixel Data Bit 5 | D7 | IO39 |
|
||||
| Pixel Data Bit 6 | D8 | IO34 |
|
||||
| Pixel Data Bit 7 | D9 | IO35 |
|
||||
| Camera Reset | RESET | IO2 |
|
||||
| Camera Power Down | PWDN | IO0 |
|
||||
| Power Supply 3.3V | 3V3 | 3V3 |
|
||||
| Ground | GND | GND |
|
||||
|
||||
> The pin mapping will be slightly different if you use other ESP32 modules.
|
||||
|
||||
In particular, if you are using a **ESP-WROVER-KIT** for your development, whose camera connector is already broken out (the one labeled Camera / JP4), please follow the steps below:
|
||||
|
||||
1. Plug your camera module, i.e. the OV2640 module in this example, on the board;
|
||||
2. Connect the 3V3 and GND pins on the camera module to those counterparts on the board.
|
||||
|
||||
The image below shows a **ESP-WROVER-KIT** development board with a **OV2640** camera module installed on it.
|
||||
|
||||
![esp_wrover_kit_with_ov2640](../../../img/esp_wrover_kit_with_ov2640.png)
|
||||
|
||||
## Flashing to ESP32
|
||||
|
||||
Please see [here](https://github.com/espressif/esp-who).
|
||||
|
||||
## Checking Detection Result
|
||||
|
||||
1. Put your camera module away from a human face for at least 0.3 meter;
|
||||
2. Open a Serial Terminal by using the command line `make monitor`;
|
||||
2. Open a Serial Terminal by using the command line `idf.py monitor -p [PORT]`;
|
||||
3. Check result at your Serial Terminal, and you will be able to see information displayed in the screenshot below:
|
||||
|
||||
![detected](../../../img/detected.png)
|
||||
|
@ -76,17 +31,18 @@ The keyword **DETECTED** appears whenever ESP32 detects a human face.
|
|||
|
||||
In this example, several parameters can be configured by customers to better support different customized scenarios. For the detailed description of these parameters, please see [Here](https://github.com/espressif/esp-face/tree/master/face_detection).
|
||||
|
||||
Besides, please see below for the recommended configuration for general-purpose scenarios:
|
||||
|
||||
```
|
||||
mtmn_config.type = FAST;
|
||||
mtmn_config.min_face = 80;
|
||||
mtmn_config.pyramid = 0.7;
|
||||
mtmn_config.pyramid = 0.707;
|
||||
mtmn_config.pyramid_times = 4;
|
||||
mtmn_config.p_threshold.score = 0.6;
|
||||
mtmn_config.p_threshold.nms = 0.7;
|
||||
mtmn_config.p_threshold.candidate_number = 20;
|
||||
mtmn_config.r_threshold.score = 0.7;
|
||||
mtmn_config.r_threshold.nms = 0.7;
|
||||
mtmn_config.r_threshold.candidate_number = 4;
|
||||
mtmn_config.r_threshold.candidate_number = 10;
|
||||
mtmn_config.o_threshold.score = 0.7;
|
||||
mtmn_config.o_threshold.nms = 0.4;
|
||||
mtmn_config.o_threshold.nms = 0.7;
|
||||
mtmn_config.o_threshold.candidate_number = 1;
|
||||
```
|
|
@ -0,0 +1,150 @@
|
|||
menu "Camera Pins"
|
||||
choice CAMERA_MODEL
|
||||
bool "Select Camera Pinout"
|
||||
default CAMERA_MODEL_WROVER_KIT
|
||||
help
|
||||
Select Camera Pinout.
|
||||
|
||||
config CAMERA_MODEL_WROVER_KIT
|
||||
bool "WROVER-KIT With OV2640 Module"
|
||||
config CAMERA_MODEL_ESP_EYE
|
||||
bool "ESP_EYE DevKit"
|
||||
config CAMERA_MODEL_M5STACK_PSRAM
|
||||
bool "M5Stack Camera With PSRAM"
|
||||
config CAMERA_MODEL_M5STACK_WIDE
|
||||
bool "M5Stack Camera F (Wide)"
|
||||
config CAMERA_MODEL_AI_THINKER
|
||||
bool "ESP32-CAM by AI-Thinker"
|
||||
config CAMERA_MODEL_CUSTOM
|
||||
bool "Custom Camera Pinout"
|
||||
endchoice
|
||||
|
||||
config CAMERA_PIN_PWDN
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Power Down pin"
|
||||
range -1 33
|
||||
default -1
|
||||
help
|
||||
Select Power Down pin or -1 for unmanaged.
|
||||
|
||||
config CAMERA_PIN_RESET
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Reset pin"
|
||||
range -1 33
|
||||
default -1
|
||||
help
|
||||
Select Camera Reset pin or -1 for software reset.
|
||||
|
||||
config CAMERA_PIN_XCLK
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "XCLK pin"
|
||||
range 0 33
|
||||
default 21
|
||||
help
|
||||
Select Camera XCLK pin.
|
||||
|
||||
config CAMERA_PIN_SIOD
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "SIOD pin"
|
||||
range 0 33
|
||||
default 26
|
||||
help
|
||||
Select Camera SIOD pin.
|
||||
|
||||
config CAMERA_PIN_SIOC
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "SIOC pin"
|
||||
range 0 33
|
||||
default 27
|
||||
help
|
||||
Select Camera SIOC pin.
|
||||
|
||||
config CAMERA_PIN_VSYNC
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "VSYNC pin"
|
||||
range 0 39
|
||||
default 25
|
||||
help
|
||||
Select Camera VSYNC pin.
|
||||
|
||||
config CAMERA_PIN_HREF
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "HREF pin"
|
||||
range 0 39
|
||||
default 23
|
||||
help
|
||||
Select Camera HREF pin.
|
||||
|
||||
config CAMERA_PIN_PCLK
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "PCLK pin"
|
||||
range 0 39
|
||||
default 25
|
||||
help
|
||||
Select Camera PCLK pin.
|
||||
|
||||
config CAMERA_PIN_Y2
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y2 pin"
|
||||
range 0 39
|
||||
default 4
|
||||
help
|
||||
Select Camera Y2 pin.
|
||||
|
||||
config CAMERA_PIN_Y3
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y3 pin"
|
||||
range 0 39
|
||||
default 5
|
||||
help
|
||||
Select Camera Y3 pin.
|
||||
|
||||
config CAMERA_PIN_Y4
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y4 pin"
|
||||
range 0 39
|
||||
default 18
|
||||
help
|
||||
Select Camera Y4 pin.
|
||||
|
||||
config CAMERA_PIN_Y5
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y5 pin"
|
||||
range 0 39
|
||||
default 19
|
||||
help
|
||||
Select Camera Y5 pin.
|
||||
|
||||
config CAMERA_PIN_Y6
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y6 pin"
|
||||
range 0 39
|
||||
default 36
|
||||
help
|
||||
Select Camera Y6 pin.
|
||||
|
||||
config CAMERA_PIN_Y7
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y7 pin"
|
||||
range 0 39
|
||||
default 39
|
||||
help
|
||||
Select Camera Y7 pin.
|
||||
|
||||
config CAMERA_PIN_Y8
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y8 pin"
|
||||
range 0 39
|
||||
default 34
|
||||
help
|
||||
Select Camera Y8 pin.
|
||||
|
||||
config CAMERA_PIN_Y9
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y9 pin"
|
||||
range 0 39
|
||||
default 35
|
||||
help
|
||||
Select Camera Y9 pin.
|
||||
|
||||
endmenu
|
|
@ -55,11 +55,14 @@ void task_process (void *arg)
|
|||
size_t frame_num = 0;
|
||||
dl_matrix3du_t *image_matrix = NULL;
|
||||
camera_fb_t *fb = NULL;
|
||||
|
||||
/* 1. Load configuration for detection */
|
||||
mtmn_config_t mtmn_config = init_config();
|
||||
|
||||
do
|
||||
{
|
||||
int64_t start_time = esp_timer_get_time();
|
||||
/* 2. Get one image with camera */
|
||||
fb = esp_camera_fb_get();
|
||||
if (!fb)
|
||||
{
|
||||
|
@ -69,7 +72,10 @@ void task_process (void *arg)
|
|||
int64_t fb_get_time = esp_timer_get_time();
|
||||
ESP_LOGI(TAG, "Get one frame in %lld ms.", (fb_get_time - start_time) / 1000);
|
||||
|
||||
/* 3. Allocate image matrix to store RGB data */
|
||||
image_matrix = dl_matrix3du_alloc(1, fb->width, fb->height, 3);
|
||||
|
||||
/* 4. Transform image to RGB */
|
||||
uint32_t res = fmt2rgb888(fb->buf, fb->len, fb->format, image_matrix->item);
|
||||
if (true != res)
|
||||
{
|
||||
|
@ -80,6 +86,7 @@ void task_process (void *arg)
|
|||
|
||||
esp_camera_fb_return(fb);
|
||||
|
||||
/* 5. Do face detection */
|
||||
box_array_t *net_boxes = face_detect(image_matrix, &mtmn_config);
|
||||
ESP_LOGI(TAG, "Detection time consumption: %lldms", (esp_timer_get_time() - fb_get_time) / 1000);
|
||||
|
||||
|
|
|
@ -52,23 +52,122 @@
|
|||
*/
|
||||
#define CAMERA_FRAME_SIZE FRAMESIZE_QVGA
|
||||
|
||||
#define PWDN_GPIO_NUM 0
|
||||
#define RESET_GPIO_NUM 2
|
||||
#define Y2_GPIO_NUM 4
|
||||
#define Y3_GPIO_NUM 5
|
||||
#define Y4_GPIO_NUM 18
|
||||
#define Y5_GPIO_NUM 19
|
||||
#define Y6_GPIO_NUM 36
|
||||
#define Y7_GPIO_NUM 39
|
||||
#define Y8_GPIO_NUM 34
|
||||
#define Y9_GPIO_NUM 35
|
||||
#if CONFIG_CAMERA_MODEL_WROVER_KIT
|
||||
#define PWDN_GPIO_NUM -1
|
||||
#define RESET_GPIO_NUM -1
|
||||
#define XCLK_GPIO_NUM 21
|
||||
#define PCLK_GPIO_NUM 22
|
||||
#define HREF_GPIO_NUM 23
|
||||
#define VSYNC_GPIO_NUM 25
|
||||
#define SIOD_GPIO_NUM 26
|
||||
#define SIOC_GPIO_NUM 27
|
||||
|
||||
#define Y9_GPIO_NUM 35
|
||||
#define Y8_GPIO_NUM 34
|
||||
#define Y7_GPIO_NUM 39
|
||||
#define Y6_GPIO_NUM 36
|
||||
#define Y5_GPIO_NUM 19
|
||||
#define Y4_GPIO_NUM 18
|
||||
#define Y3_GPIO_NUM 5
|
||||
#define Y2_GPIO_NUM 4
|
||||
#define VSYNC_GPIO_NUM 25
|
||||
#define HREF_GPIO_NUM 23
|
||||
#define PCLK_GPIO_NUM 22
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_ESP_EYE
|
||||
#define PWDN_GPIO_NUM -1
|
||||
#define RESET_GPIO_NUM -1
|
||||
#define XCLK_GPIO_NUM 4
|
||||
#define SIOD_GPIO_NUM 18
|
||||
#define SIOC_GPIO_NUM 23
|
||||
|
||||
#define Y9_GPIO_NUM 36
|
||||
#define Y8_GPIO_NUM 37
|
||||
#define Y7_GPIO_NUM 38
|
||||
#define Y6_GPIO_NUM 39
|
||||
#define Y5_GPIO_NUM 35
|
||||
#define Y4_GPIO_NUM 14
|
||||
#define Y3_GPIO_NUM 13
|
||||
#define Y2_GPIO_NUM 34
|
||||
#define VSYNC_GPIO_NUM 5
|
||||
#define HREF_GPIO_NUM 27
|
||||
#define PCLK_GPIO_NUM 25
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_M5STACK_PSRAM
|
||||
#define PWDN_GPIO_NUM -1
|
||||
#define RESET_GPIO_NUM 15
|
||||
#define XCLK_GPIO_NUM 27
|
||||
#define SIOD_GPIO_NUM 25
|
||||
#define SIOC_GPIO_NUM 23
|
||||
|
||||
#define Y9_GPIO_NUM 19
|
||||
#define Y8_GPIO_NUM 36
|
||||
#define Y7_GPIO_NUM 18
|
||||
#define Y6_GPIO_NUM 39
|
||||
#define Y5_GPIO_NUM 5
|
||||
#define Y4_GPIO_NUM 34
|
||||
#define Y3_GPIO_NUM 35
|
||||
#define Y2_GPIO_NUM 32
|
||||
#define VSYNC_GPIO_NUM 22
|
||||
#define HREF_GPIO_NUM 26
|
||||
#define PCLK_GPIO_NUM 21
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_M5STACK_WIDE
|
||||
#define PWDN_GPIO_NUM -1
|
||||
#define RESET_GPIO_NUM 15
|
||||
#define XCLK_GPIO_NUM 27
|
||||
#define SIOD_GPIO_NUM 22
|
||||
#define SIOC_GPIO_NUM 23
|
||||
|
||||
#define Y9_GPIO_NUM 19
|
||||
#define Y8_GPIO_NUM 36
|
||||
#define Y7_GPIO_NUM 18
|
||||
#define Y6_GPIO_NUM 39
|
||||
#define Y5_GPIO_NUM 5
|
||||
#define Y4_GPIO_NUM 34
|
||||
#define Y3_GPIO_NUM 35
|
||||
#define Y2_GPIO_NUM 32
|
||||
#define VSYNC_GPIO_NUM 25
|
||||
#define HREF_GPIO_NUM 26
|
||||
#define PCLK_GPIO_NUM 21
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_AI_THINKER
|
||||
#define PWDN_GPIO_NUM 32
|
||||
#define RESET_GPIO_NUM -1
|
||||
#define XCLK_GPIO_NUM 0
|
||||
#define SIOD_GPIO_NUM 26
|
||||
#define SIOC_GPIO_NUM 27
|
||||
|
||||
#define Y9_GPIO_NUM 35
|
||||
#define Y8_GPIO_NUM 34
|
||||
#define Y7_GPIO_NUM 39
|
||||
#define Y6_GPIO_NUM 36
|
||||
#define Y5_GPIO_NUM 21
|
||||
#define Y4_GPIO_NUM 19
|
||||
#define Y3_GPIO_NUM 18
|
||||
#define Y2_GPIO_NUM 5
|
||||
#define VSYNC_GPIO_NUM 25
|
||||
#define HREF_GPIO_NUM 23
|
||||
#define PCLK_GPIO_NUM 22
|
||||
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_CUSTOM
|
||||
#define PWDN_GPIO_NUM CONFIG_CAMERA_PIN_PWDN
|
||||
#define RESET_GPIO_NUM CONFIG_CAMERA_PIN_RESET
|
||||
#define XCLK_GPIO_NUM CONFIG_CAMERA_PIN_XCLK
|
||||
#define SIOD_GPIO_NUM CONFIG_CAMERA_PIN_SIOD
|
||||
#define SIOC_GPIO_NUM CONFIG_CAMERA_PIN_SIOC
|
||||
|
||||
#define Y9_GPIO_NUM CONFIG_CAMERA_PIN_Y9
|
||||
#define Y8_GPIO_NUM CONFIG_CAMERA_PIN_Y8
|
||||
#define Y7_GPIO_NUM CONFIG_CAMERA_PIN_Y7
|
||||
#define Y6_GPIO_NUM CONFIG_CAMERA_PIN_Y6
|
||||
#define Y5_GPIO_NUM CONFIG_CAMERA_PIN_Y5
|
||||
#define Y4_GPIO_NUM CONFIG_CAMERA_PIN_Y4
|
||||
#define Y3_GPIO_NUM CONFIG_CAMERA_PIN_Y3
|
||||
#define Y2_GPIO_NUM CONFIG_CAMERA_PIN_Y2
|
||||
#define VSYNC_GPIO_NUM CONFIG_CAMERA_PIN_VSYNC
|
||||
#define HREF_GPIO_NUM CONFIG_CAMERA_PIN_HREF
|
||||
#define PCLK_GPIO_NUM CONFIG_CAMERA_PIN_PCLK
|
||||
#endif
|
||||
|
||||
#define XCLK_FREQ 20000000
|
||||
|
||||
void app_camera_init();
|
||||
|
|
|
@ -1,3 +1,79 @@
|
|||
# Face Detection and Recognition with ESP-EYE
|
||||
|
||||
Please see [HERE](../../../docs/en/get-started/ESP-EYE_Getting_Started_Guide.md)
|
||||
This example demonstrates **Face Recognition** and **Voice Wakeup** with ESP-EYE.
|
||||
|
||||
For hardware preparation, please see [HERE](../../../docs/en/get-started/ESP-EYE_Getting_Started_Guide.md)
|
||||
|
||||
# Key Features
|
||||
|
||||
- Voice wakeup
|
||||
- Face detection
|
||||
- Face recognition
|
||||
|
||||
# Key Process
|
||||
|
||||
The figure below describes the workflow of ESP-EYE:
|
||||
|
||||
![esp-eye-workflow](../../../docs/_static/get-started/work_flow_en.png)
|
||||
|
||||
#### 1. Voice Wake-up
|
||||
|
||||
ESP-EYE awaits to be woken up after powering up (Red LED on and white LED off). The board wakes up after recognizing the wake-up command "Hi Lexin" ([Ləsɪ:n]), and then awaits for networking (Red LED flashing and white LED off). Subsequently, users can initiate the networking.
|
||||
|
||||
>Note: If you want to download an audio clip of our wake-up command "Hi Lexin", please click [here](https://dl.espressif.com/dl/Hi_Lexin_wake-up_commend.wav).
|
||||
|
||||
|
||||
#### 2. Networking
|
||||
|
||||
Users can connect their PCs or mobile phones to ESP-EYE's Wi-Fi (by default), with the following information:
|
||||
|
||||
- Username: esp-eye-xxxx (xxxx should be the board's MAC address)
|
||||
- Password: not needed
|
||||
|
||||
Alternatively, users can also follow the steps below to configure the username and password of the board's Wi-Fi connection:
|
||||
|
||||
- Launch your Terminal.
|
||||
- Run `make menuconfig` and complete the configuration, as instructed in the figure below:
|
||||
|
||||
![wifi connection](../../../docs/_static/get-started/wifi_connection.jpeg)
|
||||
|
||||
> Note: After reconfiguring the Wi-Fi username and password, you will have to restart from the point of downloading firmware.
|
||||
|
||||
#### 3. Face Detection
|
||||
|
||||
ESP-EYE starts the face detection after networking. Users can see the real-time image captured by the board, through their browser (address: `192.168.4.1/face_stream`). During this step, the red LED is off and the white LED is on.
|
||||
|
||||
#### 4. Face Recognition
|
||||
|
||||
After detecting a face, ESP-EYE will start the face
|
||||
recognition if there are any enrolled Face IDs stored in the board:
|
||||
|
||||
- When there is a match, the red LED on the board flashes once and the browser displays **HELLO ID XXX**.
|
||||
- When there is no match, the board shows no signs and the browser displays **WHO?**.
|
||||
|
||||
If there is no enrolled Face ID, the board continues the face-detecting process. You should enroll at least one Face ID if you want to start face
|
||||
recognition.
|
||||
|
||||
#### 5. Add/delete a Face ID
|
||||
|
||||
The users can add/delete a Face ID after the network is successfully established.
|
||||
|
||||
##### 5.1 Add a Face ID
|
||||
|
||||
![Enroll a Face ID](../../../docs/_static/get-started/face_id_enrollment_en.png)
|
||||
|
||||
- Single-click the Side Tactile Button to enroll a new Face ID. At this point, the red LED is on and the browser displays **START ENROLLING**;
|
||||
- Once you put a face in front of the camera, the face-sampling starts automatically. The red LED flashes whenever the board gets a face sample and the browser displays the ordinal number of the current face sample, i.e. **THE 1st SAMPLE** etc. By default, the board has to take three samples to add one Face ID. Users can configure the number of samples needed for one Face ID. (Please adjust your position/distance from the camera and try again if you cannot see the red LED flashing for some time).
|
||||
- After the Face ID enrollment, the red LED on the board is off and the browser displays **ENROLLED FACE ID XXX**;
|
||||
- The board enters Face Detection after the Face ID enrollment.
|
||||
|
||||
Currently, ESP-EYE can enroll up to 10 Face IDs. Please note that the maximum number of enrolled Face IDs can be configured according to how users allocate the flash memory. However, we recommend a number that is no greater than 30.
|
||||
|
||||
##### 5.2 Delete a Face ID
|
||||
|
||||
- Double-click the Side Tactile Button to delete an existing Face ID.
|
||||
- After that, the board deletes the earliest record of all the existing enrolled Face IDs. The white LED on the board flashes, and the browser displays: **XXX ID(S) LEFT**.
|
||||
|
||||
#### Troubleshooting
|
||||
|
||||
The board returns to the "awaiting to be woken up" status when there are network anomalies, such as "network disconnection" and "network timeout".
|
||||
|
|
|
@ -1,97 +1,43 @@
|
|||
# Recognition with Command Line in Single Chip
|
||||
|
||||
This example demonstrates **Human Face Recognition** with a single ESP32 chip (without using any LCD module). ESP32 firstly gets images that are captured by the camera module, then determines if there are any recognized human faces as well as displays its **Recognition Results** in the **Serial Terminal**.
|
||||
This example demonstrates **Face Recognition** with a single ESP32 chip (without using any LCD module). ESP32 firstly gets images that are captured by the camera module, then determines if there are any recognized human faces as well as displays its **Recognition Results** in the **Serial Terminal**.
|
||||
|
||||
# Preparation
|
||||
|
||||
To run this example, you need the following components:
|
||||
|
||||
* An ESP32 Module: **ESP32-WROVER**, which we highly recommend for beginners, is used in this example.
|
||||
* A Camera Module: the **OV2640** image sensor, which we highly recommend for beginners, is used in this example.
|
||||
* SDKs:
|
||||
* [ESP-IDF](https://github.com/espressif/esp-idf)
|
||||
* [ESP-WHO](https://github.com/espressif/esp-who)
|
||||
|
||||
For the detailed introduction about preparation, please see [here](https://github.com/espressif/esp-who).
|
||||
|
||||
* An ESP32 Module: Either **ESP32-WROVER-KIT** or **ESP-EYE**, which we highly recommend for beginners, is used in this example.
|
||||
* A Camera Module: Either **OV2640** or **OV3660** image sensor, which we highly recommend for beginners, is used in this example.
|
||||
|
||||
# Quick Start
|
||||
|
||||
|
||||
After you've completed the hardware settings, please follow the steps below:
|
||||
|
||||
1. **Connect** the camera to ESP32 module;
|
||||
2. **Flash Application** to ESP32;
|
||||
3. **Start Human Face Recognition** and **Check Detection Results**.
|
||||
|
||||
## Connect
|
||||
|
||||
The table below lists the specific pins used in this example for connecting the ESP32 module and the camera module.
|
||||
|
||||
| Interface | Camera Pin | Pin Mapping for ESP32-WROVER |
|
||||
| :--- | :---: | :---: |
|
||||
| SCCB Clock | SIOC | IO27 |
|
||||
| SCCB Data | SIOD | IO26 |
|
||||
| System Clock | XCLK | IO21 |
|
||||
| Vertical Sync | VSYNC | IO25 |
|
||||
| Horizontal Reference | HREF | IO23 |
|
||||
| Pixel Clock | PCLK | IO22 |
|
||||
| Pixel Data Bit 0 | D2 | IO4 |
|
||||
| Pixel Data Bit 1 | D3 | IO5 |
|
||||
| Pixel Data Bit 2 | D4 | IO18 |
|
||||
| Pixel Data Bit 3 | D5 | IO19 |
|
||||
| Pixel Data Bit 4 | D6 | IO36 |
|
||||
| Pixel Data Bit 5 | D7 | IO39 |
|
||||
| Pixel Data Bit 6 | D8 | IO34 |
|
||||
| Pixel Data Bit 7 | D9 | IO35 |
|
||||
| Camera Reset | RESET | IO2 |
|
||||
| Camera Power Down | PWDN | IO0 |
|
||||
| Power Supply 3.3V | 3V3 | 3V3 |
|
||||
| Ground | GND | GND |
|
||||
|
||||
> The pin mapping will be slightly different if you use other ESP32 modules.
|
||||
|
||||
In particular, if you are using a **ESP-WROVER-KIT** for your development, whose camera connector is already broken out (the one labeled Camera / JP4), please follow the steps below:
|
||||
|
||||
1. Plug your camera module, i.e. the OV2640 module in this example, on the board;
|
||||
2. Connect the 3V3 and GND pins on the camera module to those counterparts on the board.
|
||||
|
||||
The image below shows a **ESP-WROVER-KIT** development board with a **OV2640** camera module installed on it.
|
||||
|
||||
![esp_wrover_kit_with_ov2640](../../../img/esp_wrover_kit_with_ov2640.png)
|
||||
|
||||
## Flashing to ESP32
|
||||
|
||||
Please see [here](https://github.com/espressif/esp-who).
|
||||
1. **Connect** the camera to ESP32 module. For connection pins, please see [here](../../../docs/en/Camera_connections.md)
|
||||
2. **Flash Applications** to ESP32;
|
||||
3. **Start Human Face Detection** and **Check Detection Results**.
|
||||
|
||||
## Checking Results
|
||||
|
||||
1. Put your camera module away from a human face for at least 0.3 m;
|
||||
2. Open a Serial Terminal by using the command line `make monitor`;
|
||||
3. Check result at your Serial Terminal, and you will be able to see information as displayed in the screenshot below, which indicates the **Face Enrollment** will start soon:
|
||||
2. Open a Serial Terminal by using the command line `idf.py monitor -p [PORT]`;
|
||||
3. Once there is a face detecetd, you will be able to see information as displayed in the screenshot below, which indicates the **Face Enrollment** will start soon:
|
||||
|
||||
![login_delay2](../../../img/enroll_start_count_down.png)
|
||||
![login_delay2](../../../img/enrollment.png)
|
||||
|
||||
### Enrolling a Face ID
|
||||
|
||||
To successfully enroll a **Face ID**, ESP32 will collect a certain number of samples of a user's face, which is configurable and 3 by default. To be more specific, by default, ESP32 will collect three samples of a user's face to enroll a new **Face ID**.
|
||||
|
||||
![start_enrollment_1](../../../img/enrollment_take_1st_sample.png)
|
||||
![start_enrollment_2](../../../img/enrollment_take_2nd_sample.png)
|
||||
![start_enrollment_3](../../../img/enrollment_take_3rd_sample.png)
|
||||
![errolled_face_id](../../../img/errolled_face_id.png)
|
||||
|
||||
### Recognizing a Face ID
|
||||
|
||||
After the **Face ID Enrollment**, ESP32 starts the **Face Recognition**:
|
||||
|
||||
![start_recognition](../../../img/start_recognition.png)
|
||||
After the **Face ID Enrollment**, ESP32 starts the **Face Recognition**.
|
||||
|
||||
ESP32 checks if the newly detected face matches any existing **Face ID**, whenever it detects a face:
|
||||
|
||||
* If Yes, the Serial Terminal displays the corresponding **Face ID**:
|
||||
|
||||
![recognition_matched](../../../img/matched.png)
|
||||
![recognition_matched](../../../img/recognized.png)
|
||||
|
||||
* If No, the Serial Terminal displays `No Matched ID`:
|
||||
|
||||
|
|
|
@ -0,0 +1,150 @@
|
|||
menu "Camera Pins"
|
||||
choice CAMERA_MODEL
|
||||
bool "Select Camera Pinout"
|
||||
default CAMERA_MODEL_WROVER_KIT
|
||||
help
|
||||
Select Camera Pinout.
|
||||
|
||||
config CAMERA_MODEL_WROVER_KIT
|
||||
bool "WROVER-KIT With OV2640 Module"
|
||||
config CAMERA_MODEL_ESP_EYE
|
||||
bool "ESP_EYE DevKit"
|
||||
config CAMERA_MODEL_M5STACK_PSRAM
|
||||
bool "M5Stack Camera With PSRAM"
|
||||
config CAMERA_MODEL_M5STACK_WIDE
|
||||
bool "M5Stack Camera F (Wide)"
|
||||
config CAMERA_MODEL_AI_THINKER
|
||||
bool "ESP32-CAM by AI-Thinker"
|
||||
config CAMERA_MODEL_CUSTOM
|
||||
bool "Custom Camera Pinout"
|
||||
endchoice
|
||||
|
||||
config CAMERA_PIN_PWDN
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Power Down pin"
|
||||
range -1 33
|
||||
default -1
|
||||
help
|
||||
Select Power Down pin or -1 for unmanaged.
|
||||
|
||||
config CAMERA_PIN_RESET
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Reset pin"
|
||||
range -1 33
|
||||
default -1
|
||||
help
|
||||
Select Camera Reset pin or -1 for software reset.
|
||||
|
||||
config CAMERA_PIN_XCLK
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "XCLK pin"
|
||||
range 0 33
|
||||
default 21
|
||||
help
|
||||
Select Camera XCLK pin.
|
||||
|
||||
config CAMERA_PIN_SIOD
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "SIOD pin"
|
||||
range 0 33
|
||||
default 26
|
||||
help
|
||||
Select Camera SIOD pin.
|
||||
|
||||
config CAMERA_PIN_SIOC
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "SIOC pin"
|
||||
range 0 33
|
||||
default 27
|
||||
help
|
||||
Select Camera SIOC pin.
|
||||
|
||||
config CAMERA_PIN_VSYNC
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "VSYNC pin"
|
||||
range 0 39
|
||||
default 25
|
||||
help
|
||||
Select Camera VSYNC pin.
|
||||
|
||||
config CAMERA_PIN_HREF
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "HREF pin"
|
||||
range 0 39
|
||||
default 23
|
||||
help
|
||||
Select Camera HREF pin.
|
||||
|
||||
config CAMERA_PIN_PCLK
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "PCLK pin"
|
||||
range 0 39
|
||||
default 25
|
||||
help
|
||||
Select Camera PCLK pin.
|
||||
|
||||
config CAMERA_PIN_Y2
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y2 pin"
|
||||
range 0 39
|
||||
default 4
|
||||
help
|
||||
Select Camera Y2 pin.
|
||||
|
||||
config CAMERA_PIN_Y3
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y3 pin"
|
||||
range 0 39
|
||||
default 5
|
||||
help
|
||||
Select Camera Y3 pin.
|
||||
|
||||
config CAMERA_PIN_Y4
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y4 pin"
|
||||
range 0 39
|
||||
default 18
|
||||
help
|
||||
Select Camera Y4 pin.
|
||||
|
||||
config CAMERA_PIN_Y5
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y5 pin"
|
||||
range 0 39
|
||||
default 19
|
||||
help
|
||||
Select Camera Y5 pin.
|
||||
|
||||
config CAMERA_PIN_Y6
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y6 pin"
|
||||
range 0 39
|
||||
default 36
|
||||
help
|
||||
Select Camera Y6 pin.
|
||||
|
||||
config CAMERA_PIN_Y7
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y7 pin"
|
||||
range 0 39
|
||||
default 39
|
||||
help
|
||||
Select Camera Y7 pin.
|
||||
|
||||
config CAMERA_PIN_Y8
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y8 pin"
|
||||
range 0 39
|
||||
default 34
|
||||
help
|
||||
Select Camera Y8 pin.
|
||||
|
||||
config CAMERA_PIN_Y9
|
||||
depends on CAMERA_MODEL_CUSTOM
|
||||
int "Y9 pin"
|
||||
range 0 39
|
||||
default 35
|
||||
help
|
||||
Select Camera Y9 pin.
|
||||
|
||||
endmenu
|
|
@ -74,8 +74,11 @@ void task_process(void *arg)
|
|||
size_t frame_num = 0;
|
||||
dl_matrix3du_t *image_matrix = NULL;
|
||||
camera_fb_t *fb = NULL;
|
||||
|
||||
/* 1. Load configuration for detection */
|
||||
mtmn_config_t mtmn_config = init_config();
|
||||
|
||||
/* 2. Preallocate matrix to store aligned face 56x56 */
|
||||
dl_matrix3du_t *aligned_face = dl_matrix3du_alloc(1,
|
||||
FACE_WIDTH,
|
||||
FACE_HEIGHT,
|
||||
|
@ -89,6 +92,7 @@ void task_process(void *arg)
|
|||
do
|
||||
{
|
||||
int64_t start_time = esp_timer_get_time();
|
||||
/* 3. Get one image with camera */
|
||||
fb = esp_camera_fb_get();
|
||||
if (!fb)
|
||||
{
|
||||
|
@ -98,7 +102,10 @@ void task_process(void *arg)
|
|||
int64_t fb_get_time = esp_timer_get_time();
|
||||
ESP_LOGI(TAG, "Get one frame in %lld ms.", (fb_get_time - start_time) / 1000);
|
||||
|
||||
/* 4. Allocate image matrix to store RGB data */
|
||||
image_matrix = dl_matrix3du_alloc(1, fb->width, fb->height, 3);
|
||||
|
||||
/* 5. Transform image to RGB */
|
||||
uint32_t res = fmt2rgb888(fb->buf, fb->len, fb->format, image_matrix->item);
|
||||
if (true != res)
|
||||
{
|
||||
|
@ -109,64 +116,67 @@ void task_process(void *arg)
|
|||
|
||||
esp_camera_fb_return(fb);
|
||||
|
||||
/* 6. Do face detection */
|
||||
box_array_t *net_boxes = face_detect(image_matrix, &mtmn_config);
|
||||
ESP_LOGI(TAG, "Detection time consumption: %lldms", (esp_timer_get_time() - fb_get_time) / 1000);
|
||||
|
||||
if (net_boxes)
|
||||
{
|
||||
frame_num++;
|
||||
ESP_LOGI(TAG, "Face Detection Count: %d", frame_num);
|
||||
//ESP_LOGI(TAG, "Face Detection Count: %d", frame_num);
|
||||
|
||||
/* 5. Do face alignment */
|
||||
if (align_face(net_boxes, image_matrix, aligned_face) == ESP_OK)
|
||||
{
|
||||
//count down
|
||||
while (count_down_second > 0)
|
||||
{
|
||||
ESP_LOGE(TAG, "Face ID Enrollment Starts in %ds.", count_down_second);
|
||||
ESP_LOGI(TAG, "Face ID Enrollment Starts in %ds.\n", count_down_second);
|
||||
|
||||
vTaskDelay(1000 / portTICK_PERIOD_MS);
|
||||
|
||||
count_down_second--;
|
||||
|
||||
if (count_down_second == 0)
|
||||
ESP_LOGE(TAG, ">>> Face ID Enrollment Starts <<<");
|
||||
ESP_LOGI(TAG, "\n>>> Face ID Enrollment Starts <<<\n");
|
||||
}
|
||||
|
||||
//enroll
|
||||
/* 6. Do face enrollment */
|
||||
if (is_enrolling == 1)
|
||||
{
|
||||
left_sample_face = enroll_face(&id_list, aligned_face);
|
||||
ESP_LOGE(TAG, "Face ID Enrollment: Take the %d%s sample",
|
||||
ESP_LOGI(TAG, "Face ID Enrollment: Take the %d%s sample",
|
||||
ENROLL_CONFIRM_TIMES - left_sample_face,
|
||||
number_suffix(ENROLL_CONFIRM_TIMES - left_sample_face));
|
||||
|
||||
if (left_sample_face == 0)
|
||||
{
|
||||
next_enroll_index++;
|
||||
ESP_LOGE(TAG, "Enrolled Face ID: %d", id_list.tail);
|
||||
ESP_LOGI(TAG, "\nEnrolled Face ID: %d", id_list.tail);
|
||||
|
||||
if (id_list.count == FACE_ID_SAVE_NUMBER)
|
||||
{
|
||||
is_enrolling = 0;
|
||||
ESP_LOGE(TAG, ">>> Face Recognition Starts <<<");
|
||||
ESP_LOGI(TAG, "\n>>> Face Recognition Starts <<<\n");
|
||||
vTaskDelay(2000 / portTICK_PERIOD_MS);
|
||||
}
|
||||
else
|
||||
{
|
||||
ESP_LOGE(TAG, "Please log in another one.");
|
||||
ESP_LOGI(TAG, "Please log in another one.");
|
||||
vTaskDelay(2500 / portTICK_PERIOD_MS);
|
||||
}
|
||||
}
|
||||
}
|
||||
/* 6. Do face recognition */
|
||||
else
|
||||
{
|
||||
int64_t recog_match_time = esp_timer_get_time();
|
||||
|
||||
int matched_id = recognize_face(&id_list, aligned_face);
|
||||
if (matched_id >= 0)
|
||||
ESP_LOGE(TAG, "Matched Face ID: %d", matched_id);
|
||||
ESP_LOGI(TAG, "Matched Face ID: %d\n", matched_id);
|
||||
else
|
||||
ESP_LOGE(TAG, "No Matched Face ID");
|
||||
ESP_LOGI(TAG, "No Matched Face ID\n");
|
||||
|
||||
ESP_LOGI(TAG, "Recognition time consumption: %lldms",
|
||||
(esp_timer_get_time() - recog_match_time) / 1000);
|
||||
|
|
|
@ -55,23 +55,122 @@
|
|||
*/
|
||||
#define CAMERA_FRAME_SIZE FRAMESIZE_QVGA
|
||||
|
||||
#define PWDN_GPIO_NUM 0
|
||||
#define RESET_GPIO_NUM 2
|
||||
#define Y2_GPIO_NUM 4
|
||||
#define Y3_GPIO_NUM 5
|
||||
#define Y4_GPIO_NUM 18
|
||||
#define Y5_GPIO_NUM 19
|
||||
#define Y6_GPIO_NUM 36
|
||||
#define Y7_GPIO_NUM 39
|
||||
#define Y8_GPIO_NUM 34
|
||||
#define Y9_GPIO_NUM 35
|
||||
#if CONFIG_CAMERA_MODEL_WROVER_KIT
|
||||
#define PWDN_GPIO_NUM -1
|
||||
#define RESET_GPIO_NUM -1
|
||||
#define XCLK_GPIO_NUM 21
|
||||
#define PCLK_GPIO_NUM 22
|
||||
#define HREF_GPIO_NUM 23
|
||||
#define VSYNC_GPIO_NUM 25
|
||||
#define SIOD_GPIO_NUM 26
|
||||
#define SIOC_GPIO_NUM 27
|
||||
|
||||
#define Y9_GPIO_NUM 35
|
||||
#define Y8_GPIO_NUM 34
|
||||
#define Y7_GPIO_NUM 39
|
||||
#define Y6_GPIO_NUM 36
|
||||
#define Y5_GPIO_NUM 19
|
||||
#define Y4_GPIO_NUM 18
|
||||
#define Y3_GPIO_NUM 5
|
||||
#define Y2_GPIO_NUM 4
|
||||
#define VSYNC_GPIO_NUM 25
|
||||
#define HREF_GPIO_NUM 23
|
||||
#define PCLK_GPIO_NUM 22
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_ESP_EYE
|
||||
#define PWDN_GPIO_NUM -1
|
||||
#define RESET_GPIO_NUM -1
|
||||
#define XCLK_GPIO_NUM 4
|
||||
#define SIOD_GPIO_NUM 18
|
||||
#define SIOC_GPIO_NUM 23
|
||||
|
||||
#define Y9_GPIO_NUM 36
|
||||
#define Y8_GPIO_NUM 37
|
||||
#define Y7_GPIO_NUM 38
|
||||
#define Y6_GPIO_NUM 39
|
||||
#define Y5_GPIO_NUM 35
|
||||
#define Y4_GPIO_NUM 14
|
||||
#define Y3_GPIO_NUM 13
|
||||
#define Y2_GPIO_NUM 34
|
||||
#define VSYNC_GPIO_NUM 5
|
||||
#define HREF_GPIO_NUM 27
|
||||
#define PCLK_GPIO_NUM 25
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_M5STACK_PSRAM
|
||||
#define PWDN_GPIO_NUM -1
|
||||
#define RESET_GPIO_NUM 15
|
||||
#define XCLK_GPIO_NUM 27
|
||||
#define SIOD_GPIO_NUM 25
|
||||
#define SIOC_GPIO_NUM 23
|
||||
|
||||
#define Y9_GPIO_NUM 19
|
||||
#define Y8_GPIO_NUM 36
|
||||
#define Y7_GPIO_NUM 18
|
||||
#define Y6_GPIO_NUM 39
|
||||
#define Y5_GPIO_NUM 5
|
||||
#define Y4_GPIO_NUM 34
|
||||
#define Y3_GPIO_NUM 35
|
||||
#define Y2_GPIO_NUM 32
|
||||
#define VSYNC_GPIO_NUM 22
|
||||
#define HREF_GPIO_NUM 26
|
||||
#define PCLK_GPIO_NUM 21
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_M5STACK_WIDE
|
||||
#define PWDN_GPIO_NUM -1
|
||||
#define RESET_GPIO_NUM 15
|
||||
#define XCLK_GPIO_NUM 27
|
||||
#define SIOD_GPIO_NUM 22
|
||||
#define SIOC_GPIO_NUM 23
|
||||
|
||||
#define Y9_GPIO_NUM 19
|
||||
#define Y8_GPIO_NUM 36
|
||||
#define Y7_GPIO_NUM 18
|
||||
#define Y6_GPIO_NUM 39
|
||||
#define Y5_GPIO_NUM 5
|
||||
#define Y4_GPIO_NUM 34
|
||||
#define Y3_GPIO_NUM 35
|
||||
#define Y2_GPIO_NUM 32
|
||||
#define VSYNC_GPIO_NUM 25
|
||||
#define HREF_GPIO_NUM 26
|
||||
#define PCLK_GPIO_NUM 21
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_AI_THINKER
|
||||
#define PWDN_GPIO_NUM 32
|
||||
#define RESET_GPIO_NUM -1
|
||||
#define XCLK_GPIO_NUM 0
|
||||
#define SIOD_GPIO_NUM 26
|
||||
#define SIOC_GPIO_NUM 27
|
||||
|
||||
#define Y9_GPIO_NUM 35
|
||||
#define Y8_GPIO_NUM 34
|
||||
#define Y7_GPIO_NUM 39
|
||||
#define Y6_GPIO_NUM 36
|
||||
#define Y5_GPIO_NUM 21
|
||||
#define Y4_GPIO_NUM 19
|
||||
#define Y3_GPIO_NUM 18
|
||||
#define Y2_GPIO_NUM 5
|
||||
#define VSYNC_GPIO_NUM 25
|
||||
#define HREF_GPIO_NUM 23
|
||||
#define PCLK_GPIO_NUM 22
|
||||
|
||||
|
||||
#elif CONFIG_CAMERA_MODEL_CUSTOM
|
||||
#define PWDN_GPIO_NUM CONFIG_CAMERA_PIN_PWDN
|
||||
#define RESET_GPIO_NUM CONFIG_CAMERA_PIN_RESET
|
||||
#define XCLK_GPIO_NUM CONFIG_CAMERA_PIN_XCLK
|
||||
#define SIOD_GPIO_NUM CONFIG_CAMERA_PIN_SIOD
|
||||
#define SIOC_GPIO_NUM CONFIG_CAMERA_PIN_SIOC
|
||||
|
||||
#define Y9_GPIO_NUM CONFIG_CAMERA_PIN_Y9
|
||||
#define Y8_GPIO_NUM CONFIG_CAMERA_PIN_Y8
|
||||
#define Y7_GPIO_NUM CONFIG_CAMERA_PIN_Y7
|
||||
#define Y6_GPIO_NUM CONFIG_CAMERA_PIN_Y6
|
||||
#define Y5_GPIO_NUM CONFIG_CAMERA_PIN_Y5
|
||||
#define Y4_GPIO_NUM CONFIG_CAMERA_PIN_Y4
|
||||
#define Y3_GPIO_NUM CONFIG_CAMERA_PIN_Y3
|
||||
#define Y2_GPIO_NUM CONFIG_CAMERA_PIN_Y2
|
||||
#define VSYNC_GPIO_NUM CONFIG_CAMERA_PIN_VSYNC
|
||||
#define HREF_GPIO_NUM CONFIG_CAMERA_PIN_HREF
|
||||
#define PCLK_GPIO_NUM CONFIG_CAMERA_PIN_PCLK
|
||||
#endif
|
||||
|
||||
#define XCLK_FREQ 20000000
|
||||
|
||||
void app_camera_init();
|
||||
|
|
BIN
img/detected.png
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 21 KiB |
Before Width: | Height: | Size: 22 KiB |
After Width: | Height: | Size: 30 KiB |
Before Width: | Height: | Size: 5.6 KiB |
Before Width: | Height: | Size: 6.0 KiB |
Before Width: | Height: | Size: 6.2 KiB |
Before Width: | Height: | Size: 4.9 KiB |
Before Width: | Height: | Size: 5.4 KiB |
Before Width: | Height: | Size: 5.5 KiB |
BIN
img/matched.png
Before Width: | Height: | Size: 21 KiB |
Before Width: | Height: | Size: 20 KiB After Width: | Height: | Size: 15 KiB |
After Width: | Height: | Size: 18 KiB |