From 5970bcc138506a71ac94033ee730c816ff29bfdc Mon Sep 17 00:00:00 2001 From: Natasha Date: Thu, 10 Jan 2019 11:34:56 +0800 Subject: [PATCH] Doc/update esp eye guides --- .../ESP-EYE_Getting_Started_Guide.md | 191 ++++++++++++++++++ .../ESP-EYE_V2.0_Getting_Started_Guide.md | 190 ----------------- ...de.md => ESP-EYE_Getting_Started_Guide.md} | 9 +- .../detection_with_command_line/README.md | 18 ++ .../recognition_with_command_line/README.md | 30 ++- 5 files changed, 235 insertions(+), 203 deletions(-) create mode 100644 docs/en/get-started/ESP-EYE_Getting_Started_Guide.md delete mode 100644 docs/en/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md rename docs/zh_CN/get-started/{ESP-EYE_V2.0_Getting_Started_Guide.md => ESP-EYE_Getting_Started_Guide.md} (94%) diff --git a/docs/en/get-started/ESP-EYE_Getting_Started_Guide.md b/docs/en/get-started/ESP-EYE_Getting_Started_Guide.md new file mode 100644 index 0000000..49fbe45 --- /dev/null +++ b/docs/en/get-started/ESP-EYE_Getting_Started_Guide.md @@ -0,0 +1,191 @@ +# ESP-EYE Getting Started Guide + +[[中文]](../../zh_CN/get-started/ESP-EYE_Getting_Started_Guide.md) + +## What You Need + +* 1 × ESP-EYE V2.1 board +* 1 × Micro USB cable +* 1 × PC with Windows, Linux or Mac OS + +## Overview + +ESP-EYE is an ESP32-based development board that integrates a digital microphone, an 8 MB PSRAM and a 4 MB flash, while also providing an external 2-Megapixel camera. These features make the board ideal for applications relating to face detection, face recognition and speech recognition. Besides, the board can also support image transmission over Wi-Fi and debugging through a Micro USB port, which enables the development of advanced AI solutions. + +## Hardware Preparation + +The list and figure below describe the key components, interfaces and functions of the ESP-EYE development board: + +![ESP-EYE image](../../_static/get-started/esp-eye_callout.png) + +* **3D_PIFA Antenna** + + A 3D PIFA antenna. With the R14 resistor users can select the external IPEX antenna, whereas with the R15 resistor they can select the 3D antenna. + +* **IPEX Connector** + + Used for connecting the external IPEX antenna. With the R14 resistor users can select the external IPEX antenna, whereas with the R15 resistor they can select the 3D antenna. + +* **ESP32 Chip** + + A 2.4 GHz Wi-Fi and Bluetooth combo chip. + +* **Crystal Oscillator** + + Provides an external clock to ESP32. + +* **Flash & PSRAM** + + Provides memory for storing applications. + +* **CP2102 USB-to-UART Chip** + + Converts the USB signals to UART signals. + +* **USB Port** + + Provides power supply to the whole system. + +* **LDO Power Supply** + + Provides the required power supply to the ESP32 chip, camera and LED indicators. + +* **Side Tactile Button** + + A function key. + +* **Top Tactile Button** + + Reset/Boot button. We recommend that you do not configure this button for other uses. + +* **LED Indicators** + + The board has a red and a white indicator. Different flashing combinations of the red and white indicators reflect the different statuses of the board, e.g. waking up, networking, face detection, face recognition, face enrollment and face recognition. + +* **Camera** + + An external 2-Megapixel camera module for face detection, face recognition and Face ID enrollment. + +* **Camera Connector** + + Used for connecting the external camera. + +* **MIC** + + A digital microphone for voice control functions. + +* **SPI Port** + + A reserved port for data transmission. + + +## Software Development + +ESP-EYE supports firmware downloading with a Linux, MacOS or Windows PC. At present, users must set up a toolchain in the development environment before starting software development. + +### Preparation + +- Go through [Get Started](https://docs.espressif.com/projects/esp-idf/en/v3.1.1/get-started/index.html) to set up the toolchain in your PC. +- Connect the ESP-EYE to your PC with a Micro USB cable. +- Launch your development-environment tool, such as Terminal (Linux/MacOS) or MinGW (Windows). + +### Getting ESP-WHO + +To obtain a local copy, open your Terminal (such as Terminal in a Linux environment), go to the directory in which you want to store ESP-WHO, and clone the repository by using the `git clone` command: + +``` +git clone --recursive https://github.com/espressif/esp-who.git +``` + +ESP-WHO will be downloaded into an automatically-generated folder named `esp-who`. + +> Do not miss the `--recursive` option. If you have already cloned ESP-WHO without this option, please run another command to get all the submodules: `git submodule update --init --recursive` + +### Setting up a Path to ESP-WHO + +Please follow the instruction described in [Setup Path to ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/v3.1.1/get-started/index.html#get-started-setup-path) to configure the `IDF_PATH` variable to `esp-who/esp-idf`. + +### Downloading Firmware + +This section describes the steps for downloading firmware to ESP-EYE (taking the Linux operating system as an example): + +- Power up ESP-EYE by connecting it to your PC with a USB cable. +- Run `ls /dev/ttyUSB*` in your Terminal to check if the board has connected itself to your PC successfully. If the connection is successful, you will see a newly generated item named e.g, `/dev/ttyUSB0` on your list, after running the command. +- Go to an example directory, such as `cd esp-who/examples/single_chip/recognition_solution`. +- Run `make defconfig` to complete the default configuration. +- Run `make menuconfig` and go to `Serial flasher config` -> `Default serial port` to configure the device name, according to the name of the item you saw in the second step, e.g. `/dev/ttyUSB0`. Then, save it and exit. +- Run `make flash` to download the firmware. + +### Checking Logs + +This section describes the steps for checking the logs on your Terminal (taking the Linux operating system as an example). + +- Launch your Terminal. +- Run `make monitor`. + +> Note: The ESP-EYE development board will reboot after this operation. + +### Key Process + +The figure below describes the workflow of ESP-EYE: + +![esp-eye-workflow](../../_static/get-started/work_flow_en.png) + +#### 1. Voice Wake-up + +ESP-EYE awaits to be woken up after powering up (Red LED on and white LED off). The board wakes up after recognizing the wake-up command "Hi Lexin", and then awaits for networking (Red LED flashing and white LED off). Subsequently, users can initiate the networking. + +#### 2. Networking + +Users can connect their PCs or mobile phones to ESP-EYE's Wi-Fi (by default), with the following information: + +- Username: esp-eye-xxxx (xxxx should be the board's MAC address) +- Password: not needed + +Alternatively, users can also follow the steps below to configure the username and password of the board's Wi-Fi connection: + +- Launch your Terminal. +- Run `make menuconfig` and complete the configuration, as instructed in the figure below: + + ![wifi connection](../../_static/get-started/wifi_connection.jpeg) + +> Note: After reconfiguring the Wi-Fi username and password, you will have to restart from the point of downloading firmware. + +#### 3. Face Detection + +ESP-EYE starts the face detection after networking. Users can see the real-time image captured by the board, through their browser (address: `192.168.4.1/face_stream`). During this step, the red LED is off and the white LED is on. + +#### 4. Face Recognition + +After detecting a face, ESP-EYE will start the face +recognition if there are any enrolled Face IDs stored in the board: + +- When there is a match, the red LED on the board flashes once and the browser displays **HELLO ID XXX**. +- When there is no match, the board shows no signs and the browser displays **WHO?**. + +If there is no enrolled Face ID, the board continues the face-detecting process. You should enroll at least one Face ID if you want to start face +recognition. + +#### 5. Add/delete a Face ID + +The users can add/delete a Face ID after the network is successfully established. + +##### 5.1 Add a Face ID + +![Enroll a Face ID](../../_static/get-started/face_id_enrollment_en.png) + +- Single-click the Side Tactile Button to enroll a new Face ID. At this point, the red LED is on and the browser displays **START ENROLLING**; +- Once you put a face in front of the camera, the face-sampling starts automatically. The red LED flashes whenever the board gets a face sample and the browser displays the ordinal number of the current face sample, i.e. **THE 1st SAMPLE** etc. By default, the board has to take three samples to add one Face ID. Users can configure the number of samples needed for one Face ID. (Please adjust your position/distance from the camera and try again if you cannot see the red LED flashing for some time). +- After the Face ID enrollment, the red LED on the board is off and the browser displays **ENROLLED FACE ID XXX**; +- The board enters Face Detection after the Face ID enrollment. + +Currently, ESP-EYE can enroll up to 10 Face IDs. Please note that the maximum number of enrolled Face IDs can be configured according to how users allocate the flash memory. However, we recommend a number that is no greater than 30. + +##### 5.2 Delete a Face ID + +- Double-click the Side Tactile Button to delete an existing Face ID. +- After that, the board deletes the earliest record of all the existing enrolled Face IDs. The white LED on the board flashes, and the browser displays: **XXX ID(S) LEFT**. + +#### Troubleshooting + +The board returns to the "awaiting to be woken up" status when there are network anomalies, such as "network disconnection" and "network timeout". \ No newline at end of file diff --git a/docs/en/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md b/docs/en/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md deleted file mode 100644 index c3ae1e6..0000000 --- a/docs/en/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md +++ /dev/null @@ -1,190 +0,0 @@ -# ESP-EYE Getting Started Guide - -[[中文]](../../zh_CN/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md) - -## What You Need - -* 1 × ESP-EYE V2.0 board -* 1 × Micro USB cable -* 1 × PC loaded with Windows, Linux or Mac OS - -## Overview - -ESP-EYE is an ESP32-based development board that integrates a digital microphone, 8 MB PSRAM and 4 MB flash, as well as provides an external 2-million-pixel camera, which makes the board very suitable for applications in the fields of face detection, face recognition, and speech recognition. Besides, the board can also support image transmission over Wi-Fi and debugging using the Micro USB port, which enables the users' development of advanced AI solutions. - -## Hardware Preparation - -The list and figure below describe the key components, interfaces, and controls of the ESP-EYE development board: - -![ESP-EYE image](../../_static/get-started/esp-eye_callout.png) - -* **3D_PIFA Antenna** - - A 3D PIFA antenna. Users can choose the R14 resistor to select external IPEX antenna or choose the R15 resistor to select the 3D antenna. - -* **IPEX Connector** - - Used for connecting external IPEX antenna. Users can choose the R14 resistor to select external IPEX antenna or choose the R15 resistor to select the 3D antenna. - * **ESP32 Chip** - - A 2.4 GHz Wi-Fi and Bluetooth combo chip - -* **Crystal Oscillator** - - Provides an external clock for ESP32. - -* **Flash & PSRAM** - - Provides memory to store applications. - -* **CP2102 USB-to-UART Chip** - - Converts the USB signals to UART signals. - -* **USB Port** - - Provides the power supply to the whole system. - -* **LDO Power Supply** - - Provides the required power supplies to the ESP32 chip, camera and LED indicators. - -* **Side Tactile Button** - - A function key - -* **Top Tactile Button** - - Reset/Boot button. We recommend that you do not configure this button for other uses. - -* **LED Indicators** - - The board has a red indicator and a white indicator. Different combinations of red and white indicators reflect the different status of the board, such as waking up, networking, face detection, face recognition, face enrollment, and face recognition... - -* **Camera** - - An external 2-million-pixel camera module for face detection, face recognition and Face ID enrollment - -* **Camera Connector** - - Used to connect the external camera. - -* **MIC** - - A digital microphone for voice control functions - -* **SPI Port** - - A reserved port for data transmission - - -## Software Development - -ESP-EYE supports firmware downloading with a PC loaded with Linux, MacOs or Windows operating systems. For now, users must set up toolchain in the development environment before starting the software development. - -### Preparation - -- Go through [Get Started](https://docs.espressif.com/projects/esp-idf/en/v3.1.1/get-started/index.html) to set up toolchain in your PC; -- Connect ESP-EYE to your PC with a Micro USB cable; -- Launch your development environment tool, such as Terminal (Linux/MacOS) or MinGW (Windows). - -### Get ESP-WHO - -To obtain a local copy: open your Terminal (such as Terminal in a Linux environment), navigate to the directory you want to put ESP-WHO, and clone the repository using `git clone` command: - -``` -git clone --recursive https://github.com/espressif/esp-who.git -``` - -ESP-WHO will be downloaded into an automatically generated folder named `esp-who`. - -> Do not miss the `--recursive` option. If you have already cloned ESP-WHO without this option, please run another command to get all the submodules: `git submodule update --init --recursive` - -### Set UP Path to ESP-WHO - -Please follow the instruction described in [Setup Path to ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/v3.1.1/get-started/index.html#get-started-setup-path) to configure the `IDF_PATH` variable to `esp-who/esp-idf`. - -### Download Firmware - -This section describes the steps to download firmware to ESP-EYE (taking the Linux operating system as an example): - -- Power up ESP-EYE by connecting it to your PC with a USB cable. -- Run `ls /dev/ttyUSB*` in your Terminal to check if the board has connected itself to your PC successfully. You will see a newly generated item named something like `/dev/ttyUSB0` in your list after running the command if the connection is successful. -- Navigate to an example directory, such as `cd esp-who/examples/single_chip/recognition_solution`. -- Run `make defconfig` to complete default configuration. -- Run `make menuconfig`, and go to `Serial flasher config` -> `Default serial port` to configure the device name to the item you saw in the second step, something like `/dev/ttyUSB0`. Then, save and exit. -- Run `make flash` to download firmware. - -### Check Logs - -This section describes the steps to check logs with your Terminal (taking the Linux operating system as an example). - -- Launch your Terminal; -- Run `make monitor`. - -> Note: The ESP-EYE development board will reboot after this operation. - -### Key Process - -The figure below describes the workflow of ESP-EYE: - -![esp-eye-workflow](../../_static/get-started/work_flow_en.png) - -#### 1. Voice Wake-up - -ESP-EYE awaits to be woken up after powering up (Red LED on and white LED off). The board wakes up after recognizing the wake-up command "Hi Lexin", then awaits for networking (Red LED flashing and white LED off). Now, users can initiate the networking. - -#### 2. Networking - -Users can connect their PCs or mobile phones to ESP-EYE's Wi-Fi with the following information (by default): - -- Username: esp-eye-xxxx (xxxx indicates the board's MAC address) -- Password: no needed - -Alternatively, users can also follow the steps below to configure the username and password of the board's Wi-Fi: - -- Launch your Terminal; -- Run `make menuconfig`, and complete the configuration as instructed in the figure below: - - ![wifi connection](../../_static/get-started/wifi_connection.jpeg) - -> Note: You will have to restart from the step of downloading firmware after you have reconfigured the Wi-Fi username and password. - -#### 3. Face Detection - -ESP-EYE starts the face detection after networking. Users can see the real-time image, captured by the board, with their browser (address: `192.168.4.1/face_stream`). During this, the red LED is off and the white LED is on. - -#### 4. Face Recognition - -After detecting a face, ESP-EYE will start the face -recognition if there are any enrolled Face IDs stored in the board: - -- When there is a match: the red LED on the board flashes once, and the browser displays **HELLO ID XXX**. -- When there isn't any match: the board shows no signs, and the browser displays **WHO?**”. - -If there isn't any enrolled Face ID, the board continues face detecting. Therefore, please at least enroll one Face ID if you want to start the face -recognition. - -#### 5. Add/delete a Face ID - -The users can add/delete a Face ID after the network is successfully established. - -##### 5.1 Add a Face ID - -![Enroll a Face ID](../../_static/get-started/face_id_enrollment_en.png) - -- Single-click the Side Tactile Button to enroll a new Face ID. At this moment, the red LED is on, and the browser displays **START ENROLLING**; -- Stand in front of the camera, and the face sampling starts automatically. The red LED flashes whenever the board gets a face sample, and the browser displays the numerical order of the current face sample, such as **THE 1st SAMPLE**. By default, the board has to take three samples to add one Face ID. Users can configure the number of samples needed for one Face ID. (Please adjust your position/distance from the camera and try again if you don't see the red LED flashing for a long time). -- After the Face ID enrollment, the red LED on the board is off and the browser displays **ENROLLED FACE ID XXX**; -- The board enters Face Detection after the Face ID enrollment. - -Currently, ESP-EYE can enroll up to 10 Face IDs. Please note that the maximum number of enrolled Face IDs can be configured based on how the users allocate the flash memory. However, we recommend a number that is no greater than 30. - -##### 5.2 Delete a Face ID - -- Double-click the Side Tactile Button to delete an existing Face ID; -- After that, the board deletes the earliest record of all the existing enrolled Face ID. The white LED on the board flashes, and the browser displays: **XXX ID(S) LEFT**. - -#### Troubleshooting - -The board returns to the status of "waiting to be woken up" when there are any network anomalies, such as "network disconnection" and "network timeout". diff --git a/docs/zh_CN/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md b/docs/zh_CN/get-started/ESP-EYE_Getting_Started_Guide.md similarity index 94% rename from docs/zh_CN/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md rename to docs/zh_CN/get-started/ESP-EYE_Getting_Started_Guide.md index 7a661d6..c9fc75a 100644 --- a/docs/zh_CN/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md +++ b/docs/zh_CN/get-started/ESP-EYE_Getting_Started_Guide.md @@ -1,10 +1,10 @@ # ESP-EYE 入门指南 -[[EN]](../../en/get-started/ESP-EYE_V2.0_Getting_Started_Guide.md) +[[EN]](../../en/get-started/ESP-EYE_Getting_Started_Guide.md) ## 准备工作 -* 1 × ESP-EYE V2.0 开发板 +* 1 × ESP-EYE V2.1 开发板 * 1 × Micro USB B 电缆 * 1 × PC（Windows、Linux 或 Mac OS） @@ -192,7 +192,4 @@ ESP-EYE 开发板的工作流程如下图所示： #### 异常情况 -当出现“网络断开”或“联网超时”等异常情况时，开发板会回到“等待唤醒”状态。 - -Positioning of 5Gtowers linked by a “diber backbone” at Millbrook.( Millbrook) -Millbrook 试验场中，通过 “diber backbone” 相互连接的 5G 电信塔。（图片来源：Millbrook） \ No newline at end of file +当出现“网络断开”或“联网超时”等异常情况时，开发板会回到“等待唤醒”状态。 \ No newline at end of file diff --git a/examples/single_chip/detection_with_command_line/README.md b/examples/single_chip/detection_with_command_line/README.md index d4a8d3e..74dcb44 100644 --- a/examples/single_chip/detection_with_command_line/README.md +++ b/examples/single_chip/detection_with_command_line/README.md @@ -72,3 +72,21 @@ Please see [here](https://github.com/espressif/esp-who). The keyword **DETECTED** appears whenever ESP32 detects a human face. +## Advance Configuration + +In this example, several parameters can be configured by customers to better support different customized scenarios. For the detailed description of these parameters, please see [Here](https://github.com/espressif/esp-face/tree/master/face_detection). + +Besides, please see below for the recommended configuration for general-purpose scenarios: + +``` +mtmn_config.min_face = 80; +mtmn_config.pyramid = 0.7; +mtmn_config.p_threshold.score = 0.6; +mtmn_config.p_threshold.nms = 0.7; +mtmn_config.r_threshold.score = 0.7; +mtmn_config.r_threshold.nms = 0.7; +mtmn_config.r_threshold.candidate_number = 4; +mtmn_config.o_threshold.score = 0.7; +mtmn_config.o_threshold.nms = 0.4; +mtmn_config.o_threshold.candidate_number = 1; +``` \ No newline at end of file diff --git a/examples/single_chip/recognition_with_command_line/README.md b/examples/single_chip/recognition_with_command_line/README.md index 13bd4a1..813b02f 100644 --- a/examples/single_chip/recognition_with_command_line/README.md +++ b/examples/single_chip/recognition_with_command_line/README.md @@ -1,6 +1,6 @@ # Recognition with Command Line in Single Chip -This example demonstrates **Human Face Recognition** with a single ESP32 chip (without using any LCD module). ESP32 firstly gets images that are captured by the camera module, then determines if there are any recognized human faces as well as displays its **Recognition Results** in the **Serial Terminal**. +This example demonstrates **Human Face Recognition** with a single ESP32 chip (without using any LCD module). ESP32 firstly gets images that are captured by the camera module, then determines if there are any recognized human faces as well as displays its **Recognition Results** in the **Serial Terminal**. # Preparation @@ -26,7 +26,7 @@ After you've completed the hardware settings, please follow the steps below: ## Connect -The table below lists the specific pins used in this example for connecting the ESP32 module and the camera module. +The table below lists the specific pins used in this example for connecting the ESP32 module and the camera module. | Interface | Camera Pin | Pin Mapping for ESP32-WROVER | | :--- | :---: | :---: | @@ -49,12 +49,12 @@ The table below lists the specific pins used in this example for connecting the | Power Supply 3.3V | 3V3 | 3V3 | | Ground | GND | GND | -> The pin mapping will be slightly different if you use other ESP32 modules. +> The pin mapping will be slightly different if you use other ESP32 modules. In particular, if you are using a **ESP-WROVER-KIT** for your development, whose camera connector is already broken out (the one labeled Camera / JP4), please follow the steps below: -1. Plug your camera module, i.e. the OV2640 module in this example, on the board; -2. Connect the 3V3 and GND pins on the camera module to those counterparts on the board. +1. Plug your camera module, i.e. the OV2640 module in this example, on the board; +2. Connect the 3V3 and GND pins on the camera module to those counterparts on the board. The image below shows a **ESP-WROVER-KIT** development board with a **OV2640** camera module installed on it. @@ -66,7 +66,7 @@ Please see [here](https://github.com/espressif/esp-who). ## Checking Results -1. Put your camera module away from a human face for at least 0.3 m; +1. Put your camera module away from a human face for at least 0.3 m; 2. Open a Serial Terminal by using the command line `make monitor`; 3. Check result at your Serial Terminal, and you will be able to see information as displayed in the screenshot below, which indicates the **Face Enrollment** will start soon: @@ -74,7 +74,7 @@ Please see [here](https://github.com/espressif/esp-who). ### Enrolling a Face ID -To successfully enroll a **Face ID**, ESP32 will collect a certain number of samples of a user's face, which is configurable and 3 by default. To be more specific, by default, ESP32 will collect three samples of a user's face to enroll a new **Face ID**. +To successfully enroll a **Face ID**, ESP32 will collect a certain number of samples of a user's face, which is configurable and 3 by default. To be more specific, by default, ESP32 will collect three samples of a user's face to enroll a new **Face ID**. ![start_enrollment_1](../../../img/enrollment_take_1st_sample.png) ![start_enrollment_2](../../../img/enrollment_take_2nd_sample.png) @@ -96,3 +96,19 @@ ESP32 checks if the newly detected face matches any existing **Face ID**, whenev * If No, the Serial Terminal displays `No Matched ID`: ![recognition_no_matched](../../../img/no_matched.png) + + +## Advance Configuration + +Users can change the configuration by adjusting some macro definitions specified in the `app_facenet.h` file: + +- `ENROLL_CONFIRM_TIMES`: the number of face samples required to enroll one new **Face ID**. By default, this parameter is set to 3, indicating three face samples are required to enroll a new **Face ID**. +- `FACE_ID_SAVE_NUMBER`: the number of **Face IDs** that are allowed to be enrolled. By default, this parameter is set to 1, indicating only one **Face ID** can be stored in the RAM when the system boots up. Users can configure this parameter to a bigger value if they want to enroll more than one **Face ID**. + +Users can also store the enrolled **Face IDs** in the flash of the board, so the existing **Face IDs** won't be lost when the board powers off. To achieve this, please use the following functions, provided in `esp-face`: + +- `enroll_to_flash()`: Stores the enrolled **Face IDs** in the flash +- `read_id_from_flash()`: Reads all the enrolled **Face IDs** stored in the flash +- `delete_id_in_flash()`: Deletes the earliest enrolled **Face IDs** stored in the flash + +For the detailed description of more parameters for face recognition, please see [Here](https://github.com/espressif/esp-face/tree/master/face_recognition).