vcc_gnd
/
esp-who
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update readme

XiaochaoGONG 2018-11-19 17:20:41 +08:00
parent f2403dc556
commit 512388443e
1 changed files with 31 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

73
README.md
View File

@ -1,12 +1,12 @@
# ESP-WHO # ESP-WHO
ESP-WHO is a face detection and recognition platform that is currently based on Espressif Systems' [ESP32](https://espressif.com/en/products/hardware/esp32/overview) chip. ESP-WHO is within Espressif Systems AI framework, with the focus on face detection and recognition so far. ESP-WHO is a face detection and recognition platform that is currently based on Espressif Systems' [ESP32](https://espressif.com/en/products/hardware/esp32/overview) chip.
# Overview # Overview
ESP-WHO supports development of face detection and recognition applications based around Espressif Systems' [ESP32](https://espressif.com/en/products/hardware/esp32/overview) chip in the most convenient way. With ESP-WHO, you can easily build up face detection- and recognition-featured applications, for instance: ESP-WHO supports development of face detection and recognition applications based around Espressif Systems' [ESP32](https://espressif.com/en/products/hardware/esp32/overview) chip in the most convenient way. With ESP-WHO, you can easily build up face detection- and recognition-featured applications, for instance:
* A coffee machine that brews coffee according to your taste preference via ESP-WHO; * A coffee machine that brews coffee according to your taste preference;
* Home applicance that will shut off the electricity automatically if ESP-WHO recognizes that unsupervised children are operating them; * Home applicance that will shut off the electricity automatically when unsupervised children are operating them;
* And many more applications that suit your needs. * And other more applications that suit your needs.
In general, the ESP-WHO features will be supported as shown below: In general, the ESP-WHO features will be supported as shown below:
@ -15,48 +15,37 @@ In general, the ESP-WHO features will be supported as shown below:
In ESP-WHO, Detection, Recognition and Image Utility are at the core of the platform. In ESP-WHO, Detection, Recognition and Image Utility are at the core of the platform.
* **Image Utility** offers the fundamental image processing API for ESP32. * **Image Utility** offers fundamental image processing APIs for ESP32.
* **Detection**, or MTMN, uses [MTCNN](http://cn.arxiv.org/abs/1604.02878) and [MobileNet](https://arxiv.org/abs/1801.04381) for reference. However, we replace CNN with MobileNet block and the model size is quite smaller than other face detection models. With special training schemes, MTMN could achieve good performance in detection in some applications scenarios. * **Detection** takes images as input and give the position of face if there is a face. It is implemented with MTMN model, which refers to [MTCNN](http://cn.arxiv.org/abs/1604.02878) and [MobileNet](https://arxiv.org/abs/1801.04381). However, we replace CNN with MobileNet block and the model size is quite smaller than other face detection models.
* **Recognition** is implemented with MobileFace and will be released soon, accompanied with technique details. * **Recognition** is to identify the particular person, and it needs the results of detection. It is implemented with MobileFace model.
Both input and output are flexible. Both input and output are flexible.
* Image sources could be **input** via camera, Wi-Fi socket and etc. However, we don't provide many drivers right now, but drivers for other camera modules, Wi-Fi socket and etc will be released very soon. * Image sources could be **input** via camera. However, we don't provide many drivers right now, those for other camera modules will be released in the future.
* Results could be **output** and displayed on Command line, LCD or even website via Wi-Fi http_server. * Results could be **output** and displayed through Command line, LCD or even website via Wi-Fi http service.
# Quick Start with ESP-WHO # Quick Start with ESP-WHO
## Hardware Preparation ## Hardware Preparation
To run ESP-WHO, you need to have a development board that integrates a **ESP32 module** that has sufficient GPIO pins and more than **4 MB** external SPI RAM. In this case, we take **ESP-WROVER-KIT** as the test board, which is highly recommended for getting started with ESP-WHO. To run ESP-WHO, you need to have a development board which integrates a **ESP32 module** that has sufficient GPIO pins and more than **4 MB** external SPI RAM. In this case, we take **ESP-WROVER-KIT** as the test board, which is highly recommended for getting started with ESP-WHO.
On how to select ESP32 module for your applications, please refer to the README.md of each example. On how to configure ESP32 module for your applications, please refer to the README.md of each example.
## Software Preparation ## Software Preparation
### Image ### Image
The recommended resolution of input image is **320x240** in **RGB565** format. The recommended resolution of input image is **QVGA (320x240)**.
As for choosing camera as an image offer, make sure that the ESP32 module you choose offers specific pins your camera needs. If there are pin conflicts, please refer to the pin connection in README.md of each example. As for choosing camera as an image offer, make sure that the ESP32 module you choose offers specific pins that your camera needs. If there are pin conflicts, please refer to the pin connection in README.md of each example.
By now, we have provided the driver of **OV2640**, which is highly recommended as an image offer to get started with. More camera drivers will be released later on.
### ESP-IDF
See setup guides for detailed instructions to set up the ESP-IDF:
* [Getting Started Guide for the stable ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/stable/get-started/)
* [Getting Started Guide for the latest (master branch) ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/latest/get-started/)
In this case, we take [ESP-IDF v3.1](https://github.com/espressif/esp-idf/releases/v3.1) as the test version.
By now, we have provided the driver of **OV2640**, which is highly recommended to get started with.
### ESP-WHO ### ESP-WHO
Make sure you clone the project recursive: Make sure you clone the project recursive:
@ -67,42 +56,42 @@ git clone --recursive https://github.com/espressif/esp-who.git
If you clone project without `--recursive` flag, please go to the `esp-who` directory and run command `git submodule update --init` before doing anything. If you clone project without `--recursive` flag, please go to the `esp-who` directory and run command `git submodule update --init` before doing anything.
### ESP-IDF
See setup guides for detailed instructions to set up the ESP-IDF:
* [Getting Started Guide for the stable ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/stable/get-started/)
* [Getting Started Guide for the latest (master branch) ESP-IDF version](https://docs.espressif.com/projects/esp-idf/en/latest/get-started/)
In this case, we take [ESP-IDF v3.1](https://github.com/espressif/esp-idf/releases/v3.1) as the test version.
# Components # Components
Components such as camera drivers, image utility, face detection and etc are stored in [components](components) folder. Components 是这个SDK的主要框架，里面有camera相关驱动，算法core等，这些会被大多数的应用所包含。
See the following sections for more details about components. See the following sections for more details about components.
## Camera ## Camera
The [camera](components/camera) component contains drivers for camera devices of ESP32. Now OV2640 is supported. The [camera](components/camera) component contains drivers for camera devices of ESP32. Now OV2640 is supported.
## Image_util
The [image_util](components/image_util) component is designed for fundamental image processing of ESP32. Resizing and transforming are the major operations in ESP-WHO. ## esp-face
## esp_facenet The [esp-face](components/esp-face) component contains the APIs of ESP-WHO neural networks, including face detection and recognition framework.
The [esp_facenet](components/esp_facenet) component contains the APIs of ESP-WHO neural networks. Only face detection is released by now.
- [esp_facenet/mtmn](components/esp_facenet/mtmn): The neural network of MTMN is stored in this folder. Codes in [mtmn.c](components/esp_facenet/mtmn/mtmn.c) file provide detailed implementations of **P-Net**, **R-Net** and **O-Net** and offer the APIs.
- [esp_facenet/face_detection](components/esp_facenet/face_detection):
The face detection implementation is stored in this folder. Codes in [face_detection.c](components/esp_facenet/face_detection/face_detection.c) call net implementation APIs of [mtmn](components/esp_facenet/mtmn/mtmn.c) and also with some process such as [nms](https://en.wikipedia.org/wiki/Canny_edge_detector#Non-maximum_suppression) to implement face detection.
Face Recognition of ESP-WHO will be stored in here as well.
# Examples # Examples
The folder of [examples](examples) contains sample applications demonstrating the API features of ESP-WHO. The folder of [examples](examples) contains sample applications demonstrating the API features of ESP-WHO.
Take one Face Detection as an example. Take one Face Detection as an example.
1. Get into one example folder `esp-who/examples/single_chip/camera_with_command_line`. 1. Get into one example folder `esp-who/examples/single_chip/detection_with_command_line`.
``` ```
cd esp-who/examples/single_chip/camera_with_command_line cd esp-who/examples/single_chip/detection_with_command_line
``` ```
2. Compile with the latest [ESP-IDF](https://github.com/espressif/esp-idf). You may run `make defconfig` before `make flash` to get the default configurations. 2. Compile the project. You may run `make defconfig` before `make flash` to get the default configurations.
``` ```
make defconfig make defconfig
make flash make flash