💡 rewrite ./README.rst
121
README.md
|
@ -1,121 +0,0 @@
|
||||||
# ESP-WHO
|
|
||||||
|
|
||||||
ESP-WHO is a face detection and recognition platform that is currently based on Espressif Systems' [ESP32](https://espressif.com/en/products/hardware/esp32/overview) chip.
|
|
||||||
|
|
||||||
# Overview
|
|
||||||
ESP-WHO supports development of face detection and recognition applications based around Espressif Systems' [ESP32](https://espressif.com/en/products/hardware/esp32/overview) chip in the most convenient way. With ESP-WHO, you can easily build up face detection- and recognition-featured applications, for instance:
|
|
||||||
* A coffee machine that brews coffee according to your taste preference;
|
|
||||||
* Home applicance that will shut off the electricity automatically when unsupervised children are operating them;
|
|
||||||
* And other more applications that suit your needs.
|
|
||||||
|
|
||||||
In general, the ESP-WHO features will be supported as shown below:
|
|
||||||
|
|
||||||
![overview](img/overview.jpg)
|
|
||||||
|
|
||||||
In ESP-WHO, Detection, Recognition and Image Utility are at the core of the platform.
|
|
||||||
|
|
||||||
|
|
||||||
* **Image Utility** offers fundamental image processing APIs.
|
|
||||||
|
|
||||||
* **Detection** takes images as input and give the position of face if there is a face. It is implemented with MTMN model, which refers to [MTCNN](http://cn.arxiv.org/abs/1604.02878) and [MobileNets](https://arxiv.org/abs/1801.04381).
|
|
||||||
|
|
||||||
* **Recognition** is to identify the particular person, and it needs the results of detection. It is implemented with MobileFace model.
|
|
||||||
|
|
||||||
* **Optimization** is mainly to increase the precision of the inference, and to accelerate the whole process. But also it might change the structure of the network, update the coefficients, refactor the code, etc.
|
|
||||||
|
|
||||||
Both input and output are flexible.
|
|
||||||
|
|
||||||
* Image sources could be **input** via camera. However, we don't provide many drivers right now, those for other camera modules will be released in the future.
|
|
||||||
|
|
||||||
* Results could be **output** and displayed through Command line, LCD or even website via Wi-Fi http service.
|
|
||||||
|
|
||||||
|
|
||||||
# Quick Start with ESP-WHO
|
|
||||||
|
|
||||||
## Hardware Preparation
|
|
||||||
|
|
||||||
To run ESP-WHO, you need to have a development board which integrates a **ESP32 module** that has sufficient GPIO pins and more than **4 MB** external SPI RAM. Either [ESP-WROVER-KIT](https://www.espressif.com/en/products/hardware/esp-wrover-kit/overview) or [ESP-EYE](https://www.espressif.com/en/products/hardware/esp-eye/overview) can be a good choice as the test board.
|
|
||||||
|
|
||||||
On how to configure ESP32 module for your applications, please refer to the README.md of each example.
|
|
||||||
|
|
||||||
|
|
||||||
## Software Preparation
|
|
||||||
|
|
||||||
### Image
|
|
||||||
|
|
||||||
The recommended resolution of input image is **QVGA (320x240)**.
|
|
||||||
|
|
||||||
As for choosing camera as an image offer, make sure that the ESP32 module you choose offers specific pins that your camera needs.
|
|
||||||
|
|
||||||
By now, we have provided some drivers of cameras, which are highly recommended to get started with:
|
|
||||||
|
|
||||||
**OV2640**
|
|
||||||
|
|
||||||
**OV3660**
|
|
||||||
|
|
||||||
**OV5640**
|
|
||||||
|
|
||||||
### ESP-WHO
|
|
||||||
Make sure you clone the project recursive:
|
|
||||||
|
|
||||||
```
|
|
||||||
git clone --recursive https://github.com/espressif/esp-who.git
|
|
||||||
```
|
|
||||||
|
|
||||||
If you clone project without `--recursive` flag, please go to the `esp-who` directory and run command `git submodule update --init` before doing anything.
|
|
||||||
|
|
||||||
### ESP-IDF
|
|
||||||
See setup guides for detailed instructions to set up the ESP-IDF:
|
|
||||||
|
|
||||||
* [Getting Started Guide for ESP-IDF v4.0](https://docs.espressif.com/projects/esp-idf/en/v4.0/get-started/index.html)
|
|
||||||
|
|
||||||
We take [ESP-IDF v4.0](https://github.com/espressif/esp-idf/tree/release/v4.0) as the default version.
|
|
||||||
|
|
||||||
> Now this repository supports two versions of IDF: v3.3.1 and v4.0.0.
|
|
||||||
|
|
||||||
|
|
||||||
# Components
|
|
||||||
|
|
||||||
Components is the main framework of the SDK, with some drivers and algorithm inside.
|
|
||||||
|
|
||||||
## Camera
|
|
||||||
|
|
||||||
The [camera](https://github.com/espressif/esp32-camera/tree/master) component contains drivers for camera devices of ESP32.
|
|
||||||
|
|
||||||
|
|
||||||
## esp-face
|
|
||||||
|
|
||||||
The [esp-face](https://github.com/espressif/esp-face/tree/master) component contains the APIs of ESP-WHO neural networks, including face detection and recognition framework.
|
|
||||||
|
|
||||||
|
|
||||||
# Examples
|
|
||||||
The folder of [examples](examples) contains sample applications demonstrating the API features of ESP-WHO.
|
|
||||||
|
|
||||||
Take one Face Detection as an example.
|
|
||||||
|
|
||||||
1. Get into one example folder `esp-who/examples/single_chip/detection_with_command_line`.
|
|
||||||
```
|
|
||||||
cd esp-who/examples/single_chip/detection_with_command_line
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Compile and flash the project.
|
|
||||||
```
|
|
||||||
idf.py build
|
|
||||||
idf.py flash -p [PORT]
|
|
||||||
```
|
|
||||||
3. Advanced users can modify some options by using the command `idf.py menuconfig`.
|
|
||||||
|
|
||||||
The default configuration of all examples is based on [ESP-EYE](https://www.espressif.com/en/products/hardware/esp-eye/overview).
|
|
||||||
|
|
||||||
Check README.md of each example for more details.
|
|
||||||
|
|
||||||
|
|
||||||
# Default bin
|
|
||||||
|
|
||||||
The default bin is [HERE](docs/zh_CN/get-started/default_bin/). You can use [Flash Download Tools](https://www.espressif.com/en/support/download/other-tools) to write the default bin to the ESP-EYE.
|
|
||||||
|
|
||||||
# Resources
|
|
||||||
|
|
||||||
* [Check the Issues section on github](https://github.com/espressif/esp-who/issues) if you find a bug or have a feature request, please check existing Issues before opening a new one.
|
|
||||||
|
|
||||||
* If you're interested in contributing to ESP-WHO, please check the [Contributions Guide](https://esp-idf.readthedocs.io/en/latest/contribute/index.html).
|
|
|
@ -0,0 +1,203 @@
|
||||||
|
##################################################################################
|
||||||
|
ESP-WHO
|
||||||
|
##################################################################################
|
||||||
|
|
||||||
|
ESP-WHO 是基于乐鑫芯片的图像处理开发平台。其中包含了实际应用中可能出现的开发示例。
|
||||||
|
|
||||||
|
概述
|
||||||
|
====================================================================================================
|
||||||
|
|
||||||
|
ESP-WHO 提供了例如人脸检测、人脸识别、猫脸检测和手势识别等示例。您可以基于这些示例,衍生出丰富的实际应用。
|
||||||
|
ESP-WHO 的运行基于 ESP-IDF。`ESP-DL <https://github.com/espressif/esp-dl>`_ 为 ESP-WHO 提供了丰富的深度学习接口。
|
||||||
|
配合各种外设可以实现许多有趣的应用。
|
||||||
|
|
||||||
|
.. figure:: ./img/overview.drawio.png
|
||||||
|
:alt: overview of ESP-WHO
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
ESP-WHO 概述
|
||||||
|
|
||||||
|
..
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
准备工作
|
||||||
|
====================================================================================================
|
||||||
|
|
||||||
|
硬件
|
||||||
|
--------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
按照所有示例的一般流程,可将硬件准备分为三个部分:
|
||||||
|
|
||||||
|
+ 摄像头模块: ESP-WHO 中的所有示例用到的摄像头驱动均来自 `ESP32-Camera <https://github.com/espressif/esp32-camera>`_ ,您可从中挑选摄像头型号;您也可以选择其他摄像头,只是需要自行支持驱动。
|
||||||
|
|
||||||
|
+ 处理单元: ESP32,ESP32-S2,ESP32-S3 中的任意一款。
|
||||||
|
|
||||||
|
+ 显示方式: LCD 等。
|
||||||
|
|
||||||
|
|
||||||
|
对于新手,我们还提供了开发套件 `ESP-WROVER-KIT <https://www.espressif.com/zh-hans/products/hardware/esp-wrover-kit/overview>`_ ,`ESP-EYE <https://www.espressif.com/zh-hans/products/devkits/esp-eye/overview>`_ ,`ESP-S3-EYE <https://www.espressif.com/zh-hans/products/devkits/esp-eye/overview>`_ ,他们的概况如下:
|
||||||
|
|
||||||
|
+----------------+----------+------------------+------------------+
|
||||||
|
| 开发板名称 | 芯片型号 | 摄像头型号 | 显示方式 |
|
||||||
|
| | | +------+-----+-----+
|
||||||
|
| | | | 终端 | Web | LCD |
|
||||||
|
+----------------+----------+------------------+------+-----+-----+
|
||||||
|
| ESP-EYE | ESP32 | OV2640 或 OV3660 | Y | Y | N |
|
||||||
|
+----------------+----------+------------------+------+-----+-----+
|
||||||
|
| ESP-S3-EYE | ESP32-S3 | OV2640 或 OV3660 | Y | Y | Y |
|
||||||
|
+----------------+----------+------------------+------+-----+-----+
|
||||||
|
| ESP-WROVER-KIT | ESP32 | OV2640 或 OV3660 | Y | Y | N |
|
||||||
|
+----------------+----------+------------------+------+-----+-----+
|
||||||
|
|
||||||
|
软件
|
||||||
|
--------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
获取 ESP-IDF
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
ESP-WHO 的运行基于 ESP-IDF。关于获取 ESP-IDF 的细节,请参考 `ESP-IDF 编程指南 <https://idf.espressif.com/>`_ 。
|
||||||
|
|
||||||
|
|
||||||
|
获取 ESP-WHO
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
.. code:: shell
|
||||||
|
|
||||||
|
git clone --recursive https://github.com/espressif/esp-who.git
|
||||||
|
|
||||||
|
.. attention:: 不要忘记 ``--recursive`` 拉取 ESP-WHO 的所有 submodule
|
||||||
|
|
||||||
|
可以通过 ``git submodule update --init`` 拉取和更新 submodule
|
||||||
|
|
||||||
|
|
||||||
|
运行示例
|
||||||
|
====================================================================================================
|
||||||
|
|
||||||
|
示例对开发套件的支持情况如下表所示:
|
||||||
|
|
||||||
|
+------------------------+---------------------------------------+
|
||||||
|
| 示例名称 | 支持的开发板 |
|
||||||
|
| +---------+------------+----------------+
|
||||||
|
| | ESP-EYE | ESP-S3-EYE | ESP-WROVER-KIT |
|
||||||
|
+------------------------+---------+------------+----------------+
|
||||||
|
| Camera with Terminal | Y | Y | Y |
|
||||||
|
+------------------------+---------+------------+----------------+
|
||||||
|
| Camera with Web Server | Y | Y | Y |
|
||||||
|
+------------------------+---------+------------+----------------+
|
||||||
|
| Camera with LCD | N | Y | N |
|
||||||
|
+------------------------+---------+------------+----------------+
|
||||||
|
|
||||||
|
步骤1:设定目标芯片
|
||||||
|
--------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
打开终端,进入一个您想运行的示例,设定目标芯片。
|
||||||
|
|
||||||
|
.. code:: shell
|
||||||
|
|
||||||
|
idf.py set-target [SoC]
|
||||||
|
|
||||||
|
将 [SoC] 替换成您的 目标芯片, 例如 esp32,esp32s2,esp32s3。
|
||||||
|
|
||||||
|
|
||||||
|
步骤2:通用配置
|
||||||
|
--------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
通用配置指的是每个示例都会涉及到的配置,例如,摄像头配置、模型选择等。
|
||||||
|
您可以通过 ``idf.py menuconfig`` 打开配置界面。
|
||||||
|
依次 (Top)-> Component config -> ESP-WHO General Configuration 进入通用配置,如下图所示:
|
||||||
|
|
||||||
|
.. figure:: ./img/esp-who-general-config.png
|
||||||
|
:alt: ESP-WHO General Configuration
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
ESP-WHO 通用配置
|
||||||
|
|
||||||
|
..
|
||||||
|
|
||||||
|
|
||||||
|
步骤2.1:摄像头配置
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
进入摄像头配置,如下图所示,您可以选择:
|
||||||
|
|
||||||
|
+ 摄像头输出的图片格式,
|
||||||
|
|
||||||
|
+ 摄像头的管脚配置。
|
||||||
|
|
||||||
|
.. figure:: ./img/esp-who-general-config-camera.png
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
摄像头配置
|
||||||
|
|
||||||
|
.. attention::
|
||||||
|
摄像头的管脚配置是按照开发套件来区分的。如果管脚配置中,没有您所用的开发套件,请选择 ``Custom Camera Pinout``,正确配置对应管脚,如下图所示:
|
||||||
|
|
||||||
|
.. figure:: ./img/esp-who-general-config-camera-custom-pinout.png
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
自定义摄像头管脚
|
||||||
|
|
||||||
|
|
||||||
|
步骤2.2:指示灯配置
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
进入指示灯配置,如下图所示,您可以配置:
|
||||||
|
|
||||||
|
+ 指示灯的打开和关闭
|
||||||
|
|
||||||
|
+ 指示灯的管脚
|
||||||
|
|
||||||
|
+ 指示灯的明暗
|
||||||
|
|
||||||
|
.. figure:: ./img/esp-who-general-config-led.png
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
指示灯配置
|
||||||
|
|
||||||
|
|
||||||
|
步骤2.3:人工智能应用配置
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
进入人工智能应用配置,您可以根据提示选择您想运行的模型,如下图所示:
|
||||||
|
|
||||||
|
.. figure:: ./img/esp-who-general-config-dl.png
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
人工智能应用配置
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
步骤3:示例配置
|
||||||
|
--------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
部分示例还有自己的配置菜单。进入后,您可根据描述进行配置。示例配置菜单可以在 (TOP) -> Example Configuration 中找到。
|
||||||
|
|
||||||
|
以 ``Camera with Web Server`` 为例,配置菜单如下图所示:
|
||||||
|
|
||||||
|
.. figure:: ./img/esp-who-example-config.png
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
示例配置
|
||||||
|
|
||||||
|
|
||||||
|
步骤4:运行和监视
|
||||||
|
--------------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
.. code:: shell
|
||||||
|
|
||||||
|
idf.py flash monitor
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
开发套件的默认二进制文件
|
||||||
|
====================================================================================================
|
||||||
|
|
||||||
|
各开发套件的默认二进制文件存放在目录 `./default_bin <./default_bin>`_ 下。
|
||||||
|
您可以使用 `烧写工具 <https://www.espressif.com/zh-hans/support/download/other-tools>`_ 烧录二进制文件。
|
||||||
|
|
||||||
|
|
||||||
|
反馈
|
||||||
|
====================================================================================================
|
||||||
|
|
||||||
|
如果您在使用中发现任何问题,请提交相关 `issue <https://github.com/espressif/esp-who/issues>`_ ,我们将尽快予以答复。
|
BIN
img/detected.png
Before Width: | Height: | Size: 21 KiB |
Before Width: | Height: | Size: 30 KiB |
After Width: | Height: | Size: 46 KiB |
After Width: | Height: | Size: 65 KiB |
After Width: | Height: | Size: 33 KiB |
After Width: | Height: | Size: 45 KiB |
After Width: | Height: | Size: 43 KiB |
After Width: | Height: | Size: 30 KiB |
Before Width: | Height: | Size: 1.9 MiB |
Before Width: | Height: | Size: 3.9 MiB |
Before Width: | Height: | Size: 36 KiB |
Before Width: | Height: | Size: 42 KiB |
Before Width: | Height: | Size: 85 KiB |
Before Width: | Height: | Size: 15 KiB |
BIN
img/ov2640.jpg
Before Width: | Height: | Size: 194 KiB |
After Width: | Height: | Size: 40 KiB |
BIN
img/overview.jpg
Before Width: | Height: | Size: 37 KiB |
Before Width: | Height: | Size: 18 KiB |
Before Width: | Height: | Size: 24 KiB |
Before Width: | Height: | Size: 30 KiB |