Real-Time Human Vitals Estimation using Selfie Camera

The goal of the project was to evaluate a person’s health vitals, such as heart rate, oxygen saturation (SpO2), and respiration rate. Mobile phone selfie camera extracted these vitals using the remote photoplethysmography (rPPG) technique.

Input: mobile camera feed

Output: heart rate, oxygen saturation, respiration rate

Technologies & Skills:

• rPPG
• Computer Vision
• Face Detection
• Signal Processing
• Programming Languages: Python, C++, Java, Swift

Project Development Approach 

To measure the heart rate, it was necessary to extract the remote photoplethysmographic (rPPG) signal from the video. This signal is an innovative combination of RGB signals (pixel values from the video frames). Pixel values, specifically color components, should be extracted from the specific regions on a face. Then, the heart rate is estimated via spectral analysis of the rPPG signal.

The solution is based on the blood volume changes in tissue due to the cardiac activity that affects the optical characteristics of reflected light. The rPPG framework included the following steps: face and ROI (region of interest) detection, facial landmarks extraction and tracking, rPPG signal extraction and processing.

Solution & Results

The implemented solutions estimate human vitals from a live camera feed.

• Custom face landmarks detector and tracker.
• Specific signal extraction procedure.
• Optimized signal processing running on the edge.
• Utilization of mobile camera feed.
• Calculation of heart rate, saturation, and respiration rate.