A collaboration of researchers from MIT, Northeastern University and Qatar Computing Research Institute developed a new tool called Face-to-BMI, in which they used computer-vision algorithms and machine learning to infer a person’s BMI from social media images.
Usually, researchers from a variety of backgrounds are interested in studying obesity from all angles. Traditionally, a person would have to accurately self-report their body-mass index (BMI), or see a doctor to have it measured, and this way, state-of-the-art computer-vision techniques are used to infer a person’s BMI from social media images. They used a set of annotated images from the VisualBMI project, which are, in turn, collected from Reddit posts that link to the imgur.com service, and these usually include progress pictures from various training stages.
The BMI prediction system is composed of two stages: deep feature extraction and training a regression model. For feature extraction, they used VGG-Net, trained on general object classification, and VGG-Face, trained on a face-recognition task, and for the BMI regression, the epsilon-support vector-regression models (linear regression in feature space, with epsilon as the error function, where mistakes less than epsilon are ignored) were used due to their robust generalization behavior.
A simple experiment was used to compare the Face-to-BMI system to that of humans. Given face images of two individuals, machines and humans, were requires to tell which one is more overweight. The results have shown that humans slightly outperform the machine for small BMI differences, and there is almost no performance difference for larger BMI differences.
- Zou et al. (2016): Face-to-BMI: Using Computer Vision to Infer Body Mass Index on Social Media, at: arXiv:1703.03156 [cs.CV]