Realtime human pose estimation with TensorFlow and PoseNet
How a pretrained model finds body joints in a video feed.
- Answers
- How does realtime human pose recognition with TensorFlow and PoseNet work?
- Topics
- art · data · experiment · game · people · sensors · work
- Full article on willem.com
- https://willem.com/en/2019-12-01_realtime-human-pose-recognition-through-computer-vision/
Summary
PoseNet is a TensorFlow vision model that estimates a person's pose by locating key body joints, like elbows, hands, hips and knees. When I explored it in 2019, it ran in realtime on ordinary hardware because the model was already trained.
Full text
TensorFlow is an open source machine learning platform originally developed by Google. A tensor is a multi-dimensional array of numeric values, and images flow through a chain of steps, cropping, resizing, removing colour, comparing contours, from unanalysed to recognised. The heavy lifting happens when models are trained, so running a trained model does not need very expensive hardware. PoseNet is such a pretrained vision model: it estimates a person's pose by finding key body joints, elbows, hands, hips, knees and ankles. Body detection first uses the Single Shot MultiBox Detector (SSD), a fast algorithm that shrinks the image into simplified feature maps and predicts early, which suits large objects like human bodies. With poses detected live, a person can quite literally become the game controller or drive an interactive art installation.