Bodytracking with webcam and difference

Category: Experimentation, TouchDesigner

Movement/Difference based Body Tracking

I began experimenting with body tracking using just my Mac and webcam. Here’s the process I explored:

  1. Video Device In TOP Captured a live feed from my webcam.

  2. Cache TOP Delayed the feed by five frames with a .

  3. Composite TOP to layer the live and delayed feeds, selecting the “Difference” operation to highlight moving elements (in this case, my body’s contour).

  4. Level TOP: Adjusts the contrast and brightness of the difference output to make the moving elements more defined.

  5. Chroma Key TOP: Filters out colors that don’t match the intended mask range, isolating the moving elements further. This is typically used to remove unwanted background elements.

  6. Edge TOP: Detects the edges of the defined mask, highlighting the outline of the moving elements.

  7. Blur TOP: Smooths the edges of the mask to reduce harsh transitions, creating a cleaner and more natural-looking mask.

  8. Level Final mask TOP: A final level to make the contrast and thicker and sharper. This represents the fully processed dynamic mask; Which can now be used in the interactive installation for tasks like revealing hidden visuals.

Final difference nodes TouchDesigner Node Setup

Mask result

This method offered a rudimentary way to detect movement to create body outlines. While functional, the results are far from precise, and the limitations of this tracking method became apparent.

I then quickly connected this to the static mask reveal to make the mask dynamic

Mask connected to basic mask nodes

Masking reveal

Download the project file

MaskRevealBasic-Difference.toe

Challenges with Movement based Tracking

  • Accuracy: The generated mask was inconsistent and lacked the detail needed for clean projections.
  • Lighting: Changes in light significantly affected detection, making it unreliable for low-light environments.
  • Lack of Depth: Without depth sensing, capturing precise body contours remains a challenge.

Next Steps

Experiment with the NVIDIA bodytrack CHOP on windows