VDMX6 1.1.6 Update w/ Video Tracking, OCR, Scopes, Color Transfer, and More

[ProjectileObjects]

Hi all,

We’ve been working on a big update here with some new features, you can learn move about it on the blog here: VDMX6 Update – Video Tracking, OCR, Scopes, Color Transfer and more! — VDMX - MAC VJ SOFTWARE

Download:
https://www.vidvox.net/download/VDMX6_1.1.6.dmg
https://www.vidvox.net/download/VDMX6_Plus_1.1.6.dmg

New features and changes:

• New Video Tracking plugin: Faces, hands, and human bodies can be detected and their positions published as data-sources for controlling parameters throughout VDMX. It can also be used to generate masking images and visualizations that can be previewed and used as part of the video processing pipeline.
• New Scopes plugin: The Scopes plugin generates and displays a variety of useful visualizations of the color data for a provided video stream including waveform and vectorscope modes.
• New OCR plugin: Scan images for text and QR codes, publishes detected strings as data-sources to use with text fields, text generators, and anywhere else strings can be received.
• Added example templates for quick setup of Scopes, Face Tracking, OCR, and QR code scanning.
• New Color Transfer & Segmented Color Transfer FX: Shift the color and brightness levels of a video stream to match a reference. Segmented version (macOS 14+ only) allows for selecting different references for the foreground and background.
• New Segmented Blur FX: Applies different levels of blur to the background & foreground of a video stream. (macOS 14+)
• New kMeans Palletize FX: A Palletize style effect that uses kMeans analysis to determine the dominant colors in a video.
• Text input view in Text Generators can be configured to send & receive strings (data-sources and OSC) from the UI Inspector (click text view to inspect). Can be used with the OCR & QR code detection.
• Clock plugin has a play / pause button.
• Clock plugin can now send MIDI Clock out.
• Fixes a bug with Blur Faces / Pixelate Faces in some edge cases with multiple faces.
• Pointillize.qtz, Cube Map.qtz, and V002 Rutt Etra with Displacement.qtz crashed immediately, and are now no longer listed as QC FX
• System CoreImage convolution filters that weren’t usable in VDMX are now hidden.
• Fixed a bug where the UI builder for TD comps didn’t appear in the file inspector
• Fixed bug with attributes being applied on Vuo & TD generator comps in some circumstances
• Minor text label truncation fix in the file inspector
• Minor alert string change for .tox generators that are missing the image output ‘outputImage’
• Fix: still images exported from hap frames weren’t working and still images exported from still images were upside-down
• Fix: copy/pasting between some UI items weren’t working
• Various other bug and crash fixes
• Adds v002 Glitch Analog.fs
• Fixed 1.1.5 issue where 6+ would crash if TouchDesigner application was removed or invalid.

Notes:
TLDR - For the OCR template example to work, you’ll need to have QC enabled. Some QC FX can cause crashes, we’ve removed them, but you may have to clean up the FX in your list if you’ve installed VDMX multiple times.

You can watch these tutorials on the new OCR plugin: Introduction to OCR & QR code capture in VDMX — VDMX - MAC VJ SOFTWARE

Tutorial for video tracking: Tracking faces, bodies, and hands with VDMX — VDMX - MAC VJ SOFTWARE

More tutorials coming soon. Stay tuned!

As always, if you have any issues with this version. Use the built in bug reporter. Help menu > Report Bug.

[jeanpoole]

Thanks for the great update!

The tracking / computer-vision analysis, esp, is super-timely - I’m right in middle of trying to figure out some for a project with a drummer’s hands.

2 Questions… The first - VDMX maths related, the 2nd re: tracking options

VDMX maths
Aka scaling number ranges… How to scale one number range, to a different set of numbers?
eg Slider A - ranges from 0.2 to 0.8 (eg vertical hand tracking data)
How to make that min/max drive Slider B from 0 to 1?

Or if I wanted to use just hand movement in the lower third of the screen (0.2-0.4?), but have this minimal movement give a full 0 to 1 on the 2nd slider, how to do that?

I’ve been trying a few things (eg min/max ranges / Data FX > crop - “Values between the slider’s first and last marks are scaled up to use the slider’s full range”), but can’t get it to work.
Any ideas? What simple step(s) am I missing?

Project attached.

Tracking Options?
I’ve been researching hand-tracking + while the mac version of Touch Designer doesn’t support depth cams, that did lead me to the free Mediapipe TouchDesigner plugin (github)

(See also 30 min YT explainer: Free Motion Tracking plugin for TD: Face, Hand, Pose Tracking
The three part deep dive, and this cool Watercolor Hand Tracking Brush in TD Tutorial)

In the mediapipe implementation of computer vision / tracking - there were a few other useful bits -

• Hand gesture detection (eg fist, peace sign, open palm, thumbs up/down etc)
• Finger position data
• Pinch/zoom - distance between first finger-tip and thumb-tip (gives good control!)
• Face gesture detection (mouth open/closed, eyes etc)

Are these possible within the Apple CV / VDMX framework?
Or is it better to explore those in Touch?

Thanks so much for the update!

TrackingTest.vdmx6.zip (43.7 KB)

[ProjectileObjects]

@jeanpoole

How to scale one number range, to a different set of numbers?
eg Slider A - ranges from 0.2 to 0.8 (eg vertical hand tracking data)
How to make that min/max drive Slider B from 0 to 1?

Lots to unpack here. I guess it depends on what you are trying to do. For example, if you want a single slider to input the values from another, but move from 0 to 1, then you could click on the slider you want to do this with, edit the Num FX Chain, and add a Look Up Curve and adjust the values as desired. The white N will indicate it is being affected by the chain. If you want to use Math, you should know the range (see tutorial below).

There are some quick tutorials on Number FX chains and normalized sliders:

Applying Math Expressions to Slider Receivers — VDMX - MAC VJ SOFTWARE

Creating a 'Falling' audio level data-source using number FX chains in VDMX —...

You can also set min/max ranges on sliders and check or uncheck Scale val to fit within envelopes.

As for additional feature requests with hand tracking. I’m not sure what information can be passed in from AppleVision, but regardless, this plugin and new version of VDMX6 just launched, so before any additional features get added, we’ll mainly focus on fixing and bugs that may arise from all the new additions.

If you need more hand gesture options now, I would see if TD would work for you.