Can I overlay a video source over depth image

How-To
1

I am trying to use a depth camera to determine when people walk by a wall with far and near clip planes and then project an effect on a wall with the “depth shadow” the camera reads as they walk by.

The manual states.

" Post Effect nodes and Video Processing nodes attached to this node ( Depth Camera / Kinect Source) will operate on the colour data only, unless specifically designed to read the depth data - such as the [Depth Camera / Kinect Colour Key node"

To my understanding, this means Composite Sources will ignore the depth image data and only read the color channel. Am I incorrect? I thought this would be pretty straightforward with just compositing the two systems together but I’ve been stuck in this for a few days now. Thank you for any help.

2

Kinect Depth Comp.dfx (16.6 KB)

This is a little bit fiddly but can be done with a workaround. If you use the depth camera/kinect mesh node you can apply the clipping distances there. The workaround is that you need to put that into a render to texture to create the alpha image.

Another thing you could use is the NVIDI virtual background node, which uses more compute, but it does create a smoother outline on the silhouette then using just the depth with clipping planes