1: I found what Im pretty often need possibility to manipulate with points. For example I want to clone them as a particles emitters. To clone a lot of spheres in this case will be very very heavy. So why just not to add “Point” as Shape Type in Shape3D node?
2: We really need possibility to mask people buy Player Index for all versions of Kinect implementations. Current “Near/Far plane” solution is not to useful for most “production” cases.
3: Will be amazing to have a possibility to use Depth Camera/Kinect mesh node with pre-recorded depth video from Video In Source node or streaming from NDI(then will be possible to process video data from Kinect even if it not connected to the Notch directly).
1 Like
Hi Mark,
On the Kinect side of things:
- Player index is notoriously bad quality (lowres) and has stability issues (sometimes the mask isn’t fully complete with pixel tiles missing and sometimes the body / player index will not be detected at all).
For this reason we haven’t been in a hurry to add support for it. The only good solution for masking / segmentation would be a computer vision or ML based segmentation. The same is for K2 and K4A sensors, the K4A isn’t much of an improvement if you do a search on google for r&d done by others.
- This is already supported, you can record Kinect streams and play back - http://manual.notch.one/0.9.23/en/topic/kinect-setup. Load recorded kinect streams in using the resource browser and assign the Kinect Stream input on the Kinect Source node.
Thanks,
Neil
Thank you for your answer!
About Kinect. Well, we all know what Kinect is giving not so perfect data for body tracking to, but its not stoping people from using it. Same with masks. What they have right now is good enough for most things which people are doing with it. Near/far plane also not giving u perfect cut but u guys using it in each Kinect tutorial ))) and People not really need it for cutting video. We need Player Index for isolating point clouds or trace or using it as masks for emitters etc. In this case edges problems doesn’t really noticeable and even if its buggy no one cares. For Kinect 2 its pretty stable and me and my friends are constantly using it in TD. Just yesterday I performed with it for 8 hours without any problem(TD, K2, based on this setup https://t.me/boringspace/61 ). For Azure with last sdk update it also became to be not bad at all. Besides as I know for Azure index they are already using ML for this masking isnt it? (at least for body tracking for sure)
I tried external ML solutions to. Not mentioning what in most cases it needs Linux Its way far from realtime/good quality. Not even talking about stability and user friendlies.
About Depth video. Current problem what not every depth video is coming directly from Kinect or pre recorded from Kinect. But after all its just black and white data. So why should it be related to Kinect then? For example I easily could handle first “Player Index” problem just buy connecting Kinect to TD and streaming depth and index passes over NdI to Notch. But Depth Camera/Kinect Mesh is reject to understand it.
1 Like
Hi Mark,
Sorry for the late reply, indeed it sounds like adding player index would be useful for things like pointcloud effects etc, just not reliable if you need a hiQ capture. Will note it down as an addition for a future release.
A problem I can think of with taking a depth stream from elsewhere is knowing the intrinsic and extrinsic parameters from the config of the capture which are set on the device when originally capturing the depth and colour footage from the K4A, we cannot derive these from raw footage. There are a number of stream params that are set on the Kinect Source node and this is fed into the Kinect Mesh node to enable it to project the pointcloud, do filtering and construct the mesh. There might be another workflow in Notch that is a workaround but I don’t know about these alternative workflows. Your best bet is to send a support email in and one of our workflow specialists might be able to find a workaround for you.
You might want to add your point primitive suggestion into that support email too so we can log it and schedule for a future release if it is viable.
Thanks,
Neil
Hi Mark, jsyk. Player Index masking has been added (on K4a only) and is currently in testing for the next 0.9.23 build.
1 Like
Amazing! Thank you guys!