Following Last week

For this week, I specifically focused on configuring the depth camera (Kinect Azure) so that it can track certain parameters and use it in the visuals. Continuing the previous week visual: watercolor drawing, I used Azure’s body part recognition and selected hands to be the element that I track. TL:DR, what I am doing is basically as follows:

Essentially, how the visual works is that I am using a circle shape to be used as a pointer. This circle would then apply paint when it detects movement from the hand to the screen. The paint would then disperse and fade after some time, mimicking painting using watercolor.

How It Went

Apparently, there were a lot of things to consider when selecting certain elements from Kinect Azure. What I mean by this is that the camera by default has built-in recognition system for the entire skeleton plus all things accompanying it: hands, finger, legs, pelvis, head, etc. Plus, it is split between three coordinate systems: x,y,z meaning that I have to transform 3D information to 2D and then applying to my visual.

Challenges and Worries

One of the most challenging part about this prototype was simply understanding the amounts of information Kinect Azure spews out, and processing what to do with the copius informations. I had to scour through YouTube tutorials to get a glimpse and only after experimenting for three hours that I “kind of” understood how this specific element on body tracking works.

Second, because I was sick throughout the weekends, I wasn’t able to regroup with Yaakulya to join our design and see how things look, feel, and sound. But hearing from his side and how things have been working out, all gonna be gucci gucci.

Given that we were only able to get equipment booking fully checked out this week, we were not able to hand the projector (and supposedly cameras). Thankfully, Professor Nimrah approved it, so it’s out of the way for now. Meaning for next week, our biggest hurdle will be figuring out the floor planning and equipment hanging. This worries me a lot because once things are properly secured, every element has to be reconfigured so that it works. My biggest fear is that the tracking from the top is very different than front. Plus, how do we hide ALL THIS TECH?!

But like the old saying, blame the player not the game. Adapt, improve, overcome~

External Resources

Custom Depth Map from Kinect & RealSense Point Clouds in TouchDesigner Kinect 2 vs Kinect Azure in TouchDesigner