Failed experiments with depth cameras
I wanted to experiment with using depth cameras to render live input as point clouds. For this, I tried using both the Microsoft Kinekt and Intel Realsense cameras. I had the idea to try to do this using several applications; Notch, TouchDesigner and Unity.
The initial idea around what I wanted to try and achieve was to bring a point cloud capture of a live dancing audience into a 3D environment for further visual effects processing, also to capture live performers in the same way. It would be interesting, I think, to use this data visualisation for live projections to accompany performances, or to present a 3D space that people can experience in a VR headset where they get a feel for the spatial composition of a piece and can move around it and interact with it, this could be live or experienced after a performance.
The first barrier was my Mac operating system which I ran into issues with right away.
The next barrier was a lack of documentation and articles on this process around the web.
When I finally got a PC things were better, until the Kinect I had broke. The RealSense camera I couldn’t get to work because of too many SDK installs required.
The limited-time I had for this work meant I had to end my explorations in this area. It was simply too much time and effort to get a working system up and running.
It’s worth noting that I could have prototyped my ideas using point cloud recordings captured by other people. This would have eliminated the device set-up stage of work, and meant I could go straight to data processing and testing ideas. In hindsight, I realise I should have taken this approach. I then might have known whether the results warranted going back to the live depth camera-input setup stage and spending time getting it right.
It’s also worth noting that I think this process will become easier with the advent of LIDAR and live composite depth and image capture becoming available on mobile devices such as the iPhone 12 and newer. So I will shelve this idea for now, but watch the space for ongoing tech developments to make this easier.