Azure Kinect is a camera used to track body data which is then fed into Touchdesigner using a touchin operator. I focused mainly on the x position of the hand to drive the interaction.
Instancing
In order to create a mass array of cubes to rotate I used instancing which allows you to replicate one item multiple times. This also works with texturing. Unfortunately the extent that I wanted to instance for this project was not achieved. I could not get the network to instance rotation of the cubes so for now I have them rotating altogether.
textures
I created prints that would be mapped onto each cube to bring the organic aspect of this piece. I used cubemaps to wrap textures on each face of the cube. I managed to instance this by using position data to control which cube gets what texture. Naming conventions were important here as well as not having the network be filled with base comps. Touch doesn't recognize base within a base for this function.
PERPLEXITY
I used AI software called perplexity to guide me in some areas of my process. Ultimately, it wasn't able to give provide me real solutions to many problems I had but smaller tasks worked quite well. For instance, figuring out how to create position data to map textures.
THE SETUP.
My network in Touchesigner is shown below. Shall we take a closer look? I have sections that show the different parts of my file that create the look of this piece. First we should look at the assets.
NATURE TAKOVER
I wanted to incorporate man made techniques in art with this form of media. I think the combination of organic and mechanical is quite interesting so this is my intended goal for this project.
Collaborators- Kiu and Casper and Nathan (thank you hehe)
TOUCHDESIGNER PROJECT
INTERACTIVE CUBES
The goal of this project is to create interactive cubes that are controlled by hand movements tracking using Azure Kinect.
round 1
Printmaking
I'm in love with the process of using my hands to create and digitally modifying images and textures from prints. For this case I used Monotype printing methods to create my imagery.
Initial mark making experiments. I liked how to plant material looked the best
Monotype steps
1. prepare ink station + printing press
2. roll ink onto acrylic surface
3. remove or cover areas to create marks
4. place damp paper on top of surface
5. use printing press to create pressure and transfer image
6. done!
DIGITAL MODIFICATION
round 2
The plant textures came out great! I mimicked the way vines curl using acetone and cloth the remove ink.
Using photoshop I added some adjustments to elevate the look and feel of the prints.
Slice tool in photoshop to cut image.
INTERACTION
As you can see the interaction is controlled by the x position of the right hand. It's pretty intuitive since automatically people would use their right hand to interact with something. This is thinking generally so there is a chance that left handed people would do differently.
INITIAL WANTS
Initially I wanted to have each cube rotate separately depending on the position of the hand relative to the cube. However, I couldn't figure out how to instance this. I tried multiple methods but all came with errors.
plans of action
Since Instancing didn't work, I will do it one by one. Having less cubes will make it a lot easier and maybe make it feel more intentional.
trials and tribulations explained
coding
I tried coding with perplexity to create some solutions . It would come up with errors. This case was supposed to allow each cube to rotate in 90 degree integers.
perplexity
Since I don't know code I asked perplexity to generate some code or solutions to the problem I had.
render pick
Operator Snippet
In Touch Designer something amazing is that they have examples of how some operators are used. This was a useful one to try and modify for my needs. However, it wasn't registering a new trigger.
Final look
Roots touch concrete
the Earth knows
what we forgot.
thoughts?
Using AI is helpful but very frustrating when it comes to giving operator help in Touchdesigner!! However, with coding it's a bit better. You will find that it will give you operator names and functions that don't exist.