This project explored how complex, part-automated to completely automated design systems could change the design industry. The premise of the project was to create a standalone, local system that allowes non designers to realise their vision with no design experience.
The project began in TouchDesigner but developed into a full system complete with live transcription, localised LLMs with custom system prompts, live hand tracking and complex ComfyUI workflows that were excecuted as scripts.
First experimentation with gesture controls in TouchDesigner using the MediaPipe plugin.
Implementation of gesture controls with basic composition and rendering using TouchDiffusion.
Testing multi-subject composition.
Experimenting with input distortion for more abstract, emotive results.
I incorportated the MediaPipe image segmentation models to allow the user to generate masks live. This allows further control over the image in an intuitive, gesture-controlled way.
Input and output for element 1.
Input and output for element 2.
Input and output for element 3.
Individual elements are composed using gestures.
The composition is run through a final workflow to ensure lighting and style are consistent across elements.
