Kyle Phillips
Engineer & Creative · Google NYC
Creatability
Making creative tools accessible for everyone.
October, 2018
Creatability is a collection of experiments built collaboratively with the disability community that use Chrome + AI to explore how creative tools can be more accessible. Keyboard, Sound Canvas, Body Synth, Seeing Music, Clarion Lite, Word Synth, Sampler all explore unique modalities for creation.
One of my central contributions was an open-sourced library of Web Components solving many complex accessibility issues and providing novel modes of interaction such as using PoseNet with Tensorflow.js to enable anyone to control a cursor with their nose, wrist or many other body parts with nothing other than a basic webcam.
Creatability won the Cannes Grand Prix for Design in 2019, the Webby Awards Gold for Technical Achievements and has been exhibited across the world.
The collaborative nature of these projects were amongst some of the most rewarding for me. I will never forget the first time I watched one of our blind partners drawing with Sound Canvas or when I attended a Flaming Lips concert with Jay Allen Zimmerman and watched him pull out Seeing Music from his bag to enjoy the show.
We created these open-source components in order to encourage others to create more accessible creative experiences a couple of my favorites came from Luisa Pereira:
- Nose theremins and light painters
- Making Your Own Musical Instruments with P5.js, Arduino, and WebMIDI
My role on this project was Lead Creative Technologist, I created the components used on all experiments as well as invented the novel user interface elements alongside my Creative Director Alexander Chen and fellow technologist Yotam Mann.
Creatability was originally released as a featured collection on Experiments with Google.
Selected Press
3 References
The longer version: I'm a hands-on technical lead, responsible for architecting and building the systems that turn our ideas into reality. I'm most proud of my contributions to projects like Teachable Machine, where my primary focus was engineering the intuitive, no-code interface that has empowered millions of people to experiment with machine learning and train their own models for the first time using transfer learning. This passion for inclusivity was also the guiding force behind Creatability, a series of experiments developed in close collaboration with the disability community. For this project, I translated their needs and insights into a library of open-source web components and novel assistive interfaces, such as controlling a website through body movement using PoseNet.
- Grand Prix for Design – Creatability (2019)
- Gold, Technical Achievements – Creatability (2019)
This quality — a display that invites collective, spontaneous authorship — connects to a thread running through much of my work: the idea that the best interfaces dissolve into the experience itself. In Teachable Machine, the complexity of machine learning disappears behind a webcam and a few examples. In Creatability, the computer fades behind body position and sound. With Anypixel, the technology behind 5,880 custom circuit boards fades behind the simple pleasure of pressing something and making it light up.
AutoDraw marked a key moment in my professional development, initiating a thread of exploration into the "interface for AI tools." This work paved the way for future projects like Teachable Machine and Creatability, where the focus remained on making complex technology feel simple, playful, and human. Today, this lineage continues in my work with multimodal LLMs and the Gemini Live Web Console, as I continue to invent future UIs for creative expression.