A subset of my contributions in developing over a dozen features across 7 releases for Tilt Brush, including our first cross-platform port.
Technical Art, UX/UI, Art Direction, Pipeline, Marketing Assets, Feature Development, Interaction Design
Genre: Creation tool | Platforms: Oculus Rift, Vive, Windows MR
Languages: C#, HLSL, Cg, GLSL, Python, MEL | Tools: Unity, Maya, Adobe CC
Team Size: 10 | Duration: 17+ releases over 2 years
Tilt Brush on Oculus Rift
This was my first big project on Tilt Brush, also our first port to another platform. Much of the groundwork here gave us insight how to bring Tilt Brush to new hardware.
Much of the UX challenge involved navigating input styles. Our visuals for the Vive relied heavily on the large touch pad, which the Oculus lacked. We explored ways to extend the Touch controller's form factor beyond its small face buttons and joystick.
I created prototypes beginning with sketches and ideas, creating and editing assets, iterating on concepts in-engine, and working with engineers to bring the final prototype to ship.
I authored the Touch controller geometry so we could reuse all existing controller textures for both platforms, regardless of the very different shape of the two controller geometries. This allowed us to use the same textures and materials across both platforms.
Lights & backdrop Panels
The Lights panel allows users to create custom lighting setups by changing the orientation and color of three virtual lights. This panel went through many permutations as I looked for ways to condense lighting concepts into a single panel, integrate color pickers, and work through interaction models.
I ended up pulling from traditional drawing exercises when looking for a proxy for the scene users were lighting, and writing custom shaders to flatten widgets while the panel wasn't in use.
The Backdrop panel allows users to adjust fog density and create a custom environment. Our final design landed on a large sphere with trackball style interaction (for the skybox) and a custom button for fog, using a multi-pass shader I wrote.
The most requested feature from users has been the ability to select and manipulate strokes. I designed and implemented audio, icons, widgets and our outline shader effect.
Through the process, I parsed feedback from the team and grabbed people as they passed in the hall. The final implementation included an idea I prototype for helping users better select/deselect in 3D as a key portion of the design.
There was no documentation to reference when I began creating new brushes in Tilt Brush. I reverse engineered how brushes worked in order to create new ones, sorting out how geometry was generated on the CPU, augmented with shaders on the GPU and brought to a finish using textures and material parameters.
This brush combines 3 separate channels scrolling at different rates generated from one texture, multiplied by a gradient lookup to create its stylized effect.
Made from scans of acrylic paint + specular PBR rendering in the material.
Uses vertex displacement, custom tube brush geometry, and darkened backfaces.
This brush uses intentionally stuttery flipbook animation to recreate the feeling of hand-drawn animation and classic cartoons.
As the project has grown, so has the scope of what's possible when creating art with Tilt Brush.
To help illustrate this, I created a new splash screen for the project.
I created an environment that was larger than what most beginners set out to paint, but tried to balance it with clearly visible strokes and other pieces, to help give folks an idea of where to start and alleviate the anxiety of starting on a blank page. I also created the sound design for the sketch.
Our work launched with new improvements to the sketchbook panel, which I art directed while also working on this new transition state. I was also lucky to have the opportunity to talk about the work I did on this sketch at GDC 2018. Check out my slides here.
shader based TRANSITION
Along with designing the sketch itself, I worked with my creative director on an animation transition for when the sketch loads in. He and I wrote shaders for the effect, animating either the stroke geometry or stroke textures in.
VIsual design & art Pipeline
Creating cohesive visuals
I was asked to redesign and systematize Tilt Brush’s UI/UX across the app. I created a design system that incorporated the bespoke nature of many of Tilt Brush’s UX, redesigned and illustrated many of our icons, and created documentation to reference what the visual style of Tilt Brush is.
Touching every piece of Tilt Brush's interface for consistency, redesigning nearly all our icons, writing documentation for all new and existing patterns, and creating master files with batch scripts for quick export.
Here are a few icons I designed in the process:
I also worked with a remote UI Designer and provided art direction for her work. Here are a few icons she designed:
visual design guidelines
There were no existing guidelines for creating new assets when I joined Tilt Brush. Most elements were tuned by eye and no quick reference to what Tilt Brush's style was. I took 6 weeks to overhaul our UX design to reduce the user's mental load and help build a foundation for new work in future milestones.
Vector to Unity Pipeline
This is a workflow I developed to generate meshes from vector paths in Illustrator, translated to Maya, then exported to Unity. We've used this process for generating new panels, making models conforming to existing geometry, and translating icon designs to 3D elements.
Key art & experimental features
When I'm not working on Tilt Brush directly, I'll provide feedback to the rest of the team by creating art with experimental features. I also help spot issues with parity across the Tilt Brush toolkit and Tilt Brush's online gallery in an effort to make sure all content authored with Tilt Brush looks its best no matter where it's seen.
A fun side effect of this is I have also had the opportunity to create a few sketches for Google marketing to use with their social media accounts.