Chief of Staff for the Masses: How Meta’s Joshua To Designs Wearables with AI

Listen to this episode from Beyond The Prompt - How to use AI in your company on Spotify. In this episode, Joshua To, VP of Product Design at Meta, shares how AI is reshaping how—and where—we interact with technology. He walks us through Meta’s evolving approach to AR and wearables, why notifications are still the killer use case, and how AI is becoming the “brain behind empathy.”We dig into what it means to build interfaces that understand you, why audio might be the future’s most underrated platform, and how designing for emotion changes everything—from form factor to function. Joshua also reflects on his path from launching a clothing brand to leading design at Google and Meta, and what those worlds taught him about craft, context, and human-centered systems.This one’s for anyone designing AI into the real world—where every interface choice carries weight, and intelligence starts with listening.Key takeaways: Empathy Is the Real Intelligence — Joshua flips the definition of smart tech. It’s not just about outputs—it’s about understanding you. Context, tone, emotion—that’s what great AI will sense and respond to.Design for the Moment, Not the Feed — AR’s killer use case isn’t games—it’s restraint. Joshua shares why the best AI product might just be the one that knows not to ping you. Context-aware computing is the real unlock.Audio Is the Interface to Watch — Forget screens. The most powerful interface might be your ears. From wearables to ambient signals, Joshua explains why audio design is the next big frontier for human-centered AI.AR Isn’t a Feature—It’s a System of Consideration — Joshua reframes augmented reality as quiet, ambient infrastructure. The real power of AR isn’t spectacle—it’s subtlety. It helps you move through the world with less friction, not more.LinkedIn: Joshua To | LinkedInWebsite: Home - Joshua ToMeta: Meta Careers00:00 Intro: Fixing Notifications With AI00:54 Meet Josh: VP of Product Design at Meta02:06 From Hoodies to Hardware: Josh's Journey03:53 The Google Experience: From Ads to Product Management10:37 The Evolution of Google Glass and AR19:12 The Role of AI in Josh's Career22:25 Designing the Future: AR, VR, and Attention Management32:49 Contextually Aware Suggestions33:38 Leveraging Generative AI in Design34:52 AI's Role in Concept Art and Storyboarding41:24 AI Tools and Model Capabilities45:54 The Future of AI and Wearables51:58 Reflections and Takeaways📜 Read the transcript for this episode: Transcript of Chief of Staff for the Masses: How Meta’s Joshua To Designs Wearables with AI  For more prompts, tips, and AI tools. Check out our website: https://www.beyondtheprompt.ai/ or follow Jeremy or Henrik on Linkedin:Henrik: https://www.linkedin.com/in/werdelinJeremy: https://www.linkedin.com/in/jeremyutley Show edited by Emma Cecilie Jensen.