Meta Tech Podcast

• 72: Multimodal AI for Ray-Ban Meta glasses

  In this episode of the Meta Tech Podcast, host Pascal sits down with Shane, a research scientist at Meta, to explore the cutting-edge research behind Ray-Ban Meta glasses. Shane shares insights from his seven-year journey at Meta, where he focuses on computer vision and multimodal AI within the Wearables AI organization. Tune in to learn how Shane's team is pioneering foundational models for Ray-Ban Meta glasses, tackling unique challenges, and pushing the boundaries of AI-driven innovation. Discover how multimodal AI is transforming user experiences and get a glimpse into the future of wearable technology. Whether you're an engineer, a tech enthusiast, or simply curious about the latest advancements, there is something for everyone in this episode.  Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model - https://arxiv.org/abs/2309.16058  Be My Eyes Programme: https://www.forbes.com/sites/stevenaquino/2024/10/11/inside-the-be-my-eyes-meta-collaboration-and-the-allure-to--impact-humanity/  Meta Open Source on Threads: https://www.threads.net/@metaopensource  CacheLib: https://cachelib.org/  Meta’s AI-Powered Ray-Bans Are Life-Enhancing for the Blind - Wall Street Journal: https://www.wsj.com/tech/ai/metas-ai-powered-ray-bans-are-life-enhancing-for-the-blind-3ae38026  Timestamps Intro 0:06 OSS News 0:56 Introduction Shane 1:30 The role of research scientist over time 3:03 What's Multimodal AI? 5:45 Applying Multimodal AI in Meta's products 7:21 Acoustic modalities beyond speech 9:17 AnyMAL 12:23 Encoder zoos 13:53 0-shot performance 16:25 Iterating on models 17:28 LLM parameter size 19:29 How do we process a request from the glasses? 21:53 Processing moving images 23:44 Scaling to billions of users 26:01 Where lies the optimisation potential? 28:12 Incorporating feedback 29:08 Open-source influence 31:30 Be My Eyes Programme 33:57 Working with industry experts at Meta 36:18 Outro 38:55

  --------  

  39:34
• 71: Translating Java to Kotlin at Scale

  How do you translate roughly ten million lines of Java code to Kotlin? Clicking in your the IDE gets pretty repetitive after a while and doesn’t work if you have custom APIs and requirements for null safety. Eve and Jocelyn, two software engineers on the Mobile Infra Codebases Team have taken on this challenge and talk host Pascal through the unexpected difficulties when embarking on the journey to (close to) 100% Kotlin in our Android codebase. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Meta Engineering Blog - Translating Java to Kotlin at Scale: https://engineering.fb.com/2024/12/18/android/translating-java-to-kotlin-at-scale/  Open-source transformations: https://github.com/fbsamples/kotlin_ast_tools  Mobile @Scale Conference recordings: https://www.youtube.com/watch?v=L7xSnbrk4CI Timestamps Intro 0:06 Introduction Eve 1:11 Introduction Jocelyn 2:15 Team mission 2:44 The scale of Meta's codebase 3:40 Why is there so much code? 4:34 Why migrate to Kotlin? 5:45 Isn't Kotlin slow to compile? 7:51 Why not use Android Studio's converter? 8:28 Nullability differences 10:04 Meta Codemod Service 14:50 Kotlin codemod stages 17:07 Headless J2K 20:14 Open-source transformations 23:14 Java Nullsafe 24:47 Leveraging Linters 26:01 Fixing build errors 27:24 Unexpected challenges 29:33 State of the union 33:44 Outro 36:10 Outtakes 37:08

  --------  

  38:09
• 70: Jetpack Compose at Meta

  Introducing a new Android UI Framework like Jetpack Compose into an existing app is easy right? Import some AARs and code away. But what if your app has specific performance goals to meet, has existing design components, integrations with navigation and logging frameworks? That is where Summer and her team come in who handle large-scale migrations for Instagram. They aim to provide developers with the best possible experience when working on our code bases, even if that requires some temporary pain on the side of infrastructure teams that have to maintain multiple implementations at once. Why Summer thinks it is worth it, how they approach the rollout of a new framework and so much more is all discussed in episode 70. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Jetpack Compose: https://developer.android.com/compose Litho: https://fblitho.com/ Google Showcase: Meta built threads in only 5 months using Jetpack Compose: https://android-developers.googleblog.com/2023/10/meta-built-threads-in-only-5-months-using-jetpack-compose.html Flipper: https://fbflipper.com/ Timestamps Intro 0:06 Intro Summer 1:29 Notable differences moving from FB to IG 2:26 The Instagram Data & UI Architecture team 2:58 Why modernise? 3:44 Where has the risk paid off? 6:08 What does Compose look like? 7:49 Compose v Litho 11:15 Where does Litho still have the upper hand? 14:53 Meta contributions to Compose 16:38 Compose pitfalls 19:10 Rolling Compose out across the company 20:13 Design systems 22:12 Downsides of establishing another UI framework? 24:22 Rollout stages 28:43 Experimentation stage 32:32 Closed enrollment phase 38:15 Graduation criteria 39:38 Outro 42:20 Bants 44:04

  --------  

  44:30
• 69: To type or not to type — measuring productivity impact with DAT

  Do types actually make you more productive or is it just more typing for you to do on the keyboard? That's just one of the questions we managed to answer at least on a small scale with Diff Authoring Time or DAT, here at Meta. Want to know how we leverage metrics to run experiments on productivity in our internal codebase? Tune in to episode 69. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Hack language: https://hacklang.org/  Timestamps Intro 0:06 Henri Intro 1:45 Ian Intro 3:13 Moritz Re-Intro 3:28 DAT Recap 3:48 What is Hack? 4:20 Inner and outer loop 14:13 Experimenting with language features 17:47 Code sharing frameworks at Meta 27:43 Measuring framework productivity 29:01 Will we see more experiments? 34:23 Time savings from code sharing 37:28 Outro 39:03 Blooper 39:52

  --------  

  40:21
• 68: How to Build a Mixed Reality Headset

  How do you build your own mixed reality headset from sketch to scale? That's exactly what Alfred Jones, VP of hardware engineering at Meta Reality Labs, discussed with host Pascal. From choosing the right display technology, battery, thermal budget and of course hitting the right price point. How he manages to not fall victim to choice paralysis and so much more in episode 68. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod) or Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Caddy: https://engineering.fb.com/2024/07/18/virtual-reality/caddy-cad-mixed-reality-mr-meta/ Timestamps Intro 0:06 Alfred Introduction 1:40 Who do you work with? 3:23 Decision making frameworks 5:20 Is MR the final destination? 7:19 What makes good passthrough such a challenge? 10:18 How to build your own MR headset 13:51 Hardware design constraints 19:00 Prototype phases 22:34 Durability testing 26:23 Dogfooding at Meta 28:55 Magic wand for technical limitations 31:56 Outro 34:26

  --------  

  34:52

Weitere Technologie Podcasts

Trending Technologie Podcasts

Über Meta Tech Podcast

Meta Tech Podcast: Zugehörige Podcasts