Building vision systems with AI in the real world

Adam Davis, a senior engineering manager, presents Pixelmesh - a project designed to treat a large group of mobile phones as a single, synchronized display, similar to PixMob wristbands used at Coldplay concerts. The presenter details the technical evolution of the project, moving from an initial version using AprilTags to a more robust system utilizing Morse code for device identification via camera vision. A significant portion of the talk focuses on the role of AI in the development process, highlighting both its utility in handling boilerplate code and its limitations in understanding complex physical environments and real-world edge cases. The presentation includes a live demonstration where the presenter attempts to calibrate and control the audience's phones to create visual effects like ripples and gradients, while also showcasing interactive mobile games. The session concludes with a Q&A regarding security, observability, and the future of AI in solving physical-world engineering problems.