Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference

On-device machine studying (ML) strikes computation from the cloud to private units, defending consumer privateness and enabling clever consumer experiences. Nonetheless, becoming fashions on units with restricted sources presents a significant technical problem: practitioners must optimize fashions and stability {hardware} metrics comparable to mannequin dimension, latency, and energy. To assist practitioners create environment friendly ML fashions, we designed and developed Talaria: a mannequin visualization and optimization system. Talaria allows practitioners to compile fashions to {hardware}, interactively visualize mannequin statistics, and simulate optimizations to check the impression on inference metrics. Since its inside deployment two years in the past, we now have evaluated Talaria utilizing three methodologies: (1) a log evaluation highlighting its development of 800+ practitioners submitting 3,600+ fashions; (2) a usability survey with 26 customers assessing the utility of 20 Talaria options; and (3) a qualitative interview with the 7 most energetic customers about their expertise utilizing Talaria.