ONE Summit 2022 has ended
November 15-16, 2022
LFN Developer & Testing Forum, November 17-18, 2022
View More Details & Register
Back To Schedule
Wednesday, November 16 • 12:10pm - 12:40pm
WebAssembly Based AI as a Service on the Edge with Kubernetes - Rishit Dagli, Narayana Junior College; Incoming University of Toronto & Shivay Lamba, Meilisearch

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

WebAssembly (WASM) is being adopted at an increasing rate for edge applications. That allows WASM runtimes, such as WasmEdge (a lightweight and high-performance runtime for cloud-native, edge, and decentralized devices), to run serverless functions on the edge. Following the large-scale adoption and benefits of serverless computing, we focus on deploying these as a Function-as-a-service on edge devices. This talk gives a lot of focus to converting your multi-thrreaded rust applications (often the case with using libraries) into WASM compatible deployments.Machine Learning inference is often a computationally intensive task and edge applications could greatly benefit from the speed of WebAssembly. Unfortunately, Linux containers end up being too heavy for such tasks. Demonstrating Machine Learning deployments in such a fashion, another problem we face is that the standard WebAssembly provides very limited access to the native OS and hardware, such as multi-core CPUs, GPUs, or TPUs which is not ideal for the systems we target. The talk also shows how one could use the WebAssembly System Interface (WASI) to get security, portability, and native speed for ML models. To top it off this talk ends with a demo of deploying a Machine learning model as a serverless function using WASM deployed on an edge device.

avatar for Shivay Lamba

Shivay Lamba

Shivay Lamba, WASMEdge
Shivay Lamba is a software developer specializing in DevOps, Machine Learning and Full Stack Development. He is an Open Source Enthusiast and has been part of various programs like Google Code In and Google Summer of Code as a Mentor and is currently a MLH Fellow. He has also worked... Read More →
avatar for Rishit Dagli

Rishit Dagli

Student, Research Scientist, University of Toronto, Civo
I am a CS Freshman at University of Toronto. I love researching and working with Machine Learning, especially Computer Vision (with papers in ICLR, Nature, and ACM SIGGRAPH among others, and have served as a PC/reviewer for NeurIPS, CVPR, ICML). I also maintain/contribute extensively... Read More →

Wednesday November 16, 2022 12:10pm - 12:40pm PST
Willow A