Talk Intermediate

Building Local, Secure Gen AI apps on Mobile

Rejected

Session Description

As use cases for Generative AI permeates to edge and mobile devices, there are a lot of great use cases for building vector search / RAG applications running directly on users mobile devices. One of the common concerns with using cloud solutions for LLM inference is data security, privacy and latency.

This session addresses all of these concerns by introducing open source tooling capable of running fully fledged Generative AI privately, securely and on device in Android such as Onnx ML framework for embeddings, Gemma open source model for LLM, Sqlite Vector search.

So the biggest takeaway for the audience is learning tooling and how to build on device Gen AI applications for mobile with detailed code examples to build secure AI applications that run completely locally on users devices.

Key Takeaways

Benefits to the Ecosystem
The biggest use-case for the mobile and AI dev community is the fact that the same models can now run efficiently on device thanks to optimizations like WASM, Onnx, and just better open source SLM support. The speakers come with building real world use-cases for customers which shares success stories for running on Device AI applications

References

https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference

https://onnx.ai/

Session Categories

Introducing a FOSS project or a new version of a popular project

Tutorial about using a FOSS project

Contributing to FOSS

Technology architecture

Engineering practice - productivity, debugging

Technology / FOSS licenses, policy

Story of a FOSS project - from inception to growth

Which track are you applying for?

Main track

Speakers

Shivay Lamba Developer Experience Engineer | Couchbase

Shivay Lamba is a software developer specializing in DevOps, Machine Learning and Full Stack Development.

He is an Open Source Enthusiast and has been part of various programs like Google Code In and Google Summer of Code as a Mentor and has also been a MLH Fellow. He is actively involved in community work as well. He is a TensorflowJS SIG member, Mentor in OpenMined and CNCF Service Mesh Community, SODA Foundation and has given talks at various conferences like Github Satellite, Voice Global, Fossasia Tech Summit, TensorflowJS Show & Tell.

Reviews

Good topic, but another detailed proposal also touches upon local and secure on-device/mobile AI in a real-world, large-scale, user-facing FOSS app.

Reviewer #1 Rejected

Agreed with the other reviewer, I prefer the on device ML proposal with a real world implementation in a consumer FOSS app.

Reviewer #2 Rejected

While the topic was considered good, the reviewers noted that the conference this year has received multiple proposals focused on local and secure on-device AI, while the others provided a real-world implementation in a consumer FOSS app with more practical and tangible takeaways for the audience.

Reviewer #3 Rejected