MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation

crown jewel figure
MeMemo is the first open-source JavaScript toolkit for in-browser dense neural retrieval. We demonstrate the capabilities of MeMemo by developing RAG Playground that enables AI developers to prototype retrieval-augmented text generation (RAG) apps locally in their browsers. With RAG Playground, developers can (A) enter various user queries, (B) search for semantically similar documents from an in-browser vector database, and (C) augment a text prompt with retrieved documents. (D) This allows developers to rapidly test if in-browser large language models generate more reliable responses to the query.
Demo Video
Retrieval-augmented text generation (RAG) addresses the common limitations of large language models (LLMs), such as hallucination, by retrieving information from an updatable external knowledge base. However, existing approaches often require dedicated backend servers for data storage and retrieval, thereby limiting their applicability in use cases that require strict data privacy, such as personal finance, education, and medicine. To address the pressing need for client-side dense retrieval, we introduce MeMemo, the first open-source JavaScript toolkit that adapts the state-of-the-art approximate nearest neighbor search technique HNSW to browser environments. Developed with modern and native Web technologies, such as IndexedDB and Web Workers, our toolkit leverages client-side hardware capabilities to enable researchers and developers to efficiently search through millions of high-dimensional vectors in the browser. MeMemo enables exciting new design and research opportunities, such as private and personalized content creation and interactive prototyping, as demonstrated in our example application RAG Playground. Reflecting on our work, we discuss the opportunities and challenges for on-device dense retrieval. MeMemo is available at
MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation
  title = {{{MeMemo}}: {{On-device Retrieval Augmentation}} for {{Private}} and {{Personalized Text Generation}}},
  booktitle = {Proceedings of the 47th {{International ACM SIGIR Conference}} on {{Research}} and {{Development}} in {{Information Retrieval}}},
  author = {Wang, Zijie J. and Chau, Duen Horng},
  year = {2024},
  urldate = {2024-06-26},
  langid = {english}