JuicyTalk
Google DeepMind has released EmbeddingGemma 2, an open model that embeds text, code, images, video and audio into one 768-dimensional space. It has 740M parameters, an 8K token context window and an Apache 2.0 license. It targets on-device search, classification and privacy-first RAG. This article analyzes, compares and showcase how EmbeddingGemma 2 fits in the space.
Deployable today? Yes. Weights are live on Hugging Face and Kaggle, with Ollama, llama.cpp GGUF and LiteRT builds available now.
What an Embedding Model Does
An embedding model converts content into a vector of numbers that captures meaning. Similar items land close together, so they are easy to search and compare. In a RAG pipeline, these vectors let an LLM retrieve fresh information it was not trained on. Generating embeddings locally keeps data on the device, cuts latency and works offline.
One Vector Space for Every Modality
EmbeddingGemma 2 is built on the Gemma 4 architecture. A text query can retrieve a photo. A voice memo can retrieve a video clip. Interleaved inputs, like a product listing with text, images and a demo video, produce a single embedding.
The design is modular. It has three parts:
- Text and code backbone: 270M parameters (130M transformer plus 140M embedder)
- Vision encoder: 170M parameters, optional
- Audio encoder: 300M parameters, optional
Developers load only what they need: 270M for text, 440M for text and vision, 570M for text and audio, or 740M for everything. All setups share one vector space. A query embedded with the text-only setup can match documents embedded by the full model.
The context window is 8,192 tokens, 4x larger than version 1. That fits about 29 images, 58 video frames or 5.5 minutes of audio.
Benchmarks
Google research team reports leading scores among sub-1B multimodal embedders on MTEB Code and MAEB. Full-precision results at 768 dimensions:
| Benchmark | EmbeddingGemma 2 | EmbeddingGemma 1 |
|---|---|---|
| MTEB multilingual v2 | 61.36 | 61.15 |
| MTEB Code v1 | 78.68 | 68.76 |
| MIEB lite (image) | 64.64 | n/a |
| MMEB v2 overall | 59.01 | n/a |
| MSEB retrieval (sound) | 69.54 | n/a |
| MAEB (audio) | 49.39 | n/a |
Source: EmbeddingGemma 2 model card
Code retrieval gains 9.92 points, roughly 14%. Multilingual text quality holds steady. Bigger models still lead some boards. Qwen3-VL-Embedding-2B reports 73.2 on its own MMEB-V2 run, with about 2.7x the parameters and no audio support.
Built for Phones and Laptops
With quantization on a Pixel 11 Pro, active RAM is about 191MB for text-only weights. The full multimodal model needs about 567MB. Quantization-aware training compresses weights to INT4 and INT8. The Google AI Edge team measured 37.3 ms per image on a MacBook M5 Pro GPU, using a 70-token vision budget.
Matryoshka Representation Learning (MRL) lets developers truncate vectors to 512, 256 or 128 dimensions. Moving from 768 to 128 dimensions cuts storage up to 6x. At 256 dimensions, MTEB multilingual only slips from 61.36 to 60.41. At 128 dimensions, MMEB drops to 45.65, so Google recommends 128d mainly for text-only workloads.
