Kavic/LFM3-VL-2B-litertlm

LFM3-2B & LFM3-VL-2B (32k Context LiteRT-LM Bundles)

Official Google AI Edge LiteRT-LM (.litertlm) model bundles of LiquidAI/LFM3-VL-2B with 32,768 (32k) max context length, optimized for on-device deployment across Google AI Edge Gallery, Android, iOS, Linux, and edge accelerators.

Built for the Fulkrum Reports on-device AI intelligence agent.

---

Bundles Provided

File	Type	Max Context	Quantization	Size	Target Environment
LFM3-2B_text_int4_32k.litertlm	Text LLM	32,768 (32k)	dynamic_wi4_afp32	~970 MB	Google AI Edge Gallery & Fulkrum Reports (Runs flawlessly on both GPU & CPU with zero vision delegate errors)
LFM3-VL-2B_int4_32k.litertlm	Multimodal VLM	32,768 (32k)	dynamic_wi4_afp32 + FP32 Vision	~1.65 GB	Fulkrum Reports & Multimodal runtimes (Dual GPU Language Model + CPU Vision Encoder)
LFM3-VL-2B_int4.litertlm	Multimodal VLM	4,096 (4k)	dynamic_wi4_afp32	1.2 GB	Fulkrum Reports mobile devices
LFM3-VL-2B_int8.litertlm	Multimodal VLM	4,096 (4k)	dynamic_wi8_afp32	2.1 GB	High precision evaluation

Why is 32k context so lightweight on LFM3?

LiquidAI's tri-hybrid architecture consists of:

• 12 ShortConv layers: Fixed 6-step circular state buffers (~24 KB each, constant memory regardless of sequence length).
• 6 Sliding-Window attention layers: Constrained window size (window=256, ~2 MB each).
• Only 6 Full Attention layers: Only 6 layers accumulate KV cache!

For 32,768 tokens, the entire KV cache requires only: $$\text{Memory}=2\times 6\text{ layers}\times 2\text{ kv\_heads}\times \mathrm{32,768}\text{ tokens}\times 128\text{ head\_dim}\times 2\text{ bytes}\approx 192\text{ MB}\backslash text\{Memory\}\; =\; 2\; \backslash times\; 6\; \backslash text\{\; layers\}\; \backslash times\; 2\; \backslash text\{\; kv\backslash \_heads\}\; \backslash times\; 32\{,\}768\; \backslash text\{\; tokens\}\; \backslash times\; 128\; \backslash text\{\; head\backslash \_dim\}\; \backslash times\; 2\; \backslash text\{\; bytes\}\; \backslash approx\; 192\backslash text\{\; MB\}$$ This enables full 32k context on mobile phones with 6–8 GB of RAM without OOM!

---

📱 Google AI Edge Gallery Setup Guide

Which model file should I import in Edge Gallery?

For chatting and text evaluation in Google AI Edge Gallery, always use LFM3-2B_text_int4_32k.litertlm.

Why? Like Gemma-4-E2B-it, Edge Gallery's LLM Chat mode compiles the transformer decoder with the device's OpenCL GPU or XNNPACK CPU delegate. When a multimodal .litertlm bundle is imported into Edge Gallery, LiteRT-LM attempts to initialize a vision encoder delegate which fails compilation (vision_litert_compiled_model_executor.cc:357: Failed to modify graph with delegate). The pure text bundle bypasses the vision executor entirely and runs with complete hardware acceleration on both GPU and CPU!

---

Step 1: Download the Model to Your Phone

1. Download LFM3-2B_text_int4_32k.litertlm from the Files and versions tab.
2. Transfer or save it into your Android device's Downloads or internal storage folder.

---

Step 2: Import into Google AI Edge Gallery

1. Open the Google AI Edge Gallery app.
2. Tap the + (Add Custom Model) button.
3. Select your downloaded LFM3-2B_text_int4_32k.litertlm file.
4. On the Import Model configuration screen, set:
   • Name: LFM3-2B_text_int4_32k.litertlm
   • Model type: LLM
   • Default max tokens: 4000 (or up to 32000)
   • Default TopK: 64
   • Default TopP: 0.95
   • Default temperature: 1.00
   • Support image: OFF (Disabled)
   • Support audio: OFF (Disabled)
   • Support tiny garden: OFF
   • Support mobile actions: OFF
   • Support thinking: ON (Optional: Enables collapsible <think> chain-of-thought blocks)
   • Support speculative decoding: OFF
   • Compatible accelerators: Select [ CPU ] or [ GPU ]
5. Tap Import.

---

Step 3: Chat Settings Configuration

Once imported, tap on the model in Edge Gallery to open the chat:

1. Tap the settings gear icon (top right) to open Configure settings:
   • Max tokens: 4000 – 32000 (Supports full 32k context!)
   • TopK: 64
   • TopP: 0.95
   • Temperature: 1.00
   • Accelerator:
     • GPU (Recommended for fast generation)
     • CPU (For maximum stability across all Android chipsets)
   • Enable thinking: ON or OFF
   • Enable speculative decoding: OFF (LFM3 is a standalone hybrid model)
   • Automatically compress context: OFF
2. Tap OK and start chatting!

---

🤖 Using in Fulkrum Reports (Android)

In Fulkrum Reports, both pure text and multimodal vision modes are supported natively:

// Hybrid Execution: Decoupled GPU text generation + CPU vision encoder
val engineConfig = EngineConfig(
    modelPath = modelFile.absolutePath,
    backend = if (useGpu) Backend.GPU() else Backend.CPU(threadCount = 4),
    visionBackend = Backend.CPU(threadCount = 4),
    maxNumTokens = 32768
)

val engine = Engine(context)
engine.initialize(engineConfig)
val conversation = engine.createConversation()

---

Conversion Pipeline

Reproduce or export custom variants using scripts/convert_lfm3_vl_to_litertlm.py:

python scripts/convert_lfm3_vl_to_litertlm.py \
  --model_id LiquidAI/LFM3-VL-2B \
  --output_dir out/lfm3_32k \
  --prefill_length 128 \
  --cache_length 32768