Owli-AI Assist

Who is Owli-AI Assist for?

Owli-AI Assist is designed for blind or severely visually impaired people who want to better understand their surroundings, text, displays, or objects using artificial intelligence. It does not replace personal assistance or personal checks in safety-critical situations, but it can provide additional information in many everyday situations.

The app is designed for simple operation, large controls, voice input, speech output, and TalkBack use.

What does Assist do?

Owli-AI Assist works as a visual assistance system:

1. The camera shows a live preview.
2. You explicitly capture an image with New Scene.
3. Assist sends the image to the selected AI transport.
4. The answer is displayed as text and can be read aloud.
5. You can ask follow-up questions or attach additional images.

Typical questions include:

• What is in front of me?
• Read the text on the sign.
• Which products or objects are visible?
• What is shown on the display?
• Are there obstacles, steps, vehicles, or other relevant areas?

Current AI profiles

Assist uses selectable VLM profiles optimized for different tasks:

• Quick scene overview: short, speech-friendly description with orientation and spatial layout.
• Detailed scene description: more details, orientation markers, and structured notes.
• Document/OCR: faithful reading of signs, documents, labels, tables, or displays.
• Special short-status profiles: for short updates or auto-scan-supported scenarios.

Depending on the profile, image size, compression, model, token limits, streaming, and response style are adapted.

Follow-up questions and additional images

After capturing a scene, you can ask follow-up questions without starting over. Assist keeps the original image as context so questions like “What is written on the left sign?” or “Read the lower section” are possible.

You can also attach additional images, for example a close-up, another angle, or additional text areas.

Auto-scan

Some profiles support auto-scan. Assist can then analyze new camera frames at regular intervals and provide short updates. Auto-scan is an additional, cautious source of information and does not replace safe orientation aids or personal checks.

Voice input and speech output

Questions can be dictated. A short tap can dictate and send immediately; a long press can dictate and edit before sending. Answers can start playing early through streaming TTS while the full answer is still arriving.

The last answer can be replayed, and speech output can be stopped.

AI transport: backend, OpenRouter BYOK, and local Gemma 4

In normal production mode, Assist uses the Owli backend. This is the recommended default and requires no user setup.

Advanced AI settings offer additional expert options:

• Direct OpenRouter BYOK: store your own OpenRouter key, import it by QR code, and use it directly from the device.
• OpenRouter key info: when a direct OpenRouter key is active, the app can show key limit, usage, and expiration information.
• Local Gemma 4: on suitable devices, a local model can be downloaded and used as a transport. The app checks device suitability, storage, and model status.

Accessibility-focused operation

Assist is designed around TalkBack and simple smartphone use:

• large central controls for camera, microphone, and playback
• Reset instead of unintended recapture while a scene is active
• focus guidance after scene capture and voice input
• profile changes only when no request is running and no active scene is locked
• clear states for capture, speech recognition, and running requests
• portrait and landscape layouts with adapted controls

Report AI answer

If an answer is problematic, it can be reported directly in the app. The report includes the answer, a category, your optional comment, and limited technical metadata. Image data is not sent with the report.

Privacy and processing

The live camera preview runs locally on the device. Data is only sent when you actively use an AI function, for example New Scene, a follow-up question, or an additional image.

In the default setup, Assist processes requests through the Owli backend. For backend follow-up questions, the original scene image is sent again so the AI can answer with image context.

In optional Direct OpenRouter BYOK mode, the app sends requests with your stored OpenRouter key directly to OpenRouter. Depending on the feature, this can include the scene image, question text, and additional image attachments.

In optional local Gemma 4 mode, inference runs on-device if the model is ready and the device is suitable. Other app functions, such as help pages or feedback, may still use internet access.

If you use Report AI answer, report data is sent to the Owli backend without screenshots or image files and is currently stored for 30 days for review.

The full version is available in the Owli-AI Assist privacy policy.