Product Information
What is Osaurus?
This machine, Apple Silicon-only local LLM server. Similar to Ollama but built on Apple's MLX to maximize performance on M-series chips. SwiftUI App + SwiftNIO server with OpenAI-compatible endpoints.
Created by Dinoki, a fully local desktop AI assistant and companion.
Highlights:
- Native MLX Runtime: Optimized for Apple Silicon using MLX/MLXLLM
- Apple Silicon Only: Designed and tested for M-series Macs
- OpenAI API Compatibility: `/v1/models` and `/v1/chat/completions` (streaming and non-streaming)
- Function/Tool Calling: OpenAI-style tools + `tool_choice`, with tool calls parsing and streaming deltas
- Chat Templates: Uses model-provided Jinja `chat_template` with BOS/EOS and smart fallback
- Session Reuse (KV Cache): Faster multi-turn chats via `session_id`
- Fast Token Streaming: Server-sent events for low-latency output
- Model Manager UI: Browse, download, and manage MLX models from MLX-Community
- System Resource Monitor: Real-time CPU and RAM usage visualization
- Self-contained: SwiftUI app with embedded SwiftNIO HTTP server
How to use Osaurus?
Osaurus is a native LLM server designed for Apple Silicon chips, leveraging Apple's MLX framework for maximum performance and OpenAI API compatibility. It aims to provide efficient, localized language model capabilities for applications like desktop AI assistants.
Core Functions of Osaurus
Dark mode, menu bar location, AI-driven, OpenAI integration, Apple Silicon support
Usage Scenarios of Osaurus
- Run large language models locally on Apple Silicon Macs.
- Enable multi-turn conversations for AI assistants and accelerate response times.
- Integrate local LLM capabilities into other apps via OpenAI API-compatible interfaces.
- Locally manage, browse, and download models from the MLX community.
- Support AI assistants in performing autonomous tasks and tool calls.
- Provide personalized and privacy-first local AI capabilities for AI companion apps.
Common Questions about Osaurus
What does Osaurus do?
How do I use Osaurus?
What are the core features of Osaurus?
What are the application scenarios of Osaurus?
