basert

We have hosted the application basert in order to run this application in our online workstations with Wine or directly.

Quick description about basert:

BaseRT is a local large language model inference runtime optimized for Apple Silicon computers. It accelerates model execution through hand-written Metal kernels and requires an M1 or newer Mac running macOS 14 or later. A unified command-line interface can download models from Hugging Face, convert checkpoints, launch chats, benchmark performance, and inspect model packages. Its server implements OpenAI-compatible chat, completion, embedding, transcription, tool-calling, and multimodal endpoints. The custom .base format supports affine quantization from Q2 through Q8, optional AWQ calibration, and signed model bundles. Stable C interfaces connect the engine with Python, Node.js, Rust, and Swift applications. The repository contains the open CLI, format specifications, bindings, documentation, and benchmarks, while the prebuilt inference engine uses a separate license.

Features:

• Metal-accelerated Apple Silicon inference
• Unified model management CLI
• OpenAI-compatible local API server
• Text, image, and audio model support
• Q2 through Q8 model quantization
• Python, Node.js, Rust, and Swift bindings

Programming Language: Rust.
Categories: