openmythos

We have hosted the application openmythos in order to run this application in our online workstations with Wine or directly.

Quick description about openmythos:

OpenMythos is an experimental, open-source implementation that attempts to reconstruct a hypothesized architecture behind advanced language models using a design called a Recurrent-Depth Transformer. The project explores the idea that instead of stacking hundreds of unique transformer layers, a smaller set of layers can be reused iteratively during inference to achieve deeper reasoning without increasing parameter count. It divides computation into three main stages, including a pre-processing phase, a looped recurrent reasoning block, and a final output refinement stage, creating a structured pipeline for inference. The architecture incorporates advanced techniques such as mixture-of-experts routing, adaptive computation time, and multiple attention mechanisms to dynamically allocate compute where needed. It is highly configurable through a centralized configuration system, allowing experimentation with different architectural parameters such as loop depth, attention type.

Features:

• Recurrent-depth transformer architecture with looped computation
• Mixture-of-experts routing for dynamic specialization
• Adaptive computation time for variable reasoning depth
• Configurable attention mechanisms including GQA and MLA
• Centralized configuration for architectural experimentation
• Autoregressive generation with iterative reasoning loops

Programming Language: Python.
Categories: