Floneum

Floneum enables you to construct workflows that utilize large language models through a simple drag-and-drop interface. It loads plugins in a sandboxed environment using WebAssembly, granting them access only to the necessary resources without full system privileges. You can write plugins in any language that compiles to WebAssembly. Floneum offers ergonomic Rust wrappers, but you can also use C, Java, or Go. Features: - **Visual Interface**: Use Floneum without any programming. The visual graph editor makes it easy to combine community-built plugins with local AI models. - **Run Local Large Language Models Instantly**: Floneum requires no external dependencies, not even a GPU. It leverages LLM to run large language models locally, allowing you to process data privately without concerns about privacy. - **Plugins**: Enhance performance and tailor models for specific use cases by integrating large language models with plugins. All plugins operate in isolated environments, eliminating the need to trust any loaded plugin. They interact with the environment only in secure ways. - **Multi-Language Plugin Support**: Plugins can be written in any language that supports WebAssembly, alongside APIs accessible from any language. - **Controlled Text Generation**: Plugins can regulate the output of large language models using methods similar to JSONFORMER or guidance. This ensures the model produces valid JSON or other predefined structures, which is particularly useful for communication between language models and typed APIs.

Floneum is a toolkit that enables users to construct workflows utilizing large language models through a simple drag-and-drop interface, designed to deliver controlled, privacy-focused AI applications on consumer-grade hardware.