Text generation web UI

Gradio Web UI for Large Language Models. Its goal is to become the Automatic1111/Stable Diffusion - WebUI for text generation. Features Three interface modes: Default (two-column), Notebook, and Chat Multiple model backends: Transformers, Llama.cpp, Exllama, Exllamav2, AutoGPTQ, GPTQ-for-LLaMA, CTransformers Dropdown menu for quick switching between different models LoRA: Instant loading and unloading of LoRAs, training new LoRAs with QLoRA Precise instruction templates for chat mode, including Llama-2-Chat, Alpaca, Vicuna, WizardLM, StableLM, etc. 4-bit, 8-bit, and CPU inference via the Transformers library Use llama.cpp models with Transformers samplers (llamacpp_hf loader) Multimodal pipelines, including Llava and MiniGPT-4 Extension framework Custom chat characters Highly efficient text streaming Markdown output rendered with LaTeX, for use with Alpaca, for example API, including endpoints for WebSocket streaming (see examples)

Text generation web UI is a Gradio-based web interface for large language models, aiming to be the AUTOMATIC1111/stable-diffusion-webui of text generation. It offers support for multiple model backends, LoRA training, chat modes, and multimodal functionalities.