API, including endpoints for websocket streaming ( see the examples).Markdown output with LaTeX rendering, to use for instance with GALACTICA.Multimodal pipelines, including LLaVA and MiniGPT-4.Use llama.cpp models with transformers samplers ( llamacpp_HF loader).4-bit, 8-bit, and CPU inference through the transformers library.Precise instruction templates for chat mode, including Llama-2-chat, Alpaca, Vicuna, WizardLM, StableLM, and many others.LoRA: load and unload LoRAs on the fly, train a new LoRA using QLoRA.Dropdown menu for quickly switching between different models.Multiple model backends: transformers, llama.cpp, ExLlama, ExLlamaV2, AutoGPTQ, GPTQ-for-LLaMa, CTransformers.3 interface modes: default (two columns), notebook, and chat.Its goal is to become the AUTOMATIC1111/stable-diffusion-webui of text generation. A Gradio web UI for Large Language Models.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |