Product Information
What is Bentoml?
BentoML is a framework for building reliable, scalable, and cost-effective AI applications. It provides everything you need for model serving, application packaging, and production deployment.
Highlights:
Bento is a container for AI apps
An open standard and SDK for AI apps—package your code, inference pipelines, model files, dependencies, and runtime config in a Bento.
Auto-generated API servers supporting REST, gRPC, and long-running inference jobs.
Auto-generated Docker container images.
🏄 Freedom to build with any AI model
Import from any model hub or bring your own, built with frameworks like PyTorch, TensorFlow, Keras, Scikit-Learn, or XGBoost.
Native support for LLM inference, generative AI, embeddings, and multimodal AI apps.
Run and debug BentoML apps locally on Mac, Windows, or Linux.
🍭 Simplified modern AI app architecture
Python-first! Effortlessly scale complex AI workloads.
Enable hassle-free GPU inference.
Compose multiple models to run concurrently or sequentially across GPUs or Kubernetes clusters.
Native integrations with MLFlow, LangChain, Kubeflow, Triton, Spark, Ray, and more to complete your production AI stack.
🚀 Deploy anywhere
One-click deploy to Bentocloud, a serverless platform for hosting and operating AI apps.
Scalable BentoML deployments with 🦄 Yatai on Kubernetes.
Deploy auto-generated container images anywhere Docker runs.
How to use Bentoml?
BentoML is a framework for building reliable, scalable, and cost-effective AI applications, offering everything needed for model serving, app packaging, and production deployment, simplifying AI product development for software engineers.
Core Functions of Bentoml
AI-driven, Kubernetes
AI-Powered
Kubernetes
Usage Scenarios of Bentoml
- Package AI application code, inference pipelines, model files, and dependencies into Bento containers.
- Automatically generate API servers supporting REST API, gRPC, and Docker container images.
- Import and deploy AI models built with frameworks like PyTorch and TensorFlow.
- Support LLM inference, generative AI, embedding creation, and multimodal AI application deployment.
- Orchestrate multiple models to run concurrently or sequentially on Kubernetes clusters.
- Manage, monitor, and optimize AI model inference, including deployment automation and performance tuning.
Common Questions about Bentoml
What does BentoML do?
How do I use BentoML?
What are the core features of BentoML?
What are the application scenarios of BentoML?
