Baseten is an AI infrastructure platform. We combine applied performance research, distributed multi-cloud infrastructure, and developer tooling to run models of all modalities in production.
Get started:
Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.
A framework for efficient model inference with omni-modality models
concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3, Qwen3-MoE, DeepSeek-R1, GLM4.5, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Llava, Phi4, ...) (AAAI 2025).
A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
Ongoing research training transformer models at scale
Loading…
Loading…