Elastic Inference Nodes
Single and dual accelerator nodes serving latency-sensitive endpoints, scaled by request rate and queue depth.
- Typical Workloads
- Document recognition API, multilingual service replies, agent tool calls
- Scaling Behaviour
- Autoscaling per scenario, with minimum and maximum bounds set per tenant