Dedicated Language Inference

What is dedicated inference?

Dedicated inference offers exclusive access to a specific model, ensuring that you are the sole user of the underlying compute resources. Unlike serverless options, you won't have to share these resources with other users, guaranteeing consistent performance.

This makes it particularly suitable for applications that require low latency, have a heavy workload, or involve batch processing tasks.

What does ⚡ indicate?

The model is quantized using a methodology that balances a slight reduction in accuracy with significant reductions in computing demand and energy consumption. This approach provides a range of variants, from compact and economical options to larger, near-lossless alternatives, catering to various use cases and preferences.

Quantization offers a compelling solution for reducing inference costs while maintaining high performance standards. Leveraging quantized models enhances cost efficiency while preserving quality.

Is my data stored or used for training?

No, none of your data is stored or used for training.

Where are my models hosted?

Your models are hosted on ISO 27001 certified infrastructure, utilizing cutting-edge GPUs. The infrastructure is located within Europe, adhering to GDPR standards.

How is the quality assessed?

We utilize Eleuther's Evaluation Harness to assess the quality of models across a wide range of tasks. This allows you to compare models against each other, providing a comprehensive understanding of their performance.

However, for practical reasons, we may subsample larger evaluation datasets to 1,000 entries, which may lead to slight variations when compared to other public evaluations.