SLAs: Model Availability, Output Quality & Token Latency

The mechanism matters. Traditional SLAs measure whether the application responds to a ping. AI SLAs need to measure whether the model is available, whether the outputs are fit for purpose, and how long the user waits between prompt and response. The system can be up while the model is effectively down. A model endpoint returning a 200 OK but producing gibberish is 'available' under a standard uptime SLA. A model silently swapped for a cheaper version is 'available.' A model timing out on every third request because the inference provider is throttling is 'available.'

Standard vendor language — '99.5% system availability measured monthly. Downtime caused by third-party providers is excluded' — covers the wrapper. The model, the inference provider, the entire AI pipeline: excluded.

The enterprise counter creates two distinct tiers. System availability (99.5%, service credits as remedy) and model availability (99.0%, with a termination right as remedy for sustained failure). Model Unavailability is defined through objective, measurable criteria: the AI feature returns error responses for more than 5% of queries in a measurement window; the named model version specified in the Order Form is unavailable or has been replaced without notice; or the 95th-percentile response latency exceeds three times the baseline defined in the Order Form.

Output quality remains the largest gap between what procurement wants and what the market offers. No vendor in the market commits to output accuracy. The resolution is shifting from accuracy warranties to accuracy monitoring: the vendor regularly measures AI feature accuracy against mutually agreed benchmarks, provides quarterly accuracy reports, and if accuracy falls below the defined threshold for two consecutive quarters, either remediates or permits the customer to terminate without penalty.

The outcome-based pricing alternative is available for task-specific AI. The vendor charges per successfully completed task. No uptime commitments, no accuracy benchmarks, no credit structures. The buyer verifies resolution counts. If the system underperforms, the vendor earns less or nothing. This eliminates the two-tier SLA compromise and replaces it with a pricing structure that does not need one. It is the strongest approach available for task-specific AI.

Provider-side outages require specific handling. When the inference provider goes down, the vendor's standard SLA excludes that downtime. The enterprise counter: the vendor warrants system availability excluding only outages caused by third-party providers where the vendor has no contractual right to prioritize or supplement capacity. This narrows the carve-out to genuinely uncontrollable failures. On model change notice: the vendor must provide at least 30 days' advance notice before a material change to the underlying AI model, giving the buyer a window to test before the change takes effect.

Red Flags

• SLA covers only 'system availability' with no definition of model availability
• Third-party provider outages broadly excluded — with no limit on the vendor's contractual ability to influence those providers
• No model version pinning — vendor can silently swap the underlying model
• No advance notice obligation before material model changes
• Sole remedy for SLA failures is service credits with no termination right for sustained degradation
• No accuracy monitoring commitment — AS-IS disclaimer on output quality with no measurement program

Sample Clause (Illustrative)

"In addition to System Availability of 99.5% measured monthly, Provider shall maintain Model Availability at 99.0% measured monthly. 'Model Unavailability' means: (a) the AI Feature returns error responses or produces outputs that materially deviate from the expected output format for more than 5% of queries in a measurement window, (b) the named model version specified in the Service Order is unavailable or has been replaced without notice, or (c) the 95th-percentile response latency exceeds three times the baseline latency defined in the Service Order for a continuous period of 48 hours. Customer's remedies for System Availability failures are service credits as set forth in the SLA. Customer's remedy for sustained Model Unavailability — defined as Model Availability below 95% for two consecutive calendar months — is termination of the affected Service Order without penalty. Provider shall use commercially reasonable efforts to provide Customer with at least 30 days' advance notice before making a material change to the underlying AI model."

Creates a two-tier SLA: traditional uptime (99.5%) with service credits, and model availability (99.0%) with a termination right. Model Unavailability is defined through measurable, objective criteria — error rate, model version swap, latency — rather than trying to measure semantic drift. The buyer's remedy for sustained model degradation is not credits (impossible to price for model quality) but exit. The model change notice gives the buyer a window to test the new model against its documented use case before the change takes effect. The vendor does not warrant per-output accuracy but commits to objective performance metrics and gives the buyer an exit ramp when the model degrades below an acceptable floor.

Illustrative only. Clause language requires adaptation to your jurisdiction, deal context, and risk profile. Not legal advice.