Usage-based pricing era begins for Amazon SageMaker AI
AWS has launched a new billing system based on real-time consumption data for SageMaker AI bidirectional streaming inference. This feature allows model providers to price their services more flexibly.
Amazon Web Services (AWS) has introduced a usage-based metering feature for bidirectional streaming inferences performed on Amazon SageMaker AI.
The new system enables model providers, particularly in areas such as real-time conversational AI, to charge customers based on the amount of consumption during the live stream. This update overcomes billing difficulties previously experienced because session duration and consumption amounts were not finalized until the end of the session.
The system operates through a second WebSocket channel called 'metadata connection' added to the existing inference stream. Model servers can report metering data in JSON format in real-time through this channel. Deepgram became the first partner to use this feature to offer usage-based Speech-to-Text and Text-to-Speech billing services via AWS Marketplace. It was noted that SageMaker AI endpoint infrastructure fees will continue to apply separately from these software fees.
Sources
- aws.amazon.com · Link