Deepgram and AWS Optimize Nova-3 Audio Model with Trainium2
Deepgram ported its Nova-3 speech recognition model to AWS Trainium2 chips, achieving 11 to 70 percent higher audio processing efficiency per cost in offline batch workloads.
Deepgram partnered with AWS to adapt its Nova-3 speech-to-text model to AWS Trainium2 hardware, significantly increasing audio processing efficiency in offline batch workloads.
Deepgram ported the Nova-3 model onto PyTorch-based Trainium2 chips in collaboration with the AWS Forward Deployed Engineering team for offline batch workloads such as call center analytics and medical transcription. As a result of the optimizations, eight model replicas were run on a single Amazon EC2 trn2.3xlarge instance, delivering between 11 and 70 percent more audio processing efficiency per dollar compared to previous computing options.
During the process, layers were optimized using TorchNeuron, Neuron Explorer, and Neuron Agentic Development tools, and a lightweight Rust-based data parallel runtime was created. This improved performance by bypassing Python interpreter overhead and global interpreter lock bottlenecks.
Sources
- aws.amazon.com · Link