Trade Tests Reveal Areas Where AI Agents Struggle

Alibaba.com conducted the CommerceAgentBench test to measure the performance of AI models in real trade tasks. The results revealed that even the most successful model struggled to complete all tasks and experienced disruptions, especially in multi-step processes.

Details of the Trade Test

Alibaba.com tested 13 AI models with 107 real merchant tasks to see if the AI models could truly finish jobs completely. As a result of the tests, the best-performing model was Claude Opus 5.

Claude Opus 5 achieved the highest score by successfully completing 61.7% of the tasks. However, this rate serves as an important warning regarding the current capacity and limits of AI systems in the commercial field.

Problems Experienced in Multi-Step Workflows

Agents experienced serious difficulties in complex tasks covering multiple steps, such as supply chain costs, return disputes, and multi-leg shipping routes. This type of task caused systems to stall because it required AI models to retain information in memory over long processes and manage multiple variables simultaneously.

Results of Different Models and Infrastructures

The tested AI models were run on different infrastructures, and the success rates of the models varied according to the infrastructure used. For example, Claude Opus 5 achieved different success percentages on different platforms. The Gemini 3 Flash model, on the other hand, remained at a 29% success rate across all tested infrastructures, ranking at the bottom of the list and proving that no single model can be a leader in every area.

Merchants' AI Preferences

Research also highlights which business processes merchants approach with caution when delegating to artificial intelligence. Merchants specifically identify pricing as the area they are least willing to delegate to agents. Following pricing, fraud, disputes, and other sensitive operations requiring liability are among the main areas that merchants avoid entrusting to artificial intelligence.