
General thoughts
General thoughts
AI

If a model will classify support tickets, the relevant question is how often it routes a real ticket to the right queue, at what cost, and with what consequence when it gets the answer wrong. DeepSeek and OpenAI each offer multiple models. A comparison becomes reproducible when it names the model, the endpoint, the task and the date of the test.
DeepSeek released R1 in January 2025 as a reasoning model. R1 and V3 are distinct releases. The widely repeated $5.576 million figure comes from the V3 technical report’s estimate for its reported training stages, based on an assumed GPU-hour price. It is neither a complete account of DeepSeek’s research costs nor the cost of developing R1. Historical benchmark results belong to the model and settings that produced them. DeepSeek R1 release · DeepSeek V3 technical report
Choose one named model from each provider and record its exact identifier and test date. Check the current documentation for input types, context limits, tool calling, structured output and pricing. Verify a capability with your own request: if the workflow needs a strict JSON schema, a plausible paragraph describing the correct answer still fails the contract. Where a model has downloadable weights, evaluate that deployment separately from the provider’s hosted API. Operating it yourself changes who owns capacity, updates, latency and data handling. DeepSeek API models and pricing · OpenAI model directory
Take a representative set of resolved tickets and remove personal or confidential information. Include short requests, incomplete descriptions, ambiguous team names and tickets that should go directly to a human. Have the service owners record the expected queue and what would count as an acceptable escalation before they see either model’s output.
Give both models the same instructions, queue definitions and input. Require a queue identifier or a request for review. Track correct routing, incorrect confident answers, schema failures and unnecessary escalations separately. Record latency at the median and the 95th percentile. Include retries and the cost of human review when calculating cost per accepted ticket. A cheap request that needs two retries and manual correction can be expensive in production.
A documentation question sent to the wrong help queue causes delay. An access request routed around approval has a different consequence. Review those failure cases individually. In an orchestration workflow, keep permission checks in the application, log the model version, and give uncertain cases an explicit review path. Repeat the same evaluation when a model alias changes or your queue taxonomy evolves. The resulting scorecard is a more durable basis for choosing a provider than a ranking from a previous model generation.