[{"data":1,"prerenderedAt":297},["ShallowReactive",2],{"locale-alternates:\u002Fnew-openai-models-benchmarks-real-world-performance":3,"post-\u002Fnew-openai-models-benchmarks-real-world-performance":8},{"path":4,"alternates":5},"\u002Fnew-openai-models-benchmarks-real-world-performance",{"en":4,"tr":6,"de":7},"\u002Ftr\u002Fbenchmark-yukseliyor-benim-isimde-hatalar-artiyor","\u002Fde\u002Fneue-openai-modelle-benchmarks-arbeitsalltag",{"page":9,"translations":148,"nav":154,"related":281,"random":286},{"id":10,"title":11,"body":12,"categories":118,"category":121,"changeHistory":121,"date":122,"description":123,"disclosures":124,"draft":127,"extension":128,"firstLiveAt":121,"image":129,"imageAlt":130,"kind":131,"lang":132,"meta":133,"navigation":134,"omitGermanLocalizationDisclosure":127,"path":4,"publishedAt":121,"readingTime":135,"rights":121,"seo":136,"seoTitle":137,"slug":138,"sources":121,"stem":138,"tags":139,"translationKey":145,"type":146,"updated":121,"__hash__":147},"posts\u002Fnew-openai-models-benchmarks-real-world-performance.md","New OpenAI Models: Do Better Benchmarks Mean Better Work?",{"type":13,"value":14,"toc":110},"minimark",[15,29,32,35,40,43,46,49,53,62,65,68,71,75,78,81,84,88,96,104,107],[16,17,18,19,28],"p",{},"Artificial Analysis reports that GPT-6.1 Terra delivers better results at lower cost than its predecessor. In its ",[20,21,27],"a",{"href":22,"rel":23,"target":26},"https:\u002F\u002Fartificialanalysis.ai\u002Farticles\u002Fgpt-6-1-terra-pushes-the-cost-efficiency-frontier",[24,25],"nofollow","noopener","_blank","October 26, 2026 assessment",", the model gains six points over the previous Terra release on the Intelligence Index at maximum effort, while average cost per task falls from $1.40 to $0.40. Those results and costs apply to the tasks in that evaluation.",[16,30,31],{},"Each new model arrives with a familiar promise: better performance, greater efficiency, a lower bill. Yet that is not what I have experienced with OpenAI’s last few releases. When using GPT-5.6 and GPT-6 versions after GPT-5.5, I have encountered more errors.",[16,33,34],{},"I am not presenting a controlled comparison between GPT-6.1 Terra, the subject of that report, and the versions I have used. But there is a gap between the expectation these announcements create and my own experience. Why does switching to a supposedly better model leave me with more problems in the work I do?",[36,37,39],"h2",{"id":38},"a-lower-price-is-not-enough-reason-to-switch","A lower price is not enough reason to switch",[16,41,42],{},"I would welcome a model that does the same work, just as accurately, in less time and with fewer resources. That would be an improvement. If accuracy declines, though, a lower price alone is harder to count as a gain.",[16,44,45],{},"A response may arrive faster but still need another check. Someone may have to find the missing information or ask the model to try again. Some errors may pass unnoticed. A cheaper model call does not remove any of that work.",[16,47,48],{},"That is what bothers me about the recent GPT releases. When I cannot see the reported improvements in my own use, I cannot assume that moving to the newer version is a good decision.",[36,50,52],{"id":51},"onur-tırpan-reports-a-similar-problem-in-his-tests","Onur Tırpan reports a similar problem in his tests",[16,54,55,56,61],{},"In his ",[20,57,60],{"href":58,"rel":59,"target":26},"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=RuAlpQZ2nT0&t=660s",[24,25],"review of Opus 5.5 and GPT-6 Sol",", Onur Tırpan describes tests drawn from his own business applications. The work involves more than producing a plausible answer. It includes finding and verifying relevant documents, extracting the necessary information, excluding irrelevant material, and preparing the results for the next step. These tasks depend on one another.",[16,63,64],{},"His reported findings echo my experience: some newer GPT models cost less and finish faster, but deliver less of the accuracy his application needs. These are not the same tests or model comparisons as the Terra report above. This is a practitioner’s account of his own applications; I have not independently examined the raw test data or all the settings used.",[16,66,67],{},"What caught my attention was his distinction between uncertainty that needs a human decision and an error that passes unnoticed. If a system recognizes an ambiguous case and asks a person, its checks may be working. If it omits information or invents something and proceeds as though the result were correct, the problem is different. An average score can obscure behaviors with very different consequences for a business.",[16,69,70],{},"He also reports that Opus 5.5 improved quality, cost, and speed together. His account does not support a claim that every cheaper model gets worse. What matters is how the replacement completes a particular task and which errors it makes.",[36,72,74],{"id":73},"is-lower-cost-coming-at-the-expense-of-quality","Is lower cost coming at the expense of quality?",[16,76,77],{},"My experience with OpenAI’s recent releases sometimes leaves me with that impression. It feels as though some of the capabilities I need are being sacrificed as cost and speed improve.",[16,79,80],{},"But I cannot say what causes that impression. Seeing more errors during use does not reveal which technical decisions the provider made. Onur’s explanations about optimization and infrastructure are hypotheses too. They should not be presented as established causes of a decline in quality.",[16,82,83],{},"I do not know why I am encountering more errors with OpenAI’s latest models. But I do not need that technical explanation to choose a model. My criterion is how accurately it completes my work. If the previous model makes fewer errors on that work, continuing to use it makes more sense.",[36,85,87],{"id":86},"a-new-version-has-to-earn-its-place-in-my-work","A new version has to earn its place in my work",[16,89,90,91,95],{},"In my article on ",[20,92,94],{"href":93},"\u002Fwhy-the-same-ai-model-produces-different-results","why the same AI model produces different results across applications",", I argued that context, tools, permissions, and workflow need to be considered alongside the model itself. An update changes one part of that system. We cannot assume that behavior which worked before will survive the change.",[16,97,98,99,103],{},"I also discussed how ",[20,100,102],{"href":101},"\u002Fstripe-openrouter-which-model-for-which-job","a cheaper answer can require more human review when choosing between models",". This latest report brings me back to the same question: am I getting the same work for less money, or paying for the difference in correction effort?",[16,105,106],{},"There is no need to discard benchmark results. They need to be read with an understanding of the tasks and measures involved. But a claim of progress in an announcement cannot decide whether a business, or I, should change models.",[16,108,109],{},"A higher score than the previous model is interesting. If the replacement completes the same work with fewer errors, that gives me a reason to use it. With the last few GPT releases, I have not always seen those two improvements arrive together.",{"title":111,"searchDepth":112,"depth":112,"links":113},"",2,[114,115,116,117],{"id":38,"depth":112,"text":39},{"id":51,"depth":112,"text":52},{"id":73,"depth":112,"text":74},{"id":86,"depth":112,"text":87},[119,120],"ai","business",null,"2026-10-28","New OpenAI models promise better results at lower cost. My experience with recent GPT releases, alongside Onur Tırpan’s tests, makes me question what improves in practice.",{"aiUse":125,"aiNote":126},"ai-assisted","Evren Bal supplied the topic, personal observation, and news-and-commentary approach. AI assisted with source review, the Turkish draft, and this English adaptation.",false,"md","\u002Fimages\u002Fhero\u002Fmodel-benchmarks-real-work.avif","Beside a rising performance chart, a person compares two documents and points to a missing entry.","Commentary","en",{},true,4,{"title":11,"description":123},"New OpenAI Models: Benchmarks and Real-World Performance","new-openai-models-benchmarks-real-world-performance",[140,141,142,143,144],"ai-models","benchmarks","model-evaluation","ai-costs","model-upgrades","model-benchmarks-real-work-regressions","post","BdIAHgDPjmK_QIf3576VaJDX6y1o8bB7X9QBv3gXOAU",{"en":149,"tr":150,"de":152},{"path":4,"title":11},{"path":6,"title":151},"Yeni OpenAI Modelleri: Benchmark Başarısı İşe Yansıyor mu?",{"path":7,"title":153},"Neue OpenAI-Modelle: Besser im Benchmark, besser im Einsatz?",{"prev":155,"next":121,"others":158,"lucky":278,"readingTime":135},{"path":156,"title":157},"\u002Fdoes-ai-visibility-win-customers","Does AI Visibility Actually Win Customers?",[159,162,165,168,171,173,176,179,182,185,188,191,194,197,200,203,206,209,212,215,218,221,224,227,230,233,236,239,242,245,248,251,254,257,260,263,266,269,272,275],{"path":160,"title":161},"\u002Fopen-publishing-ai-monthly-seo-reporting-experiment","Open Publishing and an AI-Assisted Monthly SEO Reporting Experiment",{"path":163,"title":164},"\u002Fthe-threshold-collapsed-to-zero","The Threshold Collapsed: What ProductLog Taught Me About Building in Public",{"path":166,"title":167},"\u002Fai-editorial-disclosure","What an AI Disclosure Tells Readers About the Author’s Contribution",{"path":169,"title":170},"\u002Fai-value-is-not-always-more-work","AI's Value Is Not Always More Work",{"path":93,"title":172},"Why the Same AI Model Produces Different Results Across Applications",{"path":174,"title":175},"\u002Fbank-account-api-integration","Integrating One Bank Is Easy. Keeping Forty Running Is Not.",{"path":177,"title":178},"\u002Fsensitive-ai-data-processing-location","Where Should Sensitive AI Data Be Processed?",{"path":180,"title":181},"\u002Fvoice-agent-system-deployment-constraints","What to Test Before Investing in a Voice Agent",{"path":183,"title":184},"\u002Fwriting-with-ai-means-thinking-with-your-archive","Writing with AI Means Thinking About Your Archive, Too",{"path":186,"title":187},"\u002Fwordpress-to-nuxt-ai-powered-content-pipeline","From WordPress to Nuxt: Building an AI-Powered Content Pipeline",{"path":189,"title":190},"\u002Fhow-much-does-the-eu-ai-act-affect-companies-outside-the-eu","How Much Does the EU AI Act Affect Companies Outside the EU?",{"path":192,"title":193},"\u002Fhow-to-route-requests-across-multiple-ai-models","How Should You Route Requests Across Multiple AI Models?",{"path":195,"title":196},"\u002Fwhere-should-people-be-in-the-loop-while-ai-does-the-work","Where Should People Be in the Loop While AI Does the Work?",{"path":198,"title":199},"\u002Fhealthcare-ai-chatbot-triage-medical-device","AI in Healthcare: The Difference Between Chatbots, Triage, and Medical Devices",{"path":201,"title":202},"\u002Fkeeping-customers-happy-isnt-enough","Keeping Customers Happy Isn’t Enough. You Have to Follow Up.",{"path":204,"title":205},"\u002Fis-ai-only-bad-at-your-job","Is AI Only Bad at Your Job?",{"path":207,"title":208},"\u002Fai-job-listings-work-companies-need","What AI Job Listings Reveal About the Work Companies Need Done",{"path":210,"title":211},"\u002Fbringing-english-back-into-my-working-day","Bringing English Back Into My Working Day",{"path":213,"title":214},"\u002Fai-turned-a-refactor-i-wouldnt-do-into-a-one-hour-job","With AI, a Refactor I Wouldn't Bother With Took One Hour",{"path":216,"title":217},"\u002Fbeyond-the-bot-lessons-from-building-a-chat-system-for-global-patients","What We Learned Building a Healthcare Chatbot for International Patients",{"path":219,"title":220},"\u002Fdo-we-need-ai","Do We Need AI, or Are We Solving the Wrong Problem?",{"path":222,"title":223},"\u002Fseptember-2026-what-i-wrote","September 2026: What I Wrote, and Why",{"path":225,"title":226},"\u002Fhow-can-companies-create-value-from-ai","How Can Companies Create Value From AI?",{"path":228,"title":229},"\u002Fwhat-does-an-ai-visibility-score-measure","What Does an AI Visibility Score Measure?",{"path":231,"title":232},"\u002Fwhat-is-an-ai-harness","What Is an AI Harness? A Plain-English Explanation",{"path":234,"title":235},"\u002Fhow-much-authority-should-ai-have","How Much Authority Should You Give an AI System?",{"path":237,"title":238},"\u002Fis-buying-backlinks-worth-it","Would We Still Buy This Backlink If It Sent No Ranking Signal?",{"path":240,"title":241},"\u002Fai-will-spread-across-turkiyes-businesses","AI in Türkiye: What Adoption Figures Leave Out",{"path":243,"title":244},"\u002Fwhat-does-the-eu-ai-act-actually-regulate","What Is the EU AI Act, and What Does It Regulate?",{"path":246,"title":247},"\u002Fcan-an-accessibility-widget-make-a-website-accessible","Can an Accessibility Widget Make a Website Accessible?",{"path":249,"title":250},"\u002Fwho-captures-ai-productivity-gains","Who Captures the Productivity Gains from AI?",{"path":252,"title":253},"\u002Fhow-i-built-a-modern-infrastructure-using-open-source-tools-and-the-power-of-cloudflare","Why I Run camiler.org on a Single VPS",{"path":255,"title":256},"\u002Fai-smaller-teams-business-capacity","AI Makes Smaller Teams Possible. What Does the Business Gain — and Lose?",{"path":258,"title":259},"\u002Fproductlog-the-platform-i-built-for-myself-first","ProductLog: The Platform I Built for Myself First",{"path":261,"title":262},"\u002Fturkey-kvkk-data-contracts-foreign-providers","Turkey’s KVKK: Why Your Turkish Customer Needs Another Data Contract",{"path":264,"title":265},"\u002Fraising-children-in-the-age-of-artificial-intelligence","Raising Children in the Age of Artificial Intelligence",{"path":267,"title":268},"\u002Ffrom-asking-questions-to-delegating-work","From Asking Questions to Delegating Work: What the Codex Study Shows",{"path":270,"title":271},"\u002Fwhen-process-automation-actually-needs-ai","When Does Process Automation Actually Need AI?",{"path":273,"title":274},"\u002Fai-visibility-is-also-a-state-concern","AI Visibility Is Also on States’ Agendas",{"path":276,"title":277},"\u002Ftesting-a-button-treating-an-entire-website-redesign-as-a-sure-thing","Testing a Button, Treating an Entire Website Redesign as a Sure Thing",{"path":279,"title":280},"\u002Fthe-job-ai-wont-take-and-the-five-it-prevents","AI Is Reducing Hiring Without Layoffs",[282,284],{"path":231,"title":232,"date":283},"2026-10-01",{"path":93,"title":172,"date":285},"2026-09-12",[287,291,293],{"path":288,"title":289,"date":290},"\u002Fdoes-turkey-kvkk-apply-european-companies","Does Turkey’s KVKK Apply to European Companies?","2026-09-24",{"path":276,"title":277,"date":292},"2026-08-25",{"path":294,"title":295,"date":296},"\u002Fstart-with-the-business-problem-not-the-ai-model","Start With the Business Problem, Not the AI Model","2026-08-27",1793182636068]