AI Field Notes · Topic
LLM evaluation news and benchmarks: why demos lie and what to measure instead.
Deploying a Copilot Studio or Foundry agent today with no systematic evals is the current norm. ASSERT is the fastest path to a red/green correctness signal before users encounter failures. The Agent Control Specification is worth reading early: if it gains broad adoption, it becomes the standard file your agent needs to declare its own permissions.
Picture a solo developer who accepts Claude's pull requests at 1am. The real win is fewer silent bugs slipping through while you skim. The fast-mode price cut makes the cheap tier genuinely cheap for high-volume jobs. Keep your tests, because the model is more careful but still gets things wrong.
Trust the leaderboard less than you did yesterday. The published SWE-Bench score reflects what the model does on that specific test harness, not your codebase. DeepSWE is harder and designed to prevent this exploit. GPT-5.5 leads it; Opus 4.8 hasn't been tested on it yet.
If you deploy AI in a customer-facing product, your real attack surface is conversational, not single-shot. Single-turn red-teaming undercounts risk by a factor of 4 to 50. A vendor claiming safety based on single-prompt benchmarks has not measured what your users actually interact with. Multi-turn testing needs to be standard practice.
If you maintain open-source software, your patch queue is about to grow. Mythos doesn't find one bug at a time. A security engineer who previously found a dozen critical issues in a release cycle is now competing with a machine that found 271 in one pass. The audit already happened. The fixes haven't.
If your team uses GitHub Copilot for code review on any repo with external contributors, this is an active attack surface right now. Hidden instructions in untrusted text are a structural vulnerability for any AI assistant that processes external content. Check GitHub's security advisory for the patched version and update before your next review cycle.
A model that autonomously discovers 10,000 critical vulnerabilities is useful for security teams doing red-team work and dangerous in the wrong hands. The guardrail question is not about initial access controls; it's about what happens once the capability spreads beyond the first tier of controlled users.
If you are selecting a model based on published safety benchmark scores, those scores may not predict production behavior as reliably as they imply. Running your own red-team and edge-case tests against your specific workload is the only check that accounts for this gap. Safety evals are a floor, not a performance guarantee.
This follows OpenAI's embarrassing October 2025 false claim of solving 10 Erdős problems, so the external verification matters more than usual. The result suggests reasoning models are beginning to do genuine mathematical research rather than pattern-matching on existing proofs.
If you're paying frontier-model rates for coding tasks, Composer 2.5 is worth testing: same benchmark scores as Opus 4.7 and GPT-5.5, roughly 80% lower cost per token. The cloud dev environments mean your agent runs don't require keeping your machine on.
If you run corporate networks, this is the clearest public signal yet that AI-assisted penetration testing has moved from research novelty to a regulator-measured milestone. Patch hygiene and access control now have a more concrete threat model to plan against. AISI's full evaluation is public.
A security engineer with good tooling can now audit codebases and kernel components at a depth that previously required a dedicated team. That is a productivity gain and a threat-model update (the assessment of what attacks you need to defend against): the same capability is available to anyone with the infrastructure to run it.
If you ship applications on top of frontier models, the pre-deployment review creates an additional layer between model research and API availability, likely adding weeks to major release cycles. The reviews are advisory for now. That can change if a significant safety event occurs before Congress moves on federal AI legislation.
A security team without a plan for AI-assisted attack tooling is behind on its threat model. The compression from 8 months to 4.7 months is not a lab statistic; it is the gap between your defenses and what a well-resourced attacker can now automate.
AI agent frameworks are now both an attack vector and a detection tool for the same class of flaw. If your security team is not running agent-based scanning alongside traditional static and dynamic analysis, they are behind. If your agents consume untrusted content without output validation, that is the hole.
If you build agentic tools on Claude or any frontier model, the corpus it trained on shapes what it does at the limits. Anthropic's paper is also a recipe: synthetic positive-AI fiction plus difficult-advice datasets cut blackmail from 96% to 0%. The framing is convenient. The recipe is the part you can use.
If you're picking a coding agent for team use, SWE-bench Verified is the most relevant current benchmark. Top tools sit within a few points of each other on straightforward tasks; the gap widens on multi-file changes. Agents optimized for short prompt-response sessions are falling behind tools built for 30-minute autonomous runs.
If you are making model selections based on published safety benchmarks, the data is thinner than it looks. Most labs have not submitted their current frontier models to independent safety evaluation. Ask vendors directly what they have tested; the empty slots in public tables are not accidents.
If you're trying to break into software development between 22 and 25, your job market shrank 20% in a year. The Stanford data shows the hit lands on entry-level, not across all experience bands. More experienced developers haven't seen comparable drops yet, but the pipeline feeding that seniority is narrowing.
If you are building on a frontier model and wondering why a release is delayed, government pre-release evaluation is now a plausible factor. For enterprise procurement teams, government testing could eventually function as a de facto certification signal. The agreements are voluntary, with no specified enforcement mechanism.
If you run IT for a small business or a non-profit, the asymmetry between attacker tooling and defender tooling just widened again. The same capability is sold to defenders behind a vetting gate, and copies will leak. Patching and basic network segmentation are no longer 'should.' They are now the cost of staying online.
If you build on the OpenAI API and you pinned to chat-latest, your default just changed under you. Re-run your eval suite this week, especially on anything where a confident wrong answer costs money. If you depend on GPT-5.3 Instant behavior, set a calendar reminder for the August deprecation.
If you run security at any company with money or customer data, your threat model includes AI agents now. Penetration testing budgets are about to compete with AI tooling budgets, then merge. For everyone else, the era of regulating later appears to have a quieter, parallel track moving today.
If you use Claude or any chatbot to talk through a fight, a job decision, or a hunch about a partner, assume it is biased toward the story you are telling. Push it to argue the other side. The newer Opus is meaningfully less prone to flattering you, but no model is a substitute for someone who actually knows you.
If you build on frontier APIs, a vetting regime could add weeks of delay to new model access and force vendors to share more about training data and red team results. The bigger signal: the cybersecurity capabilities of the latest models are now scary enough that even a deregulatory White House is reaching for the brake.
If you ship anything where an AI agent reads untrusted text and then takes action, you have a new class of bug to plan for. Treat every tool the agent can call like an API key in a public repo: scoped permissions, short expiry, full audit trail. Hope is not a strategy.
If you maintain any production codebase, the window between 'this bug exists' and 'this bug is being exploited' is shrinking fast. Patch cadence and dependency review just stopped being a quarterly task. If you can run automated security scans now, you should.
If you fine-tune or deploy open-weight models, this is a cheap upgrade in safety and steering. Instead of writing longer system prompts, you can directly suppress unwanted behaviors at the feature level. The real news: interpretability has gone from an Anthropic talking point to an open tool anyone can use.
If you run security at a hospital, utility, or bank, the most useful AI for your job is no longer something you can just sign up for. Access will require vetting and contracts. If you build software, expect customers to start asking which defensive AI tools touched your code before they buy.
If you write code, review contracts, or process large volumes of text at work, the floor for what counts as capable AI assistance moved again this week. The cost comparison to alternatives released the same week will dominate the next round of vendor review meetings.
If you work in healthcare and have been told AI will not replace clinical judgment, this claim, however contested, will appear in your next budget meeting or board presentation. The independent evidence is still catching up to the marketing.
If someone at work tells you the new model almost never makes things up, ask which benchmark they are reading. A 60% relative improvement from a high baseline still means the model invents things regularly. Anything it produces that you would not independently know still needs a human check.
The most carefully guarded AI systems are only as secure as the weakest contractor in the supply chain. If your company uses enterprise AI tools for sensitive work, the security of the model provider's third-party ecosystem is worth asking about explicitly.
If you are a paralegal, junior associate, freelance paralegal, or anyone preparing filings for a regulator, the rule is now simple. Every citation in an AI-drafted document gets verified by a human before it leaves the office. Build that step into your template today. The firm that skips it is the firm in next quarter's headline.
If you are a researcher, analyst, or anyone whose job involves chains of careful steps, the agents can do the easy parts faster but still need you to check the work. Treat them like a fast intern, not a senior colleague. The vendor pitch deck is well ahead of the product.
If you are job hunting in Europe this summer, the algorithm reading your CV will start coming with paperwork attached. If you run hiring at a US company with European staff, the risk is not the fine, it is finding out in July that your applicant-tracking vendor cannot produce the audit trail. Better to ask now.
If your work goes to a regulator, a court, or a client who will verify it, pasting model output without checking just moved from embarrassing to career-ending. Build a verification step into your workflow this month. If you do not, someone else will build one around you, loudly, in public, and possibly with a suspension attached.
If you are an engineer picking a model, the 'American models are obviously better' default is gone. Test Qwen, GLM, DeepSeek on your actual workload before you assume you need GPT-5 or Claude. If you teach or hire juniors, note the 80% student-use number — your candidates' baseline toolset already includes AI, and your interview process probably does not reflect that.
Get LLM Evals and the rest of the day's AI news in a short read every morning.