weekly issue · August 30, 2026

How should teams test AI agents inside real business workflows?

Enterprise benchmarks now test documents, workflow controls, and user experience.

Direct answer

Test the workflow, not the model alone. Use the real enterprise documents, tools, user result, and failure path before release. CorporateBench expands evaluation across enterprise document collections, Natera shows a production appointment workflow, and AgentCore Evaluations supplies framework-independent scoring. Approval should depend on completed work and a tested recovery path.

Edited by Joe Cervino, Founder and Editor

Published August 30, 2026

CorporateBench evaluates enterprise question answering against more than 230,000 documents, while a field study analyzes 713,564 employee prompts across 15 functions.

Natera's voice agent books appointments, and AgentCore Evaluations scores agents across major frameworks. The operating gate is now the full workflow.

Get the Daily and Weekly Issues.

Subscribe to receive both issues. Unsubscribe anytime.

Thesis movement

Industry Matrix

Full public entity map from Pulse evidence versus the prior weekly window.

Key

53 of 53 signals

01
Platforms
Amazon Bedrock AgentCore
Act Now
What
Amazon Bedrock AgentCore: Natera says its Amazon Bedrock AgentCore voice agent books mobile phlebotomy appointments with 100% tool-calling accuracy and sub-seven-second latency
Do
Replay the appointment flow on AgentCore with tool accuracy, latency, and authentication recorded.Open the evidence from AWS ML Blog
Why
Natera's result depends on a specific voice architecture and progressive-trust authentication path.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Amazon Bedrock AgentCore uses AWS ML Blog: Natera says its Amazon Bedrock AgentCore voice agent books mobile phlebotomy appointments with 100% tool-calling accuracy and
Sources: AWS ML Blog
02
Platforms
Amazon EC2
Act Now
What
Amazon EC2: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by 75% while
Do
Benchmark the cited speech-recognition load on the current EC2 GPU shape before resizing.Open the evidence from AWS ML Blog
Why
The reported capacity reduction depends on EC2, CUDA MPS, and Triton running together.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Amazon EC2 uses AWS ML Blog: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by
Sources: AWS ML Blog
03
Platforms
Amazon Quick
Act Now
What
Amazon Quick: GoDaddy says its two-year migration from a legacy BI tool to Amazon Quick saves 15,000 hours annually, cut its dashboard count by 50%, reduced rendering
Do
Audit dashboard count, render time, and analyst hours before considering an Amazon Quick migration.Open the evidence from AWS ML Blog
Why
GoDaddy reports operational savings from a completed migration, which need a comparable local baseline.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Amazon Quick uses AWS ML Blog: GoDaddy says its two-year migration from a legacy BI tool to Amazon Quick saves 15,000 hours annually, cut its dashboard count by 50%
Sources: AWS ML Blog
04
Platforms
Amazon SageMaker AI
Act Now
What
Amazon SageMaker AI: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local code into any
Do
Use SageMaker AI on one script-mode workflow and verify code sync without an image rebuild.Open the evidence from AWS ML Blog
Why
The SDK change targets iteration time between local code and runtime containers.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Amazon SageMaker AI uses AWS ML Blog: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize
Sources: AWS ML Blog
05
Platforms
AWS Agent Registry
Act Now
What
AWS Agent Registry: AWS published Agentic Resource Discovery (ARD), an open specification for agent discovery, alongside AWS Agent Registry - a centralized, searchable catalog
Do
Register one agent, tool, and skill, then test ownership and search results in AWS Agent Registry.Open the evidence from AWS ML Blog
Why
The registry is meant to make agent capabilities searchable and governed across environments.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. AWS Agent Registry uses AWS ML Blog: AWS published Agentic Resource Discovery (ARD), an open specification for agent discovery, alongside AWS Agent Registry - a
Sources: AWS ML Blog
06
Platforms
Databricks AI Runtime
Act Now
What
Databricks AI Runtime: Databricks frames goodput:the share of time large-scale training performs useful work:as the metric determining training efficiency
Do
Trial Databricks AI Runtime on one fault-prone PyTorch job and record goodput through recovery.Open the evidence from Databricks AI
Why
The product claim centers on useful training time during fast, fault-tolerant execution.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Databricks AI Runtime uses Databricks AI: Databricks frames goodput:the share of time large-scale training performs useful work:as the metric determining training
Sources: Databricks AI
07
Platforms
Docker
Act Now
What
Docker: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local code into any
Do
Confirm the SageMaker code-sync loop works with the current Docker image and rollback process.Open the evidence from AWS ML Blog
Why
The cited workflow avoids rebuilding images, which changes iteration speed only if runtime parity holds.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Docker uses AWS ML Blog: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local code
Sources: AWS ML Blog
08
Tools
Dependabot
Act Now
What
Dependabot: GitHub says the GitHub Copilot app can automate repetitive Dependabot pull-request triage
Do
Let Dependabot open a controlled pull request set, then compare triage time and reviewer corrections.Open the evidence from GitHub AI Blog
Why
The cited GitHub workflow automates pull-request triage, where review quality matters as much as speed.
No material change · Evidence held steady into Act Now on 2 signals across 2 sources. Dependabot uses GitHub AI Blog: GitHub says the GitHub Copilot app can automate repetitive Dependabot pull-request triage
Sources: GitHub AI Blog · GitHub AI Blog
09
Tools
GitHub Copilot
Act Now
What
GitHub Copilot: GitHub says the GitHub Copilot app can automate repetitive Dependabot pull-request triage
Do
Pilot Copilot on repetitive Dependabot triage with merge approval kept human-owned.Open the evidence from GitHub AI Blog
Why
GitHub ties Copilot to a bounded maintenance task rather than unrestricted repository changes.
No material change · Evidence held steady into Act Now on 2 signals across 2 sources. GitHub Copilot uses GitHub AI Blog: GitHub says the GitHub Copilot app can automate repetitive Dependabot pull-request triage
Sources: GitHub AI Blog · GitHub AI Blog
10
Tools
Agent Arena
Act Now
What
Agent Arena: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude
Do
Run the same task across Agent Arena's low-cost and leading routes before choosing a default.Open the evidence from Agent Arena | AI Agent Performance Leaderboard
Why
The board shows wide cost and success differences, so a single leaderboard rank hides the tradeoff.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Agent Arena uses Agent Arena | AI Agent Performance Leaderboard: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High)
Sources: Agent Arena | AI Agent Performance Leaderboard
11
Tools
Alyx
Act Now
What
Alyx: Arize says its Signal tool found two hidden retry loops in the production agent Alyx: a duplicate task-state loop and a 43-call dataset retry that had
Do
Replay Alyx with retry ceilings and confirm both hidden loops become explicit failures.Open the evidence from Arize AI Blog
Why
Alyx produced tool activity that looked valid while retries consumed work without progress.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Alyx uses Arize AI Blog: Arize says its Signal tool found two hidden retry loops in the production agent Alyx: a duplicate task-state loop and a 43-call dataset retry
Sources: Arize AI Blog
12
Tools
Amazon Bedrock AgentCore Evaluations
Act Now
What
Amazon Bedrock AgentCore Evaluations: uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex, the OpenAI Agents SDK, Amazon Bedrock
Do
Score one existing agent through AgentCore Evaluations without changing its framework.Open the evidence from AWS ML Blog
Why
The cited release uses an OpenTelemetry contract across multiple agent frameworks.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Amazon Bedrock AgentCore Evaluations uses AWS ML Blog: uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph
Sources: AWS ML Blog
13
Tools
Claude Agent SDK
Act Now
What
Claude Agent SDK: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex
Do
Send one Claude Agent SDK trace through the cited evaluation contract and inspect scoring gaps.Open the evidence from AWS ML Blog
Why
AgentCore lists the SDK among supported frameworks, making cross-framework evaluation testable.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Claude Agent SDK uses AWS ML Blog: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with
Sources: AWS ML Blog
14
Tools
Genie One
Act Now
What
Genie One: Databricks announced Genie One features aimed at extending AI-assisted analytics from answering questions to taking action on insights
Do
Constrain one Genie One action to a reversible analytics workflow and log every write.Open the evidence from Databricks AI
Why
Databricks is moving Genie from answering questions toward taking action on the resulting data.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Genie One uses Databricks AI: Databricks announced Genie One features aimed at extending AI-assisted analytics from answering questions to taking action on insights
Sources: Databricks AI
15
Tools
Google ADK
Act Now
What
Google ADK: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex
Do
Evaluate one Google ADK agent through the shared AgentCore scoring contract.Open the evidence from AWS ML Blog
Why
The release names Google ADK as a supported framework for framework-neutral evaluation.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Google ADK uses AWS ML Blog: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with
Sources: AWS ML Blog
16
Tools
LangGraph
Act Now
What
LangGraph: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex
Do
Pass one LangGraph trace into AgentCore Evaluations and compare the resulting score fields.Open the evidence from AWS ML Blog
Why
The cited evaluation layer claims framework-neutral scoring for LangGraph agents.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. LangGraph uses AWS ML Blog: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph
Sources: AWS ML Blog
17
Tools
LlamaIndex
Act Now
What
LlamaIndex: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex
Do
Score one LlamaIndex agent with the same OpenTelemetry criteria used for another framework.Open the evidence from AWS ML Blog
Why
Shared criteria make a framework comparison possible without changing the business task.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. LlamaIndex uses AWS ML Blog: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with
Sources: AWS ML Blog
18
Tools
ModelBuilder
Act Now
What
ModelBuilder: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local code into any
Do
Build one existing SageMaker model path with ModelBuilder and record packaging differences.Open the evidence from AWS ML Blog
Why
The new SDK pairs ModelBuilder with a unified script-mode workflow.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. ModelBuilder uses AWS ML Blog: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local
Sources: AWS ML Blog
19
Tools
ModelTrainer
Act Now
What
ModelTrainer: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local code into any
Do
Move one training script through ModelTrainer and verify inputs, outputs, and failure recovery.Open the evidence from AWS ML Blog
Why
ModelTrainer is part of SageMaker's attempt to unify previously separate script workflows.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. ModelTrainer uses AWS ML Blog: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local
Sources: AWS ML Blog
20
Tools
OpenAI Agents SDK
Act Now
What
OpenAI Agents SDK: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex
Do
Evaluate one OpenAI Agents SDK trace with the same OpenTelemetry rubric used for other frameworks.Open the evidence from AWS ML Blog
Why
The cited AgentCore release includes the SDK in its framework-neutral evaluation path.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. OpenAI Agents SDK uses AWS ML Blog: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with
Sources: AWS ML Blog
21
Tools
Signal
Act Now
What
Signal: Arize says its Signal tool found two hidden retry loops in the production agent Alyx: a duplicate task-state loop and a 43-call dataset retry that had
Do
Run Signal on a known retry failure and require loop counts to surface as errors.Open the evidence from Arize AI Blog
Why
Arize reports that Signal exposed retries previously disguised as valid tool activity.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Signal uses Arize AI Blog: Arize says its Signal tool found two hidden retry loops in the production agent Alyx: a duplicate task-state loop and a 43-call dataset retry
Sources: Arize AI Blog
22
Tools
SourceCode
Act Now
What
SourceCode: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local code into any
Do
Sync one local change with SourceCode and verify the running container matches the reviewed commit.Open the evidence from AWS ML Blog
Why
SourceCode promises faster iteration by changing runtime code without rebuilding the image.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. SourceCode uses AWS ML Blog: Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local
Sources: AWS ML Blog
23
Tools
Strands Agents
Act Now
What
Strands Agents: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex
Do
Score one Strands Agents workflow through the cited AgentCore evaluator before comparing frameworks.Open the evidence from AWS ML Blog
Why
AgentCore names Strands among the frameworks supported by its shared scoring contract.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Strands Agents uses AWS ML Blog: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with
Sources: AWS ML Blog
24
Tools
Triton Inference Server
Act Now
What
Triton Inference Server: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by 75% while
Do
Stress-test Triton with CUDA MPS and preserve the current server route as rollback.Open the evidence from AWS ML Blog
Why
The cited AWS deployment couples Triton serving with GPU sharing to reach its capacity result.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Triton Inference Server uses AWS ML Blog: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech
Sources: AWS ML Blog
25
Projects
OpenClaw
Act Now
What
OpenClaw: GitHub describes OpenClaw as the fastest-growing project in GitHub history
Do
Review OpenClaw's published security lessons before adopting its patterns in a critical path.Open the evidence from GitHub AI Blog
Why
OpenClaw's rapid growth makes its early security work more useful than its adoption count alone.
No material change · Evidence held steady into Act Now on 2 signals across 2 sources. OpenClaw uses GitHub AI Blog: GitHub describes OpenClaw as the fastest-growing project in GitHub history
Sources: GitHub AI Blog · GitHub AI Blog
26
Companies
GitHub
Act Now
What
GitHub: published the lessons its team learned evaluating LLMs for real-world secret scanning, framing pre-production LLM evaluation as its own discipline that
Do
Adopt GitHub's pre-production evaluation discipline on one secret-scanning model change.Open the evidence from GitHub AI Blog
Why
GitHub treats evaluation as a release gate for a real-world security workflow.
No material change · Evidence held steady into Act Now on 3 signals across 3 sources. GitHub uses GitHub AI Blog: published the lessons its team learned evaluating LLMs for real-world secret scanning, framing pre-production LLM evaluation as its own
Sources: GitHub AI Blog · GitHub AI Blog · GitHub AI Blog
27
Companies
OpenAI
Act Now
What
OpenAI: TechCrunch reports OpenAI is building AI agents for everything and openly questions adoption : 'Will everyone use them?' : framing agent proliferation from
Do
Measure completed agent work and repeat use before expanding OpenAI agent access.Open the evidence from AI News & Artificial Intelligence | TechCrunch
Why
OpenAI is increasing agent supply while the cited report leaves broad user demand unresolved.
No material change · Evidence held steady into Act Now on 3 signals across 3 sources. OpenAI uses AI News & Artificial Intelligence | TechCrunch: TechCrunch reports OpenAI is building AI agents for everything and openly questions adoption : 'Will everyone
Sources: AI News & Artificial Intelligence | TechCrunch · Arize AI Blog · Agent Arena | AI Agent Performance Leaderboard
28
Models
Claude Opus 4.8
Act Now
What
Claude Opus 4.8: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude
Do
Keep Claude Opus 4.8 only where its measured task result clears the current cost threshold.Open the evidence from Agent Arena | AI Agent Performance Leaderboard
Why
Agent Arena places cost and net task improvement on the same board, exposing route economics.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Claude Opus 4.8 uses Agent Arena | AI Agent Performance Leaderboard: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro
Sources: Agent Arena | AI Agent Performance Leaderboard
29
Models
Claude Opus 5
Act Now
What
Claude Opus 5: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude
Do
Test Claude Opus 5 against the cheaper cited routes before assigning it routine agent work.Open the evidence from Agent Arena | AI Agent Performance Leaderboard
Why
Its listed task cost is materially higher than several alternatives on the same benchmark.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Claude Opus 5 uses Agent Arena | AI Agent Performance Leaderboard: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High)
Sources: Agent Arena | AI Agent Performance Leaderboard
30
Models
Kimi K3
Act Now
What
Kimi K3: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude
Do
Benchmark Kimi K3 on the same agent task and budget used for the current default.Open the evidence from Agent Arena | AI Agent Performance Leaderboard
Why
Its Agent Arena placement must be read with both task success and cost, not rank alone.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Kimi K3 uses Agent Arena | AI Agent Performance Leaderboard: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at
Sources: Agent Arena | AI Agent Performance Leaderboard
31
Companies
Amazon Web Services
Act Now
What
Amazon Web Services: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by 75% while
Do
Reproduce the cited speech workload on the current AWS route before changing GPU capacity.Open the evidence from AWS ML Blog
Why
AWS reports lower infrastructure use on a specific MPS and Triton configuration, not a general cloud result.
No material change · Evidence held steady into Act Now on 2 signals across 2 sources. Amazon Web Services uses AWS ML Blog: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech
Sources: AWS ML Blog · AWS ML Blog
32
Companies
Arize
Act Now
What
Arize: Stuart Sy of OpenAI, interviewed by Arize, argues better models do not fix every agent failure because the bottleneck has moved off the model and onto
Do
Instrument one failed agent run for context, evaluation, and observability gaps with Arize.Open the evidence from Arize AI Blog
Why
Arize's cited interview places the current bottleneck outside the model itself.
No material change · Evidence held steady into Act Now on 2 signals across 2 sources. Arize uses Arize AI Blog: Stuart Sy of OpenAI, interviewed by Arize, argues better models do not fix every agent failure because the bottleneck has moved off the model
Sources: Arize AI Blog · Arize AI Blog
33
Companies
Databricks
Act Now
What
Databricks: frames goodput:the share of time large-scale training performs useful work:as the metric determining training efficiency
Do
Track useful training time beside uptime in the next Databricks training run.Open the evidence from Databricks AI
Why
Databricks defines goodput as productive training time, changing which failure time teams should measure.
No material change · Evidence held steady into Act Now on 2 signals across 2 sources. Databricks uses Databricks AI: frames goodput:the share of time large-scale training performs useful work:as the metric determining training efficiency
Sources: Databricks AI · Databricks AI
34
Projects
PyTorch
Act Now
What
PyTorch: Databricks frames goodput:the share of time large-scale training performs useful work:as the metric determining training efficiency
Do
Measure PyTorch goodput during one fault-tolerant training run instead of reporting uptime alone.Open the evidence from Databricks AI
Why
Databricks ties its runtime claim to the share of training time that produces useful work.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. PyTorch uses Databricks AI: Databricks frames goodput:the share of time large-scale training performs useful work:as the metric determining training efficiency
Sources: Databricks AI
35
Research
CorporateBench
Act Now
What
CorporateBench: is a human-validated, multi-task question-answering benchmark for enterprise document collections whose evaluation corpora exceed 230,000 documents
Do
Run CorporateBench's method against a permission-safe sample of internal enterprise documents.Open the evidence from arXiv AI + CL
Why
The benchmark targets document complexity and confidentiality absent from simple synthetic tests.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. CorporateBench uses arXiv AI + CL: is a human-validated, multi-task question-answering benchmark for enterprise document collections whose evaluation corpora exceed
Sources: arXiv AI + CL
36
Research
HarnessLens
Act Now
What
HarnessLens: targets two failure modes in agent-harness evolution: wasting verification rollouts on unrelated behaviors and allowing aggregate scores to hide specific
Do
Apply HarnessLens to one harness change and inspect which unrelated rollouts it avoids.Open the evidence from arXiv AI + CL
Why
The paper targets wasted verification and regressions concealed by aggregate scores.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. HarnessLens uses arXiv AI + CL: targets two failure modes in agent-harness evolution: wasting verification rollouts on unrelated behaviors and allowing aggregate scores
Sources: arXiv AI + CL
37
Research
SARA
Act Now
What
SARA: A paper on tool-augmented LLM agents argues that untrusted runtime observations can become action-driving commands and cause real-world side effects beyond
Do
Use SARA's separation of observation and authorization in one tool-action threat test.Open the evidence from arXiv AI + CL
Why
The paper shows how untrusted runtime text can drive actions beyond the user's intent.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. SARA uses arXiv AI + CL: A paper on tool-augmented LLM agents argues that untrusted runtime observations can become action-driving commands and cause real-world side
Sources: arXiv AI + CL
38
Technologies
Agentic Resource Discovery
Act Now
What
Agentic Resource Discovery: AWS published Agentic Resource Discovery (ARD), an open specification for agent discovery, alongside AWS Agent Registry - a centralized, searchable catalog
Do
Test Agentic Resource Discovery across two environments and verify every returned capability owner.Open the evidence from AWS ML Blog
Why
The specification addresses discovery across environments, where stale ownership can create unsafe routing.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Agentic Resource Discovery uses AWS ML Blog: AWS published Agentic Resource Discovery (ARD), an open specification for agent discovery, alongside AWS Agent Registry - a
Sources: AWS ML Blog
39
Technologies
CUDA MPS
Act Now
What
CUDA MPS: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by 75% while
Do
Load-test CUDA MPS concurrency on the current speech model before consolidating GPU workers.Open the evidence from AWS ML Blog
Why
The AWS result attributes lower GPU use to MPS combined with Triton under a measured workload.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. CUDA MPS uses AWS ML Blog: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by 75%
Sources: AWS ML Blog
40
Technologies
OpenTelemetry
Act Now
What
OpenTelemetry: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex
Do
Map the current agent trace into AgentCore's OpenTelemetry scoring fields before changing tools.Open the evidence from AWS ML Blog
Why
OpenTelemetry is the contract that lets the cited evaluator compare agents across frameworks.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. OpenTelemetry uses AWS ML Blog: Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with
Sources: AWS ML Blog
41
Companies
Alice
Act Now
What
Alice: Raises $140M at Near $1B Valuation as AI Security Revenue Nears $100M ARR - The SaaS Sentinel - Funding round: Series of funding totaling $140 million led
Do
Test Alice on one model-policy violation set before changing the current security monitor.Open the evidence from saassentinel.com
Why
Alice sells model testing and runtime monitoring, so detection quality must precede vendor adoption.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Alice uses saassentinel.com: Raises $140M at Near $1B Valuation as AI Security Revenue Nears $100M ARR - The SaaS Sentinel - Funding round: Series of funding totaling
Sources: saassentinel.com
42
Companies
Anthropic
Act Now
What
Anthropic: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude
Do
Compare Anthropic's cited Agent Arena route on the team's actual task and cost ceiling.Open the evidence from Agent Arena | AI Agent Performance Leaderboard
Why
The leaderboard evidence shows model cost and task success moving independently.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Anthropic uses Agent Arena | AI Agent Performance Leaderboard: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at
Sources: Agent Arena | AI Agent Performance Leaderboard
43
Companies
Conviva
Act Now
What
Conviva: CEO Keith Zubchevich said enterprises deploying more AI agents must evaluate the experience those agents create, not merely whether they work, because
Do
Add user-intent drift and conversation quality to the next customer-agent evaluation with Conviva.Open the evidence from SiliconANGLE theCUBE
Why
Conviva argues that functional completion misses the experience created by dynamic agent conversations.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Conviva uses SiliconANGLE theCUBE: CEO Keith Zubchevich said enterprises deploying more AI agents must evaluate the experience those agents create, not merely whether
Sources: SiliconANGLE theCUBE
44
Companies
DeepSeek
Act Now
What
DeepSeek: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude
Do
Validate DeepSeek's cited cost and success rates on an owned task before rerouting production.Open the evidence from Agent Arena | AI Agent Performance Leaderboard
Why
Agent Arena shows a low-cost route with measurable improvement, but the result is benchmark-specific.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. DeepSeek uses Agent Arena | AI Agent Performance Leaderboard: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at
Sources: Agent Arena | AI Agent Performance Leaderboard
45
Companies
GoDaddy
Act Now
What
GoDaddy: says its two-year migration from a legacy BI tool to Amazon Quick saves 15,000 hours annually, cut its dashboard count by 50%, reduced rendering times to
Do
Compare GoDaddy's migration measures with the current BI dashboard and labor baseline.Open the evidence from AWS ML Blog
Why
The cited savings combine fewer dashboards, faster rendering, and broader self-service access.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. GoDaddy uses AWS ML Blog: says its two-year migration from a legacy BI tool to Amazon Quick saves 15,000 hours annually, cut its dashboard count by 50%, reduced rendering
Sources: AWS ML Blog
46
Companies
Google DeepMind
Act Now
What
Google DeepMind: announced a pilot that it describes as the world's first double-blind AI evaluations
Do
Use Google DeepMind's double-blind setup on one model comparison with evaluator identity hidden.Open the evidence from Google DeepMind Blog
Why
The pilot tests whether brand knowledge changes evaluation of frontier-model outputs.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Google DeepMind uses Google DeepMind Blog: announced a pilot that it describes as the world's first double-blind AI evaluations
Sources: Google DeepMind Blog
47
Companies
Instinct
Act Now
What
Instinct: Viral AI startup Instinct has raised $350M at a $2.5B valuation | TechCrunch Summary: - Funding round/status: Series B funding; the latest round brings
Do
Keep Instinct in a permission-limited trial until the private-beta assistant proves repeat use.Open the evidence from techcrunch.com
Why
The funding evidence is strong while the product remains private and depends on broad app access.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Instinct uses techcrunch.com: Viral AI startup Instinct has raised $350M at a $2.5B valuation | TechCrunch Summary: - Funding round/status: Series B funding; the latest
Sources: techcrunch.com
48
Companies
Keenable
Act Now
What
Keenable: Startup Keenable Announces $26M in Funding To Build Web Search Infrastructure Built For AI Agents Keenable announced a seed funding round of $26 million
Do
Compare Keenable's agent index with the current search route on source grounding and recall.Open the evidence from theaiinsider.tech
Why
Keenable is building machine-focused retrieval, which should be judged on agent answers rather than human browsing.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Keenable uses theaiinsider.tech: Startup Keenable Announces $26M in Funding To Build Web Search Infrastructure Built For AI Agents Keenable announced a seed funding round
Sources: theaiinsider.tech
49
Companies
Moonshot
Act Now
What
Moonshot: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude
Do
Test Moonshot's cited Kimi route against the current default on the same agent workload.Open the evidence from Agent Arena | AI Agent Performance Leaderboard
Why
Agent Arena's shared task board exposes whether lower cost survives an owned workload.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Moonshot uses Agent Arena | AI Agent Performance Leaderboard: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at
Sources: Agent Arena | AI Agent Performance Leaderboard
50
Companies
Natera
Act Now
What
Natera: says its Amazon Bedrock AgentCore voice agent books mobile phlebotomy appointments with 100% tool-calling accuracy and sub-seven-second latency using a
Do
Audit Natera's appointment-booking measures against an equivalent healthcare handoff path.Open the evidence from AWS ML Blog
Why
The result combines tool accuracy, response latency, and progressive authentication in one workflow.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Natera uses AWS ML Blog: says its Amazon Bedrock AgentCore voice agent books mobile phlebotomy appointments with 100% tool-calling accuracy and sub-seven-second latency
Sources: AWS ML Blog
51
Companies
NVIDIA
Act Now
What
NVIDIA: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by 75% while
Do
Recreate NVIDIA's MPS and Triton configuration before reducing speech-recognition GPU capacity.Open the evidence from AWS ML Blog
Why
The reported infrastructure reduction is tied to a concrete serving configuration and latency target.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. NVIDIA uses AWS ML Blog: AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by 75%
Sources: AWS ML Blog
52
Companies
Rillet
Act Now
What
Rillet: turned an unsolicited board update into a $1 billion accounting startup in 48 hours - Startup Fortune Summary: - Funding round and status: $100 million
Do
Compare Rillet's real-time ledger workflow with one owned close or reconciliation process.Open the evidence from startupfortune.com
Why
Rillet's funding follows adoption of AI-native accounting inside a core finance system.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Rillet uses startupfortune.com: turned an unsolicited board update into a $1 billion accounting startup in 48 hours - Startup Fortune Summary: - Funding round and status
Sources: startupfortune.com
53
Companies
Tencent
Act Now
What
Tencent: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude
Do
Evaluate Tencent's cited route on the same Agent Arena task mix used for the current provider.Open the evidence from Agent Arena | AI Agent Performance Leaderboard
Why
A shared leaderboard result is useful only when the task mix matches the operating workload.
No material change · Evidence held steady into Act Now on 1 signal across 1 source. Tencent uses Agent Arena | AI Agent Performance Leaderboard: On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at
Sources: Agent Arena | AI Agent Performance Leaderboard

Act nowValidateMonitorDeprioritizeEvidence supportActionability

Focused view: Act now

01 / PlatformsAct Now

Amazon Bedrock AgentCore

Replay the appointment flow on AgentCore with tool accuracy, latency, and authentication recorded.

Open the evidence from AWS ML Blog

Movement: No material change
Evidence: 41 → 41
Actionability: 81 → 81

Evidence held steady into Act Now on 1 signal across 1 source. Amazon Bedrock AgentCore uses AWS ML Blog: Natera says its Amazon Bedrock AgentCore voice agent books mobile phlebotomy appointments with 100% tool-calling accuracy and

What should a production agent test cover?

CorporateBench tests enterprise answers across more than 230,000 documents, while current field evidence tracks how agents behave inside real work.

The operating decision is workflow-level approval: test the data, tool action, user result, and recovery path before expanding authority.

How quickly is agent access spreading across the business?

OpenAI is building agents for a wider set of tasks while public coverage still questions whether broad adoption will follow.

The useful measure is completed work with a visible review path, not the number of agents announced.

Agent distribution

OpenAI is building an agent for nearly every task
TechCrunch reports OpenAI is building AI agents for everything and openly questions adoption : 'Will everyone use them?' : framing agent proliferation from the frontier labs as a supply-side push whose demand side is unproven.
Sources: AI News & Artificial Intelligence | TechCrunch

Which enterprise tests expose failures beyond model scores?

Current research tests enterprise documents, employee prompts, tool-driven side effects, operational control, and evaluation bias.

Production teams should measure the full task inside its real context instead of treating a model score as a release decision.

Enterprise evaluation

CorporateBench tests enterprise answers across 230,000 documents
CorporateBench is a human-validated, multi-task question-answering benchmark for enterprise document collections whose evaluation corpora exceed 230,000 documents. Its authors identify confidentiality and overly simple synthetic data as obstacles to evaluating LLMs on corporate communications.
Sources: arXiv AI + CL
Polished market panels push model confidence from 6.5% to 54%
Across 12 frontier models, adding an increasingly professional-looking market panel raised commitment to a directional call on a provably unpredictable question from 6.5% to 54.0%. The paper says fully fabricated panels produced a similar increase, showing that polished evidence can induce action even when its numbers are invented.
Sources: arXiv AI + CL
DeepMind pilots double-blind frontier model evaluations
Google DeepMind announced a pilot that it describes as the world's first double-blind AI evaluations.
Sources: Google DeepMind Blog

Workplace evidence

Employee study analyzes 713,564 prompts across 15 functions
A field study analyzed 713,564 employee prompts and corresponding LLM responses from nearly 4,000 back-office employees across 15 functions over eight months in 2025. It reports that senior employees used generative AI more sophisticatedly, consistent with domain expertise shaping effective use.
Sources: arXiv AI + CL

Agent safety

Tool observations can become commands outside user intent
A paper on tool-augmented LLM agents argues that untrusted runtime observations can become action-driving commands and cause real-world side effects beyond user intent when action induction is conflated with execution authorization. It proposes SARA to separate those functions.
Sources: arXiv AI + CL

Applied operations

GoDaddy analytics migration saves 15,000 hours annually
GoDaddy says its two-year migration from a legacy BI tool to Amazon Quick saves 15,000 hours annually, cut its dashboard count by 50%, reduced rendering times to under five seconds, and made AI-powered self-service analytics available to every employee.
Sources: AWS ML Blog
NVIDIA MPS cuts speech-recognition GPU needs by 75%
AWS reports that NVIDIA CUDA Multi-Process Service with Triton on Amazon EC2 reduced GPU infrastructure for automatic speech recognition by 75% while sustaining sub-second latency at 92.1 requests per second per GPU.
Sources: AWS ML Blog
Databricks measures useful training time through goodput
Databricks frames goodput:the share of time large-scale training performs useful work:as the metric determining training efficiency. Its AI Runtime product is positioned around fast, fault-tolerant PyTorch training.
Sources: Databricks AI
Databricks moves Genie from answers toward action
Databricks announced Genie One features aimed at extending AI-assisted analytics from answering questions to taking action on insights.
Sources: Databricks AI

Operational reasoning

Models face operational control tests beyond text
Researchers evaluated LLMs on inventory control, queueing-network control, and assortment optimization, including a setting where a model received only the problem class and broad parameter ranges and had to return an algorithm. The paper reports that LLMs can design near-optimal operations-research algorithms.
Sources: arXiv AI + CL

What evidence should expand an AI employee's role?

Natera reports an appointment-booking voice agent, while Conviva adds user experience to evaluation and shopping research shows preference drift.

The role should expand only after task completion, user impact, and escalation are measured together.

Customer operations

Natera voice agent books mobile blood-draw appointments
Who for: healthcare teams using Bedrock AgentCore for appointment workflows.
Natera says its Amazon Bedrock AgentCore voice agent books mobile phlebotomy appointments with 100% tool-calling accuracy and sub-seven-second latency using a dual-WebSocket bridge, event-driven latency masking, and progressive-trust authentication.
Sources: AWS ML Blog

Experience evaluation

Conviva adds user experience to agent evaluation
Who for: enterprises measuring customer-facing agent experiences.
Conviva CEO Keith Zubchevich said enterprises deploying more AI agents must evaluate the experience those agents create, not merely whether they work, because dynamic conversations and evolving intent change the observability model.
Sources: SiliconANGLE theCUBE

Agent behavior

Small page changes alter agent shopping preferences
Research on agentic shopping found that small changes such as page-viewing order and memory altered AI purchasing preferences unpredictably, preventing consistent prediction or marketing influence over what an agent chose.
Sources: @emollick on X

Which funded AI software startups passed this week's qualification gate?

Rillet, Instinct, Alice, and Keenable met the exact-window funding, AI product, SaaS, and competitor-exclusion rules.

Each product still needs a clear buyer, permission boundary, and measured result before adoption.

Funded AI SaaS

Rillet raises $100 million for AI-native accounting software
Who for: finance teams replacing legacy accounting systems.
Rillet closed a $100 million Series C led by Iconiq Capital, bringing total funding above $200 million at a $1 billion valuation. Its AI-native accounting software runs a real-time general ledger with native integrations.
funded · ai SaaS
Sources: startupfortune.com
Instinct raises $250 million for a personal AI assistant
Who for: consumers granting an assistant access to daily apps and devices.
Instinct raised $250 million in a Series B co-led by Index Ventures and Benchmark. The round brings total funding to $350 million at a $2.5 billion valuation while the personal assistant remains in private beta.
funded · ai SaaS
Sources: techcrunch.com
Alice raises $140 million for AI security software
Who for: teams testing models and monitoring deployed AI systems.
Alice raised $140 million led by Apax Digital, bringing total funding to $280 million at a valuation near $1 billion. Its software tests AI models and monitors deployed systems for policy violations.
funded · ai SaaS
Sources: saassentinel.com
Keenable raises $26 million for agent-focused web search
Who for: product teams building agents that need a web index.
Keenable announced $26 million in funding to build web-search infrastructure for AI agents. The software gives agents an index designed for machine consumption rather than human browsing.
funded · ai SaaS
Sources: theaiinsider.tech

Which resources can test the next agent workflow?

The resource set covers agent evaluation, cost comparison, discovery, observability, and developer workflows.

Start with the failure already visible in production, run one bounded test, and keep the current route as a fallback.

Evaluation

Arize maps agent failures beyond the model
Stuart Sy of OpenAI, interviewed by Arize, argues better models do not fix every agent failure because the bottleneck has moved off the model and onto context, evals, and observability.
Sources: Arize AI Blog
GitHub explains how to evaluate LLMs before production
GitHub published the lessons its team learned evaluating LLMs for real-world secret scanning, framing pre-production LLM evaluation as its own discipline that gates whether an LLM feature ships.
Sources: GitHub AI Blog
AgentCore Evaluations scores agents across major frameworks
Amazon Bedrock AgentCore Evaluations uses OpenTelemetry as a framework-agnostic scoring contract and can evaluate agents built with LangGraph, LlamaIndex, the OpenAI Agents SDK, Google ADK, the Claude Agent SDK, or Strands Agents.
Sources: AWS ML Blog
HarnessLens isolates regressions in evolving agent systems
HarnessLens targets two failure modes in agent-harness evolution: wasting verification rollouts on unrelated behaviors and allowing aggregate scores to hide specific regressions. The proposed framework uses behavior-aware, budget-conscious verification to evolve harnesses.
Sources: arXiv AI + CL

Benchmarks

Agent Arena compares agent cost and task performance
On the same Agent Arena board, P50 cost per task spans two orders of magnitude - DeepSeek V4 Pro (High) at $0.24 and GPT 5.6 Luna at $0.07 against Claude Opus 5 at $2.55 - while MIT-licensed DeepSeek V4 Pro (High) posts 6.97% net improvement and 13.49% confirmed success versus the leader's 12.73% and 15.97%.
Sources: Agent Arena | AI Agent Performance Leaderboard

Open source

GitHub profiles OpenClaw growth and security work
GitHub describes OpenClaw as the fastest-growing project in GitHub history. Its maintainers are publishing lessons from the project's first six months, including work required to secure the rapidly adopted project.
Sources: GitHub AI Blog · GitHub AI Blog

Developer workflows

GitHub Copilot automates Dependabot pull-request triage
GitHub says the GitHub Copilot app can automate repetitive Dependabot pull-request triage.
Sources: GitHub AI Blog · GitHub AI Blog
SageMaker SDK v3 syncs local code into containers
Amazon SageMaker Python SDK v3 unifies script-mode workflows through ModelTrainer and ModelBuilder, while SourceCode can synchronize local code into any container at runtime so teams can iterate without rebuilding Docker images.
Sources: AWS ML Blog

Agent discovery

AWS publishes an open specification for agent discovery
AWS published Agentic Resource Discovery (ARD), an open specification for agent discovery, alongside AWS Agent Registry - a centralized, searchable catalog for agents, tools and skills. AWS positions the pair as enabling cross-environment agent discovery and governance at scale.
Sources: AWS ML Blog

Observability

Arize Signal finds hidden retry loops in production
Arize says its Signal tool found two hidden retry loops in the production agent Alyx: a duplicate task-state loop and a 43-call dataset retry that had appeared as valid tool activity or an OK root span.
Sources: Arize AI Blog

Methods and provenance

28 accepted stories · 28 public sources · 13 email headlines

Every headline above retains its full source set. The email is a ranked projection of this issue.

Use only candidates and source metadata prepared from the exact Weekly publication window.
Exclude Reddit evidence and sources without precise in-window publication time.
Treat duplicate funding coverage as one event and exclude managed-automation competitors from Startup Highlights.
Keep search context separate from story evidence and retain every qualifying source attached to accepted stories.

Continue with: About Joe Cervino and Anti Enterprises · Browse the Anti-Newsletter archive

Next issue

Keep the signal coming.

Get the Daily and Weekly Issues in your inbox.

Subscribe to receive both issues. Unsubscribe anytime.