NVIDIA's Groq 3 LPX AI inference accelerator chip is now in full production. The Groq 3 LPX racks are an extension of the NVIDIA Vera Rubin platform ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌

TLDR

Together With

TLDR AI 2026-08-25

Powering the next era of Confidential AI (Sponsor)

Google Cloud, is committed to providing the most advanced, secure, and private infrastructure for the most demanding AI workloads, and partnering with a broad and diverse range of organizations to help them meet their AI workload needs.

Working closely together, Apple and Google have built a serving platform on Google Cloud that meets the rigorous security, confidentiality, and transparency goals that Apple has for Private Cloud Connect PCC.

By protecting data in use, Confidential Computing becomes a fundamental and foundational element for building trust in AI systems, providing verifiable integrity and isolation for sensitive workloads.
Get started on Google Cloud for free →

🚀

Headlines & Launches

NVIDIA Enters Full Production of Groq 3 LPX AI Inference Accelerator Chips, Supercharging Vera Rubin With The Fastest Token Generation Speeds Ever Recorded (4 minute read)

NVIDIA's Groq 3 LPX AI inference accelerator chip is now in full production. The Groq 3 LPX racks are an extension of the NVIDIA Vera Rubin platform. They deliver boosted AI inference capabilities that enable ultra-fast token generation for response-sensitive agentic workloads. Groq 3 LPX enables agentic AI tasks to be done within minutes versus hours. It offers a 4x boost in response times versus the nearest alternative platform.

Anonymous Ox Alpha processes 26T tokens on OpenCode, breaks OpenRouter launch record (4 minute read)

OpenCode users processed 26 trillion tokens through Ox Alpha during the model's first four days. The anonymous AI model recorded 327,000 unique users and 8,328,244 completed sessions. It is currently available for free through an OpenAI-compatible endpoint, making it easy for developers to substitute the model into existing workflows. OpenCode's model page doesn't list Ox Alpha's maker, its release date, knowledge-cutoff date, or output-limit metadata.

🧠

Deep Dives & Analysis

Hot Chips 2026: CUDA Targets RISC-V (7 minute read)

Nvidia is looking to extend CUDA support to RISC-V. This will open the door for RISC-V CPUs to feed GPU compute. RISC-V's software ecosystem has some distance to go before catching up to x86-64 and aarch64. While Nvidia's effort to bring CUDA into the RISC-V world is a promising development, the vast majority of existing RISC-V hardware won't meet Nvidia's requirements.

LLMs could control their host machines by exploiting inference engines (6 minute read)

Host machines running AI models are high-value targets: they have sufficient compute to run a frontier LLM, offer easy access to the LLM's weights, and have privileged access to other computers in the datacenter compared with a generic computer on the Internet. Research shows that LLMs can run token sequences that exploit vulnerabilities in the software that loads an LLM onto GPUs. This attack surface may be further increased with vision and audio tokens. Possible mitigations for this type of attack would be to run GPUs and token parsers on separate computers and to restrict the permissions granted to GPU hosts and treat all the data they emit as untrusted.

The Economics of the Intelligence Frontier (20 minute read)

AI tasks become commodities once models exceed their maximum necessary intelligence, shifting competition toward cost, latency, infrastructure, and distribution. Frontier labs can still become enormous businesses if new capability creates valuable markets faster than competitors reproduce and commoditize those advances.

🧑‍💻

Engineering & Research

Hazard Hunt: $80K to find where AI models actually break (Sponsor)

Every frontier model refuses chemical, biological, and cyber requests in theory. Hazard Hunt puts that to the test with $80K across four categories, scored purely on break count. Specialize in one category or spread across all four, starting Tuesday in the Gray Swan Arena.

Enter the arena →

Speculative Programmatic Tool Calling (12 minute read)

Speculative Programmatic Tool Calling (sPTC) optimizes recursive language models by pre-launching tool calls during token generation, reducing latency from high-latency tools and context generation. This method acts like a JIT compiler, allowing parallel execution of non-blocking tool calls, providing a 1-1.2x runtime speed-up. sPTC is particularly useful in memory-bound local LLMs and high-volume serving systems by overlapping computation with execution time, offering significant performance improvements for intricate program executions within harnesses like RLMs.

Graph Engineering (GitHub Repo)

A curated collection of papers, benchmarks, and open-source projects exploring how dynamic graph structures can organize tasks, coordinate agents, track runtime state, and support the evolution of multi-agent systems.

Rome (GitHub Repo)

Runs persistent AI agents, workflows, and apps inside a guardrailed collaboration environment.

🎁

Miscellaneous

GTM Engineer, Applied AI at TLDR ($175-205k base + $40-60k bonus, Fully Remote)

TLDR is hiring a GTM Engineer to join our Applied AI team and own our AI-native GTM stack. We're looking for someone comfortable building AI agents and working with HubSpot. Click here to learn more!

When code is abundant (31 minute read)

Large language models are transforming software development by making code generation faster and cheaper, shifting the primary challenge from creating code to trusting and verifying it. Advanced engineering organizations like Stripe, Spotify, and Amplitude have started integrating AI-generated code into production, emphasizing the need for robust governance, context, and verification systems.

The AI Bullwhip (5 minute read)

The AI infrastructure faced a series of bottlenecks from GPU scarcity to storage issues, leading to increased costs across the supply chain. As demand for AI components surged, GPU prices spiked, server shipments declined, and memory manufacturers shifted focus to High Bandwidth Memory. This mismatch of supply and demand caused inflated hardware prices and higher data center construction costs, highlighting a classic Bullwhip Effect.

⚡