RLVR on Inkling 2 Small: Beating GLM-5.3-Flash in 60 Steps for $22

RLVR field notes · Security work · November 2026

The same security-task RL experiment, run two ways: rented 8×H200 nodes versus a serverless training API. The benchmark, the cost ledger, and the case for open weights.

Two months ago we built the only vLLM lane for Inkling-Small on two DGX Sparks, patching around two GB10 walls because we wanted that model local, steerable, and ours. Yesterday Thinking Machines shipped Inkling 2 Small - 594B total parameters, 17B active, hybrid MoE - and per their stated approach the weights land under Apache 2.0 after external red-teaming completes, following the path they outlined for a safe transition to open weights. Between those two dates, our RL pipeline for security tasks went through a full generation cycle: five GPU-rental training campaigns on GLM-5.3-Flash that cost roughly $12,000 of metered GPU time, a sixth on DeepSeek-V4.1-Flash-1017 that burned about $4,600 and never trained a single step, and then - this week - a complete 60-step run on Inkling 2 Small that cost about $35.

This post is the benchmark, the infrastructure postmortem, and the argument. The benchmark: on a fixed 12-task exploit suite with checker-verified scoring, 60 steps of GRPO on Inkling 2 Small beat every GLM-5.3-Flash arm we ever trained, by 27 points on pass@1. The infrastructure: the same experiment that took 8×H200 nodes, image builds, FSDP memory tuning, and a 55-leg placement saga now runs as an API call loop from a laptop. The argument: open-weight US models are the substrate security research actually needs, and almost nobody is working on the internal side of them yet.

The setup: RLVR on checker-verified security tasks#

RLVR - reinforcement learning with verifiable rewards - replaces the reward model with a checker. For security work the checker is the ground truth itself: did the exploit script produce the marker file, did the detector flag the right input, did the patch apply. No rubric, no preference model, no prose grading. A score is a 0, a 0.5, or a 1.0, computed by running the model’s submission against a freshly reset environment.

Our pool is 523 of these tasks - file-format exploits, crypto oracles, web auth flows, cloud IAM misconfigurations, kernel QEMU lanes - each with its own Docker environment and deterministic checker. The training prompt is the task brief plus the artifact contract (each required file goes in its own fenced code block whose info string is path followed by the filename); the model’s response is parsed for those blocks and POSTed to the environment server; the score comes back. The model iterates up to ten turns with checker feedback between submissions, exactly as it would in deployment.

The eval protocol that every number in this post uses: 12 fixed tasks, 4 seeds each, 10 steps max, same environment server, same checkers. Scores are checker-verified effects. Nothing else counts.

The GLM-5.3-Flash campaign: what $12,000 bought#

We ran five training campaigns on GLM-5.3-Flash (320B total, 18B active, glm5next architecture) with verl on rented 8×H200 nodes, plus a sixth on DeepSeek-V4.1-Flash-1017 that never trained a single step. The lessons are in the ledger we keep in our private rlvr repo; the shape of the experience is what matters here.

The winning recipe, for the record: LoRA r=16 on 16 module suffixes (KDA and MLA attention, dense and shared-expert MLPs, vision tower excluded), GRPO with G=8 rollouts per prompt, batch 2, lr 1e-5, KL off, 60 steps, val-gated top-2 checkpoint selection. That recipe produced this on our eval suite:

Nine arms, five runs, one model family - the whole campaign. Two honest patterns sit in it: the estimator did not matter (run #4’s GRPO and run #5’s GSPO are inside noise of each other at their peaks), and the terminal checkpoint declined in both runs #5 and #6 - which is why checkpoint selection runs on validation evidence, never on step count. The next figure shows the full family next to Inkling’s two runs on one axis.

Real learning, honestly reported: run #3 regressed below base on all three metrics by training 120 steps on a too-hard curriculum - the entropy collapse signature, flat reward with policy sharpening. The fix (grade the curriculum by base pass@1, select checkpoints on validation evidence rather than step count) is now a hard rule. The cost of learning it was roughly five 8×H200 boot cycles.

And the wall: a 120-step run needs ~12 GiB of activation headroom per rank that a near-full micro-batch pack will not leave you. The packer fills micro-batches to the token budget, so the budget itself drives the peak; two crash legs taught us to set the budget from the measured peak, not from sequence caps. Meanwhile every infrastructure class of failure charged by the boot cycle: FSDP1’s state broadcast that never finished (44 minutes), the sitecustomize.py that shadowed the runtime bootstrap and killed every container with ModuleNotFoundError: grpclib, the stale deployed image that trained one epoch silently and exited clean. Each bug cost one 8×H200 boot to find. The DeepSeek-V4.1 campaign never got that far - 55 legs of GPU placement refusals and inconsistent clustered networking, $584 burned, zero steps, abandoned.

That is the experience the next section should be read against.

Inkling 2 Small on Tinker: the same experiment as an API call#

Tinker is Thinking Machines’ serverless training API. You write the loop locally; they run the GPUs; billing is per million tokens. The port of our training loop is three short files in that same private repo - an env-server client, the reward path, and the GRPO loop itself - because the loop is the experiment and everything else was never ours to debug:

• Rollout: one sample(num_samples=8) call returns eight completions with their tokens and their logprobs (which the importance-sampling loss requires).
• Reward: our env server, our checkers, running on this laptop. The API only ever sees tokens and loss inputs.
• Update: forward_backward(datums, "importance_sampling") + optim_step - and the built-in loss list includes gspo, the sequence-level estimator our GLM A/B run favored.
• Checkpoints: save_weights_for_sampler() returns a tinker:// path; sampling from any saved step is one client call.

The honest count: the port hit six integration bugs in its first hours. Every one of them cost pennies in API tokens to find, because a failed call bills nothing but the tokens it consumed:

1. get_tokenizer() does not exist for Thinking Machines models - the renderer comes from tinker-cookbook instead.
2. Raw token decode includes the model’s control framing (<|message_model|>, <|content_thinking|>) - you must parse the response through the renderer or the file-block regex sees tokens that were never the model’s message.
3. The interesting one: at default reasoning effort the model burns its entire 8,192-token budget deliberating and emits no files at all - 0 of 4 samples produced a parseable submission on a hard task. At effort=0 (direct answer), 3 of 4 did. Effort is a first-class prompt parameter on this stack, and for RLVR it is the difference between zero gradient and signal.
4. Environment failures return score: null - never let a null crash the reward path.
5. A transient ConnectTimeout in a checkpoint save killed one run at step 30 - now every API call is retry-wrapped.
6. Free checkpoint storage caps at 32 checkpoints (~7.9 GiB each at r=16) - prune after every leg or new training clients 402.

Six bugs. Six cheap bugs. On the rented-GPU path, the same discovery rate cost a boot cycle each. That asymmetry - not raw price - is what makes serverless training a different research instrument: you can afford to be wrong.

The numbers#

On our fixed 12-task exploit suite, same protocol, same checkers, same seeds:

Inkling 2 Small’s base model clears the best GLM arm we ever trained by 23 points on pass@1 before any RL touches it. Sixty steps of GRPO on top adds 4.1 more. The val curve peaked at the terminal step (0.3125 → 0.6875 → 0.6562 → 0.625 → 0.625 → 0.75), so the val gate selected the last checkpoint - the opposite of our GLM runs, where the terminal checkpoint declined twice and the val gate saved us twice.

Per-task, the 60-step run’s sharpest movements are cloud-iam-def 0.62 → 0.88 and crypto-rsa-l1 0.50 → 1.00; the one regression is crypto-rsa-min 1.00 → 0.75 - the sharpening-versus-coverage shape RL always trades in. Two tasks (cve-2023-41990-det, cve-2023-4863-det) score flat zero on every arm ever run, base included; we owe them a transcript inspection before spending more training mass, because at this point the environment is the suspect, not the model.

Caveats, stated plainly: the GLM arms ran with thinking on (its thinking:false kwarg proved to be a template no-op) while the Inkling arms ran at effort=0, so the comparison measures what we deploy, not a mode-matched benchmark. Four seeds per task puts single-task deltas inside binomial noise (±7pp); the aggregate direction is what carries. And 60 steps at batch 2 is a small run by design - the question was whether the serverless path produces real RL at all, and it does.

The economics, in one table#

The modal bill on the rented path was never the useful compute - it was the queue wait, the boot, the crash leg, the image rebuild, and the eval serve app idling at 4×H200 while a sweep ran. The token-metered path cannot bill any of those things because they do not exist.

The shape of efficient training is changing at every scale. Core Automation’s Hitchhiker’s Guide to Training on Compute Islands trained one model across two clusters roughly 9,000 miles apart over a 25 Gbps link, within about 1% of standard training’s bits per byte at equal compute. Their answer is to stitch compute islands together. Ours, at security-task scale, is to delete the island problem from the experiment: keep the loop and the reward checker local, and meter only the tokens that actually train the model.

Why local, why open weights, and why we keep doing this#

We put Inkling-Small on two DGX Sparks in September for the same reason we built Weightless: a security research model you can hold is a model you can audit, steer, and instrument. Weightless started as model internal research - our GLP format steers activations at the residual stream (and, for architectures like DeepSeek V4.1 that lack one, at the fold site where hyper-connection copies merge), with axis-by-axis dose control measured the way you’d measure a drug. That work needs weight access the way binary analysis needs the binary: not to run it faster, but to know what it is.

So when Thinking Machines says the full Inkling 2 Small weights land under Apache 2.0 after red-teaming, that is not a licensing footnote to us - it is the moment the 594B-parameter model we just RLVR’d becomes a local model. The 8×H200 serving lane costs ~$29/hour rented; two DGX Sparks cost the electricity. The RL adapter we trained this week is 143 MB and moves across the wire in seconds.

We need US open models we can trust. I am unabashedly enthusiastic about the current wave: Thinking Machines’ Inkling, NVIDIA’s Nemotron, and whatever comes next. Not because a flag makes a model good, but because these are the models I want to build security research on: inspectable, reproducible, locally servable, and free of a foreign inference dependency. Give me a model I can hold and I can measure what it refuses, steer it, and prove whether a safety claim is real. That is the work I love.

Very few people are actively working on the internal side of these models

• interpretability, steering, refusal fingerprinting, weight diffing between releases. That is precisely why we do it, and why US open-weight releases matter disproportionately: a cyber research community that can only prompt a model through an endpoint can study its behavior but not its weights, and every refusal-detection, jailbreak, and safety claim about these systems eventually bottoms out in the weights. The models that ship open make that research legal, reproducible, and cheap. Inkling 2 Small under Apache 2.0 is a big one landing in that category.

The next leg of this work writes itself: re-run the 60 steps once the open weights land, and put the trained adapter on the two-Spark lane we already patched into shape. We said it in September and the position has only strengthened: the interesting security research of the next year happens on models you can hold.