GPU speed, without leaving Rust’s safety model
Rust’s ownership rules are valuable precisely when many actors touch shared state. A GPU kernel is one of the hardest places to preserve them: host code launches asynchronous work, device programs execute in parallel, and a mistaken destination index can turn into a race with undefined behavior.
cuTile Rust narrows the problem. Programmers write tile operations with single-threaded semantics. Mutable output tensors are partitioned into disjoint regions before launch. Shared inputs stay immutable. The generated launch boundary carries those distinctions from host Rust into device-side tensor views.
Split mutable memory before the GPU sees it, preserve ordering inside each tile, and make every escape hatch explicit.
The result is not “Rust makes every GPU program safe.” The safe surface covers the tensor access patterns cuTile Rust can express. Raw pointers and unchecked access remain available inside explicit unsafe code when a kernel needs lower-level control.
Ownership crosses the launch boundary
The familiar Rust rule is aliasing XOR mutability: one &mut T, or many &T, but not both. cuTile Rust maps that distinction onto a tile grid.
A host program partitions a mutable output, then passes each tile program an exclusive sub-tensor. Immutable inputs are broadcast as shared tensor views. The generated launcher holds those values while work is in flight and recovers them after synchronization with the same ownership form.
The paper’s element-wise add example contains no unsafe: host code makes 128-element output partitions, and the kernel loads matching input tiles before storing their sum. A proc macro generates the typed launch interface and enforces the accepted kernel parameter grammar at compile time.
What does not become safe automatically?
The model does not prove arbitrary per-thread protocols, unrestricted raw-pointer arithmetic, or every possible synchronization pattern. Unsupported patterns remain explicit unsafe opt-outs. The safety claim is about the accepted tensor and partition API under the paper’s launch and Tile IR assumptions.
Disjointness outside. Ordering inside.
Partitioning handles conflicts between tile programs. A second problem remains inside one tile program: GPU compilers may reorder operations unless the IR carries an ordering edge.
Tile IR uses tokens. Operations on an exclusive mutable view form a token chain, so a load followed by a store has a happens-before relationship. Reads through shared immutable views carry no such token dependency and can be reordered for performance.
The paper gives a concrete race: a cuTile Python attention permutation swaps two destination indices. Different tile programs then write the same destination, and 17–35% of elements differ across runs in the reported configurations. In cuTile Rust’s partition view, the programmer does not choose that destination index; the owned sub-tensor is the destination.
The paper proves data-race freedom for kernels written in the safe cuTile Rust API, assuming the generated launcher and kernel entry realize the stated disjoint partitions and token threading. Unsafe opt-outs are excluded from that theorem.
Build once, choose how to execute
Host-side GPU work is represented by DeviceOp: a lazy, typed operation that owns or borrows its operands and composes before submission. The same operation can be driven synchronously, awaited by an async executor, or captured as a CUDA graph.
- Sync blocks until the stream completes and returns recovered tensors.
- Async yields the host task while GPU work runs, allowing other I/O or control work to proceed.
- Graph captures a repeated sequence and replays it with one driver call.
The paper reports about 7.3 μs per operation for individually synchronized launches, about 3.4 μs for chained sync or amortized async, and about 0.8 μs for graph replay on the RTX 5090 experiment. Those are measurements for the described setup, not universal constants.
Why scoped graph capture helps borrowing
A recorded operation is added to the capture stream but does not execute as an independent kernel during capture. A borrow can therefore be released between record calls while the graph retains stream order. The implementation restricts graph capture to non-allocating operations so replay does not depend on unstable allocation addresses.
Safety without a visible tax
The central performance question is whether ownership-aware access costs more than manual pointer access. On the NVIDIA B200, the paper compares safe Rust, unsafe Rust, cuTile Python, and cuBLAS across memory-bound element-wise addition and compute-bound GEMM.
At N = 2^28, safe and unsafe Rust both reach 7.02 TB/s for element-wise add, while cuTile Python reaches 7.01 TB/s against a reported 7.68 TB/s peak. At M = N = K = 8192, safe Rust reaches 2.07 PFlop/s, 96.4% of cuBLAS; unsafe Rust is within 0.3% of it, and cuTile Python reaches 2.04 PFlop/s.
The end-to-end case study is Grout, a Qwen3 inference engine built on cuTile Rust. In the single-request sweep, Grout reaches 154.7 generated tokens/s for Qwen3-4B on an RTX 5090 at generation length 8192 and 80.1 tokens/s for Qwen3-32B on a B200, compared with 77.5 for vLLM and 76.5 for SGLang in that B200 configuration.
The benchmarks use specific GPUs, model sizes, tile shapes, clocks, and batch-1 workloads. Grout is a specialized case study, not a general-purpose serving-stack replacement. The paper also reports GEMM gaps at some sizes and uses cuBLAS for model GEMMs.
The useful boundary
cuTile Rust’s correspondence is strongest where a kernel naturally looks like:
- partition an output into disjoint tiles;
- read shared inputs;
- perform tile operations;
- write the owned output view;
- compose the launch with an explicit execution mode.
It gives up some SIMT-level control in exchange for tile-level reasoning. Explicit warp primitives, shared-memory protocols, and patterns outside the tensor API still need unsafe code. The API is young, and the paper identifies broader safe coverage, async heterogeneous scheduling, and cross-device ownership as future work.
The durable idea is not that a type system erases GPU complexity. It is that a carefully chosen abstraction can move the most common race invariants into ordinary Rust ownership, keep the fast path close to the backend, and leave the remaining danger visible.
Read the source
Melih Elibol, Jared Roesch, Isaac Gelado, Eric Buehler, and Michael Garland. “Fearless Concurrency on the GPU.” arXiv:2606.15991, 2026. Paper · CUDA Tile · cuTile Python · Grout