Introducing CUDA Rust: Two Tracks for Writing GPU Kernels

What this is

CUDA Rust is NVIDIA's move to let GPU kernels be written in native Rust rather than launched from Rust into code written in another language. It was announced in September 2026, alongside a statement that NVIDIA will keep growing and maturing CUDA Rust into 2027 and beyond, positioning it alongside the mature CUDA C++ and CUDA Python toolchains (source). It's aimed at people already building the Rust side of the AI systems stack — inference engines, serving infrastructure, drivers, agent runtimes — who currently have to drop out of Rust the moment they need to write the kernel itself. There are two tracks, matching the two models CUDA already exposes: SIMT, via cuda-oxide, where you write what one thread does and launch thousands; and Tile, via cutile-rs, where you write what one tile of data does and the Tile IR compiler decides how that maps onto the hardware. NVIDIA's own guidance is to reach for Tile first and drop to SIMT when you need direct control over memory and threads.

Install

cutile-rs is the lighter of the two to get running: it's a published crate, needs no nightly toolchain, and needs no LLVM of your own — just a supported GPU, a compatible CUDA toolkit, a recent stable Rust toolchain, and Linux.

cargo new vecadd_demo
cd vecadd_demo
cargo add cutile

The SIMT track (cuda-oxide) is installed differently, as a custom rustc codegen backend driven by a Cargo subcommand, and needs a pinned nightly toolchain, clang with libclang headers, and a CUDA toolkit:

cargo +nightly install --git https://github.com/NVlabs/cuda-oxide.git cargo-oxide

Hello world

Both tracks ship the same example — elementwise addition over 1,024 floats — so you can compare them directly. Here is the Tile version. Paste it into src/main.rs and run cargo run:

use cutile::prelude::*;

// The macro captures this module's AST into the host binary. The kernel is
// JIT-compiled through CUDA Tile IR the first time it is actually launched.
#[cutile::module]
mod kernel {
    use cutile::core::*;

    #[cutile::entry()]
    fn add<const B: i32>(
        // B is the tile width, a static dimension. A different B produces a
        // different specialization.
        z: &mut Tensor<f32, { [B] }>, // exclusive output, one sub-tensor of B elements
        x: &Tensor<f32, { [-1] }>,    // shared input; -1 is a dynamic dimension, resolved at launch
        y: &Tensor<f32, { [-1] }>,
    ) {
        // This body runs once per mut sub-tensor, as a single logical thread.
        // Tile kernels load tiles, not scalars, from x and y.
        let tx = load_tile_like(x, z); // the slice of x lining up with this sub-tensor of z
        let ty = load_tile_like(y, z);
        z.store(tx + ty); // elementwise across the whole tile
    }
}

fn main() -> Result<(), Error> {
    let device = Device::new(0)?;
    let stream = device.new_stream()?;

    // These are lazy. Nothing has touched the GPU yet.
    let x = api::ones::<f32>(&[1024]);
    let y = api::ones::<f32>(&[1024]);

    // Partitioning does three things at once: gives each tile exclusive
    // ownership of its own 128-element chunk, fixes the grid at 1024/128 = 8
    // tiles, and supplies B.
    let z = api::zeros::<f32>(&[1024]).partition([128]);

    let c: Vec<f32> = kernel::add(z, x, y) // takes ownership of all three tensors
        .first()                           // ...and returns them; pick the output back out
        .unpartition()                     // drop the host-side partition wrapper; no data moves
        .to_host_vec()                     // record the copy back
        .sync_on(&stream)?;                // and only now does any of it run

    let errors = c.iter().filter(|&&v| (v - 2.0).abs() > 1e-5).count();
    if errors == 0 {
        println!("PASSED: all {} elements correct", c.len());
    } else {
        eprintln!("FAILED: {errors} errors");
    }
    Ok(())
}

The equivalent SIMT program (cuda-oxide) is built the same way, via cargo oxide new vecadd_demo && cargo oxide run, but needs a DisjointSlice<f32> type on the output parameter to make the same exclusivity guarantee that Tile gets from .partition() — full listing at the NVIDIA blog post.

The one concept you must understand first

Everything about both tracks follows from what unit of work the kernel body describes. In SIMT, the function body is what one thread does, and you're responsible for indexing into flat memory correctly for that thread — which is why cuda-oxide needs a type like DisjointSlice to prove, at compile time, that no two threads can write the same element. In Tile, the function body is what one tile does — a whole sub-tensor treated as a single logical unit — and the compiler, not you, decides how many real GPU threads back that tile and how the mapping changes across architectures. That's the whole reason NVIDIA's guidance is to reach for Tile first: your source code doesn't encode architecture-specific choices, so it doesn't need updating when the target GPU does. You only drop to SIMT when you need that lower-level control over memory and threads yourself (source).

Next steps

• cutile-rs on GitHub — the crate itself, for the Tile track's API surface beyond this one kernel.
• Tile IR compiler documentation — the compiler that both cutile-rs and the C++/Python tile frontends target; useful once you want to understand what's happening between your #[cutile::entry] function and PTX.
• cuda-oxide on GitHub — the SIMT track's repo, for when a kernel needs direct control over memory and threads that the Tile model abstracts away.

The sources do not give version numbers, pricing, or benchmark figures for either track, and do not describe the promised inter-language interop beyond stating that it's planned.