GPUMODE A100 CONV2D V2 #1

November 2026

Big-filter convolutions are a ton of math. The ranked conv2d_v2 shape is one 256×256 image with 128 input and 128 output channels, a 32×32 filter, all FP32. That's 849 billion multiply-adds. PyTorch's deterministic FP32 path takes 2.55 s on an A100, and TF32 fails the 1e-3 tolerance bc a 10-bit mantissa can't hold sums that long.

I'm writing up the tricks I used to get #1 on A100. TLDR: move to the frequency domain, then spend basically all the effort on moving fewer bytes.

Only one shape counts

Only the last of the five benchmark shapes is ranked, so the submission dispatches on shape. The ranked shape gets its own CUDA path, and everything else is Helion and just has to pass.

An FFT doesn't sum in the same order as a direct convolution, so it only passes if the reference is close to exact. On A100 it is. On H100, L4 and B200 the reference's own FP32 rounding drifts past the tolerance, so an accurate FFT answer fails there.[1] Everything below is A100 only.

Frequency domain

In the frequency domain, the convolution becomes a small matrix product at each frequency: output channel co at frequency f is the sum over input channels ci of X[ci, f] · conj(W[co, ci, f]). Filter size doesn't show up at all.

A full-image FFT would need a 4.3 GB weight spectrum, so I use overlap-save[2]: cut the image into overlapping 88×88 tiles, transform each one, and keep the 57×57 block that wrap-around doesn't corrupt. 88 is the smallest tile that covers the 225×225 output with 4×4 tiles, and 88 = 8 × 11 keeps a hand-written FFT short.

256 × 256 input tile T = 88, placed every 57 pixels 88 − 32 + 1 = 57 exact outputs per side 4 × 57 = 228 ≥ 225 outputs, so 4 × 4 tiles 16 tiles × 128 channels = 2,048 FFTs 88 × 45 = 3,960 distinct bins per tile blue: one tile and the 57 × 57 block it keeps orange: the last tile runs past the image edge and is filled with zeros
One channel. Each 88×88 tile gets one FFT and keeps only its top-left 57×57 block after the inverse. The gray grid is those blocks covering the 225×225 output.

What's left is about 4.2 billion real multiply-adds of channel mixing, 200× less than direct. Even with 3 products per multiply (next section) that's around 80 µs of tensor-core work. The rest of the ~850 µs is data movement: every intermediate array is 65 MB or more, and the A100 only has 40 MB of L2. So most of what follows either removes an array or writes it in the exact order its reader wants.

FP32 accuracy from FP16 tensor cores

On A100, FP16 tensor cores are 2× faster than TF32, but FP16 only has 11 significand bits. So I split each FP32 value into a high FP16 part rounded to 11 bits, and a low FP16 part holding the rest:

ab≈ahibhi+ahiblo+alobhi ab \approx a_\text{hi} b_\text{hi} + a_\text{hi} b_\text{lo} + a_\text{lo} b_\text{hi}

3 FP16 products give about 22 bits, and the alobloa_\text{lo} b_\text{lo} term is too small to matter. The split is two integer ops:

static __device__ __forceinline__ float rn11r(float v) {
    unsigned u = f2u(v);
    u = (u + 0x1000u) & 0xFFFFE000u;  // keep 11 bits
    return u2f(u);  // hi; lo = v - hi is exact in fp32
}

What actually made it pass: the small cross terms get their own accumulators and are added in once per chunk. In one shared accumulator they get rounded against the big running sum, and a couple of outputs land just outside tolerance. FP16's range is small too, so the image spectrum is scaled by 1/16 and the trig tables by 16 (powers of two, so exact). Together with the 88-point tiles this cut 29%, and the worst error ends up at 0.5–0.65 of the tolerance.

Never writing the weight spectrum

The weight spectrum is 519 MB. Just writing it and reading it back is over half a millisecond, so it never gets written.

A 2-D DFT is rows, then columns. The row half runs up front and writes 189 MB (hi + lo). The column half runs inside the mixer kernel, one piece at a time in shared memory, and the mixer uses each piece right away. Spectrum rows b, 88 − b, 44 + b and 44 − b come from the same even/odd partial sums, so one pass gives four rows.

I also store every intermediate array in the exact register pattern mma.sync.m16n8k16 expects across a warp's 32 lanes[3] (here), so each lane loads its operands with one aligned 16-byte load. These two changes cut another 29%.

Hand-written FFT

cuFFT is great, but around it I still needed a gather to cut the tiles, a transpose, and a scatter and crop after the inverse. Each of those is a full pass over 65 MB or more.

My forward kernel does all of it in one pass. It loads a tile with cp.async, which writes zeros when the source size is 0, so the padding past the image edge is free:

// bytes to copy: 4 inside the image, 0 writes zeros
const int ok = (Y < 256 && X < 256) ? 4 : 0;
asm volatile("cp.async.ca.shared.global [%0], [%1], 4, %2;"
             :: "r"(smaddr(sm + r * RS + j)),
                "l"(src + min(X, 255)), "r"(ok));

Two real rows go through as one complex FFT, so 44 FFTs cover 88 rows. Each 88-point transform is 8-point DFTs, a twiddle, then 11-point DFTs. The inverse keeps a whole column in registers and only stores the 57 rows that survive the crop, so the crop is free and the second pass does 57 rows instead of 88.

The inverse went from 310 to 145 µs and the forward side from 390 to 304 µs, and the score dropped 19%. The kernels are CUDA source strings compiled with NVRTC[4] on first use, so the submission stays one Python file.

Two kernels, one launch

The image FFT and the weight row transform don't depend on each other and use different parts of the SM. FFT blocks sit on shared memory and wait on latency, and weight-row blocks run tensor cores out of registers. Back to back, each one leaves the other's resources idle.

So one launch runs both. Even blocks do the FFT, odd blocks do the weight rows, and launch bounds[5] fit two blocks per SM, which in practice means one of each.

const int g = blockIdx.x / (FPR + 1);
const int p = blockIdx.x % (FPR + 1);
if (p < FPR) fwd_body(g * FPR + p, x, z);  // one tile fft
else rows_body(g, wt, tab, tp);            // 8 filter pairs

The first stage went from 358 to 296 µs, and tuning how much work each block gets brought it to 266. That 15% is what took #1.

one launch · 266 µs even blocks: tile FFTs odd blocks: filter rows transpose to fragment order (Gluon) filter column DFT + channel mixer ≈ 390 µs inverse FFT and crop · ≈ 150 µs full weight spectrum 519 MB, never written z · 65 MB xq · 65 MB yf · 65 MB output tp 189 MB
The ranked path: 4 launches. Every array in between is bigger than L2, so each one is a full trip to memory. The weight spectrum only ever exists one piece at a time, in shared memory.

What didn't work

Right now the first launch takes 266 µs, the fused mixer 380–400 µs and the inverse ~150 µs. The mixer is L2-bound: it reads the row transforms 3 times and the image spectra 4 times.

Idea Result What profiling showed
Prefetching, L2 hints or 2 CTAs per SM in the mixer 470–892 vs ~400 µs L2 traffic is the limit, so hiding latency doesn't help
FFT writes the mixer's layout directly, no transpose 1,114 vs 876 µs scattered 8-byte stores from the FFT blocks cost more than the transpose
cuFFT strided transforms instead of transposes 4,445 vs 2,916 µs strided FFT reads are slow
cuFFT for the inverse, plus a crop copy 540 vs 311 µs an extra full pass, and it transforms all 88 rows
More threads or registers per FFT block 202–211 vs 199 µs registers and FFT math weren't the limit

Final results

The final kernel scored 851.5 µs, #1 on A100.[6] That's about 3,000× faster than PyTorch's deterministic FP32 reference (2.55 s).

Main takeaways:

Notes

  1. My H100 entry runs the FFT, then redoes the risky outputs in the reference's exact summation order. Slower, but it passes. ↩︎

  2. The standard way to do a long convolution as FFTs over overlapping blocks. Here the blocks overlap by 31, the filter size minus one. ↩︎

  3. Called the MMA fragment layout. The PTX docs list it for every mma shape. ↩︎

  4. NVIDIA's runtime compiler. It turns CUDA C++ source into PTX inside the running process. ↩︎

  5. __launch_bounds__(threads, min_blocks) caps registers so that many blocks fit on one SM. ↩︎

  6. Submission 972398, 851.51 µs. The previous #1 was 855.04 µs. ↩︎