Automating CPU-to-GPU Acceleration — V2

Introduction
This is an update to the earlier Gpu Automation post. The core idea is unchanged: a [GpuAccelerate] decorator on an ordinary C# method turns it into a GPU-accelerated system, generated by a Roslyn-based build tool — no shader code, no buffer management, no kernel boilerplate written by hand. Since the first version, the generator gained two more patterns, a runtime dispatcher that measures real cost and decides GPU vs. CPU per call, automatic handling of a race condition that shows up in any pair-interaction system, automatic inlining of helper methods, and a couple of safety rails for working alongside generated code. This post covers what changed.

The Problem
As entity counts grow, CPU-side systems like these become a bottleneck — a simple per-entity update, and an O(n²) per-entity inner query for collision checks:

The Idea
A [GpuAccelerate] decorator on an ordinary C# method is the entire integration surface. Four things describe the system:

●      Name — name of the generated system (Input/Shader/Kernel files)
●      Pattern — detected automatically
●      RW — read-write GPU buffers uploaded every frame (e.g. positions, outputs)
●      RO — read-only buffers uploaded once (e.g. directions, speeds, weights)
●      Uniforms — scalar values passed per-dispatch (e.g. deltaTime, threshold)

How It's Built
GpuGenerator.exe --scan <dir> --out <dir> is a standalone tool built on Roslyn — a real C# compiler. For every accelerated system, it now outputs four files: Input, Shader, Kernel, and Dispatcher.
The pipeline runs in four stages:
●  Scan — Roslyn parses every .cs file and finds [GpuAccelerate]-tagged methods.
●  Classify — AstWalker and AstClassifier walk the method body and tag each statement: Math, Loop, IfPair, and so on.
●  Detect Pattern — a PatternDetectionVisitor infers Map, AtomicPairs, or CrossPairs purely from the shape of the classified statements.
●  Generate — a matching visitor emits the GPU shader, kernel, entity struct, and dispatcher.

Patterns
The compiler infers the shape from the code; nothing needs to be tagged by hand.

The rule came from data, not guesswork: systems across three validated repositories were surveyed and classified by shape, and a PatternDetectionVisitor turned that survey into a mechanical rule — math only maps to Map, a loop over one set maps to AtomicPairs, two loops over different sets map to CrossPairs, anything else falls to Generic.

Why it matters: one real ~15-line CPU method (collision detection) generates roughly 200 lines of GPU dispatch, shader, and kernel code automatically, across four files — code that would be tedious to write and keep in sync by hand.

Adaptive Dispatch
Every generated dispatcher owns an AdaptiveThreshold, a five-phase model that chooses GPU or CPU-parallel per call, based on live-measured timing rather than a fixed assumption.
●  CpuWarmup — run CPU for 10 frames to get a clean baseline.
●  CpuLearn — recompute the threshold every frame from live timing.
●  FirstGpuProbe — one throwaway GPU dispatch to discard the cold shader-compile cost.
●  Validate — confirm GPU is actually faster; if not, raise the threshold and re-learn.
●  Stable — use GPU past the threshold; re-validate every n frames, and revert if performance drops below half.

Symmetric Pair Rewriting
A classic CPU idiom — visit each pair once, react on both sides — is a GPU data race. A neighbor loop that checks nIdx < i to avoid processing a pair twice, then updates Force on both particles[i] and particles[nIdx], is safe on a single CPU thread but breaks the moment each entity gets its own GPU thread: two threads can write to the same Force[i] slot at the same time.

GpuGenerator detects this shape and rewrites it automatically. The SymmetricPairRewriter splits the branch into two halves so every thread only ever writes its own output slot — one half handles nIdx < i, the other handles nIdx > i — removing the race without the developer needing to think about GPU memory semantics at all.

Reusing Code Automatically
Accelerated methods often lean on small helper functions for shared math. GpuGenerator inlines these into the generated shader on its own: any helper method it can see in the source, called from inside an accelerated method, is automatically inlined into the generated shader. A [GpuInline] attribute can be added to a helper as an explicit way to force the inlining, but in the common case nothing needs to be added by hand.

Safety Rails
●  Freeze = true — the generator never regenerates this system again, even if the source method changes. A permanent opt-out.
●  SkipIfTouched = true — if the generated files are edited by hand, GpuGenerator detects that (via a timestamp check) and will not regenerate them on the next build; it only keeps regenerating while the generated files are still untouched. A one-time opt-out, triggered as soon as generated code is manually modified.

Benchmark Results
Position Update [Map]: Across the whole tested range — 0 to 10,000 entities — CPU stays ahead of GPU. There's no crossover in this range; for a pattern this cheap per-entity, GPU dispatch overhead isn't worth paying even at 10,000 entities, and the adaptive dispatcher correctly keeps using CPU-parallel throughout.

Collision Detection [AtomicPairs]: Here GPU wins early. CPU degrades quadratically while GPU stays comparatively flat. The AdaptiveThreshold model puts the crossover at 1,299 entities, matching where the two measured curves actually cross.

Trigger Detection [CrossPairs]: Measured at two trigger-volume counts. More triggers per entity means more CPU work per call, so the GPU's fixed dispatch cost amortizes sooner — the threshold drops as the per-call workload grows, exactly as the linear-class formula predicts.
1,000 triggers — threshold: 1,111 entities

 5,000 triggers — threshold: 278 entities

Limitations
●  The decorator must be placed on a pure CPU computation method.
●  The Generic pattern emits a commented stub for shapes the classifier cannot detect, so a developer has to complete the implementation manually.
●  All benchmark results are on a single machine.

Future Work
●  Testing on different systems and expanding to other systems.
●  Determine if more patterns would be needed.
●  Broaden validation across different hardware and operating systems beyond the current single-machine setup.

Conclusion
The [GpuAccelerate] decorator turns a CPU computation method into a dual-path GPU + CPU-parallel system, built on the real Roslyn compiler. Map, AtomicPairs, CrossPairs, and Generic are auto-detected from code shape, and an AdaptiveThreshold decides GPU vs. CPU per call from live-measured cost rather than a fixed assumption. Correctness and reuse are automatic too — symmetric pair rewriting and [GpuInline] resolution both work without requiring manual attributes.