Skip to content

gpu-guide.md

Latest commit

 

History

History
315 lines (221 loc) · 8.04 KB

gpu-guide.md

File metadata and controls

315 lines (221 loc) · 8.04 KB
Raw

GPU Acceleration Guide

Cross-platform GPU computing based on wgpu: Vulkan / DirectX 12 / Metal, calling Rust kernels via Java FFM.

Contents

Overview

YiShape-DL GPU acceleration is provided by yishape-math-gpu, included as a Maven transitive dependency in yishape-dl — no separate declaration needed.

Features:

  • Cross-vendor: NVIDIA, AMD, Intel, Apple Silicon (Metal)
  • No CUDA: No CUDA/cuDNN toolchain installation required
  • Transparent fallback: Automatically falls back to HPC SIMD / Java CPU when GPU is unavailable or operator scale is too small
  • Precision: double at Java API layer, float32 in GPU kernels (consistent with PyTorch default)

Accelerated operators include: GEMM, convolution, activations, LayerNorm, BatchNorm, pooling, Depthwise Conv, Attention, etc.

Dependencies

<!-- GPU/HPC/Math all included in one -->
<dependency>
    <groupId>com.yishape.lab</groupId>
    <artifactId>yishape-dl</artifactId>
    <version>0.5.0</version>
</dependency>

Native libraries are packaged inside the yishape-math-gpu JAR and auto-loaded by OS/architecture:

libs/
├── windows-x86_64/yishape_math_gpu.dll
├── linux-x86_64/yishape_math_gpu.so
└── darwin-aarch64/yishape_math_gpu.dylib  (etc.)

Quick Start

1. Detect GPU

// DL facade detection (recommended)
System.out.println("GPU available: " + DL.device.isGpuAvailable());
System.out.println("Current device: " + DL.device.current());

2. Enable GPU Training

import com.yishape.lab.yishape.dl.DL;

DL.device.set(DL.device.gpuIfAvailable());

var model = DL.nn.sequential(
    DL.nn.linear(784, 256), DL.nn.relu(),
    DL.nn.linear(256, 10));
var trainer = DL.train.adam(model, DL.loss.crossEntropy());

trainer.fit(loader, 10, DL.train.logger());

3. Model Migration

model.toDevice(DL.device.gpu());
// ...
model.toDevice(DL.device.cpu());

Device Management

API Behavior
DL.device.gpuIfAvailable() Use GPU if available, else CPU (recommended)
DL.device.gpu() Force GPU, throws IllegalStateException if unavailable
DL.device.cpu() Force CPU
DL.device.set(device) Set current thread's device context
DL.device.current() Query current device
DL.device.isGpuAvailable() Whether GPU is available
DL.device.isGPU() Whether current device is GPU
DL.device.isCPU() Whether current device is CPU
DL.device.reset() Reset to CPU

Runtime Toggle

import com.yishape.lab.gpu.GpuSwitch;

GpuSwitch.disable();  // Globally disable GPU, immediate CPU fallback
GpuSwitch.enable();   // Restore

JVM Launch Parameters

java -Dyishape.gpu=false -jar app.jar
java -Dyishape.gpu.gemm.minFlops=1000000 -jar app.jar

Small-matrix GEMM can avoid GPU launch overhead via the minFlops threshold.

Execution Path and Fallback

Request computation
    ↓
GPU available && scale ≥ threshold ?
    ├─ Yes → wgpu compute shader (float32)
    └─ No  → HPC Rust SIMD
              └─ Still not met → Java SISD (double)

Trainer batch GEMM benefits significantly from GPU when batch size is large enough.

DL Training Integration

Image Classification

var dataset = DL.dataset.imageFolder("datasets/animals", 224);
var split = DL.train.split(dataset, 0.8);

DL.device.set(DL.device.gpuIfAvailable());

var model = DL.train.simpleCNN(dataset.numClasses(), 224);
var trainer = DL.train.adam(model, DL.loss.crossEntropy());

trainer.fit(
    DL.train.dataLoader(split.train(), 32, true), 20,
    DL.train.logger(),
    DL.train.validation(DL.train.dataLoader(split.val(), 32)));

GPU vs CPU Benchmark

var trainer = DL.train.adam(model, DL.loss.crossEntropy());

DL.device.set(DL.device.cpu());
long t0 = System.currentTimeMillis();
trainer.fit(loader, 5);
long cpuMs = System.currentTimeMillis() - t0;

DL.device.set(DL.device.gpuIfAvailable());
t0 = System.currentTimeMillis();
trainer.fit(loader, 5);
long gpuMs = System.currentTimeMillis() - t0;

System.out.printf("CPU: %d ms, GPU: %d ms, speedup: %.2fx%n",
    cpuMs, gpuMs, (double) cpuMs / gpuMs);

Note: The following is an internal low-level API, intended only for custom operator development or deep performance tuning. For daily training and inference, use the DL.nn.* / DL.device.* facade — no direct calls needed.

Internal GPU API (Advanced)

import com.yishape.lab.gpu.YishapeGpu;

// Matrix multiply
double[][] C = YishapeGpu.matMul(A, B);

// Element-wise
double[] sum = YishapeGpu.add(a, b);
double[] prod = YishapeGpu.mul(a, b);

// Activations
double[] r = YishapeGpu.relu(x);
double[] g = YishapeGpu.gelu(x);

// Normalization
double[] out = YishapeGpu.layerNorm(x, gamma, beta, outer, dim);

// Buffers
GpuBuffer buf = YishapeGpu.alloc(1024);
buf.upload(data);
double[] result = buf.download();
buf.free();

JVM Tuning Parameters

Property Description Example
yishape.gpu false to disable GPU -Dyishape.gpu=false
yishape.gpu.gemm.minFlops Minimum FLOPs threshold for GEMM -Dyishape.gpu.gemm.minFlops=1e6

Multi-platform Support

Platform Backend GPU
Windows Vulkan / DX12 NVIDIA, AMD, Intel
Linux Vulkan NVIDIA, AMD (RADV), Intel (ANV)
macOS Metal Apple Silicon, AMD

Requirements

  • JDK 25+ (FFM API)
  • Latest GPU driver
  • Developers compiling native libraries from source need Rust toolchain (end users only need the JAR)

PyTorch Comparison

Feature PyTorch YiShape-DL
Detect GPU torch.cuda.is_available() DL.device.isGpuAvailable()
Device name torch.cuda.get_device_name(0) DL.device.current().name()
Model to GPU model.cuda() model.toDevice(DL.device.gpu())
Auto-select cuda if available else cpu DL.device.gpuIfAvailable()
Disable GPU CUDA_VISIBLE_DEVICES="" -Dyishape.gpu=false / GpuSwitch.disable()
Precision float32 default GPU float32, CPU double

Key difference: PyTorch GPU is CUDA-bound (NVIDIA only); YiShape covers multiple vendors via wgpu, and automatically falls back instead of throwing errors when GPU is unavailable.

FAQ

"No suitable GPU adapter found"

Cause: Outdated driver or no Vulkan/Metal support.
Resolution: Update drivers; the code automatically falls back to CPU — no catch needed.

Slight GPU vs CPU result differences

GPU uses float32, CPU defaults to double. Acceptable for deep learning training, consistent with PyTorch behavior.

OOM / Out of Memory

  1. Reduce batch size
  2. GpuSwitch.disable() for temporary CPU fallback
  3. Close other GPU-consuming programs

Multi-GPU

Current version uses the default high-performance adapter. Multi-GPU parallelism is planned.

Apple Silicon

Metal backend supports M-series chips; speedup depends on batch size and model scale.

API Cheat Sheet

// Detection
DL.device.isGpuAvailable()
DL.device.current()

// Device
DL.device.gpu()
DL.device.gpuIfAvailable()
DL.device.cpu()
DL.device.set(device)
DL.device.isGPU()
DL.device.isCPU()
DL.device.reset()

// Model
model.toDevice(DL.device.gpu())
model.toDevice(DL.device.cpu())

// Toggle
GpuSwitch.enable() / disable()

// Internal low-level API (custom operator development only)
// YishapeGpu.matMul(A, B) / YishapeGpu.relu(x) / ...

Related Documents