Popular repositories Loading

1. Neural-network Neural-network Public

2. trt-resnet50-benchmark trt-resnet50-benchmark Public

   ResNet-50 inference with TensorRT 11: FP32 vs FP16 vs INT8 (ModelOpt) on a GTX 1650 Ti, with profiling and a root-caused INT8 NaN issue

   Python

3. llm-decode-cuda llm-decode-cuda Public

   From-scratch C++/CUDA LLM inference engine (SmolLM2-360M) on a GTX 1650 Ti: custom GEMV at 97% of bandwidth, split-KV attention, INT8 weights, cuBLAS autotuning, verified against HuggingFace

   Cuda