If you are searching "YOLOv8 vs TensorRT", the useful answer up front is that they are not competitors. YOLOv8 is a family of object-detection models you train. TensorRT is NVIDIA's inference runtime that executes a trained model fast on a specific GPU. The real question is not which one to pick; it is whether your trained model, exported and optimized through TensorRT, meets your latency, power and accuracy budget on the hardware you actually have.

This guide walks through the pipeline I shipped for railway defect detection on NVIDIA Jetson Orin, when TensorRT pays off, what INT8 quantization really costs, and the limits nobody puts on the box. Everything here comes from building the railway inspection system, not from benchmarks run on a datacenter GPU with a batch of 64.

What each piece actually is

YOLOv8 is the model family: convolutional detection networks with anchor-free heads, trained supervised on labeled images (faults, defects, anomalies). You get weights and an architecture. Out of the box on a desktop GPU it is fast; on edge hardware with a power envelope, "fast" is not automatically "inside budget".

TensorRT is a compiler and runtime. It takes an exported graph (commonly ONNX), fuses layers, selects kernels for the target GPU, and emits an engine: a serialized, hardware-specific plan that executes with minimal per-call overhead. Engines are the unit of deployment and the unit of lock-in.

ONNX is the interchange format between training and inference. You train in PyTorch, export to ONNX, and hand that to TensorRT. Keeping this boundary clean is what lets you retrain without redesigning the deployment.

The pipeline that shipped: training to sub-100ms on the track

The railway inspection system runs on a Jetson Orin mounted on inspection hardware, watching track footage for defects. The full chain:

  1. PyTorch training of the YOLOv8 detector on labeled track imagery.
  2. ONNX export of the trained graph for TensorRT ingestion.
  3. TensorRT engine build targeting the Orin's GPU, with layer fusion and kernel selection handled by the builder.
  4. INT8 quantization via quantization-aware training (QAT), so the model learns weights that survive 8-bit representation instead of being crushed after the fact.
  5. Hand-written CUDA kernels for the pre and post-processing stages where TensorRT cannot help you, because NMS and custom decode logic still need to be fast.
  6. GStreamer hardware decode to pull video frames in without burning CPU on software decoding.

Measured results on the deployed system: 98.7% accuracy, 4.2x faster than manual inspection, under 100ms per inference, and INT8 delivering roughly 2x the throughput of FP16. Those numbers are from a fixed railway-track context with controlled lighting and motion; treat them as a data point for that setting, not as a universal constant for your video.

Measuring correctly on device

Two measurement traps catch teams repeatedly. First, benchmarking on files instead of the live decode path: GStreamer hardware decode changes the frame timing and memory pools, so an engine that looks fast over pre-decoded frames can miss budget in situ. Second, averaging over best-case frames: defects are rare, so a loop over clean frames flatters latency and hides jitter. Profile the full loop (decode, pre-process, inference, post-process) on real footage, log the tail latencies, and validate accuracy and speed together. A faster engine that drops a defect class is not faster; it is worse.

When TensorRT pays off, and when it does not

TensorRT is worth the build pipeline when all of these hold:

  • The target GPU is fixed and known (a Jetson Orin on a train, not "whatever the cloud gives us").
  • You need sustained batch-1, real-time inference (continuous video, not overnight batch).
  • Latency or power is a hard constraint: sub-100ms budgets, fanless operation, battery or generator-limited deployments.
  • The model architecture is stable for weeks or months, so engine builds amortize.

It is the wrong choice when you retrain weekly (every retrain means re-export and re-validate), when you are CPU-only (TensorRT does not help; use ONNX Runtime or a quantized CPU path), or when you are still in exploration and do not yet know which model family you will ship. The build step also forces discipline: fixed input shapes, explicit precision decisions, and hardware-specific validation. If that discipline is absent, the engine becomes a liability rather than an asset.

FP16 vs INT8, and quantization-aware training

FP16 is the safe default: minimal accuracy loss, roughly half the memory bandwidth of FP32, and a straightforward engine build. INT8 roughly doubles throughput over FP16 on the Orin, at the cost of quantization error, which matters most on small objects, low-contrast textures and the long tail of rare defect classes.

Post-training quantization (PTQ) calibrates on sample data and rounds weights; it is fast to try and fine for tolerant workloads. QAT is the commitment: you simulate quantization during training so the network adapts its weights to 8-bit representation. QAT recovers most of the accuracy PTQ loses, and it is what made sub-100ms with 98.7% accuracy simultaneously possible in our case. The price is training infrastructure and curated calibration data that actually represents deployment conditions: the lighting, motion blur and camera angle you will see on the track, not the tidy images from a labeling tool.

The real enemies on track are not the model's FLOPs. They are motion blur from vibration and speed, and low-light or harsh-glare exposure. Histogram equalization on the input pipeline was the single most effective mitigation for low-light frames, restoring contrast the detector could lock onto. No amount of TensorRT tuning fixes a frame with no usable signal.

The decision checklist

Answer these honestly before committing to an engine pipeline:

  1. Is the target hardware fixed and NVIDIA? If no, stop; TensorRT does not apply.
  2. Is the latency budget under 100ms or power a hard cap? If no, ONNX Runtime on the same GPU is simpler and good enough for many workloads.
  3. Is the model stable, or is the team retraining weekly? If it churns, stay on ONNX Runtime/PyTorch until it settles.
  4. Do you have representative calibration and validation data from the deployment environment? If not, budget data collection first; INT8 without it is a coin flip.
  5. Is there a GStreamer or equivalent hardware decode path for your camera source? If decoding is the bottleneck, kernel-tuning the network yields nothing.

All five yes: TensorRT with QAT INT8. Some no: stay on ONNX Runtime or PyTorch, and revisit when the answers change.

Honest limitations of the TensorRT path

  • INT8 needs representative calibration data. Calibrating on desk imagery and deploying on railhead footage is how you ship a model that passes in the lab and misses in the field.
  • Engines are tied to one GPU architecture and TensorRT version. An Orin engine is not an A100 engine; a TensorRT upgrade can force rebuilds and revalidation. Plan for this in CI and in your release checklist.
  • Retraining means re-exporting. Every model refresh re-runs export, engine build, QAT validation and on-hardware profiling. If your refresh cycle is faster than your validation pipeline, you will ship unvalidated engines or stop refreshing.
  • The win is workload-specific. The roughly 2x INT8-over-FP16 throughput and under-100ms latency are facts about this system on this track context. Measure on your own footage before promising numbers to anyone.

What I would tell a team starting this tomorrow

Train a baseline in PyTorch, export to ONNX, and measure on the actual Orin before writing a single CUDA kernel. Most teams discover the bottleneck is decode or pre-processing, not the network. Add INT8-QAT only after FP16 is validated end to end, because debugging quantization and pipeline issues simultaneously is miserable. Invest in histogram equalization and motion-blur mitigation as input-stage infrastructure, and keep a small holdout of ugly real-world frames as the final gate. The engine is the easy part; the data is the product.

If you want an on-device vision system built, measured and handed over with its runbook, get in touch. I ship these systems fixed-scope, in your repo and your cloud.