Schedule a Meeting →  

    Home  ›  Blog  ›  145 FPS on NVIDIA Jetson  

   

How We Achieve 145 FPS on NVIDIA Jetson Orin

   

Published by AeroAI.Dynamics — Engineering Intelligence. For Industry.

   

When we tell manufacturers that our vision system runs at 145+ FPS with 6.8 ms latency, the first reaction is usually disbelief. "How is that possible on an edge device?" The answer isn't magic — it's engineering optimisation at every layer of the stack.

   

Here's the technical breakdown of how we achieve real-time performance on the NVIDIA Jetson Orin platform.

   

The Hardware Foundation: NVIDIA Jetson Orin

   

The Jetson Orin delivers 67 TOPS (Trillion Operations Per Second) of AI performance. But raw TOPS don't tell the whole story. What matters is how you use them.

   

Key hardware features we leverage:

   

     
• 2048 CUDA Cores — massive parallel processing for neural network inference
     
• 64 Tensor Cores — dedicated hardware for matrix operations (the backbone of AI)
     
• DLA (Deep Learning Accelerator) — secondary inference engine for background tasks
     
• Shared Memory Architecture — CPU and GPU share memory, eliminating data transfer bottlenecks
   

   

Layer 1: Model Architecture

   

We don't use off-the-shelf models. Our detection architecture is custom-designed for industrial inspection:

   

     
• Optimized backbone: We use a lightweight CNN backbone that maintains accuracy while reducing parameter count by 40% compared to standard ResNet-50
     
• Input resolution 640×640: Balanced for speed vs. detail — enough to catch sub-millimeter defects without overloading the GPU
     
• Single-shot detection: No region proposal network (RPN) — we detect objects in one pass, cutting latency in half
   

   

Layer 2: TensorRT Optimization

   

This is where the real speed comes from. NVIDIA TensorRT is an inference optimizer that takes a trained model and rebuilds it for maximum performance on Jetson hardware.

   

What TensorRT does to our model:

   

     
• Layer fusion: Combines multiple neural network layers into one operation (convolution + batch norm + activation → single kernel)
     
• Precision calibration: Converts from FP32 to INT8 precision — 4× smaller model, 2× faster inference, with negligible accuracy loss (<0.5%)
     
• Kernel auto-tuning: TensorRT tests hundreds of kernel implementations and picks the fastest one for our specific model architecture
     
• Dynamic tensor memory: Reuses memory buffers across layers, reducing allocation overhead to near zero
   

   

Result: our inference time dropped from 22 ms to 6.8 ms after TensorRT optimization — a 3.2× speedup.

   

Layer 3: Pipeline Architecture

   

Speed isn't just about the neural network. We designed a streaming pipeline that overlaps image capture, preprocessing, inference, and output — so the GPU never waits.

   

     
• Frame 1: GPU runs inference
     
• Simultaneously: CPU captures Frame 2 from camera, preprocesses it (resize, normalize)
     
• Frame 2: GPU runs inference — no idle time
     
• Simultaneously: CPU sends results from Frame 1 to PLC via OPC UA
   

   

This pipelining effectively doubles throughput compared to sequential processing.

   

The Result: 145 FPS, 6.8 ms, 94.2% Accuracy

   

Every component of our stack — from the custom model architecture to TensorRT to the streaming pipeline — is designed to squeeze maximum performance from the Jetson Orin.

   

The numbers speak for themselves:

   

     
• 145+ FPS — fast enough for high-speed production lines
     
• 6.8 ms latency — real-time defect detection with instant PLC trigger
     
• 94.2% accuracy — measured on real factory data, not lab benchmarks
   

   

Want to see it in action on your production line?

    Schedule a Demo →