Review: The New NVIDIA Jetson Orin Nano

Last Update: Dec 22, 2024

I wrote a book! Check out A Quick Guide to Coding with AI.
Become a super programmer!
Learn how to use Generative AI coding tools as a force multiplier for your career.

Hello, friends! If you’re a reader of this blog you’ve probably heard about NVIDIA’s Jetson. It’s a great platform for prototyping apps and putting AI at the edge.

I got lucky and got my hands on the newest, very affordable Jetson, the Jetson Orin Nano.

Today, we’ll dive into everything from unboxing this little gem to testing its performance with AI models. Ready to explore what makes this device so cool? Let’s get started!

If you’d rather see a video version of this review here it is.

What’s Inside the Box?

First impressions matter, and the Orin Nano’s packaging is as clean and straightforward as you’d expect. Here’s what you get:

• A robust power supply with both European and American plugs.
• A quick start manual (kinda)
• And, of course, the star of the show: the Jetson Orin Nano.

You do need to purchase an SD card for it, aas it’s not shipped with one.

The device is sleek, lightweight, and full of potential. It includes HDMI, USB, Ethernet, GPIO, and video input ports. Whether you’re making an AI robot or building computer vision stuff (I know I will), this device has you covered.

The Specs: Why It’s Called a “Raspberry Pi on Steroids”

Let’s talk hardware. Look at what’s packed into this small device:

• CPU: A six-core ARM Cortex 64-bit processor.
• GPU: NVIDIA Ampere architecture, tuned for AI workloads.
• RAM: 8 GB of high-speed memory.
• Connectivity: Ethernet and wireless ready.
• Storage Options: NVMe support for all your AI model needs.

Think of it as a supercharged Raspberry Pi with a serious focus on AI.

Setting It Up: The Good, The Tricky, and The Rewarding

Getting the Orin Nano up and running wasn’t exactly plug-and-play, but it wasn’t rocket science either. Here’s how it went down:

1. Firmware Update: Before diving in, I updated the firmware to version 36.x. This step required temporarily installing Jetpack 5.1.3. Then, you update the firmware from there. (Pro tip: don’t skip this. You want the new firmware and Jetpack)
2. Jetpack 6.1 Installation: After the firmware update, I installed Jetpack 6.1, NVIDIA’s tailored OS for the Jetson lineup. It’s smooth, intuitive, and optimized for AI tasks.

If you’re purchasing this devcie, then burning SD cards and flashing firmware shouldn’t be a problem.

Performance Tests: How Does It Handle AI Models?

Here’s where the fun begins—I tested the Orin Nano with AI models using Ollama, a platform designed for running large language models (LLMs).

Check out Ollama here

Here’s what I discovered:

• 1 Billion Parameter Models: Smooth sailing! These models ran effortlessly, perfect for tasks like chatbots or real-time applications.
• 3 Billion Parameter Models: Still impressive, though with slightly longer loading times.
• 7 Billion Parameter Models: Houston, we have a problem. The Orin Nano hit its limits here, occasionally freezing or locking up.

While it’s not designed for massive models, this performance is impressive for a device in this price range.

You can view the performance tests I did in this video.

Pros and Cons: The Real Deal

What I Loved:

1. Compact and Lightweight: Perfect for edge AI projects where space is a premium.
2. Impressive Performance: At $250, you’re getting serious bang for your buck.
3. Quiet Operation: Even under load, it’s quiet—a big plus for home projects.
4. User-Friendly OS: Jetpack is intuitive and packed with features.

What Needs Improvement:

1. Limits on Larger Models: It’s not a deal-breaker, but don’t expect it to handle anything beyond 7 billion parameters comfortably.
2. Occasional Glitches: Some random lockups and an odd “system throttled due to overcurrent” error. I’m looking into it. Manageable, but worth noting.

Final Verdict: Is It Worth It?

Absolutely. The NVIDIA Jetson Orin Nano is a fantastic choice for developers exploring AI at the edge or prototyping innovative solutions.

It’s not built for heavy-duty production workloads, but for $250, it’s hard to beat the value and potential.

What’s Next?

I’ll explore the Orin Nano’s features more in upcoming projects, so stay tuned. If you’re interested in edge AI or want to try out large language models, this device is a perfect start.

Have questions or suggestions for tests? Just drop a comment or message me on social media. Let’s see what this small device can really do!

Happy experimenting! 🚀

Questions, comments? Want more details? Yell at me!!

Published: Dec 21, 2024 by Jeremy Morgan. Contact me before republishing this content.