I have been all-in on mini PCs now for the past couple of years and have since gotten rid of my enterprise 2U servers and that create a lot of heat and noise as well as consume tons of power. One of the draws of mini PCs is the fact they are extremely powerful for their size and very power efficient when compared to an enterprise server. Most newer mini PCs today come with NVMe storage and we all know that NVMe storage is blazing fast when compared to other types of storage. But there is a hidden truth that many buyers discover too late. Note all NVMe slots in mini PCs are created equal.
So what exactly are we talking about?
Many may not realize that when they buy popular mini PCs on the market, many of these are throttling NVMe performance as they wire the slots in ways that may not give you the full potential of the NVMe drive itself. Also, as you would expect, this information is not exactly something the manufacturer emphasizes in their product pages.
However, as you can imagine, this can make a huge difference between an NVMe drive running at full speed and one that performs just a little bit better than a SATA SSD drive. So you have the possibility of losing massive amounts of performance if the M.2 slot is not wired to a full speed bus.
The basics of NVMe and PCIe
To help us understand why this is a big deal, you need to understand a bit about NVMe and its general performance and how it achieves that performance. NVMe is not a type of physical connector. Instead it is a protocol that uses PCI Express lanes for storage access. Each of these PCIe lanes provides the bandwidth needed for storage. The more lanes you have and the newer the PCI generation, the faster your NVMe drive can go.
As a general comparison of the different drive generations and lanes, below is a look at the relative speeds they can achieve:
- PCIe Gen 3 x4 can deliver around 3500 MB/s
- PCIe Gen 4 x4 doubles that to over 7000 MB/s
- PCIe Gen 5 x4, which is becoming more mainstream and can theoretically deliver speeds of 14,000 MB/s+
Keep in mind the above is the maximum performance of the drive itself. However, for the drive to be able to achieve that it needs the 4 lanes and the generation of PCIe lane that is given.
If your NVMe drive is plugged into a slot that only supports PCIe Gen 3 x1, the theoretical maximum is just under 1000 MB/s. PCIe Gen 2 is even slower. All of this means that even though the drive itself is capable of those incredible speeds
What happens in mini PCs when it comes to NVMe
Mini PCs are one of the best developments for home lab builders. They are able to cram a lot of CPU memory, networking, and storage performance into a small form factor. However, the tradeoff with these devices is they have to balance all those devices with the available PCIe lanes manufacturers have to work with.
Manufacturers of these mini PCs will often advertise that these PCs have multiple NVMe slots. This is true. However, the part that gets left off the marketing is that only one of the ports runs at full PCIe Gen 3 or 4 speed due to only one of the slots having 4 lanes. The other M.2 NVMe slots may only have 1 or 2 lanes.
In some mini PCs and designs, the slots are all capped so that none of them have full access to the lanes they typically need for full performance. This is due to the fact that the chipset of the motherboard only offers a limited number of lanes to use.
A great example of a mini PC that I think is great, but that is limited in the number of lanes provided to the NVMe storage it can house is the Beelink ME Mini. It is I think a great mini PC as a low-power Proxmox node.
You can check out my review of the little mini PC here: Beelink ME Mini NAS Review: Ultra-Compact Storage with 24TB Potential.
But consider this. It can house 6 NVMe drives. But, the kicker with this is that all of the NVMe drives run at PCIe Gen 3 x1 speeds with only 1 out of the 6 slots being an x2 slot. So, again, only 1 of the NVMe drives gets 2 lanes of bandwidth while the others get 1 lane.
Be aware of configuration settings
You also need to pay close attention to the configuration details out of the box. Some settings may not be configured optimally for NVMe speed. One example of this is the Minisforum MS-A2 that by default has (2) of the NVMe M.2 slots set to PCIe Gen 3 x 4 but can be set to PCIe Gen 4 x4. So unless you change this setting you will be losing tons of performance on these slots if you have a Gen 4 drive.
Why do mini PC manufacturers do this?
Actually there are a few reasons why these types of mini PCs arrive with these types of limitations with NVMe. Many of these limitations can be explained by one or more of the following things.
1. Chipset limitations
Many of the mobile processors that are used inside of mini PCs have fewer PCIe lanes that desktop motherboards. If the CPU only has 12 to 16 lanes of bandwidth to work with and you have multiple devices that will be sharing these lanes, including graphics cards, network adapters, USB controllers, and storage, then when you do the math, often NVMe lanes need to be limited. This is especially true if a mini PC has multiple NVMe slots available or advertised as such.
Storage often gets the short end of this stick since manufacturers know, the every day person is probably not going to be able to tell the difference between a PCIe Gen 3 NVMe drive with 2 lanes of bandwidth instead of the 4 they should have.
2. Cost control on mini PCs
Manufacturers also have to look at how to control costs. By limiting the number of NVMe lanes or using older PCIe standards, they can shave off costs of manufacturing and still be able to advertise it has NVMe support.
3. Thermal and power design
The faster an NVMe drive is, the more heat it will produce. Mini PCs pack a lot of hardware inside a very tight space and have limited cooling capacity available. Vendors sometimes restrict the bandwidth to throttle performance so that thermals are kept under control.
So how does this affect real world performance?
When looking at things on paper, you might think at first it doesn’t matter. After all, if you have an NVMe drive that is limited to 1 lane such as a PCIe Gen 3 drive, you are still getting 1000 MB/s and that is still faster than most SATA SSDs. But the real difference comes down to certain scenarios, and these are especially relevant for home labs. However IOPs is king in a home lab or other storage intensive workflows.
Think about the following use cases where you need all the IOPs you can get:
- Virtual machines: If you run Proxmox or VMware ESXi on a mini PC, your storage performance has a huge impact on how responsive your home lab VMs will feel. Limited NVMe lanes will bottleneck disk I/O.
- Media editing: Tasks like video editing definitely benefit from high sequential read and write speeds. A Gen 4 NVMe on a desktop can fly through 4K footage. The same drive in a bandwidth, lane-restricted slot might crawl.
- File servers: If you are using a mini PC as a NAS or home lab storage node, storage throughput determines how quickly clients can transfer large files.
- Future proofing: You may not need full Gen 4 performance today, but as drives become cheaper and faster, having slot limitations will mean your mini PC will not age well when it comes to storage performance.
The bottom line is that if you go for a full speed drive of the PCIe generation that is supported and the lanes are limited, you will definitely be paying extra for an NVMe drive whose potential is never realized.
Pay attention to the details before you buy a mini PC or NAS
The trick is knowing that you need to look at these specs before you buy. Here are a few things that I personally consider before I pull the trigger on a new mini PC:
- Check the official specs carefully: Don’t just look at sales ads or other information that is just trying to see it. Look at the official “whitepaper” specs. Look not just for NVMe support, but for the PCIe version and lanes. These are normally notated in the specs as the number of lanes in the format “x1 or x2 or x4”. In my opinion, these days, it is not a good sign if the vendor does not tell you exactly what specs are involved with the NVMe drives.
- Look for independent reviews: Look for reviews of the mini PCs. Most tech reviewers are familiar with these limitations and will call these out if the NVMe slots are capped at lower speeds.
- Read forums: Communities of enthusiasts are some of teh first to share their real world testing results that help to shed light on these limitations.
- Understand the CPU and chipset: As we mentioned earlier, mobile processors that are used in many of the mini PCs may only support a fraction of the PCIe bandwidth you would get in a desktop variant.
- Ask before buying: If the vendor has an official forum or support line/email address, ask directly what PCIe versions and lanes are supported by the NVMe slots.
Below is a “specs” page for a mini PC that shows you need to look at the details before purchasing to fully understand the storage speeds.
Is NVMe worthless in mini PCs?
No, not at all. The point of what we are saying is that you need to go into purchasing your next mini PC with eyes wide open. Be sure to get what you pay for and not assume that you can go out and also buy the latest and greatest NVMe drive and it perform up to its spec. In fact, having all the information up front will save you a lot of money on the NVMe drive. If your mini PC only supports PCIe Gen 3, why buy a brand new Gen 4 drive? If it only supports PCIe Gen 3 x1, why buy the most expensive Gen 3 NVMe drive you can buy?
However, I love NVMe in the home lab for mini PCs, especially in a full bandwidth slot. It definitely helps to run more VMs, containers, and other resources with tons of performance to spare. Also, I have found you can get away with running less memory and having overall memory pressure when you have NVMe and you still “feel” like the box is performant as it helps to offset the performance hit of memory paging.
Check out my post on how to add an NVMe storage drive to Proxmox: Proxmox add disk storage space – NVMe drive.
Optimize for the limitations of your mini PC
There are also some things you can do to make the performance better, even for mini PCs that have limited NVMe drive bandwidth or generations. Think about doing the following:
- Use the fastest slot for your OS and applications that need the best performance
- If secondary slots are slower, use these for workloads that may be ok with slower storage, like for bulk storage
- If you have a restricted NVMe slot, you might consider using external drives through USB 3.2 Gen 2 or Thunderbolt storage. These connections on a mini PC can sometimes outperform restricted internal slots
- For virtualized workloads, avoid overcommitting too many VMs on NVMe storage drives that are limited as this will definitely make any performance limitations much more evident.
PCIe slot devices for full-bandwidth NVMe M.2 slots?
Also, something to mention, there are also cards you can buy that will turn a PCIe x16 slot into additional M.2 slots that are full speed. Below is an example of one of those add-in cards. The RITOP card below provides (2) NVMe slots, one on each side, and then also keeps the PCIe slot so you can plug in cards to the slot. However, you need bifurcation support for this to work.
What should you look for if you need to avoid NVMe limitations?
If you need top tier NVMe performance and limitations are not acceptable for your tasks or home lab workloads, look for these types of machines instead:
- Mini workstations like Intel NUC Pro models or Asus NUC 14 Pro often have at least one full speed PCIe Gen 4 slot
- Small form factor desktops give you more PCIe lanes and usually have more expansion options even if these are compact
- Dedicated NAS devices can have very high sequential performance with multiple drives in RAID configurations which may be an alternative to having NVMe drives that are limited
Wrapping up
NVMe drives in mini PCs are a great development that gives us access to blazing fast storage performance. However, do understand and keep in mind that you need to pay attention to the storage configuration details of mini PCs and NAS devices on the market. Since many of these use the mobile architectures and try to fit many different types of hardware in the same platform, there is only a finite amount of PCIe bandwidth available. Many manufacturers choose to lower the lanes available for the NVMe slot or slots so they can add other devices.
The key takeaway is understand these limitations, look at the specs as the “devil is in the details” and make sure you know your storage capabilities before you buy the latest and fastest NVMe drive. How about you? Have you been bitten by this before? What do you think about the limitations on NVMe performance from mini PC manufacturers?
About The Author
