25 GPUs for Local AI Ranked by Value, and a 2018 Server Card Takes Top Tier

A 96 GB RTX Pro 6000 now lists at $16,000 on Nvidia’s own store, which TweakTown reports is 87% above its $8,565 launch price. A 32 GB AMD Instinct MI50, a server card from 2018, sells for about $649 on eBay. In the snapshot behind this ranking, the cheapest Pro 6000 costs $17,000, or about $177 per gigabyte of memory, and the MI50 costs about $20. I want to walk through a grade for 25 cards, from S down to F, built on 2 numbers, and explain why some of the newest cards sit at the bottom of the list.

The prices and speeds come from a November 3, 2026 snapshot of retail and marketplace listings plus llama.cpp results that owners post from their own cards. I could not check every listing live again, and card prices have moved every week this year, so read each figure as a dated reading. The first 20 cards come from that snapshot. I added 5 more, marked with an asterisk in the first chart, using the freshest dated listings I could verify, which come from TechSpot’s price tracker for 4 of them and GPUPrix for the RTX 4060 Ti. Those 5 have no owner speed results, so I graded them on memory price, bandwidth and software. Specs, launch prices and software support dates I checked against vendor and press sources, and the links are below.

2 numbers decide every grade

The first number is dollars per gigabyte of memory. Memory decides what a card can run at all. Think of it as the size of a desk. The model has to fit on the desk with room left over for the conversation, which grows as you keep chatting, and a model that does not fit cannot run however fast the card is. My yardstick is the used RTX 3090, which works out to $72 per gigabyte in this snapshot. Anything well under that is cheap memory, and anything well over it is expensive.

The second number is dollars per token per second. A token is roughly 0.75 of a word, and tokens per second is how fast the AI types. This number says how fast a card feels for the money. The speeds come from public llama.cpp results, which let NVIDIA, AMD and Intel cards be compared on one scale, and I used owner reports on a Qwen model where they exist because that is closer to real use.

My reference model is a 27B parameter Qwen model compressed to 4 bits. The raw weights come to 13.5 GB by my arithmetic (27 billion parameters at half a byte each), and the file in the snapshot is about 18 GB once the compression format’s overhead is counted. As a cross check, the Q4_K_M file for Qwen3.5 27B on Hugging Face is listed at 17.98 GB, and the model supports a 262,144 token context window according to llmrun. A 24 GB card runs it with room to spare. A 16 GB card needs heavier compression, which costs some quality.

The grades work like this. S gives the most model for the least money with no real catch. A is excellent with 1 catch. B is a good buy for the right person. C is fine, but something else does the same job for less. D is a good card at a bad price, and F is a trap, a card that looks like value until you try to load a model on it.

Dollars per gigabyte of memory for 25 cards, colored by tier. The dashed line marks the used RTX 3090 yardstick at $72. Starred cards were added from Jul to Oct 2026 listings. Source: Nov 3 2026 listing snapshot, vendor specs, TechSpot, GPUPrix

Memory sets the ceiling and bandwidth sets the speed

Generating text for 1 user works by reading every weight in the model out of memory to produce each new token. That makes single user generation limited by memory bandwidth, and Inventive HQ’s rule of thumb puts the ceiling at bandwidth divided by the bytes read per token. I ran that on 3 cards using the 18 GB file. The RTX 3090 moves 936 GB/s, so its ceiling is about 52 tokens per second, and owners report 40, which is 77% of it. The Arc Pro B70 moves 608 GB/s for a ceiling near 34, and owners report 26, also 77%. The RTX 5090 moves 1,792 GB/s for a ceiling near 100 against 75 reported, or 75%. Real speeds landing in that band is what the bandwidth math predicts.

The same math explains the Tesla P40. NVIDIA’s product brief lists 24 GB of GDDR5 with up to 347 GB/s, so the ceiling on the 18 GB file is about 19 tokens per second before any overhead. The snapshot has no owner number for it, and I would expect something under that ceiling.

Context length matters too. The KV cache holds the conversation so far, and every new token reads it again. Inventive HQ’s speed calculator estimates an RTX 4090 running an 8B model at about 116 tokens per second with 4,096 tokens of context and about 29 at 131,072. A card with a fast bus and a small memory pool suffers on both counts as chats grow.

Memory bandwidth against dollars per gigabyte for 7 cards. The MI50 combines near 1 TB/s with about $20 per gigabyte. Source: NVIDIA, AMD, Intel, TechSpot, TweakTown

What owners actually measure

Here are the owner reported speeds from the snapshot. Quantization and setups differ by card, so they are a guide and a lab benchmark would be tighter. A fast reader at 250 words a minute needs about 5 to 6 tokens per second, by my arithmetic at 0.75 of a word per token, so every card in the chart outruns reading speed. The RTX 5090 leads at 75. The used RTX 4090 reaches 46 and the used RTX 3090 reaches 40, which is about 15% more speed for twice the money. The Arc A770 trails at 9, which is readable and slow.

Owner reported tokens per second on a 27B Qwen model. Quantization differs by card, and the MI50 result uses 8 bit weights. Source: Owner posted llama.cpp results, Nov 2026 snapshot

F tier is where good money buys too little memory

The RTX 5060 with 8 GB costs about $460 new. An owner measured 97 tokens per second on the short llama.cpp benchmark, the cheapest speed in the whole set, and that number is the only good thing the card does here. The 27B Qwen model does not fit on it even with heavy compression. Smaller models run fine, so keep one if you own it and look elsewhere before buying for this job.

The RTX 5070 with 12 GB costs about $900. That is $75 per gigabyte, above the used 3090 yardstick, for half the memory. The RTX 5070 Ti with 16 GB costs about $1,350 at $84 per gigabyte.

The RTX 5080 annoys me most. The cheapest 16 GB listing in the snapshot was a marketplace posting at $2,300, or $144 per gigabyte, while the RTX 5060 Ti with the same 16 GB costs about $790. That is $1,510 extra for a card about twice as fast that loads exactly the same models. Model size sets what you can run at all, and the 5080 earns its F on that basis.

Two more cards joined the F pile. The RTX 4070 Ti Super with 16 GB costs about $1,299 on TechSpot’s tracker as of August 9, 2026, which is $81 per gigabyte for the same 16 GB the $790 5060 Ti offers. Its 256 bit bus with 21 Gbps memory works out to 672 GB/s by my arithmetic. GPUPrix showed a lower $880 listing, which would be $55 per gigabyte and move the card up, so check the live price. The RTX 4070 Ti with 12 GB costs $1,222 on TechSpot’s tracker as of July 2, 2026. That is $102 per gigabyte for 12 GB at 504 GB/s, the highest memory price of any 12 GB card here.

D tier holds good cards at prices that undo them

The used RTX 4090 with 24 GB costs about $3,500. In the review used for the snapshot it ran Qwen at 46 tokens per second, and a used 3090 with the same 24 GB did 40 for about half the money. The same models fit on both.

The RTX Pro 6000 is the strangest grade on the list. It is the only card here with 96 GB, and StorageReview lists it at 1.79 TB/s of bandwidth, the same as the RTX 5090. In the review behind the snapshot, the full power version runs a 70B class Llama model on a single card at 32 tokens per second, which nothing else here can do. At $177 per gigabyte it costs more than double the yardstick. If you need a model that size on 1 card for work, it is your only answer here, and for everyone else it is a very nice D.

The Intel Arc Pro B70 with 32 GB is a D because of this week’s price. PC Gamer reported its launch at $949 in March 2026, the snapshot saw it at $1,300 at Newegg earlier, and the cheapest listing now is $1,946 from a marketplace seller. At that price it costs $61 per gigabyte for 26 tokens per second on Qwen, which is more money for less speed than the Radeon AI Pro R9700 in B tier. StorageReview calls it promising and, at times, frustrating. At $1,300 I would move it up 2 tiers.

The RTX 4060 Ti with 16 GB lands in D at about $843, the lowest new listing GPUPrix tracked on October 21, 2026. That is $53 per gigabyte and 69% above its $499 launch price. The RTX 5060 Ti has the same 16 GB for about $53 less in my snapshot, with 448 GB/s against this card’s 288 GB/s.

C tier splits into fast but pricey and cheap but slow

The RTX 5090 with 32 GB costs about $4,921, against a $1,999 launch price. It is the fastest card on the list at 75 tokens per second on Qwen, far faster than anyone reads, and its 32 GB fits models no 24 GB card can hold. The price works out to $154 per gigabyte, more than double the yardstick. Per token per second it costs about $66 against about $43 for a used 3090, roughly 50% more. A buyer who wants only speed will be happy with it, and the value grade is a C.

The rest of C has cheap memory and slow cards. The used Intel Arc A770 with 16 GB costs about $325, close to $20 per gigabyte, and one owner gets about 9 tokens per second on Qwen. The used Tesla T4 with 16 GB costs about $430, or $27 per gigabyte, and sips power at 70 W, but it is the slowest card in the benchmarks here. It is also sold as a passively cooled card, which means a server pushing air through it, and a desk at home is a poor substitute.

The AMD RX 9060 XT with 16 GB costs about $650. One owner gets about 21 tokens per second on a heavier compression of Qwen, which is usable. For $140 more, the RTX 5060 Ti in B tier does the same job at 29.

The AMD RX 7600 XT with 16 GB costs about $595 on TechSpot’s tracker as of August 12, 2026, which is $37 per gigabyte and $55 below the RX 9060 XT’s snapshot price. It launched at $329 on a 128 bit bus, and its 288 GB/s is 64% of the 5060 Ti’s 448 GB/s. A 16 GB card needs the Q3_K_M version of the 27B model, which InsiderLLM lists at about 14 GB, so the bandwidth rule of thumb puts its ceiling near 20 tokens per second. That is usable, and the 5060 Ti has a ceiling more than 50% higher for $195 more.

B tier has 6 buys for the right person

The Tesla P40 with 24 GB costs about $400 used, and at $16.67 per gigabyte it has the cheapest memory of all 25 cards. Qwen fits with room for a long conversation. It misses S tier because software is moving on without it. NVIDIA’s CUDA toolkit notes say offline compilation and library support for Maxwell, Pascal and Volta were removed in CUDA 13, so you stay on CUDA 12. vLLM’s installation guide lists compute capability 7.0 or higher as its GPU requirement, and the P40 is 6.1, so that serving tool is off the table while llama.cpp still runs it. The P40 is also a 250 W board that ships passively cooled, so plan for a fan. I would buy one as a cheap second card.

The Intel Arc Pro B60 with 24 GB costs about $1,000, roughly $42 per gigabyte, and Qwen fits with room to talk. Typical owner speeds sit around 22 tokens per second, comfortably faster than reading. The catch is Intel’s software. Intel archived its Extension for PyTorch and set end of life for March 2026, and it recommends using PyTorch directly because Intel GPU support has been upstreamed into it. Builds of llama.cpp fill the gap for local chat.

The RTX 5060 Ti with 16 GB costs about $790 and runs Qwen at about 29 tokens per second with heavier compression. Nobody gets excited about it. It is new, it is NVIDIA, and everything works on day one, which saves weekends of troubleshooting.

The AMD RX 9070 XT with 16 GB costs about $920. One owner gets 36.5 tokens per second on Qwen with a custom setup. AMD’s ROCm support table lists it as supported, and the plain RX 9070 at about $700 has the same 16 GB, which makes it a fair buy for anyone dodging the NVIDIA premium.

The Radeon AI Pro R9700 with 32 GB costs about $1,800 in the snapshot, the cheapest new 32 GB card I found. AMD launched it at $1,299 on October 27, 2025, and Qwen runs at 29.8 tokens per second. It stays in B because a used card in A tier gets you most of that speed for less.

The Intel Arc B580 with 12 GB costs about $369 on TechSpot’s price tracker as of August 26, 2026, which is $31 per gigabyte and almost the same as the used RTX 3060 at $370. Its bus delivers 456 GB/s against 360 GB/s on the 3060, a 27% edge, and it launched at $249. It sits in B and the 3060 sits in S because of software, since Intel archived its PyTorch extension while the 3060 runs everything on day one. Its 12 GB cannot hold the 18 GB Qwen file, which makes it a card for smaller models.

A tier is 2 used cards with 1 catch each

The used RTX 3090 with 24 GB costs about $1,727. Every guide recommends it, and it earns that. You get 24 GB, CUDA and about 40 tokens per second on Qwen, several times reading speed, and every tool runs on it. It launched at $1,499 in September 2020, so a used card now sells above its original price. The $72 per gigabyte yardstick keeps it at A. Prices vary by tracker, and one price tracker showed renewed 3090s from $1,830 on October 22.

The used RX 7900 XTX with 24 GB costs about $1,259. It is the one I would look at hardest. It has the same 24 GB for $468 less than the 3090, owners report about the same speed on Qwen, and on the short llama.cpp benchmark it is faster. The catch is software, because image tools and some training tools still lean on NVIDIA. For chatting and coding with a local model, it is the better buy.

S tier is a 2018 server card and a used RTX 3060

Neither S card is new. The AMD Instinct MI50 with 32 GB costs about $649 on eBay. AMD’s own page lists HBM2 with 1 TB/s of peak bandwidth, and the price works out to about $20 per gigabyte, less than a third of the yardstick. One owner runs Qwen at an 8 bit setting at 22.8 tokens per second, a quality level no 16 GB card can hold at all. The ceiling for a roughly 28 GB file at 1 TB/s is near 37, so that result sits at about 62% of it.

The listing leaves out the software story. AMD’s ROCm 6.0 release notes put the MI50 in maintenance mode, say 5.7 was the last release with full support, and set bug fixes to end in Q2 2024. Current ROCm tables list the MI50 as unsupported. It still runs through llama.cpp, and owners maintain their own builds. It is a passively cooled 300 W server card, most snapshot listings ship from China, and the 1 US listing was $750. If you can live with that, nothing here gives you more model for the money.

The used RTX 3060 with 12 GB costs about $370. In the hardware review used for the snapshot, it runs a 14B Qwen model at 23 to 31 tokens per second depending on how long the chat gets, which is a useful model at a speed you would be happy with. Techgage’s launch table lists 12 GB at 360 GB/s and a $329 price. Per token per second on the short test, it is the cheapest speed of any card here with more than 8 GB. For adding memory cheaply next to a bigger card, I think it makes a sensible second card, since it is the lowest price you will pay for 12 more gigabytes on a card that runs everything.

What the spec sheets say about the cards I keep returning to

Prices shift weekly, so I checked the fixed numbers against manufacturer and review sources. The RX 7900 XTX has 24 GB of GDDR6 on a 384 bit bus, 960 GB/s of bandwidth, a 355 W rating and a $999 launch price. The RTX 3090 offers 936 GB/s at 350 W and launched at $1,499, according to Tom’s Hardware’s comparison table. That puts the two within 3% of each other on the number that sets generation speed, which fits what owners report.

The Arc Pro B60 carries 24 GB of GDDR6 at 456 GB/s, and Club386 reports a configurable board power of 120 W to 200 W. That is about 49% of the 3090’s bandwidth at a much lower power draw, so I treat it as a quiet, cool 24 GB card with half the 3090’s ceiling.

The RTX 5060 Ti 16 GB launched at $429 with 448 GB/s and a 180 W rating. The $790 in my snapshot is 84% above that launch price, and TechSpot’s price tracker shows $664 in August 2026 after a $429 low in November 2025.

The RTX 5090 has 32 GB at 1,792 GB/s with a 575 W rating and a $1,999 launch price. Its bandwidth is 91% above the 3090’s, and Windows Central suggested at least a 1,000 W power supply for it at announcement. At $4,921 in the snapshot, the card sits 146% above its launch price. Power draw belongs in the budget for any card on this list, because a 575 W card can need a new power supply on top of the card itself.

Software support decides how long a cheap card lasts

A card keeps working after support ends. NVIDIA’s developer blog says CUDA 13 means you cannot build new applications for Maxwell, Pascal and Volta with its compiler, while applications built for those cards keep full support through the life of the 580 driver, sometime in 2028. PyTorch keeps a CUDA 12.6 legacy build for those architectures. The P40 sits in that window, and so does anything with a similar age on the NVIDIA side.

The AMD and Intel stories run on a similar clock. AMD announced the MI50’s maintenance mode in ROCm 5.6, and Intel’s Extension for PyTorch ended in March 2026. I drop a card 1 tier when its software is leaving, since a cheap card with a short remaining life costs more than its sticker.

5 dates that set how long older cards stay useful. Orange dots mark the AMD and Intel steps. Source: AMD ROCm notes, Tom’s Hardware, Intel, NVIDIA

Why old cards beat new ones right now

Memory has become expensive, and older server cards were built with a lot of it. In this snapshot the cheapest gigabyte was $16.67 on the P40 and the most expensive was $177 on the RTX Pro 6000, which is more than 10 times as much. One tracker, Capital and Compute, citing TrendForce, puts GDDR6 spot prices at about $2.50 per gigabyte in September 2025 and $12.45 by October 5, 2026. 3DCenter’s German market data shows average card prices up 66% for NVIDIA, 46% for AMD and 38% for Intel between November 2025 and October 2026.

Cards that shipped before the shortage carry their memory at old prices, and used server cards sit furthest from the new price curve. Newer cards still win on speed and software support. For most people running a local model, speed past reading pace is a bonus, and memory is the wall.

Launch price against the November 2026 snapshot price for 5 cards. The 3090 is a used price. Source: TechSpot, Intel, AMD, TweakTown

How to grade any card yourself

Your card may not be on the list, and an old GTX 1080 is a good example. It still runs small models fine. You can grade any card in 3 steps. First, check that the model you want fits in the card’s memory with room to talk, because if it does not, the card is an F for that job however fast it is. A GTX 1080 has 8 GB, so the 27B model fails this step. Second, divide today’s price by gigabytes and compare it with the 3090’s $72 per gigabyte. Third, check that the speed is comfortably faster than you read, which is about 5 to 6 tokens per second.

A card that passes all 3 for less than the yardstick is an S or an A. Drop it 1 tier if its software is being left behind.

What I would buy

Under $400, I would buy a used RTX 3060 12 GB at around $370. For the biggest models at the lowest cost, and if you are comfortable with Linux, I would buy the MI50 at about $649. For 24 GB that works on a budget, the used RX 7900 XTX at about $1,259 is my pick. For 24 GB with the best compatibility, including image tools, it is the used RTX 3090 at about $1,727. If speed is all that matters, the RTX 5090 is the answer, with your eyes open about the price.

If I were starting from nothing this week, I would buy the 3060 first and save for a 24 GB card as my second one.

2 things from this generalize

First, price the model before the card. The 27B Qwen file is about 18 GB at 4 bits, which is about 33% above the raw 13.5 GB of weights, and that gap is the room the format and the conversation need. Any card quote that skips the file size answers a different question.

Second, redo the dollars per gigabyte sum on the day you buy and check the support date for the card’s software. Prices moved weekly all year, and a $949 card was listing at $1,946 within months.

This week, write down the 1 model you want to run, add a few gigabytes to its file size for the conversation, and divide each candidate card’s current listing price by its memory. Anything above $72 per gigabyte needs a reason.