Jump to content

The B580 does about 35T/s in Qwen 2 7B Q8, which is very competent.

 

LLM are a lot easier to accelerate. The real deal is pytorch. If Intel can figure out arc pytorch binaries, it'll leap ahead of AMD, and with NPUs inside processors for copilot, there might be something of value here.

 

But only if intel figure out good binaries for the ML frameworks.

Link to post
Share on other sites

50 minutes ago, BoomerDutch said:

I'm waiting patiently for their software to get baked enough before I'd jump in.

 

Their software stack seems to be easier to use than AMD's one, but not as good as CUDA still.

33 minutes ago, 05032-Mendicant-Bias said:

The real deal is pytorch.

https://github.com/intel/intel-extension-for-pytorch

 

I've used it for CPUs in production (AVX-512 + AMX) and it worked fine. I've seen reports of folks using it for inference with their Arc GPUs without much issues.

Major "issue" is that the performance is often subpar, like a B580 being slower than a 3060, but that's also a matter of how well priced the product is.

35 minutes ago, 05032-Mendicant-Bias said:

But only if intel figure out good binaries for the ML frameworks.

For LLMs in specific, they have integrations and pre-built binaries in place for most engines out there:

https://github.com/intel/ipex-llm

 

Getting it up and running with an XPU is way easier than ROCm.

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to post
Share on other sites

I like the talk about card form factors in the video, but I feel like rather than fanning out into multiple different configurations, it's be better to figure out what's the way to go and make all of the cards somewhat unified in this aspect. Not talking strictly about numbers of fans, but connector type and the gpu bracket being somewhat standardised across all models.

 

The fact that there are interesting models for every scenario doesn't mean you will be able to get this model in your region or be fast enough to grab the specific one that fits your SFF case. And then if it didn't become a standard, the next gen will be completely different.

 

Underdog GPU vendors should try to unify card dimensions and power delivery across all models to show that they commit to some standard - us, the SFF crowd will like that. There are people going specifically for those low profile dual slot quadro A2000 or RTX 4060 for such reasons because you're almost guaranteed to have the same form factor and you can remove the bracket.

 

If you want more interesting SFF cases like this to be made, removing the bracket and standardised mounting points and dimensions should be a thing:

 

 

 

Link to post
Share on other sites

4 hours ago, James said:

Sure, we all want an RTX 5090, but what Intel is cooking up with their ‘pro’ series of Arc graphics cards- the B50 and B60 - has FAR more potential to change the industry.

 

 

 

To think intel somehow is gonna beat amd.. wild.

Useful threads: PSU Tier List | Motherboard Tier List | Graphics Card Cooling Tier List ❤️

Baby: MPG X570 GAMING PLUS | AMD Ryzen 9 5900x /w PBO | Thermalright Peerless Assasin 120 SE ARGB | ASUS TUF Gaming GeForce RTX™ 4090 OC Edition 24GB | Corsair Vengeance RGB PRO 32GB DDR4 (4x8GB) 3600 MHz | Corsair RM1000xKingston FURY Renegade 2TB | WD_BLACK SN750 | Samsung EVO 850 | Kingston A400 |  PNY CS900 | Lian Li O11 Dynamic White | Display(s): Samsung Oddesy G7, ASUS TUF GAMING VG27AQZ 27" & MSI G274F,  MSI G274F 27"

Link to post
Share on other sites

11 hours ago, MultiGamerClub said:

To think intel somehow is gonna beat amd.. wild.

I really want to give AMD the a fair shake, but AMD still can't release good ROCm binaries, and that's after my friends tried and failed to make ROCm work, it's become a bit of a meme in my circle, me rebuilding the stack to get some new comfyui node to accelerate the latest new model. And it turns out I bought the best supported AMD card, the 7900XTX.

 

E.g. the official instructions for WSL involve me manually deleting a shared object file to get the acceleration running! I keep seeing people in the issues of the repo getting acceleration to run by moving shared object files in oblique sub directories.

 

AMD now is working on windows ROCm binaries and DirectML.

 

For reference, a few months ago DirectML was losing between 90 to 95% performance in diffusion on my system, and now AMD advertised a 3X upgrade, losing 50% to 75% performance in DirectML ONNX acceleration under windows to ROCm.

 

I'm really hopeful Intel will figure out the acceleration a lot faster, given how quickly the Arc drivers matured. I got a B570 for my nephew near MSRP, and he can play Minecraft RTX no problem. On just a second generation card, that's nuts. Intel is cooking!

Link to post
Share on other sites

14 hours ago, 05032-Mendicant-Bias said:

I really want to give AMD the a fair shake, but AMD still can't release good ROCm binaries, and that's after my friends tried and failed to make ROCm work, it's become a bit of a meme in my circle, me rebuilding the stack to get some new comfyui node to accelerate the latest new model. And it turns out I bought the best supported AMD card, the 7900XTX.

 

E.g. the official instructions for WSL involve me manually deleting a shared object file to get the acceleration running! I keep seeing people in the issues of the repo getting acceleration to run by moving shared object files in oblique sub directories.

 

AMD now is working on windows ROCm binaries and DirectML.

 

For reference, a few months ago DirectML was losing between 90 to 95% performance in diffusion on my system, and now AMD advertised a 3X upgrade, losing 50% to 75% performance in DirectML ONNX acceleration under windows to ROCm.

 

I'm really hopeful Intel will figure out the acceleration a lot faster, given how quickly the Arc drivers matured. I got a B570 for my nephew near MSRP, and he can play Minecraft RTX no problem. On just a second generation card, that's nuts. Intel is cooking!

ROC(KS)? NODES? BEST? 7900 XTX?

Dude i sold the 7900 XTX because of driver issues and swapped to a 4090

All tho i don't think my brother needed a 7900 XTX when he already bought the 6950XT like 2 months before but.. hey ho so it goes.

 

I do hope AMD gets their ass in gear because they've suffered enough, all tho i really hope AMD can fix their drivers..

 

So many after 20 years of driver issues we can finally say..

 

AMD knows drivers, that line is something i never think i will write seriously until they know how to.

 

Until then im sticking to the 4090 for a while.. Maybe a 8900 XTX or a 9900 XT doesnt sound so bad in 5-6 years.

If they got that 32GB Memory or 48GB Memory for less of what the 4090 costs.. I'll probably switch tbh.


The only real reason i upgraded to a 4090 was for VRChat to not suck absolutely shit in performance XD

(Which is still does.. Should've kept my 3080Ti and my 2300$ :))))

Useful threads: PSU Tier List | Motherboard Tier List | Graphics Card Cooling Tier List ❤️

Baby: MPG X570 GAMING PLUS | AMD Ryzen 9 5900x /w PBO | Thermalright Peerless Assasin 120 SE ARGB | ASUS TUF Gaming GeForce RTX™ 4090 OC Edition 24GB | Corsair Vengeance RGB PRO 32GB DDR4 (4x8GB) 3600 MHz | Corsair RM1000xKingston FURY Renegade 2TB | WD_BLACK SN750 | Samsung EVO 850 | Kingston A400 |  PNY CS900 | Lian Li O11 Dynamic White | Display(s): Samsung Oddesy G7, ASUS TUF GAMING VG27AQZ 27" & MSI G274F,  MSI G274F 27"

Link to post
Share on other sites

8 hours ago, MultiGamerClub said:

Dude i sold the 7900 XTX because of driver issues and swapped to a 4090

I get it. 

 

For me the drivers themselves are working fine. Gaming works without issue, but I'm not doing ray tracing. I can believe VR doesn't work as well.

 

Acceleration wise, LLM work fine on AMD, anything pytorch based is a nightmare.

 

The problem is that 7900XTX is 930 €, while the RTX 4090 is more like 2 500€  3 200 € (wow, it has gotten even more expensive), more expensive than the RTX 5090 32GB that is around 2 600 €. I guess because of teething issues with the RTX5090 and because the RTX 4090 is no longer in production? Anyway it's wildly above what I consider a fair price for those cards. between 1 500€ and 1 800€.

 

The Intel GPU is supposed to be around 500 $ and still give you 24 GB VRAM.

 

The market is hungry for cheaper, high VRAM option with good drivers. Someone trying to gain market share has the opportunity served to them on a silver platter to put a foot in the door.

Link to post
Share on other sites

On 5/21/2025 at 9:14 AM, 05032-Mendicant-Bias said:

I get it. 

 

For me the drivers themselves are working fine. Gaming works without issue, but I'm not doing ray tracing. I can believe VR doesn't work as well.

 

Acceleration wise, LLM work fine on AMD, anything pytorch based is a nightmare.

 

The problem is that 7900XTX is 930 €, while the RTX 4090 is more like 2 500€  3 200 € (wow, it has gotten even more expensive), more expensive than the RTX 5090 32GB that is around 2 600 €. I guess because of teething issues with the RTX5090 and because the RTX 4090 is no longer in production? Anyway it's wildly above what I consider a fair price for those cards. between 1 500€ and 1 800€.

 

The Intel GPU is supposed to be around 500 $ and still give you 24 GB VRAM.

 

The market is hungry for cheaper, high VRAM option with good drivers. Someone trying to gain market share has the opportunity served to them on a silver platter to put a foot in the door.

Drivers have come a long way i suppose, since i sold the 7900 xtx to my brother hes had no issues playing any game with it.

 

VR did run okay if it wasnt for the one single bug i found that amd didnt fix since the 7900 XTX came out.. Think it was a few months out by then.

 

I paid about 1175€ in todays conversion for it, it was the second cheapest 7900 XTX in norway.. Insane price.. Not as insane as the 4090 tho haha

 

Can't wait for the Intel B60 gpu's to come around eventually or something similar so friends can finally not look at prices and just "ewwwww, im gonna stick to my 360 until its disconnected"

 

I really do hope the market gets better.. Because the prices i see nvidia and amd realese we just slap 30 to 40% DOUBLE THE PRICE up in norway at times.. Its really really insane.

Useful threads: PSU Tier List | Motherboard Tier List | Graphics Card Cooling Tier List ❤️

Baby: MPG X570 GAMING PLUS | AMD Ryzen 9 5900x /w PBO | Thermalright Peerless Assasin 120 SE ARGB | ASUS TUF Gaming GeForce RTX™ 4090 OC Edition 24GB | Corsair Vengeance RGB PRO 32GB DDR4 (4x8GB) 3600 MHz | Corsair RM1000xKingston FURY Renegade 2TB | WD_BLACK SN750 | Samsung EVO 850 | Kingston A400 |  PNY CS900 | Lian Li O11 Dynamic White | Display(s): Samsung Oddesy G7, ASUS TUF GAMING VG27AQZ 27" & MSI G274F,  MSI G274F 27"

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×