Jump to content

Tinycorp has a shot to clean Nvidia's clock

Hey, just felt like giving a little heads up. Tinycorp, seems to be getting the hard stuff right when it comes to building out AI tech stacks (software mostly), and this has been the great bulk of what kills most hardware companies in terms of those that require chip tape outs. Keep an eye out on them. They're committing to a "no IP protection" building out of the tech stack from taped out chips, the computers they go in, all the way to the libraries models are built with. It'll be a while (likely, read years), but they have a real shot.


Obviously, if true, it would provide a solid foundation for anyone wanting to have more competition in the GPU marketplace and sidestep AMD's incompetence (Love ya team red, but...come on). Of course, plans change and particular details could radically swing the utility calculations of building off of this AI focused bit of kit, but, at this early stage, it's the best Nostradamus moment you can hope for. Just a friendly FYI 馃槈

Link to comment
Share on other sites

Link to post
Share on other sites

鈥o how exactly is a software focused company that does AI going to compete with a hardware vendor that provides the necessary hardware for AI?

Even if they're looking to expand into making their own chips, a company founded a year ago has an extremely long way to go to even consider competing with a well established trillion dollar company. I'd wait a few more years to see if they manage to stick around and actually have some real accomplishments and not just claims before I'd start considering them viable competition.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, Eigenvektor said:

鈥o how exactly is a software focused company that does AI going to compete with a hardware vendor that provides the necessary hardware for AI?

Even if they're looking to expand into making their own chips, a company founded a year ago has an extremely long way to go to even consider competing with a well established trillion dollar company. I'd wait a few more years to see if they manage to stick around and actually have some real accomplishments and not just claims before I'd start considering them viable competition.

Glad to answer your question. 馃檪So, over the last 10ish years of going through various different hardware firms that have been taping stuff out, you'll find post mortem analyses occasionally done. Sometimes it's educated speculation. Other times, it's former employees spilling internal details. If you get really lucky, the company itself will write it's own cause of death analysis.

What you'll typically hear, most often, actually, is that, for every 10 people that will be willing to do advanced EE work in Verilog or VHDL, you'll find 1 person willing to program your chip. It's actually much harder to find engineers willing to program for your chip than engineers willing to design your chip. I'll spare you the complexities of the details, but think of it as the chicken and egg issue of growing a software product in a start-ups network effects.

To get around this, tinygrad has started with software and built out a library that competes, rather well actually, with pytorch. It's in pure Python, about 5000ish lines of code (1000 if you're talking about pure functionality with the rest being for performance optimization). Is already trading blows in terms of actual computational speed. Tinygrad also feels fairly close to using pytorch. It's not a drop in solution; It's not like you changing out your .py files to .mojo to switch languages, but it's fairly close.

It's probably the most fun library I've ever contributed too and, while not at v1 yet, is almost certainly gonna make it.

The thing is, proof of capability here is proof of ability over the hardest hurdle. Get past this hump, and it's very believable to think that you'll be able to do native support for any mainline library/framework for AI research and be able to do hardware validation on the chip level.

Further, it's not actually true that they are a software company. They've just shipped their first hardware product. Sure, it's basically a glorified pre-built, but you can add: logistics, hardware QA, Hardware Validation, etc. to the list of abilities they've built and demonstrated native ability to handle. This, of course, will all be the same skills used to get your taped out chips into peoples hands when you get to that level and is no small feat.

The founder himself also is doing this as a retread for his other hardware company, Comma.ai. I'll spare you the details about it here (too long a post without it) but the gist of what's useful is. 1)they've been around for quite a while with an impressive list of accomplishments to their name) 2) they sell their own custom hardware with all that entails 3) they're now using tinygrad's output in the input to their production. Needless to say, all excellent signs. Oh, and did I mention that both of these companies are profitable. A rarity for sure, but an excellent sign.

So, ya, you'll have to wait a few years before you'll see many more people making this kind of prediction, but as my original post said, "at this early stage, it's the best Nostradamus prediction you can hope for". In my professional analysis, it's a sure enough shot (not a sure shot, to make the difference explicit) that I'm willing to speculate openly.

Link to comment
Share on other sites

Link to post
Share on other sites

4 hours ago, Eigenvektor said:

鈥o how exactly is a software focused company that does AI going to compete with a hardware vendor that provides the necessary hardware for AI?

Even if they're looking to expand into making their own chips, a company founded a year ago has an extremely long way to go to even consider competing with a well established trillion dollar company. I'd wait a few more years to see if they manage to stick around and actually have some real accomplishments and not just claims before I'd start considering them viable competition.

Because just because companies make hardware, doesn't mean their software (or firmware) is good.

Take commodity hardware and improve the soft/firmware.

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, ToboRobot said:

Because just because companies make hardware, doesn't mean their software (or firmware) is good.

Take commodity hardware and improve the soft/firmware.

so for Tiny corp to do well
Nvidia has to do well
Edit: Sidenote, Tinycorp doesn't sound like a company, its sounds like a company tycoon game on steam

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, ToboRobot said:

Because just because companies make hardware, doesn't mean their software (or firmware) is good.

Sure. But if your software runs on their hardware, you're not competing with them. You're increasing their sales. And unless your software negates any difference in terms of performance, Nvidia will remain the prime choice for AI hardware. (And by that I don't mean it runs equally bad on all of them)

3 hours ago, Sirgeorge said:

I'm not sure how any of this is proof of them having a credible shot at defeating Nvidia (at what exactly?). What makes them so special that they will be able to compete in a way that AMD or Intel, both decidedly not startups in that business, couldn't do better (and still fail)?

Your post sounds like someone artificially trying to pump up their stock more than anything else.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites

Link to post
Share on other sites

16 minutes ago, Eigenvektor said:

Sure. But if your software runs on their hardware, you're not competing with them. You're increasing their sales. And unless your software negates any difference in terms of performance, Nvidia will remain the prime choice for AI hardware. (And by that I don't mean it runs equally bad on all of them)

unless you take their hardware, fix the software, and resell it to their customers who see the value.

It's about delivering value to capture margin.

The price of compute is X.聽 Hardware and software are fighting for profit, but if you make hardware that is good, but your software isn't there is a problem.聽 nVidia is able to command higher prices because their hardware and software are so good.聽 AMD therefore isn't able to price their hardware as high.聽 TinyCorp therefore sees the potential to deliver good software for AMD hardware and to get paid for doing so.

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, ToboRobot said:

unless you take their hardware, fix the software, and resell it to their customers who see the value.

Sure. But it's not like AMD's hardware is on the same level as Nvidia's and only a software fix away from being on the same level. Unless you're saying it would make AMD's hardware a better deal in terms of price/performance even when you now have two companies wanting to profit from it.

鈥h and Tiny Corp launches Nvidia-powered AI computer because 'it just works'

Quote

After a weeks-long struggle to get its AMD RX 7900XTX-based TinyBox working on open source firmware, Tiny Corp says it will be launching an Nvidia RTX 4090 version for users who want something that "just works."

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites

Link to post
Share on other sites

8 hours ago, Sirgeorge said:

AI tech stacks (software mostly)

8 hours ago, Sirgeorge said:

competition in the GPU marketplace and sidestep AMD's

You are aware AI accelerators and AI software doesn't actually have anything to do with the GPU market right? There is a very big difference between GPUs that have AI acceleration capabilities and AI accelerators as well as the software frameworks for AI.

If you want competition in the GPU market then you want to be looking at a company developing GPUs, like Intel for example.

Link to comment
Share on other sites

Link to post
Share on other sites

All I can say is if they ever become a "Threat" to Nvidia, they'll either be snapped up and brought under Nvidia's umbrella or crushed like a bug.

Nvidia is known for doing such things in the past and I seriously doubt they'll abandon their ways of business, esp since they are now a company that's worth more than any other in the world.

No - I don't think they'll abandon what got them to that point so easily.

"If you ever need anything please don't hesitate to ask someone else first"..... Nirvana
"Whadda ya mean I ain't kind? Just not your kind"..... Megadeth
Speaking of things being "All Inclusive", Hell itself is too.

Link to comment
Share on other sites

Link to post
Share on other sites

6 hours ago, leadeater said:

If you want competition in the GPU market then you want to be looking at a company developing GPUs, like Intel for example.

oddly enough the only reason I picked up Intel's gpu is because the company聽 that made it went with something different
Like just the name SPARKLE was fascinating to me (even if I havent used that pc at all)

Link to comment
Share on other sites

Link to post
Share on other sites

10 hours ago, BrandonLatzig said:

so for Tiny corp to do well
Nvidia has to do well
Edit: Sidenote, Tinycorp doesn't sound like a company, its sounds like a company tycoon game on steam

Tinycorp does not do well on the back of Nvidia. Tinycorp does well on the back of execution. They managed to build out the tech stack from bare metal to working benchmarking on AMD GPUS, for example. It demonstrates that they can support any GPU, and given they are planning to tape out their own chips, Nvidia is a placeholder for the time being. All-in-all, it's a great placeholder and serves well for now. Also, yes the company does sound like a placeholder videogame name. This is a very amusing observation. Thank you for making it. 馃檪

10 hours ago, Eigenvektor said:

Sure. But if your software runs on their hardware, you're not competing with them. You're increasing their sales. And unless your software negates any difference in terms of performance, Nvidia will remain the prime choice for AI hardware. (And by that I don't mean it runs equally bad on all of them)

I'm not sure how any of this is proof of them having a credible shot at defeating Nvidia (at what exactly?). What makes them so special that they will be able to compete in a way that AMD or Intel, both decidedly not startups in that business, couldn't do better (and still fail)?

Your post sounds like someone artificially trying to pump up their stock more than anything else.

Yes, if they were a pure software company then it would follow that their success would be a strict subset of the success of Nvidia. This is fundamentally not the case though. They are planning on taping out their own chips. This is the crux of the point. The big reason other chip builders failed is because they put out a piece of hardware, perhaps even a good piece of hardware and it tanked because no one wanted to learn how to program for it. The solution, in the case of AI, is to build out a good software stack on hardware people currently use where acquiring new knowledge to program with that software (in this case a library) is trivial, and then swap out someone else's hardware, in this case Nvidia, for your own. Of course, you'll always be able to use tinygrad on any other hardware, but gradually building out your abilities software first and then selling hardware you taped out last is an excellent strategy compared to the alternative.

Also, I didn't say they would "defeat" Nvidia. I said they would "clean their clock". My apologize for the confusion. By that I meant that they would erode the big moat Nvidia had over compute by virtue of network effects from previous adoption of the Nvidia stack by making learning a new stack trivial on new hardware from a company charging a fraction with comparable performance. It would ultimately arbitrage away Nvidia's profit by virtue of increasing competition and decreasing the tolerance people have for higher prices.

What makes them special, is the ability to execute (their code is beautiful by-the-by) and they've even been able to reverse engineer AMD's tech stack down to bare metal showing that they can support the entire stack from ANY piece of hardware up to the level of just general model building using basic Python code using their library (or any other for that matter). If you can already do a better job then AMD in terms of supporting a GPU, imagine what you can do when you don't have to deal with the muck of reverse engineering because you're supporting your own GPU?

Finally, they aren't a publicly traded company and I would kindly ask you to judge the post based off of what it is rather than what it sounds like. Thank you. 馃檪

9 hours ago, ToboRobot said:

unless you take their hardware, fix the software, and resell it to their customers who see the value.

It's about delivering value to capture margin.

The price of compute is X.聽 Hardware and software are fighting for profit, but if you make hardware that is good, but your software isn't there is a problem.聽 nVidia is able to command higher prices because their hardware and software are so good.聽 AMD therefore isn't able to price their hardware as high.聽 TinyCorp therefore sees the potential to deliver good software for AMD hardware and to get paid for doing so.

I thank you for summing up a good chunk of what they do in far fewer words. Your brevity is beautiful. I would like to add that this is part of their strategy, but it seems almost incidental? Long term, they want to handle the whole stack from tape out to library because that way they don't have to fight a hardware manufacturer for basic insight into a chip design.

9 hours ago, Eigenvektor said:

Sure. But it's not like AMD's hardware is on the same level as Nvidia's and only a software fix away from being on the same level. Unless you're saying it would make AMD's hardware a better deal in terms of price/performance even when you now have two companies wanting to profit from it.

鈥h and Tiny Corp launches Nvidia-powered AI computer because 'it just works'

As stated above, anything related to AMD is incidental. In regards to the post on launching on Nvidia over Intel/AMD, they tried really hard to avoid it. Initially, they even rejected an Nvidia offer. After over a year of AMD saying "we'll see" (read NO) to requests to open up the firmware and allowing Tinycorp to just fix the bugs that are in it to make AMD hardware work because they were, on a hardware level, "comparable enough" to Nvidia chips in terms of power (not my opinion, I believe this was the CEOs statement at one point in time, don't quote me). They ultimately decided that paying the Nvidia tax and reverse engineering AMD were easier options. They offer both AMD and Nvidia options at this time (I believe they were planning all three at one point and AMD may be a temporary offering because the Nvidia systems were about 67% more expensive and people initially were expecting to pay AMD system prices).

All-in-all, it's a temporary arangement. They won't be using this hardware for long. But the point is that they'll be making an alternative pipeline with cheaper prices with good support that doesn't require learning someone else's unique tech stack. Oh, and they won't protect their own IP so anyone willing to do their own spin-up on the hardware can because they aren't interested in tearing down Nvidia's moat just to build their own.

9 hours ago, leadeater said:

You are aware AI accelerators and AI software doesn't actually have anything to do with the GPU market right? There is a very big difference between GPUs that have AI acceleration capabilities and AI accelerators as well as the software frameworks for AI.

If you want competition in the GPU market then you want to be looking at a company developing GPUs, like Intel for example.

I will mention again that the title of this post is structured as "X company will clean Y company's clock". It doesn't actually matter the exact implementation of how it gets done. What matters is when people who have been in the GPGPU compute world look at the available options for AI applications and see that better alternatives exist. GPGPU work has been done long prior to Nvidia's refocusing on GPGPU for AI workloads. In fact, when I first got into it, Nvidia was still touting how awesome their larger 32/64fp capabilities are. Getting into the jive of offering stuff for folks that benefit from 4int 8fp, etc. and tensor optimized computation just makes GPGPU more tantalizing for AI applications, but you had been able to use GPUs for general purpose compute for a long time prior. They became better accelerators for the task, but they were always accelerators when compared to the old CPU approach to it.

The fact is that GPUs have been co-opted for general purpose compute for a while and thus anything affecting this section of the market affects GPUs in general. If the competition for GPUs for this application decreases because of more options for this particular area, then GPU offerings for other areas will be more favorable to the purchasers because there will be fewer of them.

9 hours ago, Beerzerker said:

All I can say is if they ever become a "Threat" to Nvidia, they'll either be snapped up and brought under Nvidia's umbrella or crushed like a bug.

Nvidia is known for doing such things in the past and I seriously doubt they'll abandon their ways of business, esp since they are now a company that's worth more than any other in the world.

No - I don't think they'll abandon what got them to that point so easily.

I completely understand your point and agree with your last line that they won't abandon their ways. I would like to offer some push back on some of your other points though.

1) The CEO is an ideolog who won't sell (long track record of sticking to guns). 2) The other company in relation here is Comma.ai and they've been going toe-to-toe with all the big wigs in autonomous driving. Being able to keep up that game for as long as they have is impressive and demonstrates an ability to deal with any worries of volatility. 3) the AI space isn't the consumer space. People will actually bother reading the technical papers. When the benchmarks show comparable performance and engineers are reading over the paltry modifications to start training on their hardware over others, it's gonna be a much easier sell when the big difference shows itself to be price, and not in Nvidia's favor. 4) Past a certain point it's just pain illegal to be anti-competitive and we're finally seeing push back on such practices both legally and culturally.

2 hours ago, BrandonLatzig said:

oddly enough the only reason I picked up Intel's gpu is because the company聽 that made it went with something different
Like just the name SPARKLE was fascinating to me (even if I havent used that pc at all)

I too love "ooo shiny".

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, Sirgeorge said:

I will mention again that the title of this post is structured as "X company will clean Y company's clock". It doesn't actually matter the exact implementation of how it gets done. What matters is when people who have been in the GPGPU compute world look at the available options for AI applications and see that better alternatives exist. GPGPU work has been done long prior to Nvidia's refocusing on GPGPU for AI workloads. In fact, when I first got into it, Nvidia was still touting how awesome their larger 32/64fp capabilities are. Getting into the jive of offering stuff for folks that benefit from 4int 8fp, etc. and tensor optimized computation just makes GPGPU more tantalizing for AI applications, but you had been able to use GPUs for general purpose compute for a long time prior. They became better accelerators for the task, but they were always accelerators when compared to the old CPU approach to it.

The fact is that GPUs have been co-opted for general purpose compute for a while and thus anything affecting this section of the market affects GPUs in general. If the competition for GPUs for this application decreases because of more options for this particular area, then GPU offerings for other areas will be more favorable to the purchasers because there will be fewer of them.

That's all well and good but you talked about AI specifically and there are a lot of AI startups software wise and hardware silicon makers that all made exclusively AI products that have nothing at all to do with GPUs, then you talked about GPUs.

The point is you are either talking about "AI" or GPUs, they are not the same thing. Lots of companies have come and gone in the "AI" space that have had and would have had nothing to do with GPUs or GPGPU because that's not actually what the focus on AI actually is.

GPGPU is classically FP32 and FP64 which is not the data widths and types used for AI and ML anymore for the majority.

So no it's not going to help the GPU market because it does nothing to address the available market options, does not increase them, and it does not address the issue of silicon wafer production contention. In fact an increase in AI chips being made and the increasing demand for them makes that worse not better.

3 hours ago, Sirgeorge said:

Initially, they even rejected an Nvidia offer.

Offer of what? All I've seen is them saying they won't use Nvidia. To Nvidia they are nothing so I doubt Nvidia offered anything, choosing not to use Nvidia doesn't mean anything really so unless there literally was an actual business proposal from Nvidia to Tin Corp then there was no Nvidia offer.

Link to comment
Share on other sites

Link to post
Share on other sites

4 hours ago, Sirgeorge said:

1) The CEO is an ideolog who won't sell (long track record of sticking to guns).聽

He also has a track record of wasting his time in LTT forums, though, where there isn't much of interest for his company聽馃槙

Link to comment
Share on other sites

Link to post
Share on other sites

5 hours ago, leadeater said:

That's all well and good but you talked about AI specifically and there are a lot of AI startups software wise and hardware silicon makers that all made exclusively AI products that have nothing at all to do with GPUs, then you talked about GPUs.

The point is you are either talking about "AI" or GPUs, they are not the same thing. Lots of companies have come and gone in the "AI" space that have had and would have had nothing to do with GPUs or GPGPU because that's not actually what the focus on AI actually is.

GPGPU is classically FP32 and FP64 which is not the data widths and types used for AI and ML anymore for the majority.

So no it's not going to help the GPU market because it does nothing to address the available market options, does not increase them, and it does not address the issue of silicon wafer production contention. In fact an increase in AI chips being made and the increasing demand for them makes that worse not better.

Offer of what? All I've seen is them saying they won't use Nvidia. To Nvidia they are nothing so I doubt Nvidia offered anything, choosing not to use Nvidia doesn't mean anything really so unless there literally was an actual business proposal from Nvidia to Tin Corp then there was no Nvidia offer.

I'll take the point from a different angle. So, from the perspective of an AI Researcher/Engineer there is a need. You're trying to engage in AI work on this new (old) fangled thing called neural nets and unlike your K-nearest neighbor, decision trees, etc. it's taking a lot more compute. Your old approach of good old fashioned CPU compute isn't really gonna work here. What do you have on the table? Oh, there's this whole GPU thing. What it suffers from in terms of single thread performance it makes up for in terms of numerosity of cores. I know they're not exactly built for it, but I wonder if you can do matmuls with them. Oh? It turns out you can, and it can get you lots of performance for cheap (by professional terms). I think we'll start using it. Well, what exactly should we use? To talk with it. After all, no one here wants to "code in peripheral", yuk!

It turns out that if we wanna get performance while not being experts in bare metal programming, we'll have to use some of these particular tools that are available and at our disposal. Well, there's a couple of tool sets available to us including this whole "Open" family of stuff, but it's all weird and requires graphical specific stuff, yuk! Oh? What's this? CUDA? You're telling me I can use C with this stuff, and it's just kinda a natural extension of C? Sure! I can do that. After all, for decades we've been using Lisps for AI, but we're really getting into this whole Python thing and the default implementation is source-to-source compiled with C with easy calling of C. Let's do that! Wait, this thing doesn't really run on certain kinds of GPUs. Nvidia? Ok, we'll use Nvidia stuff since they make cards that work with this rocking CUDA thing.

--years of network effects and new found industry standardization later--

AI Guys: Hey, so a firm that got started 7 years ago suddenly made a killer app for AI and we suddenly have like a bajillion dollars as our industry suddenly got lots of VC money. Money isn't really any object for us and we have to work with stuff that we know at scale. Can we have more of your GPUs?

Nvidia: Ya sure, that'll be a bajillion dollars please. Oh, and since you wanna buy all the gpus, we'll kinda feel happy to just not care about what we charge on our whole gaming devision. Hey, does anyone remember when our stock was trading for 20ish bucks a share? I sure don't.

AMD/Intel/Accelerates (doesn't matter who): Hey, wanna buy our stuff?

AI Guys: well, can we use CUDA/Python/C with your stuff with fairly minimal learning curve and get comparable performance to Nvidia chips on them? If so, then sure, we'd like to get more [things that crunch numbers well enough], but we're kinda expert limited and we can't/won't really learn a new proprietary way of doing things.

The response back has thus far been....well...not that good and so no one adopts anything else at scale. Occasionally you get experiments and some people will buy your stuff if your AMD and you are selling for half the price, but not much better.

This is a key point to understand. GPUs are incidental to the AI researcher. They got to be in the right place at the right time to be the solution to a problem and the particular formulation of what made Nvidia GPUs win out was that they would accelerate an AI workload as compared to the old CPU approach to things and would do so without having to pick up a new set of highly technical skills. Now, any replacer has to get Nvidia levels of performance from hardware (not been too difficult), BUT also have a software tech stack from bare metal up to the programming language so it's as simple as learning to use a new library at the most complicated. Otherwise, the, very gilded pockets, AI researchers will just go back to Nvidia.聽

This is a situation where a technical need found a home in a specific company's offerings and that company then got vendor lock-in due to network effects of a tech stack. Suddenly there was a lot of money servicing that particular demand and then GPU prices went through the roof for anyone else (beyond what they were before).聽

In that case, a company that can actually provide an alternative to Nvidia with a fraction of the cost will cause a massive change in the power dynamics between Nvidia and all of their customers. The majority of their customers (by money) will move over the demand and the majority of their customers (by numerical value) will suddenly have much fewer "infinite pocket" fellow customers to compete with.

I believe you have a category error here. The original point and content of the post is in reference to a swiping away of what is, at this point, the majority of the demand (by money value) from Nvidia by an up-and-comer. It's a bit murky what the actual architecture will be on the opposition, but the point is that Nvidia will "have it's clock cleaned". Does it really matter what particular type of mouse trap it is (though no one can say exactly what kind it will be yet). The end result will be, regardless, a more competitive edge for who is left buying Nvidia GPUs post "clock cleaning".

On note of GPGPU, ya, classically it would have been. Not really the case for most of the number crunching going on these days. Most of what I use is FP16 with some attempts to dive as deep as FP/INT4. I've not seen other researchers fairing well at those lower precisions either, but the 32/64 width was kinda an artifact of its time.

Oh, and here's the proof on the Nvidia offer:聽

I will give Nvidia credit. They aren't so poor in decision making as to think of tinygrad as nothing. They've been playing this game for a long time and know better then to think in this fashion. Good on them in this regard. It's the little guys that grow big and so it's the little guys that are the biggest deal. 馃檪

3 hours ago, SpaceGhostC2C said:

He also has a track record of wasting his time in LTT forums, though, where there isn't much of interest for his company聽馃槙

Does he? All I knew is he had some kind of gripe with LTT. Don't know what. Not my horse not my course. Could you give me some more info on that. That's new info for me.

Link to comment
Share on other sites

Link to post
Share on other sites

On 6/23/2024 at 11:34 AM, leadeater said:

You are aware AI accelerators and AI software doesn't actually have anything to do with the GPU market right? There is a very big difference between GPUs that have AI acceleration capabilities and AI accelerators as well as the software frameworks for AI.

Could you elaborate on that? From my POV "AI accelerators" are just either regular GPUs or GPUs with some cut down stuff meant for GPGPU (like the x100 chips).

Unless you're talking about NPUs or TPUs, which I'd argue only have a minor share for places where you're actually training models.

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to comment
Share on other sites

Link to post
Share on other sites

27 minutes ago, igormp said:

Could you elaborate on that? From my POV "AI accelerators" are just either regular GPUs or GPUs with some cut down stuff meant for GPGPU (like the x100 chips).

Unless you're talking about NPUs or TPUs, which I'd argue only have a minor share for places where you're actually training models.

All the chip startups that have failed have been this or ASIC and not "GPU". That's what is the context when talking about companies that have come in and tried to make AI hardware and failed because they weren't making GPUs or even hardware designed or capable of doing general FP32 or FP64.

Most certainly it is done on GPUs, Nvidia GPUs, but that's pretty much the point. You have that and then everything else not that which could be CPU, ASIC, NPU, TPU etc etc.

Anyone bringing anything new in to the AI market hardware wise aren't making GPUs with the exception of Intel.聽

image.png.97a788c208b1336bcfbac1c729fadd26.png

That is 3 GPUs in this list? Since Intel's is not a GPU. 2 if you only count the M4 NPU and exclude it's GPU.

Link to comment
Share on other sites

Link to post
Share on other sites

On 6/24/2024 at 9:09 PM, Sirgeorge said:

Oh, and here's the proof on the Nvidia offer:聽

That's Nvidia marketing and customer relations, I would be worried if those employed by Nvidia in those positions weren't doing their literal job. Telling a company that they should use your hardware is not the same thing as a business offer between the two since all Nvidia is doing is trying to convince another company that they should make products using Nvidia products.

An "offer" is a business exchange in which Nvidia would be going to Tiny Corp with something more than a pitch to use their products.

What I am pointing to is the misrepresentation of what actually transpired, of which was not a formal meeting of two companies to try and form a contractual agreement between them initiated by Nvidia with some incentive to try and lure Tiny Corp in to using their products.聽 That's not what happened.

Link to comment
Share on other sites

Link to post
Share on other sites

17 minutes ago, leadeater said:

All the chip startups that have failed have been this or ASIC and not "GPU". That's what is the context when talking about companies that have come in and tried to make AI hardware and failed because they weren't making GPUs or even hardware designed or capable of doing general FP32 or FP64.

Most certainly it is done on GPUs, Nvidia GPUs, but that's pretty much the point. You have that and then everything else not that which could be CPU, ASIC, NPU, TPU etc etc.

Anyone bringing anything new in to the AI market hardware wise aren't making GPUs with the exception of Intel.聽

image.png.97a788c208b1336bcfbac1c729fadd26.png

That is 3 GPUs in this list? Since Intel's is not a GPU. 2 if you only count the M4 NPU and exclude it's GPU.

Ah, I get what you mean. But I believe you got their idea wrong at first. Tinycorp is focusing on building a new software stack that's agnostic to the hw underneath, and being realistic, Nvidia is the major player with AMD being a distant competitor due to their lackluster software stack.

If Tinycorp's stack manages to get traction (which I don't have much hopes for since a small framework doesn't solve all of your problems suddenly), then AMD would be a suitable platform to compete with Nvidia, or at least that how I see OP's POV.

The M4 NPU is pretty useless for training btw, and its GPU is okay-ish, but still doable, so I guess 3 GPUs in that list after all.

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to comment
Share on other sites

Link to post
Share on other sites

25 minutes ago, igormp said:

Ah, I get what you mean. But I believe you got their idea wrong at first. Tinycorp is focusing on building a new software stack that's agnostic to the hw underneath, and being realistic, Nvidia is the major player with AMD being a distant competitor due to their lackluster software stack.

If Tinycorp's stack manages to get traction (which I don't have much hopes for since a small framework doesn't solve all of your problems suddenly), then AMD would be a suitable platform to compete with Nvidia, or at least that how I see OP's POV.

I'm aware that Tiny Corp is just software I just really don't see it as having any impact on the GPU market since AMD already is a suitable platform to use as it is, just more difficult and not all AMD hardware usable (until now).

Where I see meaningful change on that front coming from is AMD itself and it's partners along with their high end users. We've got some very large users coming in and adopting AMD now who will be contributing a lot in to making AMD's hardware actually more readily useable. On the other front AMD itself has made Radeon useable with ROCm and supporting multiple GPUs so now you aren't locked in to only using CDNA based hardware.

The bigger issue as I was pointing to is that the AI/ML market itself has little long term interest in being tied to either Nvidia or AMD and classic GPUs so as much as AMD GPUs are being more readily usable the market is moving away from actually even wanting to use them. AMD isn't in a luxurious position where they have a large userbase of people using their products for AI in workstations that creates demand in servers and clusters etc so if you are going to take a risk why choose AMD if Gaudi 3 is suitable for your needs or any of the others.

AMD has a limited window of opportunity before becoming irrelevant for AI, Xilinx might save them partially though, never know.

Ultimately Tiny Corp has all the same credibility as every other now since failed company, deliver first then make extraordinary claims. Anyone coming in claiming that X will upend Nvidia in a large way like this honestly can't be taken too seriously.

Link to comment
Share on other sites

Link to post
Share on other sites

27 minutes ago, leadeater said:

The bigger issue as I was pointing to is that the AI/ML market itself has little long term interest in being tied to either Nvidia or AMD and classic GPUs so as much as AMD GPUs are being more readily usable the market is moving away from actually even wanting to use them.

Ah, I see your point now, agreed on that.

30 minutes ago, leadeater said:

Ultimately Tiny Corp has all the same credibility as every other now since failed company, deliver first then make extraordinary claims. Anyone coming in claiming that X will upend Nvidia in a large way like this honestly can't be taken too seriously.

I mean, tinycorp is not claiming anything HW-wise, they just want to build a hw-agnostic framework (their plan is to run on amd, nvidia, intel and AS for now) that's fast and simple, and do seem to be making good progress on that.

That AMD claim was more on OP's part and I agree is not something that should be taken serious.

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, leadeater said:

I'm aware that Tiny Corp is just software I just really don't see it as having any impact on the GPU market since AMD already is a suitable platform to use as it is, just more difficult and not all AMD hardware usable (until now).

Where I see meaningful change on that front coming from is AMD itself and it's partners along with their high end users. We've got some very large users coming in and adopting AMD now who will be contributing a lot in to making AMD's hardware actually more readily useable. On the other front AMD itself has made Radeon useable with ROCm and supporting multiple GPUs so now you aren't locked in to only using CDNA based hardware.

The bigger issue as I was pointing to is that the AI/ML market itself has little long term interest in being tied to either Nvidia or AMD and classic GPUs so as much as AMD GPUs are being more readily usable the market is moving away from actually even wanting to use them. AMD isn't in a luxurious position where they have a large userbase of people using their products for AI in workstations that creates demand in servers and clusters etc so if you are going to take a risk why choose AMD if Gaudi 3 is suitable for your needs or any of the others.

AMD has a limited window of opportunity before becoming irrelevant for AI, Xilinx might save them partially though, never know.

Ultimately Tiny Corp has all the same credibility as every other now since failed company, deliver first then make extraordinary claims. Anyone coming in claiming that X will upend Nvidia in a large way like this honestly can't be taken too seriously.

AMD is NOT a suitable platform to use as it. I have seen some really horrendous "baby's first driver" level mistakes that have left me flabbergasted looking at what AMD is doing. We're talking basic run conditions. It's made it obvious that there's gotta be some real Italian levels of spaghetti code in there. Unfortunately, the stack is not as open as I once thought and actually making contributions at the level of the issues is...well...not really possible unless you basically get to partner with AMD (which is not really good enough in this case for obvious reasons). AMD has also made it very obvious that they aren't gonna fix this.

I agree on the AMD relevance to AI part and I genuinely would love to see them get their stuff together. Honestly, they need to, at a minimum, open source their entire software stack. That'll probably be just about the only thing they absolutely have to do, and, given the track record, anything else is at most critical but insufficient.

Tinycorp has more credibility than most because they, unlike the vast majority of alternatives, actually picked the only viable method of making this work (software first, then firmware, then hardware) and have advanced all 3 thus far in the right sequence to the right degree. It's been rather wonderful to see. Also, I've seen what they've produced. I've seen their meetings (public on discord every Monday, if I remember correctly). I've bothered to read and understand their entire codebase (even contributed to it on occasion). This is...without giving too much away of what I do these days, a professional concern of mine which warrants professional study and the purpose of the post was as a kind of tongue and cheek way to wink to those that would have absolutely no business knowing better what the betting odds are this early on because the only ones that have a shot are the kind of people that can play insider baseball. Most of those are gonna be the kind of people who would do so because their meal ticket comes from it. It was in no way said with such certainty as to be a guarantee and nor was it ever claimed to be taken in such a manor. The appropriate qualifiers have been issued throughout the entire piece.

2 hours ago, igormp said:

Ah, I see your point now, agreed on that.

I mean, tinycorp is not claiming anything HW-wise, they just want to build a hw-agnostic framework (their plan is to run on amd, nvidia, intel and AS for now) that's fast and simple, and do seem to be making good progress on that.

That AMD claim was more on OP's part and I agree is not something that should be taken serious.

The long term plan does involve chip tape-outs and the actual hardware architecture proposed has varied somewhat. Full blown GPUs have been talked about (along with other alternatives). Regardless, if they execute well, it'll give Nvidia a much worse hand when it comes to selling their chips to this new sector as opposed to their old ones. I would also like to highlight again that throughout the entire process I never claimed X WILL do Y. I went to painstaking effort to have the structure of my post and subtext be, "hey, not something you'll known on the outside looking in, but here's a hint as to what may finally disrupt the current status quo". I wouldn't have added the phrase, "has a shot" in the title if I was giving a gurantee. 馃槈

2 hours ago, leadeater said:

That's Nvidia marketing and customer relations, I would be worried if those employed by Nvidia in those positions weren't doing their literal job. Telling a company that they should use your hardware is not the same thing as a business offer between the two since all Nvidia is doing is trying to convince another company that they should make products using Nvidia products.

An "offer" is a business exchange in which Nvidia would be going to Tiny Corp with something more than a pitch to use their products.

What I am pointing to is the misrepresentation of what actually transpired, of which was not a formal meeting of two companies to try and form a contractual agreement between them initiated by Nvidia with some incentive to try and lure Tiny Corp in to using their products.聽 That's not what happened.

I think you misread my claim in context. The line in which I used the term "offer" (first usage of it at that), was in response to EigenVektor, "...oh and Tiny Corp lauches Nvidia-powered AI computer because 'it just works'" with my response being, "In regards to the post on launching on Nvidia over Intel/AMD, they tried really hard to avoid it. Initially, they even rejected an Nvidia offer."

In this context, my evidence as to this offer is in line with the actual usage of the term "offer" meant because the claim was that Tinycorp initially rejected an offer from Nvidia to use their chips in Tinycorp's tinybox (yes, the theme they picked on the naming convention is on the nose). It's an official interaction with a portion of Nvidia to offer to supply Tinycorp with GPUs for their hardware that they sell. In context it has a different usage of the term, with a different definition, then the one you choose to use later in the thread.

Link to comment
Share on other sites

Link to post
Share on other sites

24 minutes ago, Sirgeorge said:

unless you basically get to partner with AMD

Trust me, even if you do manage to be one of their partners, it's still not easy and really frustrating聽馃珷

33 minutes ago, Sirgeorge said:

Regardless, if they execute well, it'll give Nvidia a much worse hand when it comes to selling their chips to this new sector as opposed to their old ones.

Doubt, but then again that's just your opinion and neither of us have a crystal ball, so it could go either way.

1 hour ago, Sirgeorge said:

I would also like to highlight again that throughout the entire process I never claimed X WILL do Y.

Claiming that "Nvidia will have it's clock cleaned" kinda implies many things.

1 hour ago, Sirgeorge said:

"hey, not something you'll known on the outside looking in, but here's a hint as to what may finally disrupt the current status quo". I wouldn't have added the phrase, "has a shot" in the title if I was giving a gurantee. 馃槈

You did so in a really click-baity way IMO, as seen by how most of us understood your point.

Still, I don't think it'll be really disruptive anytime soon as per other reasons stated. They won't be able to offer the level of support that Nvidia does, unless they bring their own hardware to the table with really great support, and such hw will only be relevant if they also make it supported in other frameworks such as pytorch.

AMD has a shitty software stack, tinygrad being able to run on top of AMD won't magically make AMD have an expert on site that can help them with tinygrad issues, whereas that's possible with nvidia.

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to comment
Share on other sites

Link to post
Share on other sites

  • 2 weeks later...

It seems to me that Nvidia will be the main manufacturer of video cards for many more years. They are like Apple in the world of video cards.

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now