DeepSeek isn't that wrong

[PDifolco]

Asked my local Deepseek about Tienanmen

 

[05032-Mendicant-Bias]

There is some more censorship about chinese history on the deepseek online interface that runs the full fat 600B model.

 

The llama and qwen distill are pretty much uncensored. I can't really run the full fat V3 or R1 models locally to test them... yet...

[SorryBella]

18 minutes ago, PDifolco said:

local Deepseek

If you saw the videos documenting these, the output still goes until the query mentioned banned context. basically, online 600B model is the one thats censored usually mid query.

[PDifolco]

7 minutes ago, SorryBella said:

If you saw the videos documenting these, the output still goes until the query mentioned banned context. basically, online 600B model is the one thats censored usually mid query.

Yeah that's why I tested an offline distilled one, can run the 32B easily but the 70B is pretty slow

I really like the "reasoning steps" approach of this model

[SorryBella]

Just now, PDifolco said:

Yeah that's why I tested an offline distilled one, can run the 32B easily but the 70B is pretty slow

I really like the "reasoning steps" approach of this model

Im not that excited with the summarizing, its still similar to copilot and ChatGPT in its language. Im more excited to see how it could contextualize itself as a writing character. I havent seen any character.ai clone sites using DeepSeek proxy yet.

[PDifolco]

Another interesting answer, didn't know models were performance capped for certain tasks

 

[Haswellx86]

You got the 32 billion parameter model running on your 24GB 7900XTX? How much is that quantized? 

[PDifolco]

4 minutes ago, Haswellx86 said:

You got the 32 billion parameter model running on your 24GB 7900XTX? How much is that quantized? 

I suppose it's using as much VRAM as it can then the rest goes to slower RAM, it's using around 20GB RAM

Even the 70B model works but it's very slow (several minutes per query)

[MiszS]

19 minutes ago, Haswellx86 said:

You got the 32 billion parameter model running on your 24GB 7900XTX? How much is that quantized? 

I can use the Q4 model fully on my 3090 

[venomtail]

Guys, is it worth downloading the 500GB biggest model? Makes me wanna dabble in it myself. 

[PDifolco]

26 minutes ago, venomtail said:

Guys, is it worth downloading the 500GB biggest model? Makes me wanna dabble in it myself. 

You won't be able to load it with less than 500GB VRAM or RAM... That's datacenter level hardware 

And then need a ton of CPU/GPU power to get answers in a decent time

[venomtail]

Just now, PDifolco said:

You won't be able to load it with less than 500GB VRAM or RAM... That's datacenter level hardware 

And then need a ton of CPU/GPU power to get answers in a decent time

I get how that works now. One missing link on the capacity mentioned.