Jump to content

Local LLM setup

Havel
 Share
Posted

My wife has been using the hiwaifu ai chatbot app for awhile now to role play some pretty depraved stuff and she wants me to figure out how to run it locally for her on our server to give it potentially better memory and performance

 

I am completely and utterly ignorant on how to even begin to setup an LLM, all I know is that she prefers this specific chatbot because it's basically uncesored and will let you get into some seriously dark roleplay

 

If there's any information y'all could give me in the right direction I'm all ears

Link to comment
https://linustechtips.com/topic/1613648-local-llm-setup/
Share on other sites
Link to post
Share on other sites
Posted
28 minutes ago, Havel said:

potentially better memory and performance

While running an LLM isn't as demanding as training one, you'll still need a good amount of RAM and a fast GPU (with enough VRAM).

 

If you want to run an LLM on local hardware you'll first need to find one that's open source.

https://getstream.io/blog/best-local-llm-tools/

 

This sounds like it might go in the right direction and comes with setup instructions

https://github.com/HRNPH/AIwaifu

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
https://linustechtips.com/topic/1613648-local-llm-setup/#findComment-16739227
Share on other sites
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing Programs, Apps and Websites

×