Plex Media Server - Optimisation - 1 core?

[Dravinian]

Hi All,

 

I am optimising my plex server media files.  This involves essentially encoding them prior to transmission to limit the transcode overhead.

 

So, I started doing it on a 3900x, a 12 core 24 thread PC as my FreeNAS server was very slow with its 4c 8t i7 4790k, and I thought, well it is a lot of media, let me utilise that 3900x I have laying around.

 

Nope..

 

Plex uses 5% of my CPU while optimising - by comparison it was using 60% - 80% of my i7.

 

So, I thought I would ask here after google offered no answers....is there a way to up this? To actually utilise the power of the CPU to get this job done before Christmas?

 

Sure, I suppose I could encode in a different program, but I like that Plex does this, arranges them, files them and knows where they are...it also encodes the subtitles out of the video so that they do not impact on any transcoding that is still required (some devices will still need transcoding).

 

Anyone got any experience of this?

[IronSoldier]

Just use a 3rd party tool like Handbrake. It can usually utilise up to 6 cores but it depends on video codes too when it comes to transcoding.

 

[Korben]

Yeah, I agree with @IronSoldier. It is limited by Plex. Plex doesn't make use of multi-core CPUs, it doesn't scale. I guess, it's because it's a "one-fits-all"-decision.

But if you're using a 3900X for your Plex Server, the "transcoding  overhead" shouldn't be an issue either, does it? In the settings for transcoding you can at least change the level of background encoding to "ultra"... 

[Dravinian]

2 minutes ago, Korben said:

Yeah, I agree with @IronSoldier. It is limited by Plex. Plex doesn't make use of multi-core CPUs, it doesn't scale. I guess, it's because it's a "one-fits-all"-decision.

But if you're using a 3900X for your Plex Server, the "transcoding  overhead" shouldn't be an issue either, does it? In the settings for transcoding you can at least change the level of background encoding to "ultra"... 

Thing is, I am not using the 3900x for the Plex Server, I am temporarily hosting the Server (which is on FreeNAS on an i7 4790k) on the 3900x machine just to do the opitmisation - because I thought it could get the job done faster.

 

As mentioned in my original tweet, yeah I have handbrake, I know I can use a 3rd party tool, but as mentioned, i like the fact that plex does it for plex, separates out the SRT/Subtitle files, and then orders it in the collection so it knows which version to use.

 

I am essentially doubling the collection, as I intend to keep the 4k versions for my personal viewing at home on a 4k TV, but for those watching on older laptops and mobile phones etc, the issue is they can't watcch the 4k version (the machines won't play them) so I was transcoding to them through the i7, and hw encoding doesn't work on 4k media files on that old of a CPU - also on FreeNAS there is no GPU passthrough so you can't transcode with GPU support.

It is a real pain how problematic the whole system is to be honest, it is not exactly plug and play.

[Korben]

14 minutes ago, Dravinian said:

It is a real pain how problematic the whole system is to be honest, it is not exactly plug and play.

I actually disagree. I tried some other Media servers (Kodi, Synology) and they are far worse. Plex is quiet a breeze in that regard. 

Yes, it has it's limits. But name a software that doesn't (you would already use it, wouldn't you).

[Dravinian]

4 minutes ago, Korben said:

I actually disagree. I tried some other Media servers (Kodi, Synology) and they are far worse. Plex is quiet a breeze in that regard. 

Yes, it has it's limits. But name a software that doesn't (you would already use it, wouldn't you).

I didn't mean Plex, I meant the whole running a media server.

Things you take for granted in Windows just don't exist outside of Windows, like GPU hardware acceleration, iGPU acceleration support - even the trash bin to retrieve accidentally deleted files doesn't exist, little in the way of GUIs and a lot of shell/ssh to get things done.

 

Sure if you work on servers, this will come as no shock and part and parcel of the system, but for those moving to creating one these, it is not exactly plug and play.

[Korben]

5 minutes ago, Dravinian said:

Things you take for granted in Windows just don't exist outside of Windows, like GPU hardware acceleration, iGPU acceleration support - even the trash bin to retrieve accidentally deleted files doesn't exist, little in the way of GUIs and a lot of shell/ssh to get things done.

Again, I disagree. And I don't even know where to start...

Even if Plex runs on Windows, it has the same limitations (hence the "one-fits-all" comment). Linux support GPU and iGPU acceleration. Even a trashbin... MacOS (ok, it's linux too) does it....

But we digress. 

[Donut417]

2 hours ago, Dravinian said:

Anyone got any experience of this?

This is why it’s advisable to put your media in the correct format before watching it thru Plex. Have you looked in to using a GPU? You do have to pay for Plex pass for this option, but GPUs can do this a bit better from what I have read. 

[Dravinian]

1 hour ago, Korben said:

Again, I disagree. And I don't even know where to start...

Even if Plex runs on Windows, it has the same limitations (hence the "one-fits-all" comment). Linux support GPU and iGPU acceleration. Even a trashbin... MacOS (ok, it's linux too) does it....

But we digress. 

None of those are server operating systems, they are operating systems, but they are not specifically for a server. None of this is a problem on Windows either, but that isn't really the point is it?

 

When you are running a server with server software, then you encounter the problems I mentioned which make it less of a plug and play experience.

[Dravinian]

22 minutes ago, Donut417 said:

This is why it’s advisable to put your media in the correct format before watching it thru Plex. Have you looked in to using a GPU? You do have to pay for Plex pass for this option, but GPUs can do this a bit better from what I have read. 

This entire thread is about the process of putting media into the correct format before watching it.

[Donut417]

1 minute ago, Dravinian said:

This entire thread is about the process of putting media into the correct format before watching it.

Do it with handbreak. Don’t do it in Plex. 

[Dravinian]

1 minute ago, Donut417 said:

Do it with handbreak. Don’t do it in Plex. 

Yeah I know, but then I am not getting the Plex integration that I was looking for...such as offsetting SRT files.

 

It is slow and that is irritating, but the fact that it has having literally no impact on my machine, I can just queue them up and let them run.

 

I was just hoping there was a clever way to increase the CPU usage so I could get it done faster, but that doesn't appear to be the case.

[Korben]

1 hour ago, Dravinian said:

None of those are server operating systems, they are operating systems, but they are not specifically for a server. None of this is a problem on Windows either, but that isn't really the point is it?

 

When you are running a server with server software, then you encounter the problems I mentioned which make it less of a plug and play experience.

I don't understand you. Really. Linux is not a server operating system? Are you kidding me? I'll stop here. This is getting redicoulus.

[Dravinian]

I run a VM with linux,  you can only ssh into, as it has no GUI as it runs on FreeNAS as a base [which is mentioned in post 1 - context]....one of the things I mentioned was having no GUI.  It has no trash bin so when you delete something, it is gone for good....something I also mentioned.

 

I don't understand you, I mean you keep taking one thing out of context within a discussion and being incredulous about it...but it wouldn't be incredulous if you just took the whole context.

 

Look I can do it too..

 

5 minutes ago, Korben said:

Linux is not a server

No!!!!! Linux is not a server, it is an operating system????? I don't understand, what's going on why would you say that??

 

See how stupid that is?

 

[Donut417]

1 hour ago, Dravinian said:

you can only ssh into,

Thats not accurate. While Server versions of Ubuntu for example don't have a GUI by default, you can defiantly download those packages. The same thing with the Desktop version of Ubuntu, You can download the server packages. Using Remote Desktop software you can easily administer a Linux machine via the GUI. The only big difference is Linux has a powerful command line and depending on how your server is setup it might be easier to SSH in and do things via the CLI. But technically speaking, you can have a GUI. 

 

[Dravinian]

6 minutes ago, Donut417 said:

Thats not accurate. While Server versions of Ubuntu for example don't have a GUI by default, you can defiantly download those packages. The same thing with the Desktop version of Ubuntu, You can download the server packages. Using Remote Desktop software you can easily administer a Linux machine via the GUI. The only big difference is Linux has a powerful command line and depending on how your server is setup it might be easier to SSH in and do things via the CLI. But technically speaking, you can have a GUI. 

 

Thanks, nice to know.

[Jarsky]

Perhaps try changing your transcoding settings to "Make my CPU hurt"...may make a difference?

 

I understand this thread is about optimising (pre-transcoding) your content, but assuming your server is Linux which has the VA-API, have you considered just getting a GPU and letting people transcode in real time?  You can get used GTX1060's for around US$120, and you can do a basic driver patch to remove the nvenc stream limit. 

[Dravinian]

23 minutes ago, Jarsky said:

Perhaps try changing your transcoding settings to "Make my CPU hurt"...may make a difference?

 

I understand this thread is about optimising (pre-transcoding) your content, but assuming your server is Linux which has the VA-API, have you considered just getting a GPU and letting people transcode in real time?  You can get used GTX1060's for around US$120, and you can do a basic driver patch to remove the nvenc stream limit. 

I am not sure the transcoding settings impact on the optimisation settings, but I will have a look later tonight - Plex is a real pain where if you cancel an optimisation part-way through, it has no memory that it has done any of it...so will just do everything again if you transcode the collection again.  So you have to manually do the items in a collection if you don't let it finish the entire collection - given the size, I want to avoid that.

 

Currently have it running on Windows because my server has no passthrough (FreeNAS).  Considering options for changing that, Linux is definitely amongst those options.

 

Having it on Windows means having access to the 2080ti, which has made things a lot easier in the short-term and I now have less complaining, but I would rather not have it running on this machine long-term, so pre-transcoding everything allows it to go back to the FreeNAS server in due course.

 

Just that due course will be about 4 months at this rate.

[Jarsky]

20 minutes ago, Dravinian said:

Currently have it running on Windows because my server has no passthrough (FreeNAS).  Considering options for changing that, Linux is definitely amongst those options.

 

OK yeah FreeNAS is an issue when it comes to Plex and hardware transcoding. Importing ZFS arrays from FreeNAS to ZoL (ZFS on Linux) isn't always the smoothest thing either...there can be technical challenges with that, although compatibility is far better since ZoL 0.8.3. Debian/Ubuntu is generally the best for keeping up to date on ZFS. 

 

The biggest thing you'd lose would be the nice GUI that FreeNAS has, though Cockpit has made huge progress in the last year, especially with the ZFS Manager so you can have a really functional GUI for the server as well...I updated my test server recently and quite impressed with how far Cockpit has come now.  Something to keep an eye on is UnRAID too, as theyve said they will be working on doing ZFS array support as well. Not sure what release that would be in the pipeline for though, their upcoming 6.9.0 release is focused on multi pool support (which im running on my UnRAID box), so perhaps 7.0...so probably at least a year away from seeing that. 

[Dravinian]

1 minute ago, Jarsky said:

 

OK yeah FreeNAS is an issue when it comes to Plex and hardware transcoding. Importing ZFS arrays from FreeNAS to ZoL (ZFS on Linux) isn't always the smoothest thing either...there can be technical challenges with that, although compatibility is far better since ZoL 0.8.3. Debian/Ubuntu is generally the best for keeping up to date on ZFS. 

 

The biggest thing you'd lose would be the nice GUI that FreeNAS has, though Cockpit has made huge progress in the last year, especially with the ZFS Manager so you can have a really functional GUI for the server as well...I updated my test server recently and quite impressed with how far Cockpit has come now.  Something to keep an eye on is UnRAID too, as theyve said they will be working on doing ZFS array support as well. Not sure what release that would be in the pipeline for though, their upcoming 6.9.0 release is focused on multi pool support (which im running on my UnRAID box), so perhaps 7.0...so probably at least a year away from seeing that. 

TrueNAS is just about to go live, so I thought I would let that release, give it a month or two to iron out any bugs, and then move over to that to see if there is any improvement - as I feel it will be a smoother experience.

 

If that doesn't work, then I will look to see whether I can move the entire server to a new platform. I was also thinking about doing some research into ProxMox as I have heard good things about it, but my knowledge is a bit limited at the moment so would have to invest a bit of time to see whether it is suitable.

Cheers for the info on Cockpit, I will do some research on that too.

[Jarsky]

1 minute ago, Dravinian said:

TrueNAS is just about to go live, so I thought I would let that release, give it a month or two to iron out any bugs, and then move over to that to see if there is any improvement - as I feel it will be a smoother experience.

 

trueNAS Core has a lot of cool stuff, but its still BSD and fundamentally its the same...nothing will be changing as far as GPU support (FreeNAS actually does support PCIe passthrough, just not GPU)

 

1 minute ago, Dravinian said:

If that doesn't work, then I will look to see whether I can move the entire server to a new platform. I was also thinking about doing some research into ProxMox as I have heard good things about it, but my knowledge is a bit limited at the moment so would have to invest a bit of time to see whether it is suitable.

Cheers for the info on Cockpit, I will do some research on that too.

 

Proxmox is fantastic as a very powerful open source hypervisor solution. If you're only going to run a few VM's or want to run some Docker containers, then its not really worth the headache. Its primary focus is as a hypervisor like VMWare ESXi. 

My test server actually runs Debian Buster with Proxmox and Cockpit. Cockpit is a web interface initiative from Red Hat Inc to replace the KVM GUI Manager so it has a lot of support behind it, and will be compatible with all GNU (Linux). It's much "easier" than Proxmox for a couple of machines, zfs management and general server admin. It has a docker manager as well, which is a nice replacement for using something like Portainer. 

[OddOod]

On 7/25/2020 at 6:34 AM, Korben said:

Plex doesn't make use of multi-core CPUs

It does when streaming. I've seen six cores pin to 80 when downresing from h265 4k to mpeg4 SD

[Jarsky]

1 hour ago, OddOod said:

It does when streaming. I've seen six cores pin to 80 when downresing from h265 4k to mpeg4 SD

He's talking about for Optimization (Pre-transcoding).

[OddOod]

7 hours ago, Jarsky said:

He's talking about for Optimization (Pre-transcoding).

Fair. I considered doing this myself but I have such a diversity of endpoints it seemed pointless. Though I have been considering using handbrake to re-encode everything to 265, max quality, but have been afraid of doing it in an automated fashion 

 

[NelizMastr]

On 7/25/2020 at 7:10 PM, Donut417 said:

Thats not accurate. While Server versions of Ubuntu for example don't have a GUI by default, you can defiantly download those packages. The same thing with the Desktop version of Ubuntu, You can download the server packages. Using Remote Desktop software you can easily administer a Linux machine via the GUI. The only big difference is Linux has a powerful command line and depending on how your server is setup it might be easier to SSH in and do things via the CLI. But technically speaking, you can have a GUI. 

 

Exactly. My Plex media server runs on Ubuntu Server 18.04, which itself is a VM in ESXi, and has access to 4GB of RAM, 2 CPU cores and a Quadro P600 for transcoding media. It chews through H265 and 4K like a boss, even if it's limited to 2 streams (which is fine for me). A P1000 Quadro will unlock unlimited streams. A Plex Pass is required, though.