Low Power Handbrake Video Encoding PC/Server

[BelgianNoise]

I host a Plex media server on a QNAP NAS. Over the past 3 years my library has become bigger and bigger and cost of storage is getting rather high.
So I recently switched from x264 to x265 encoding. Encoding times are now over 60min for a 720p video of 60min.

I am trying to find the most power efficient solution to encode my media.

Here is a list of the devices I own and their restive power usage and how long it takes them to encode:

• Desktop: Ryzen 9 3900X
  • I only list CPU cause I only do software encodes because the goal is to not have Plex transcode on the fly
  • Idle:  165 W
  • during encode:  240 W
  • encoding time: 70min
  • total power while not in use by me:  ~280 Wh
  • total power while in use by me:  ~90 Wh
• NAS: Celeron N5105 - 4-core - 2 GHz cpu specs (limited to 50% CPU)
  • idle:  18 W
  • idle with active drives:  42 W
  • during encode:  47.5 W
  • encoding time:  720min = 12 hours
  • total power while not in use (drives would be inactive):  480 Wh
  • total power while is use (drives active because other processes):  66 Wh
• Laptop: i7 7700HQ
  • idle: 14 W
  • during encode:  75 W
  • encoding time:  240min = 4 hours
  • total power:  300 Wh

 

Looking at these number I concluded that encoding on my desktop while I am using it is the most efficient. Using my weak NAS could technically be more efficient but since there are periods that the drives shouldn't have to be active otherwise this would average out higher then my desktop.

My ultimate goal is to have a folder I can just copy video into and the server to encode it automatically. I found that this is baked in in this docker image 'jlesage/handbrake' but docker on WSL 2 refuses to use a lot of CPU so it limits itself for a reason I could not figure out over the past days.

So I need help with:

1. What is the most power efficient and suitable hardware for this ? I am willing to pick up some used components and I dont want the encode to take 12 hours per video haha. This has become more about learning about power usage and less about saving money. (on electricity)
2. I have 2 Pi 3B's and could maybe get one or two more for very cheap. Would turning them into a cluster or combined PC work for this ? I am finding very few information on this on the internet, most of the blogs / videos are about running multiple containers in a kubernetes cluster which would def not fit this use case. (also don't think that Handbrake/ffmpeg runs amazingly on ARM)
3. Are there any encoding settings I could use that significantly improve my encode times ? I kinda don't want to increase my bitrate but I amopen to hear your suggested settings with explanations and possibly also the exported handwbrake preset so I can import and try using it.
4. Edit: why tf is my idle power usage to high on my desktop ?

 

Sometimes I have a huge queue of encodes and I have to leave my PC on during the night and/or work which greatly increases the cost.

I would also like to move this workload of of my desktop so I don't need to go crazy on my CPU anymore when i buy a new PC.

(I understand that Handbrake does not use the 12 cores but I wanted to be able to also game and stuff on my PC while encoding)

 

Thanks in advance for the help.

[BelgianNoise]

I think the best solution would be to have an i3 13100T (or 12100T, but the T is important) with an M.2 runnin Linux.
Sadly this adds up to over € 400 wich is more then I wish to spend on it.

 

Edit: a machine with a 4600G would only be about €300, but it would consume more power and have less performance if I am reading the numbers correctly online.

[m9x3mos]

8 hours ago, BelgianNoise said:

I think the best solution would be to have an i3 13100T (or 12100T, but the T is important) with an M.2 runnin Linux.
Sadly this adds up to over € 400 wich is more then I wish to spend on it.

 

Edit: a machine with a 4600G would only be about €300, but it would consume more power and have less performance if I am reading the numbers correctly online.

If you don't mind buying used and slightly older hardware, you can get a 8500t Lenovo mini pc that has m.2 for around 200usd here so that should fall under 400 euros depending what the used market looks like in your region. 

[Domrockt]

Did you consider those Aliexpress Erying Mainboard CPU Combos?

 

Aliexpress Erying

 

get a 11/12th gen low power and use that. Should be fine.

[BelgianNoise]

8 hours ago, m9x3mos said:

If you don't mind buying used and slightly older hardware, you can get a 8500t Lenovo mini pc that has m.2 for around 200usd here so that should fall under 400 euros depending what the used market looks like in your region. 

Great idea! These mini PCs are sadly not that popular over here and the used marked is mainly intel 4th or 6th gen CPUs which is not really worth it performance wise imo. (These are around €200).
Looked a bit further into 8th gen too, single core performance leaves a lot to be desired.

[BelgianNoise]

4 hours ago, Domrockt said:

Did you consider those Aliexpress Erying Mainboard CPU Combos?

 

Aliexpress Erying

 

get a 11/12th gen low power and use that. Should be fine.

I am def gonna have to look further into this. Single core performance on 11th gen seems to be comparable with the 4600G but it looks like it uses just as much power under load. (looked at the 11600H specifically cause this was the only one in the same price range)

[BelgianNoise]

This whole power consumption thing is mainly because I want it to not waste power while idle.
Is there a nice solution so that I can just wake the pc easily ?
Can I make it wake up in intervals so it can check for new files to encode and just go to sleep if there isn't any new files ?

[Bersella AI]

I've heard that someone undervolted their 12th/13th gen Core processors then gained excellent efficiency. For example, in this SMZDM.com article (written in Chinese), an engineering sample of Core i9-12900F (QX7J), paired with a Jingyue-branded mobo, has been undervolted by 250 mV (but recommended -200 mV) in Core VID & Ring Bus. Then it was stressed by AIDA64, crunching only ~50W of power. Very impressive. Furthermore, these engineering samples & these lesser-known mobos can be grabbed in highly reasonable prices, albeit with defects on PCIe connectivity.

[Bersella AI]

5 hours ago, BelgianNoise said:

This whole power consumption thing is mainly because I want it to not waste power while idle.
Is there a nice solution so that I can just wake the pc easily ?
Can I make it wake up in intervals so it can check for new files to encode and just go to sleep if there isn't any new files ?

Personally speaking, it could be achieved by setting up a power schedule in such server, i.e. making it able to periodically turn on by itself. Once ready, it reads the rendering queue from the host, and processes them; once completed, it will turns off and wait until the next scheduled power-on.

[m9x3mos]

6 hours ago, BelgianNoise said:

Great idea! These mini PCs are sadly not that popular over here and the used marked is mainly intel 4th or 6th gen CPUs which is not really worth it performance wise imo. (These are around €200).
Looked a bit further into 8th gen too, single core performance leaves a lot to be desired.

Any of the t processors will be slower than their non t counter parts. This is partly where the power savings come from. 

The 8500t chip is 6 core though and any time I have used handbrake it does use all cores when transcoding with the cpu.

Since it has a built in gpu, depending on the codec, quick sync works for transcoding which is much faster and efficient then straight cpu encoding. 

[m9x3mos]

1 hour ago, Bersella AI said:

Personally speaking, it could be achieved by setting up a power schedule in such server, i.e. making it able to periodically turn on by itself. Once ready, it reads the rendering queue from the host, and processes them; once completed, it will turns off and wait until the next scheduled power-on.

I know with many consumer boards you can set a wake alarm for at least once a day to do that. 

I have a few Systems that boot daily like that and a task schedule to shut it back down to help make sure wake on Lan will still work after a power outage when I am away from home. 

[CerealExperimentsLain]

(Encoding Watts - Idle Wats) * Hours = watt hours used to do each encode, assuming the device would have otherwise been idle for that time.

 

Desktop: 87wh

 

NAS: 354wh

 

Laptop: 244wh

Your desktop uses far less watt hours to do an encode by a massive margin.  That margin would probably be vastly improved it that machine didn't have a big desktop GPU in it that's idle when the whole system is idle or when encoding.  

[Electronics Wizardy]

On 5/5/2023 at 9:55 AM, BelgianNoise said:

Desktop: Ryzen 9 3900X

• I only list CPU cause I only do software encodes because the goal is to not have Plex transcode on the fly
• Idle:  165 W

Whats the full hardware specs? The other stuff is probably using a good amount of power when idle.

 

Do you need h.265? AV1 currently offers better compression for a given amount of cpu time, or less cpu time for a given file size.

 

GPU encoding would be much more efficent

On 5/5/2023 at 9:55 AM, BelgianNoise said:

• I only list CPU cause I only do software encodes because the goal is to not have Plex transcode on the fly
•  

Im not sure what you mean here? GPU can create video files plex can use for direct play.

 

On 5/5/2023 at 9:55 AM, BelgianNoise said:

My ultimate goal is to have a folder I can just copy video into and the server to encode it automatically. I found that this is baked in in this docker image 'jlesage/handbrake' but docker on WSL 2 refuses to use a lot of CPU so it limits itself for a reason I could not figure out over the past days.

Id just use a little powershell script here, and ffmpeg. THen you can use gpu encoding/decoding here, and you don't have to deal with the performance hit of using a vm.

[BelgianNoise]

4 hours ago, Electronics Wizardy said:

Do you need h.265? AV1 currently offers better compression for a given amount of cpu time, or less cpu time for a given file size.

From what I know not many devices can handle AV1 so Plex would still need to transcode. Most people watch through a browser.

4 hours ago, Electronics Wizardy said:

m not sure what you mean here? GPU can create video files plex can use for direct play.

I tried encoding useing Nvidia NVENC but that did not yield great results unless I increased the bitrate by quite a lot.

[BelgianNoise]

4 hours ago, CerealExperimentsLain said:

Your desktop uses far less watt hours to do an encode by a massive margin

Looking at the numbers this way makes no sense at all.
My NAS' harddrives are gonna be running 8hr per day anyway and there is days I will only use my desktop for 1 hour, which would make me leave it running without actively using it. Meaning that these encodes are responsible for the whole 240W cause I would otherwise just put my desktop in sleep.

[BelgianNoise]

4 hours ago, m9x3mos said:

Since it has a built in gpu, depending on the codec, quick sync works for transcoding which is much faster and efficient then straight cpu encoding. 

I've been encoding on AMD now for years so not sure what QuickSync does exactly. Does it speed up the normal x265 encode ? Or does it also just encode differently leading to some devices not being able to play it without a re encode or transcode ?

[BelgianNoise]

4 hours ago, Electronics Wizardy said:

Whats the full hardware specs? The other stuff is probably using a good amount of power when idle.

3900X - GTX 1070 - some MSI X570 mobo with wifi - 4 x 8 GB - Corsair H150i - 2 x M.2 - 2 x HDD - 15 W worth of stupid RGB

 

4 hours ago, Electronics Wizardy said:

THen you can use gpu encoding/decoding here, and you don't have to deal with the performance hit of using a vm.

I am so down to use GPU encoding but I just can't get the same video quality with a comparable bitrate to the way I encode now. Could you link me some info about that ?

my bitrates right now for reference: 720p = 600 kbps | 1080p = 1200kbps. (I use avg bitrate + 2 pass and not RF values)
From what I read online this is on the low side but I found it works for me and it was comparable to most YIFY x265 releases.

[m9x3mos]

2 hours ago, BelgianNoise said:

I've been encoding on AMD now for years so not sure what QuickSync does exactly. Does it speed up the normal x265 encode ? Or does it also just encode differently leading to some devices not being able to play it without a re encode or transcode ?

It is hardware level h264 level encoding the same you would do with a graphics card would. It is far and above faster than using just cpu cores to do the encode that way. 

[CerealExperimentsLain]

17 hours ago, BelgianNoise said:

I am so down to use GPU encoding but I just can't get the same video quality with a comparable bitrate to the way I encode now. Could you link me some info about that ?

my bitrates right now for reference: 720p = 600 kbps | 1080p = 1200kbps. (I use avg bitrate + 2 pass and not RF values)
From what I read online this is on the low side but I found it works for me and it was comparable to most YIFY x265 releases.

You can't.  Hardware encoding is fast as heck and relatively low power for the work it performs, but it's not as efficient in terms of 'Quality Per Megabyte' as software encoding.  The hardware encoder on any GPU is a fixed ASIC that is built for speed and can't really be updated.  A software encoder can always be updated to improve things but the CPU is just not as fast.  If your objective is 'Quality Per Megabyte' your only option is software.

 

On the other hand, if this is plex, you could use real time hardware transcoding and just brute force the quality with high bandwidth.

[Captain Tight-Pants]

On 5/7/2023 at 11:00 AM, CerealExperimentsLain said:

You can't.  Hardware encoding is fast as heck and relatively low power for the work it performs, but it's not as efficient in terms of 'Quality Per Megabyte' as software encoding.  The hardware encoder on any GPU is a fixed ASIC that is built for speed and can't really be updated.  A software encoder can always be updated to improve things but the CPU is just not as fast.  If your objective is 'Quality Per Megabyte' your only option is software.

 

On the other hand, if this is plex, you could use real time hardware transcoding and just brute force the quality with high bandwidth.

Ditto what this person said.  GPU hardware video transcoding is designed and intended for fast, potentially real-time transcoding.  Works great for on-the-fly transcoding for Plex, and also for things like streaming to YouTube or Twitch.  However, if the goal is to save disk space without losing much quality then it can't compete with regular software CPU transcoding.

On 5/6/2023 at 5:32 PM, BelgianNoise said:

I've been encoding on AMD now for years so not sure what QuickSync does exactly. Does it speed up the normal x265 encode ? Or does it also just encode differently leading to some devices not being able to play it without a re encode or transcode ?

QuickSync is Intel's hardware video transcoding option, same idea as NVENC on Nvidia GPUs.  QuickSync is included on most consumer Intel CPUs, and I believe also the new Intel ARC GPUs.  You can definitely still use an Intel CPU that has QuickSync for regular software CPU transcoding, you just need to not select the QuickSync option for transcoding (I believe it's labeled as QSV in Handbrake)

 

As far as automation is concerned, I have looked into this a little bit for myself, and while I have not personally tried any of these yet, there are a few programs already created with this in mind.  HBBatchBeast, Tdarr, and Unmanic are all programs that are designed for bulk transcoding, potentially with multiple "nodes" handling the transcoding duties, so you could potentially have multiple machines transcoding at the same time to get your library converted faster.  I believe they are all cross-platform with Windows, MacOS, and Linux/Docker versions.

[BelgianNoise]

59 minutes ago, Captain Tight-Pants said:

HBBatchBeast, Tdarr, and Unmanic are all programs that are designed for bulk transcoding

I have looked into HBBatchBeast but it seemed rather rough, home projecty and unmaintained.
Def gonna check out the other ones, thanks a lot.

[tikker]

On 5/5/2023 at 6:55 PM, BelgianNoise said:

Edit: why tf is my idle power usage to high on my desktop ?

What do you consider idle in this scenario? If there are things like browsers and what not running the CPU, and maybe GPU will be used which may be preventing them from really going into an idle state. Ryzen 3000 also seems to just idle high (~50 W) from a quick google.

 

On 5/5/2023 at 6:55 PM, BelgianNoise said:

Sometimes I have a huge queue of encodes and I have to leave my PC on during the night and/or work which greatly increases the cost.

I would also like to move this workload of of my desktop so I don't need to go crazy on my CPU anymore when i buy a new PC.

(I understand that Handbrake does not use the 12 cores but I wanted to be able to also game and stuff on my PC while encoding)

Handbrake has options to put your PC to sleep or shutdown once the encode has finished under the "When done" settings section. It doesn't automate things completely like you are after, but that could at least solve the problem of idling for prolonged times. As your wattages show, with 240 W your desktop is the most efficient per encode at ~278 Wh per encode (add ~50 Wh if the NAS drives need to be active) while your NAS needs 570 Wh per encode. If we're talking about greatly increasing the cost then it could be worth considering that, going by the listed wattages, encoding with the NAS costs (almost) twice as much per encode compared to using your desktop.

 

It took me a hot minute to realise your "while in use" numbers try to isolate the encoding energy usage (I think?), but I think that is a confusing way to look at it. While it is true that for the NAS, for example, the encoding aspect only uses 66 Wh (more) over 12 hours of encoding vs. just being active with drives, it doesn't change the fact that there is still 47.5 W of NAS power draw active during that 12 hours so the true "while in use" energy consumption is 570 Wh per encode. Similarly your desktop will be using that 240-280 W for encoding whether you are using it or not, so the true energy consumption for a 1.16 h encode is 278-325 Wh (+ maybe 50 Wh for the NAS drives).

[10leej]

Honestly I'd just bite the bullet and let the Desktop do all the encoding, yes it's gonna take a while, but unless your encoding 24/7/365 and really you only have to run the encode once. I know you want something power efficient, but it might actually be more power efficient in the long run to use the more powerful system to just get the job done quicker. This depends on the amount of files of course that need encoded.

 

Another note, AV1 has a much better compression algorithm than x265 and it might actually be a quicker encode. If space is the concern for this then I'd use that codec.

[BelgianNoise]

1 hour ago, 10leej said:

Another note, AV1 has a much better compression algorithm than x265 and it might actually be a quicker encode. If space is the concern for this then I'd use that codec.

Yep, just tried that yesterday. Results looked promising, doesn't play through the browser tho. It flat out jsut errors.
 

 

1 hour ago, 10leej said:

Honestly I'd just bite the bullet and let the Desktop do all the encoding

def looking like the best option atm.
Edit: if luck is on my side i might be getting an ITX 3200G system for real cheap soon, that would solve the whole thing.

 

2 hours ago, tikker said:

As your wattages show, with 240 W your desktop is the most efficient per encode at ~278 Wh per encode (add ~50 Wh if the NAS drives need to be active) while your NAS needs 570 Wh per encode

I might have written it down really unreadable, my bad. But I am not sure why you are adding NAS drives to my desktops power consumption.
Also your trail of thought seems to assume that the main use of my desktop is encoding and me using the pc is the "free" power. But its the other way around, when I am using the pc anyways, the ~150W it is using anyway counts towards me gaming, browsing or whatever.

[10leej]

36 minutes ago, BelgianNoise said:

Yep, just tried that yesterday. Results looked promising, doesn't play through the browser tho. It flat out jsut errors.

Ah, sounds like the media server doesn't have hardware codec support for it for trans-coding. That's a shame.