Adobe Media Encoder - Multithreading

[MaverickCreative]

We just tested a transcode from Phantom4 Pro 4k H264 to Cineform 4k on 3 different machines in the office. 4790k 4c/8t 4ghz was the fastest. 8c/16t 3.6ghz  Ryzen 1800x next fastest. Xeon 24c/56t 2.2ghz slowest. What?! How do we get AME (Adobe Media Encoder) to use all available resources so we can offload transcoding to the most powerful machine? We are looking to invest in a dual AMD 7401 server for 3D rendering and I was hoping to use it to transcode also. With 96 threads it should blow through this, but our finding is that AME prefers clock speed over cores. Any thoughts or help appreciated. Willing to use alternative software as long as we can still use Cineform.

[TheGlenlivet]

12 minutes ago, MaverickCreative said:

We just tested a transcode from Phantom4 Pro 4k H264 to Cineform 4k on 3 different machines in the office. 4790k 4c/8t 4ghz was the fastest. 8c/16t 3.6ghz  Ryzen 1800x next fastest. Xeon 24c/56t 2.2ghz slowest. What?! How do we get AME (Adobe Media Encoder) to use all available resources so we can offload transcoding to the most powerful machine? We are looking to invest in a dual AMD 7401 server for 3D rendering and I was hoping to use it to transcode also. With 96 threads it should blow through this, but our finding is that AME prefers clock speed over cores. Any thoughts or help appreciated. Willing to use alternative software as long as we can still use Cineform.

https://forums.adobe.com/message/4688869#4688869

I saw you had the same post in the adobe forum too.  Did you see this?  It looks like there are other bottlenecks in the system with software or storage that limit the encoding speed.

[MaverickCreative]

I did read that post, but I'm still unsure why we can't utilize all the CPU cores to perform the task. Why can't media encoder prefetch several frames, decode/encode on multiple threads doing several frames at a time instead of "waiting" for a frame? Or am I misinterpreting the response?

[TheGlenlivet]

"If the Exporter is doing a compression scheme that is hard on the cpu (eg h.264) it will gobble up all the remaining cpu potential. Thus you'll see 100% on all CPUs. "

He says that h.264 should push the cores to the max and I have seen the same on my 8 thread machines, but I have only ever taken from, and written to sata SSD or NVME m.2 devices so I might have the best case scenario when it comes to read/write bottlenecks. 

He also created a test where he eliminated the discs on the output side by just dropping the encoded data, but I don't know all the details of how he did it, so maybe there is something in the software that limits the speed or thread count.

What are you using for storage of the original and output data?