ramdisk How can I boost ramdisk speed?

[ZachTheDoggo]

Hello, 

I currently want to know how to boost my RamDisk speed on Linux. I picked Linux since it has two ram functions builtin to the operating system. TMPFS and RAMFS. I have used the ramfs filesystem a while back on ddr4-3200mhz ram, and although fast, I would say speeds were comparable to one being on an SSD. I was expecting atleast 17gbs a second since DDR4 3200MHZ has the capability to go to 25.6GB, but the number I got was somewhat low and unexpected. 

 

1. Is it possible to achieve atleast half of the throughput for the ramdisk. I don't expect full performance but I do atleast expect it to compete with my nvme ssd.

2. What are some tips that I can do on Linux to achieve better throughput for data for the ramdisk.


Your probably wondering why do I need a ramfs or ramdisk, it's because I just want to see speed and eye watering performance numbers if possible. I don't plan using it for important data.

Edited October 16, 2023 by ZachTheDoggo
Title change

[Electronics Wizardy]

How are you testing these speeds? I have gotten about 80GB/s with tmpfs on linux on a 12900k + 5000(ish) ddr5.

 

There is a good chance your limited by something else, as its pretty hard to test high speed io at times.

[ZachTheDoggo]

21 hours ago, Electronics Wizardy said:

How are you testing these speeds? I have gotten about 80GB/s with tmpfs on linux on a 12900k + 5000(ish) ddr5.

 

There is a good chance your limited by something else, as its pretty hard to test high speed io at times.

I don't know the exact number, I was testing the speed using OS ISO Images, such as Windows 10, and Ubuntu, Linux Mint and I believe ubuntu server. I don't know the exact number I was getting but it did pause for 2 seconds to transfer the ISO's. At that time I did use a PC with an 8th gen i7 in the U series of chips and around 40GB of ram, it was not dual channel since one is soldered to motherboard, and one is upgradable. 
 

1. How can you benchmark your ramdisk in Ubuntu?

2. How did you create your ramdisk that way I can replicate your results. 

When you reply, i'm going to try and replicate the results on my new computer that has an i9 mobile chip and 32gb of DDR5 ram.

I do appreciate your help, 

Thank you 


 

Edited October 17, 2023 by ZachTheDoggo
Added I do appreciate your help & thank you message

[Electronics Wizardy]

3 hours ago, ZachTheDoggo said:

I don't know the exact number, I was testing the speed using OS ISO Images, such as Windows 10, and Ubuntu, Linux Mint and I believe ubuntu server. I don't know the exact number I was getting but it did pause for 2 seconds to transfer the ISO's. At that time I did use a PC with an 8th gen i7 in the U series of chips and around 40GB of ram, it was not dual channel since one is soldered to motherboard, and one is upgradable. 
 

1. How can you benchmark your ramdisk in Ubuntu?

2. How did you create your ramdisk that way I can replicate your results. 

When you reply, i'm going to try and replicate the results on my new computer that has an i9 mobile chip and 32gb of DDR5 ram.

I do appreciate your help, 

Thank you 


 

What do you mean by using ISO images? Are you copying them? Can you show the exact command you were using? What is your cpu usage when doing this? Do any cores get close to 100%

 

If your doing a copy your reading and writing, so thats will hurt speeds.

 

For my speeds, I was using fio to test disk speeds, and just make a tmpfs with default settings.

[ZachTheDoggo]

55 minutes ago, Electronics Wizardy said:

What do you mean by using ISO images? Are you copying them? Can you show the exact command you were using? What is your cpu usage when doing this? Do any cores get close to 100%

 

If your doing a copy your reading and writing, so thats will hurt speeds.

 

For my speeds, I was using fio to test disk speeds, and just make a tmpfs with default settings.

I was using copy paste and move file command in terminal. I was moving the files and copy and pasting the ISO's in the ramdisk. When I first performed the test I did so using an nvme ssd so to have a more accurate test I created two ramdisks and moved and copied and pasted files, I got similar results, but still wasn't impressed. 

I'm going to try and do this again but with my new laptop which has a lot more cores and can handle a lot more power and with the 5400mhz ram. I'll reply again when i'm done testing the ISO test on the new laptop.

If you can could you give me an example on how to use the fio command for the ramdisk? I just want to see what settings I need to configure to get a replica or similar results, it's also my first time using the command to benchmark. Also, if possible how much room did you give the ramdisk for your test and what would you recommend would be a good min or sweet spot?

P.S. I did create the ramdisk using mount, I don't know if this changes or impacts the ramdisk.

[Electronics Wizardy]

56 minutes ago, ZachTheDoggo said:

I was using copy paste and move file command in terminal. I was moving the files and copy and pasting the ISO's in the ramdisk. When I first performed the test I did so using an nvme ssd so to have a more accurate test I created two ramdisks and moved and copied and pasted files, I got similar results, but still wasn't impressed. 

I'm going to try and do this again but with my new laptop which has a lot more cores and can handle a lot more power and with the 5400mhz ram. I'll reply again when i'm done testing the ISO test on the new laptop.

If you can could you give me an example on how to use the fio command for the ramdisk? I just want to see what settings I need to configure to get a replica or similar results, it's also my first time using the command to benchmark. Also, if possible how much room did you give the ramdisk for your test and what would you recommend would be a good min or sweet spot?

P.S. I did create the ramdisk using mount, I don't know if this changes or impacts the ramdisk.

What speeds are you getting using the copy command?

 

I'm pretty sure this is a copy can't use the full speed or some other io issue, not the ram disk its self. Also is there a reason why this speed matters? Normally I wouldn't store iso images on a ram disk.

 

I think for fio I used

 

fio --name=ramdiskTest --direct=0 --bs=1M --numjobs=8 --filename=test.dat --size=8G --rw=read

 

Change the variables as needed for your config.

[ZachTheDoggo]

21 minutes ago, Electronics Wizardy said:

What speeds are you getting using the copy command?

 

I'm pretty sure this is a copy can't use the full speed or some other io issue, not the ram disk its self. Also is there a reason why this speed matters? Normally I wouldn't store iso images on a ram disk.

 

I think for fio I used

 

fio --name=ramdiskTest --direct=0 --bs=1M --numjobs=8 --filename=test.dat --size=8G --rw=read

 

Change the variables as needed for your config.

Just finished the test using copy and paste in debian xfce environment here are the results. I copied 2 Ubuntu Isos, one is the LTS Ubuntu and the other is Ubuntu Server. I did use the time and copy paste command.

 

NVME SSD: 

real 0m16.031s
user 0m0.005s
sys 0m15.700s

 

RAMDISK TO RAMDISK:

real 0m5.162s
user 0m0.008s
sys 0m2.604s

 

The results are even faster if I delete the previous files in the ramdisk and restart the test.

Also, your absoluetly right I wouldn't store iso files on a ramdisk either, I just chose ISOs since I can do a test on how much speed I can get the drive since an ISO file a lot of data depending on the OS.

The reason why i'm testing a ramdisk based solution is curiosity, i'm curious how much speed I can get with my ramdisk.

I did notice something concerning, when I ran the ramdisk. It was only utilizing one core at near 100 or 100% when transferring files to the ramdisk. Do you know if you can change this in settings of the ramdisk?

I'm going to try fio shortly as well. 

Edited October 17, 2023 by ZachTheDoggo

[Electronics Wizardy]

9 minutes ago, ZachTheDoggo said:

Just finished the test using copy and paste in debian xfce environment here are the results. I copied 2 Ubuntu Isos, one is the LTS Ubuntu and the other is Ubuntu Server. I did use the time and copy paste command.

How big is this file? What is your copying speed in GB/s

 

10 minutes ago, ZachTheDoggo said:

I did notice something concerning, when I ran the ramdisk. It was only utilizing one core at near 100 or 100% when transferring files to the ramdisk. Do you know if you can change this in Linux or no?

Was this a specific process or a kernel task using 100% of one core.

 

This sounds like a single thread limiting your speed.

 

 

[cmndr]

You're effectively CPU limited. 
The memory controller and whatever cores are in use can only crunch through data so fast. 

From a practical limit... you're probably fine with just a NAND SSD. If you need lower access times maybe buy a used optane drive off ebay - they're actually persistent. 

[ZachTheDoggo]

5 hours ago, Electronics Wizardy said:

How big is this file? What is your copying speed in GB/s

 

Was this a specific process or a kernel task using 100% of one core.

 

This sounds like a single thread limiting your speed.

 

 

6.7GB would be the file. Give me a moment to get the copying speed in GB/s. 

Nvme SSD to Ramdisk:

220MB/s - 240MB/s

 

Ramdisk to Ramdisk:

Stayed around 440's Mb's, hit a max of 575MB.

 

I preformed the test using 18 cores so far it is using one core only and it was the copy paste command causing the 100% of the core. I am using a VM for this task, I didn't give it any threads so far. Should I try and see if giving it more threads could help? 

Also, conducted the test using 8GB ramdisks since I have a limited 32gb of ram at the moment. I can do this test however on my old laptop which has 40GB DDR4 ram and see if that might help.

Edited October 18, 2023 by ZachTheDoggo
Added copying speed.

[ZachTheDoggo]

2 hours ago, cmndr said:

You're effectively CPU limited. 
The memory controller and whatever cores are in use can only crunch through data so fast. 

From a practical limit... you're probably fine with just a NAND SSD. If you need lower access times maybe buy a used optane drive off ebay - they're actually persistent. 

What's weird is I have an i9 CPU, It has 24 cores and 32 threads. That should be enough processing power for the Ramdisk, and I believe it is because if it was then why is it only using one of the cores out of the bunch that have either a low percentage or nothing at all. Now granted, i'm using 18 cores in a vm with no threads. Threading might be the issue, and I might retry the test using multithreads, if you guys suggest.

[Electronics Wizardy]

23 minutes ago, ZachTheDoggo said:

6.7GB would be the file. Give me a moment to get the copying speed in GB/s. 

Nvme SSD to Ramdisk:

220MB/s - 240MB/s

 

Ramdisk to Ramdisk:

Stayed around 440's Mb's, hit a max of 575MB.

 

I preformed the test using 18 cores so far it is using one core only and it was the copy paste command causing the 100% of the core. I am using a VM for this task, I didn't give it any threads so far. Should I try and see if giving it more threads could help? 

Also, conducted the test using 8GB ramdisks since I have a limited 32gb of ram at the moment. I can do this test however on my old laptop which has 40GB DDR4 ram and see if that might help.

Those speeds seem very slow.

 

What task is using up all the cpu?

 

 

[ZachTheDoggo]

17 minutes ago, Electronics Wizardy said:

Those speeds seem very slow.

 

What task is using up all the cpu?

 

 

I believe it was the copy and paste command for some odd reason, it only uses one core, and it goes to 100% and not using or atleast seems like its not using other cores.

I'll check again and verify.

 

Should I try the test again but this time enabling settings in virtualbox to use all cores + threads?

 

Edited October 18, 2023 by ZachTheDoggo

[Electronics Wizardy]

3 minutes ago, ZachTheDoggo said:

I believe it was the copy and paste command for some odd reason, it only uses one core, and it goes to 100% and not using or atleast seems like its not using other cores.

I'll check again and verify.

 

Should I try the test again but this time enabling settings in virtualbox to use all cores + threads?

 

Is this a vm? That can slow things down a reasonle amount. Try running this on the host.

 

What CPU do you have?

[ZachTheDoggo]

4 minutes ago, Electronics Wizardy said:

Is this a vm? That can slow things down a reasonle amount. Try running this on the host.

 

What CPU do you have?

Alright, i'll try it on a host. 

I have an i9-13950HX. 

Edited October 18, 2023 by ZachTheDoggo

[Ralphred]

Something to be aware of is the amount of ram that is used normally to complete a copy command. Try copying something to a known slow drive (like a usb stick in a low speed port) and see how long after the command completes a sync command takes to complete << this is the remainder of the ram cached data being written.

 

To benchmark raw speed you are better off using dd, and probably /dev/zero as a source if you are hitting a cpu bottleneck. There will also be "monitorable threads" related to the kernels fs processing if there is a bottleneck. There is a disk bench-marking tool too, though for the love of God I can't remember it's name, I just remember using it to test the ntfs-3g fuse vs ntfs3 drivers.

[ZachTheDoggo]

10 hours ago, Electronics Wizardy said:

Is this a vm? That can slow things down a reasonle amount. Try running this on the host.

 

What CPU do you have?

I'm going to try running it on a host later today again, I tried last night speeds were faster than on the VM and this is with my old computer as the host with DDR4-3200mhz, 40GB RAM and i7-8565u. 

I also ran fio command yesterday too I think my total was around 20GB/s and 16GB/s, going to try again later today. What's weird is I don't get nowhere close to that number in terms of moving and copying files.

Last night, I was also doing research online to see if people achieved such speeds on file transfers vs benchmarks. So far I haven't seen a lot of information about it. 

* When you created your ramdisk did you try moving files and copying files too?

I also want to make sure im creating the ramdisk correctly would this be the right usage?

mkdir /tmp/ramdisk

sudo mount -t tmpfs -o size=20g tmpfs /tmp/ramdisk

Like I said, i'll try again later in the day. But, I'm starting to think the realworld number for file transfers via ramdisk is low compared to benchmarking them due to limited information online.

Edited October 18, 2023 by ZachTheDoggo

[ZachTheDoggo]

5 hours ago, Ralphred said:

Something to be aware of is the amount of ram that is used normally to complete a copy command. Try copying something to a known slow drive (like a usb stick in a low speed port) and see how long after the command completes a sync command takes to complete << this is the remainder of the ram cached data being written.

 

To benchmark raw speed you are better off using dd, and probably /dev/zero as a source if you are hitting a cpu bottleneck. There will also be "monitorable threads" related to the kernels fs processing if there is a bottleneck. There is a disk bench-marking tool too, though for the love of God I can't remember it's name, I just remember using it to test the ntfs-3g fuse vs ntfs3 drivers.

Thank you, I appreciate it.

[cmndr]

16 hours ago, ZachTheDoggo said:

What's weird is I have an i9 CPU, It has 24 cores and 32 threads. That should be enough processing power for the Ramdisk, and I believe it is because if it was then why is it only using one of the cores out of the bunch that have either a low percentage or nothing at all. Now granted, i'm using 18 cores in a vm with no threads. Threading might be the issue, and I might retry the test using multithreads, if you guys suggest.

I'm going to preface this with "gamers" took the term "bottleneck" and bastardized it. All bottleneck means is that for one task in a given time frame a part is a limit. 
You could have the fastest CPU in the world and it would STILL be the bottleneck in a cinebench benchmark run. 

----

Your CPU could be MUCH faster and it'd still be the limit for this. It's not designed to maximize this type of task, it's designed to do real world things. 

Your system (and essentially ALL von-neumann architecture designs - aka basically every computer and phone in use) doesn't have "in memory compute" and the data effectively needs to be shuffled around and restructured to be usable. This is relatively slow. The number of cores doesn't matter. Clock speed barely matters. It's going to be a question of how close the trace lengths are, how aggressive the memory controller is and how well structured the cache is. A CPU would need to be fundamentally rearchitected to NOT be the bottleneck in this use case. 


As an aside, if you set up something like PrimoCache or use ZFS, you'll get most of the benefits of a RAMDisk with a lot less jank. Getting meta-data (think the location of blocks of data) and the hottest blocks into RAM makes SSDs (especially HDDs) run a lot faster.