How memory management in Windows (and others) work

Mira Yurizaki · September 14, 2016

A random, perhaps somewhat useless topic on the topic of how modern operating systems manages memory.

So first a primer on the history of memory management.

Spoiler

In the beginning, memory was mapped directly. Each byte could be addressed individually and the amount of memory you could have in the system was determined by the processor's address size. So an 8-bit address size means there are 256 bytes of addressable memory. 16-bits would give you 65,356 bytes (64KB) of addressable memory. 24-bits got you roughly 16 million bytes (16MB). And 32-bit gets you roughly 4 billion bytes (4GB).

Memory addressing however means any and all memory. The 8-bit Guy in his explanation of cartridges pointed something out that I wasn't aware of: older computers would not have the maximum amount of memory installed on the computer. A MOS 6502 has a 16-bit memory address space, but most early computers only stuck in somewhere between peanuts for RAM (like 128 bytes on the Atari 2600) to 8K. In addition to the system ROM, this still left a hole in the memory space. The cartridge would take up part of this hole and the CPU could address the memory directly in the cartridge.

Anyway the video's pretty cool so spend the time watching it.

As computers got more powerful, RAM more plentiful, and people wanting to do more stuff, there came two problems: Memory fragmentation and programs able to access anything in memory, even though it shouldn't.

Memory fragmentation is similar to disk fragmentation. As memory gets used and freed up, it can leave "holes". Let's consider the following:

The boxes in blue are free memory spaces. When memory is freed up in the second line, a program that wants four or less spots can occupy the "hole" left behind. But if a program wants five spots, then the processor won't let it. The program will then think there's not enough memory. But once you defragment the memory, like in the third line, those five spots can be used. The other issue of programs accessing other program's memory spaces is probably more obvious as to why this is a problem. Both of these problems had their own solution to them.

Fragmentation was solved by a process known as memory paging. With this, memory is divided up into pages which for most processors is 4KB. When a program wants memory, the processor will allocate at least just enough pages to satisfy the program's memory requirement. So yes, even if a program wants only 500 bytes of memory, the processor will give it the full 4KB page. While in a technical sense this still can create fragmentation (imagine in the picture above, those blocks were 4KB chunks instead of one byte), it has a perk. For example, if the orange blocks and green blocks were part of the same program, the last orange block could have a byte that says "go here for the next page". Or there could be a table that has which programs own which page. 4KB pages is a lot less to manage than single bytes.

Preventing programs from accessing each other's memory space was solved by a system known as virtual memory. In this, the hardware makes the software except the OS itself believe that the entire address space is available, even if physically it's not. In a 32-bit system, the software believes there's 4GB of memory addresses, even if there's 2GB of RAM installed. The OS will assign virtual addresses to the programs and keeps a table of how this maps into physical addresses. A quirk with this is that when a program tries to access a virtual address it wasn't assigned, it doesn't get resolved to a physical address and so the program throws an error about it (this is known as a segmentation fault)

Both paging and virtual memory are handed by a memory management unit (or MMU) and managed by the operating system.

Today most of the address space is taken up by system RAM (which is why everything these days loads to RAM). However, some devices can eat into this address space as well. For example, the BIOS/UEFI firmware is a ROM that takes up some memory (probably the first 640KB, because IBM PC compatibility shenanigans). If the system supports memory mapped IO (MMIO), then peripherals like the PCI bus and video card memory can also take up memory space. This is why when you have 4GB installed on a 32-bit system, some of the RAM is unavailable. The system lets the other hardware take those addresses.

That's enough history, let's talk about how memory management works! (or the burning question: why shouldn't I turn off the page file when I got like a bazillion GB of memory?)

Spoiler

When a program requests memory, the operating system generally gives it more than it requested. For example, if the program requested 3KB of space in memory, the OS will reserve 4KB, since this is one page. However, the program may not be able to see the extra 1KB it got or depending on how it was programmed, may not make use of it (this is a good read about this issue: https://blogs.msdn.microsoft.com/oldnewthing/20120316-00/?p=8083/). Another thing is that a program can go ahead and reserve space ahead of time in the virtual memory system. In Windows this is called a "commit", as in, Windows promises to give you this memory when you need it.

So the reason why it's a bad idea to turn off the page file is a lot of programs like to make a memory commit. A memory commit means they have that space in RAM, but they may not actually use it. You can end up in a situation where you can have most of your RAM committed, but not all of that committed memory is in use. Let's take this for example:

Windows says 6.2GB of memory has been committed, but only 4.8GB is actually in use. Also notice that the 6.2GB is out of 18.3GB, and I have 16GB of RAM installed. This means that Windows created a page file of at least 2.3GB in size (which it has, as I found a 2GB page file on my D: drive and a 0.5GB page file on my C: drive).

Let's say I only have 8GB of RAM installed and no page file. This means if a program wants to commit 2GB of memory space, it can't (6.2 + 2 = 8.2GB, oops) and it'll throw an "out of memory error". If it manages to squeeze in by not committing that much but still fill up the 8GB of RAM, then if other programs start using memory and want to commit more, they'll start throwing "out of memory" errors. Windows isn't going to let the programs use the space reserved but not in use because the OS promised those programs it would have that space ready for them. It would be like if you reserved a table for 10 at a restaurant but you show up with five (the general rule in the US being half the party must be there before they let you in). If the restaurant decided to fill in those five empty seats for other waiting customers, you'd be pretty ticked off.

While you might get away with not having a page file with 16GB of memory, games are consuming more memory. And with those who like to have dozens of Chrome tabs up... Well you get the idea. So leave the page file up. If you have a page file, then these unused portions of committed memory get paged out and programs can have their data in RAM. Paging out an unused commit is a very cheap operation because there's no data to copy, only the page table gets updated. If you are running low on available RAM space and data in RAM starts paging out, that's when you have a problem.

In any case, what's with all the other things in this tab about my memory usage? Operating systems like to mark portions of memory depending on what's going on with the program that's using it. Windows marks RAM usage in this way:

Non-paged pool: This is data that will not be paged out. It's usually core operating system components.
Paged pool: Data that can be paged out, or is paged out.
Committed: How much memory is reserved and how much memory can be reserved. How much can be reserved can be expanded as necessary.
In use: How much memory is actively being used by programs.
Cached/Standby: Data that was in use by programs. If a program needs that data, the OS can quickly put it back into service. For example, if have an email client that you've closed, then a few minutes later open it up again, it will appear to load faster because the memory the email client used was only converted to "Cached." Windows will just convert the memory back into "In use". However, if programs need memory and there's not enough free space to do so, it will start overwriting cached memory instead.
Free: RAM that has not been touched at all.
Available: How much free space and cached space is available in RAM. This is physically what's available for commits.

Mac OS has a similar setup, but is a bit simplified (though probably for the user's benefit)

Wired: Data in memory that cannot be paged out. Similar to the non-paged pool in Windows.
Active: Memory that's actively being used by programs.
Inactive: Memory that's not being used by programs. It's the same as "Cached" in Windows.
Free: RAM that has not been touched at all
VM Size: Similar to the second number in "Committed" in Windows. Mac OS X tends to have a large amount of VM size (but it won't actually use this space up unless it needs to)

Linux has various tools to look at memory, but the popular ones just use "Free", "Used" and stats about the swap file, where data from RAM goes into the storage. A quirk about Linux is that it always "reserves" memory even if there's not enough in virtual memory, a feature known as "overcomitting". Basically when a program requests memory from Linux, it'll go "Here you go!" but when the program actually goes to use it, there's a problem because that memory space doesn't actually exist. It's like if there are 100 locker spaces available, each numbered sequentially. Linux will say to a program "Here, you can have locker 200", but when the program goes to find it, they'll find it doesn't exist.

Some other reading:

https://blogs.msdn.microsoft.com/oldnewthing/20091002-00/?p=16513

https://support.apple.com/en-us/HT201538

https://www.etalabs.net/overcommit.html

SecGuy · September 14, 2016

Looks like someone is an upcoming computer scientist

Good post.

Mira Yurizaki · January 8, 2017

Time to bump this old thing to further explain why you shouldn't disable the page file. So before I make this point, let me summarize what happens when applications use memory:

An application reserves memory to the OS by making a commit.
If there's enough available memory, the OS will grant that commit size to the application.
The commit however, is merely a reservation. The OS will allow the application to use it when it wants, but no sooner.
The OS will not use an application's committed memory pool for anything else.

So I decided randomly to check out Skyrim SE for it's commit vs. actual use. This is a vanilla Skyrim SE build and at a point when you first reach Riverwood

Skyrim has made a 4.2GB commit, but it's actually only using about 2GB. Skyrim is leaving a 2GB hole of memory that is not being used and cannot be used. If I disable the page file, this means Skyrim is effectively using all 4.2GB, even though it's really only using half of it.

So why would games request way more memory than they're using? Performance reasons. Committing memory can be a costly process. If the game is constantly asking the OS for memory, it would slow down.

h3F8LV4R2Ngf · January 9, 2017

You may wish to revisit the sections on ~~Mac OS~~ macOS and Linux: your descriptions of their workings can fairly be described as "simplified to the point of incorrect". While I'm less sure how critical to be of your Windows description, your terminology and overall description differ materially from the Russinovich, Solomon, and Lonesec's in Windows Internals, 6th Ed. Part 2, Chapter 10.

The Chimp · February 26

On 9/14/2016 at 6:45 AM, Mira Yurizaki said:

A random, perhaps somewhat useless topic on the topic of how modern operating systems manages memory.

So first a primer on the history of memory management.

Reveal hidden contents

In the beginning, memory was mapped directly. Each byte could be addressed individually and the amount of memory you could have in the system was determined by the processor's address size. So an 8-bit address size means there are 256 bytes of addressable memory. 16-bits would give you 65,356 bytes (64KB) of addressable memory. 24-bits got you roughly 16 million bytes (16MB). And 32-bit gets you roughly 4 billion bytes (4GB).

Memory addressing however means any and all memory. The 8-bit Guy in his explanation of cartridges pointed something out that I wasn't aware of: older computers would not have the maximum amount of memory installed on the computer. A MOS 6502 has a 16-bit memory address space, but most early computers only stuck in somewhere between peanuts for RAM (like 128 bytes on the Atari 2600) to 8K. In addition to the system ROM, this still left a hole in the memory space. The cartridge would take up part of this hole and the CPU could address the memory directly in the cartridge.

Anyway the video's pretty cool so spend the time watching it.

As computers got more powerful, RAM more plentiful, and people wanting to do more stuff, there came two problems: Memory fragmentation and programs able to access anything in memory, even though it shouldn't.

Memory fragmentation is similar to disk fragmentation. As memory gets used and freed up, it can leave "holes". Let's consider the following:

The boxes in blue are free memory spaces. When memory is freed up in the second line, a program that wants four or less spots can occupy the "hole" left behind. But if a program wants five spots, then the processor won't let it. The program will then think there's not enough memory. But once you defragment the memory, like in the third line, those five spots can be used. The other issue of programs accessing other program's memory spaces is probably more obvious as to why this is a problem. Both of these problems had their own solution to them.

Fragmentation was solved by a process known as memory paging. With this, memory is divided up into pages which for most processors is 4KB. When a program wants memory, the processor will allocate at least just enough pages to satisfy the program's memory requirement. So yes, even if a program wants only 500 bytes of memory, the processor will give it the full 4KB page. While in a technical sense this still can create fragmentation (imagine in the picture above, those blocks were 4KB chunks instead of one byte), it has a perk. For example, if the orange blocks and green blocks were part of the same program, the last orange block could have a byte that says "go here for the next page". Or there could be a table that has which programs own which page. 4KB pages is a lot less to manage than single bytes.

Preventing programs from accessing each other's memory space was solved by a system known as virtual memory. In this, the hardware makes the software except the OS itself believe that the entire address space is available, even if physically it's not. In a 32-bit system, the software believes there's 4GB of memory addresses, even if there's 2GB of RAM installed. The OS will assign virtual addresses to the programs and keeps a table of how this maps into physical addresses. A quirk with this is that when a program tries to access a virtual address it wasn't assigned, it doesn't get resolved to a physical address and so the program throws an error about it (this is known as a segmentation fault)

Both paging and virtual memory are handed by a memory management unit (or MMU) and managed by the operating system.

Today most of the address space is taken up by system RAM (which is why everything these days loads to RAM). However, some devices can eat into this address space as well. For example, the BIOS/UEFI firmware is a ROM that takes up some memory (probably the first 640KB, because IBM PC compatibility shenanigans). If the system supports memory mapped IO (MMIO), then peripherals like the PCI bus and video card memory can also take up memory space. This is why when you have 4GB installed on a 32-bit system, some of the RAM is unavailable. The system lets the other hardware take those addresses.

That's enough history, let's talk about how memory management works! (or the burning question: why shouldn't I turn off the page file when I got like a bazillion GB of memory?)

Reveal hidden contents

When a program requests memory, the operating system generally gives it more than it requested. For example, if the program requested 3KB of space in memory, the OS will reserve 4KB, since this is one page. However, the program may not be able to see the extra 1KB it got or depending on how it was programmed, may not make use of it (this is a good read about this issue: https://blogs.msdn.microsoft.com/oldnewthing/20120316-00/?p=8083/). Another thing is that a program can go ahead and reserve space ahead of time in the virtual memory system. In Windows this is called a "commit", as in, Windows promises to give you this memory when you need it.

So the reason why it's a bad idea to turn off the page file is a lot of programs like to make a memory commit. A memory commit means they have that space in RAM, but they may not actually use it. You can end up in a situation where you can have most of your RAM committed, but not all of that committed memory is in use. Let's take this for example:

Windows says 6.2GB of memory has been committed, but only 4.8GB is actually in use. Also notice that the 6.2GB is out of 18.3GB, and I have 16GB of RAM installed. This means that Windows created a page file of at least 2.3GB in size (which it has, as I found a 2GB page file on my D: drive and a 0.5GB page file on my C: drive).

Let's say I only have 8GB of RAM installed and no page file. This means if a program wants to commit 2GB of memory space, it can't (6.2 + 2 = 8.2GB, oops) and it'll throw an "out of memory error". If it manages to squeeze in by not committing that much but still fill up the 8GB of RAM, then if other programs start using memory and want to commit more, they'll start throwing "out of memory" errors. Windows isn't going to let the programs use the space reserved but not in use because the OS promised those programs it would have that space ready for them. It would be like if you reserved a table for 10 at a restaurant but you show up with five (the general rule in the US being half the party must be there before they let you in). If the restaurant decided to fill in those five empty seats for other waiting customers, you'd be pretty ticked off.

While you might get away with not having a page file with 16GB of memory, games are consuming more memory. And with those who like to have dozens of Chrome tabs up... Well you get the idea. So leave the page file up. If you have a page file, then these unused portions of committed memory get paged out and programs can have their data in RAM. Paging out an unused commit is a very cheap operation because there's no data to copy, only the page table gets updated. If you are running low on available RAM space and data in RAM starts paging out, that's when you have a problem.

In any case, what's with all the other things in this tab about my memory usage? Operating systems like to mark portions of memory depending on what's going on with the program that's using it. Windows marks RAM usage in this way:

Non-paged pool: This is data that will not be paged out. It's usually core operating system components.

Paged pool: Data that can be paged out, or is paged out.

Committed: How much memory is reserved and how much memory can be reserved. How much can be reserved can be expanded as necessary.

In use: How much memory is actively being used by programs.

Cached/Standby: Data that was in use by programs. If a program needs that data, the OS can quickly put it back into service. For example, if have an email client that you've closed, then a few minutes later open it up again, it will appear to load faster because the memory the email client used was only converted to "Cached." Windows will just convert the memory back into "In use". However, if programs need memory and there's not enough free space to do so, it will start overwriting cached memory instead.

Free: RAM that has not been touched at all.

Available: How much free space and cached space is available in RAM. This is physically what's available for commits.

Mac OS has a similar setup, but is a bit simplified (though probably for the user's benefit)

Wired: Data in memory that cannot be paged out. Similar to the non-paged pool in Windows.

Active: Memory that's actively being used by programs.

Inactive: Memory that's not being used by programs. It's the same as "Cached" in Windows.

Free: RAM that has not been touched at all

VM Size: Similar to the second number in "Committed" in Windows. Mac OS X tends to have a large amount of VM size (but it won't actually use this space up unless it needs to)

Linux has various tools to look at memory, but the popular ones just use "Free", "Used" and stats about the swap file, where data from RAM goes into the storage. A quirk about Linux is that it always "reserves" memory even if there's not enough in virtual memory, a feature known as "overcomitting". Basically when a program requests memory from Linux, it'll go "Here you go!" but when the program actually goes to use it, there's a problem because that memory space doesn't actually exist. It's like if there are 100 locker spaces available, each numbered sequentially. Linux will say to a program "Here, you can have locker 200", but when the program goes to find it, they'll find it doesn't exist.

Some other reading:

https://blogs.msdn.microsoft.com/oldnewthing/20091002-00/?p=16513

https://support.apple.com/en-us/HT201538

https://www.etalabs.net/overcommit.html