Richarch Sincovec shares some of what he has learned from researching the Windows XP paging file system as it relates to a Windows XP-based digital audio workstation.
by Richard Sincovec
Original Source: http://www.well.com/~sinc/DAW_Paging.html
Copied from Original Source with Richard's permission by Bob Hammond (Techead) (See SONAR Forum thread)
Version 1.02 - Last Updated August 12, 2005
A search of the web will reveal that there is a consistent recommendation to turn off the paging file (pagefile.sys) for Windows XP systems used as Digital Audio Workstations (DAWs). Does turning the pagefile off actually help DAW reliability or performance in any way, or is this merely an oft-repeated myth? There is a lot of disagreement on this subject The basis for the confusion seems to be that having the pagefile on CAN cause disruption to the audio stream. But should you turn it off? Probably not. To understand why, it is necessary to understand some of the hidden workings of the Windows XP kernel.
In the early days of home computers, programs had to manage their own memory. The program would "reserve" specific memory addresses for various purposes, and then read and write directly into that memory. This worked fine for computers with 8-bit CPUs and 16KB (that's KILObytes) of RAM, but as computers became more powerful, problems began to arise.
First off, as memory prices came down, people began using more and more RAM in systems. Also, now that computers had more power, people wanted to be able to run multiple programs at once. Not to mention, processors and motherboards were improved, and ran much faster. Now people wanted to be able to use their faster processors to run multiple programs at the same time. A program could no longer count on running in the same place every time, and could not explicitly reserve specific memory addresses - some other program might already be using those memory addresses. Things were turning into a real headache. It became obvious that the operating system needed to take an active role in all this memory management stuff.
Memory management has grown and improved considerably over the years. This paper deals with memory management as it currently exists in Windows XP. Memory management was different in earlier versions of the operating system, and will undoubtedly be different in later versions, but this is how things currently stand (in 2005).
In Windows XP, programs do not directly address memory. Each program running on a Windows XP system runs in its own little logical world, known as a ''process''. The size of this little logical world expands as needed. When a program is launched, Windows creates a process for the application, which means (among other things) that it reserves a section of physical memory for the program. Windows then loads the program into this memory and starts running it. When it starts, the program tells Windows how much memory it thinks it will use for data, and Windows also reserves a portion of the physical memory for this data. When the program accesses the data, it refers to the data relative to the start of the data block, and Windows converts this address into the actual physical address of the RAM. The program never directly addresses a memory register, and in fact, never really knows where in RAM anything is physically stored.
Actually, when executing code, the CPU never directly addresses the primary RAM in a system, either. Instead, the CPU runs off a smaller piece of RAM known as the "cache". The cache is very fast RAM located right near (or integrated with) the main CPU, and it the only RAM the CPU ever directly accesses. RAM is divided into 4KB chunks known as "pages". When the CPU needs to access a certain page of RAM, it first checks to see if the page is already in the cache. If not, it loads the appropriate page from the main RAM into the cache. Copying needed code from main memory into the cache is called a ''soft page fault''. It causes a slight delay in processing, which is why larger amounts of cache memory can result in better performance of the CPU. Using memory in this way is called using ''virtual memory''.
Now that the OS is using virtual memory, though, we're free to do some further optimization tricks. First off, we can continue the original idea of "windowing memory", except instead of using additional RAM to hold the "inactive" memory, we'll use the hard disc as "overflow storage" for RAM. We'll set aside a special part of the hard drive to use as "swap space". In Windows XP, this special part of the disc is a file named ''pagefile.sys''. Now, when the system starts to get low on memory, it can copy the least-recently-used data in RAM to disc. This RAM is now free to be used again. Then, later, when the data is needed again, it can be reloaded from disc into RAM. This is called a ''hard page fault''. Since hard page faults require accessing the disc, and disc access is much slower than RAM access, an excessive number of hard page faults are undesirable.
Windows XP takes this optimization further, though, and also swaps code in and out of RAM. Originally, when a user wanted to run a particular program, the entire program needed to first be loaded into RAM. Early versions of Windows introduced a concept called a ''dynamic link library'', or DLL. Essentially, this allowed the developer to split code into chunks, and these chunks could be loaded as needed. Multiple programs could even share the same DLL. This allowed for much more efficient use of both RAM and disc space, sped up launching and perceived performance of applications, and made programs more resilient by encouraging the use of object-oriented principles in code design. But Windows XP can also ''page'' code, just like it pages data. If a program loads a DLL and then never uses it (or doesn't use it for a long time), Windows will mark it as a candidate to be paged out. Since the code already exists on disc, it is NOT copied out to the pagefile - that would be a waste of resources. But this implies we have two different types of paging going on: paging of data, and paging of code. Data is paged to/from the pagefile, while code is paged directly from the executable file.
There is an important point here. Since there are two types of paging going on - data and code - disabling the pagefile IS NOT the same thing as disabling page faults, or disabling the virtual memory system. Page faults are integral to the functioning of virtual memory. When you load an application, that application is loaded into memory by a page fault. So, as you can see, page faults are not necessarily bad - the goal is not to eliminate page faults. The goal is to minimize page faults, and try to prevent them from happening unnecessarily (especially during a recording session). In particular, we are worried about ''hard'' page faults - we don't really care much about the soft ones. Soft page faults are so fast that they aren't worth worrying about. A hard page fault, however, will pull the heads of the disc drive away from our audio data (assuming we only have one hard drive), and can disrupt the flow of audio data. This is bad.
OK, so how do we minimize page faults, and barring that, what can we do to decrease chances that a page fault will cause a disruption to our recording? The best answer is simple: ''install more RAM''. If the computer never runs out of RAM, then paging WILL NOT HAPPEN. Pagefile.sys will remain basically untouched, and there will be no issues. Luckily, digital recording programs do not go through lots of RAM while they are running. In a recording situation, the DAW streams audio directly between the hard disc and the audio interface; it does not store that data in memory. This means you will not run out of RAM during a recording session, and the DAW software will not request more RAM during the session. So, as long as the DAW is the only software running on your system, and all your DAW software can run easily in the available physical RAM, and your software does not suffer from memory leaks that eventually crash your computer, there will never be a paging request during recording.
Keep in mind that the softsynths and effects that you use count as separate programs. Each one of these allocates its own block of memory. But, as with the main DAW, these units are designed for real-time data processing. That means that they do not constantly request more memory. When they are started, they generally request all the memory they will ever use (assuming there are no coding errors that cause memory leaks, of course). That means that all required memory is allocated BEFORE you ever hit the record button. When you hit record, it doesn't matter how long you record - the system will not require more memory for the duration. It will usually only require more memory if you make some configuration change, such as add more softsynths or effects, or start up some other program, such as an external wave editor. As long as the total memory required by all your sound software is below the available memory, there will not be any paging. 1GB of RAM seems to be sufficient for Cakewalk's Sonar 4 program to run a significant number of "normal" effects and softsynths with no paging.
But what if paging occurs? For example, what if you are using a large soundfont or sample bank that takes up a lot of memory, and/or you are synching audio and video, and/or you are running external programs such as additional sound editors (or web browsers or whatever). You may trigger paging. Is this bad? The answer is a very definite ''maybe''.
As an example, here's what might happen if you have your DAW up, and then go into a graphics editing program. Let's say you do a bunch of graphics editing - so much so, that you use up all physical RAM and trigger paging. Your DAW software might be paged out to make room for your graphics. Now, without closing the graphics program or saving anything, you go back into the DAW. But now the DAW software needs to be swapped back in to physical RAM, and something else needs to be swapped out.
In order to understand what happens next, we need to look at the concept of ''memory pools''. We already saw that memory used for code and memory used for data is paged differently. But memory used for code is also split into separate groups called ''pools''. Data that is loaded from disc and not changed is in one pool, while data that is changed (or new data that has been created) is in another pool. When we switch back from our graphics program to our DAW software, the processor needs to immediately swap the DAW back into physical RAM. It will do this as fast as absolutely possible, by freeing up some other RAM that is "ready for swapping". By "ready for swapping", I mean that that memory is already stored on disc somewhere. Windows constantly tracks memory usage, and it uses this information to determine the data that is least likely to be needed any time soon. That RAM is "zeroed out" (covered over by all 0's) to make sure old data doesn't accidentally get mixed in with new, and then our DAW software is swapped back in. This can cause a brief, intense flurry of paging, easily enough to disrupt the DAW if it is actively recording. (This is why you may get dropouts if you switch between programs while recording is going on.)
This isn't much of a problem, though - it is a quick burst of paging in response to a specific user action, and that's it. But what happens next is the potential problem.
The graphics program has a lot of unsaved data in it. Since the graphics program is no longer the active process, Windows view it as less important. Since it is less important, it becomes a candidate for swapping if the active process should need more memory. But there's a problem - when the system needs more memory, it generally needs it NOW. If it has to wait for the system to write data in the memory out to disc, then zero it out and make it available for use, it can cause a noticeable delay. So, when the graphics program becomes inactive, the worker thread will start slowly writing the unsaved data to pagefile.sys. This is just a small trickle of data, usually not noticeable (except for the slight flickering of the hard drive activity light). However, this can actively hurt your DAW. If you have only one physical hard drive, then every write to the page file pulls the read/write head away from the audio data and over to pagefile.sys, and each time this happens, there is potential for clicking or dropout. This background writing (which I'll call "backwriting" - my own invented term) will continue until the complete "changed pool" of data in the inactive thread has been written to the pagefile, then it will stop. This can take a few seconds, or many seconds, depending on how much data is in the pool. And the whole time, the DAW is in danger of audio stream disruption.
From this, it may appear that turning off the pagefile would be the way to go. But actually, the answer is not really.
First off, backwriting only happens when the system has run out of physical memory and started paging. If you have reached this point, then turning off the pagefile.sys will actually harm you. If you have the pagefile turned off and you run out of physical memory, then instead of paging, you get "Out of Virtual Memory" errors and application crashes. Paging ''may'' impact your recording; an application crash will ''definitely'' impact your recording.
There is another factor. Remember, when a program starts, it tells Windows how much memory it expects to use. If it runs out of memory and needs to request more, it is a very expensive operation. Therefore, most programs will request more memory than they will ever use. If the pagefile is disabled, then this memory is locked up by the program, regardless of whether or not the program ever uses it. However, if Windows is allowed to use a pagefile, then it will notice that the process is not using all its allocated RAM, and it will swap those unused pages out to make them available for other processes. Because of this one fact, you may be able to load many more effects and softsynths if you have paging enabled than if you do not. By disabling the pagefile.sys, you have crippled Windows and made it less-responsive and less-able to respond to bursts in resource usage. Consequently, you have reduced its perceived performance.
If you should add enough effects and softsynths to trigger paging, be aware that you may go through a period where backwriting is occurring, as mentioned earlier. So, if you do something that will trigger paging, you may want to give your DAW a few seconds (or a minute) to "settle down" before you start recording. When the hard drive light stops flickering, the backwriting operation is done, and you may start recording without worry. But even if you start recording while backwriting is occurring, it might not actually cause you any issues at all. I tried a test run on my system with a single 7200 RPM internal hard drive, using a project with 12 tracks playing, 8 tracks recording, and some 30+ softsynths and effects running. First, I tried recording with no backwriting occuring. Drive usage flickered pretty constantly around 5-10%, with a couple of spikes up to 40%. Then I started up a graphics program with a large picture, doubled the resolution of the picture, and applied an effect. This used a lot of memory. When I switched back to the DAW, there was the initial flurry of paging, then backwriting started at its normal pace. This time, there seemed to be a slight increase in disc usage (no more than 3%), but there were a good twice as many spikes, and the spikes went slightly higher. There was no disruption to the audio, but with a less-responsive drive or more tracks, there might have been.
So, the only time you are likely to have disc disruptions caused by paging is if (1) you have a single-drive system, and (2) you are pushing the limits of your disc transfer rate, due to a slow disc drive and/or having a very large number of active tracks, and (3) you have active backwriting going on. If you are running on a system that is sensitive to backwriting, then all you need to know is when it occurs, and how to avoid triggering it or how to wait for it once it's triggered. There is no need to turn the pagefile off. And of course, you should consider upgrading your RAM (so that paging doesn't occur in the first place) and/or adding a second physical drive dedicated to audio only (so that even when backwriting occurs, it does not affect flow of the audio stream).
If you are running a computer with a relatively weak CPU, or are running softsynths or effects that really tap your CPU, you may still be tempted to turn off the pagefile. However, this doesn't really gain you anything.
Remember, the pagefile and faulting are merely aspects of the overall memory management system. The purpose of the memory management system is to move data between disc, RAM, and cache. If you disable the pagefile, you throw off the delicate balance of this system, and reduce its efficiency. You may save a minimal amount of CPU because the background system thread that does backwriting will have less to do. But this thread hardly hits the CPU at all - it isn't even noticeable in the CPU. And the virtual memory management system itself CANNOT be turned off - if that were to happen, the entire computer would completely cease to function. When you turn off the pagefile.sys file, you are merely removing options for the OS, and reducing its choices for how to manage its resources. This can HURT performance much more than it can help it.
The answer is simple: it depends on your usage. The minimum pagefile size should probably not be set any smaller than 100MB. Also, the sum of RAM and pagefile storage should total at least 500MB - in other words, if you have 256MB of RAM on your system, set the pagefile to at least 250MB. Other than that, the minimum pagefile size should only be big enough to handle any demands you regularly place on your system - any larger, and you are basically wasting disc space. Contrary to popular belief, there is no need to keep the size of the pagefile larger than the size of installed RAM, or set it to 1.5x the size of RAM, or anything else. The best setting is completely dependent on YOUR usage.
One way to determine the best setting for your computer is simple: set your minimum pagefile size to some small value, like 100MB, and set the maximum pagefile size to some large number, like 2048MB. Then use your computer for a while. Occasionally check the size of the pagefile, using Windows Explorer. If the pagefile suddenly gets bigger at any point, Windows expanded it. Increase the minimum pagefile size by about the same amount. Repeat the process, slowly increasing the minimum pagefile size until your pagefile size remains constant. (Tip: if you set your pagefile size to 100MB and it doesn't get any larger as you use your computer, that is a good indication that adding more RAM to your system may not help performance of your DAW in any way.)
Another way is to determine pagefile settings is to download a utility like the [http://billsway.com/notes_public/WinXP_Tweaks/ WinXP-2K-Pagefile Utility], and use it to monitor your pagefile usage over a number of days. Then add a bit to the maximum usage reported by the utility, and use that as your minimum pagefile size.
The maximum pagefile size should be a nice large number, probably at least 1.5x the minimum pagefile size, or 700-800MB, whichever is larger. But once again, only experimentation and monitoring of the pagefile usage will tell you the best settings for you. If you use Fast User Switching (which you should probably not, with a DAW), then the maximum should be increased. A good starting point is to increase the maximum by .5x the amount of RAM on your system for each user. The maximum amount of virtual memory allowed in Windows XP is 4GB.
There are some ways to squeak optimal performance out of the pagefile. The first thing is to try to make sure that it doesn't get "fragmented". When Windows is first installed, the pagefile gets created on a blank (or nearly blank) disc. This means that the pagefile is contiguous on the disc - the whole file is placed pretty much in one place on disc. If the pagefile is spread out over the disc in many little pieces ("fragmented"), then it can take much, much longer to access. A common way the pagefile can become "fragmented" is by changing the minimum pagefile size.
As an example, say a user buys a new system with 512MB of RAM. The computer is initially configured with a pagefile.sys custom size of 768MB minimum and 1024MB maximum. Now let's say the user decides to increase the pagefile size to 1024MB minimum and 2048MB maximum. The pagefile.sys file is just that - a file. So when the minimum pagefile is increased, the pagefile.sys is made larger on disc. However, chances are good that other files have now been written to disc, and the expanded part of the pagefile cannot be contiguous with the old pieces. Instead, it is placed in empty pieces of the hard drive, and can end up spread out in many different pieces, spread widely over the disc. The pagefile has become "fragmented".
Essentially, we'd like to make sure the pagefile is defragmented. But there's a problem. Assuming it exists, the pagefile is always in use, as long as Windows is running. This means it cannot be defragmented using the Windows defragment utility, because the utility cannot move files that are in use, and the pagefile is always in use. (For this reason, the pagefile appears as a green "unmovable file" in the Windows defragment utility, as do a few other files, including the Windows Registry). To get around this issue:
If you ever increase the minimum size of the pagefile, you should probably defragment the pagefile. But once it is defragmented, it will stay that way, even between reboots.
But remember, we can configure both a minimum and maximum size for the pagefile. When Windows first starts running, the pagefile is set to the minimum size. But if Windows runs low on pagefile space, it will increase the size of the pagefile, up to the maximum allowed. Unless the pagefile is on its own disc partition, this will cause the pagefile to become fragmented. For this reason, some people recommend that the minimum and maximum pagefile sizes be set the same. This is generally not a good idea.
As was stated above, the ideal setting for the minimum size of the pagefile should be large enough that pagefile expansion usually does not occur. Also, when the computer is reboot, the pagefile returns to its minimum size. These two points mean that there is little need to worry about the extended part of the swapfile being fragmented. On the other hand, if you set the minimum and maximum to the same size, you are removing Windows' ability to increase the pagefile if it should be needed. Think of it as a "safety net", there for those rare instances when it may be needed. Assuming you have set your minimum swapfile size appropriately, keeping a larger maximum swapfile size will have no negative impact on performance, and may save you from occasional "Out of Virtual Memory" errors.
One option is to create a partition on your hard drive that contains only the pagefile. As the pagefile size changes, there are no other files on the partition, so the file never becomes fragmented, even when it is enlarged. But since this involves reformatting your boot drive and reinstalling everything, it is a rather heavy-handed choice, and there is some question as to whether the gain is worth the effort. If you are building (or rebuilding) your system from scratch, this can be the best choice. But it results in minimal gain if you set your minimum pagefile size to an appropriate value. Remember, the goal is to set this setting so that Windows rarely needs to resize the pagefile. If Windows is pretty much always running with the pagefile at its minimum size, you don't need to worry about the pagefile becoming fragmented. Even if it should occur, the pagefile will return to minimum size the next time you reboot the computer As long as the pagefile is not fragmented when it is at its minimum size, you are usually fine.
Another optimization is to add a second hard drive. In this case, the pagefile and application code should be located on the primary drive, and the secondary drive should be dedicated to AUDIO DATA ONLY. This keeps paging from interfering with the audio streams. And if you want to completely maximize your system performance, the best option is to have three disc drives. One disc is the main drive, one disc is for AUDIO DATA ONLY, and the third disc is partitioned into two logical drives - one small partition for the pagefile.sys (just big enough to hold the pagefile at its maximum size), and one partition for the rest of the disc. This causes code paging to hit the primary drive, while data paging hits the pagefile drive, and the audio drive is free from disruption. When you configure a different physical disc (other than the boot disc) to hold the pagefile.sys, follow the instructions provided here: http://support.microsoft.com/default.aspx?scid=kb;en-us;248993 . Also, something to be aware of is that there appears to be an issue where Windows may ignore these settings if the pagefile is completely removed from the main drive (drive C:) and moved to another drive. The recommendation is to leave a minimal pagefile (2MB minimum, 50MB maximum, or larger maximum if you want debug information after system crashes) on the boot drive, while creating a normal-sized pagefile on the other drive. If you are still concerned about fragmentation when Windows expands the pagefile, then partitioning the pagefile disc, and putting ONLY THE PAGEFILE in one partition on the disc, ensures that the pagefile cannot become fragmented when (if) it expands. The other partition on the pagefile disc can be used for spare storage (but NOT as your active audio drive for a recording session). Another option is to use multiple partitions on multiple discs. In this case, the multiple partitions can all be used simultaneously, similar to a RAID0 striping disc array. If you do this, just make sure you aren't using a mixture of very fast and very slow drives, and of course, make sure you do NOT use your Audio drive for this purpose. Finally, DO NOT put the pagefile on multiple partitions of the same disc. This will merely force the drive head to bounce back and forth between partitions when trying to read the pagefile, which can reduce performance.
Microsoft recommends that you do not use the "Custom Size" setting for the pagefile size, but instead let the system manage it. However, I would only recommend this option if you have a dedicated partition for the pagefile. Otherwise, the pagefile may become "permanently" fragmented as Windows automatically changes the minimum size around.
It may be worth noting that if you have three drives in your system, you will probably get better performance by having one drive as the system drive (including the pagefile), and use the other two drives in a RAID0 striping array. This tends to work better for DAWs than a three-drive system that uses one drive as the Audio drive, one drive for the pagefile, one drive for the rest of the system. Another option that may work best if you use samples heavily is to have one drive dedicated to the system (including pagefile), one drive as the primary Audio drive, and one drive that holds the samples.
You may be saying "But <insert name of guru or web site here> says I should turn off the pagefile for DAWs! Why should I believe YOU, some faceless entity with an unpronounceable name?" The answer: Don't believe me. Do your own testing.
The Performance Monitor in Windows XP is pretty useful. Start it up, configure it to display the parameters you are interested in monitoring, and test it out. Some key parameters to look at are Page Writes/Sec, Pages Output/Sec, %Committed Bytes, and Available MB in the Memory object, the %Usage in the PagingFile object, %Processor Time in the Processor object, and the Avg Disc Queue and %Idle Time for the PhysicalDisc object. If you disable paging, you may find that the Paging File and Memory objects go away. If you can't find these objects in the Performance Monitor, follow the instructions found here: http://support.microsoft.com/default.aspx?scid=kb;en-us;248993 .
The content provided on this web site is public information which was originally posted in the Cakewalk User Forums and on the old Project5 Wiki. Project5™ and Cakewalk™ are registered trademarks of Cakewalk, Inc. Other product names or companies names may be trademarks of their respective companies whether marked as such or not. Portions of this web site are copyright © 2024 Robert J. Hammond ( A member of the Technetos family of web sites ).