Tips and Tricks: Advanced Windows Extensions

January 2008

Situation
You have a brand new Windows based server with more than 4GB of RAM for your Oracle Database.  

Problem
How do you address the memory above 4GB to get the biggest bang for your buck and still maintain a stable environment?

Solution
Windows supports AWE or Advanced Windowing Extensions and PAE or Physical Address Extension with the “/AWE” or “/PAE” switches in the BOOT.INI file to address upper memory. PAE is normally enabled on Windows 2003 by default so if you install more than 4GB, the OS can see and use it. You’ll see “PAE Enabled” in the system properties page. AWE on the other hand allows a process to map upper memory from lower memory. This brings up the 3GB switch. A process in windows can normally address 2GB of RAM max. The 3GB switch removes 1GB of lower memory and allocates it to processes or “user” memory. The problem with enabling 3GB is the possible starvation of system page table entries, PTEs. Windows will become unstable if free PTEs drops below approximately ten thousand. It has been my experience that the free PTEs will drop from 180,000 to 40,000 when the 3GB switch is used. That gives about 30,000 PTEs to work with for the entire system. This can be monitored with perfmon to determine if it will be a problem. PTEs in-use rises with increased utilization so monitor it during peak usage.

What does this mean for Oracle? Oracle will take advantage of memory above 4GB for buffer cache only and a portion of lower memory will be reserved to address it so you have to balance the demand for the buffer pools and the lower memory “window size” for addressing upper memory.  You will also have to balance the demand for the Kernel pools; paged pool, nonpaged pool, and page table entries.

Tips
Windows settings:
  • All boot.ini settings should be placed at the end of the startup line for the OS.
  • PAE may not be necessary in the boot.ini check to see if “Physical Address Extension” is listed on the “System Properties” page for the server.
  • AWE is required for the boot.ini – This allows the Oracle process to address memory above 4GB.
  • 3GB may be necessary if you want room for the SGA and PGA in lower memory, minus the buffer cache of course.
  • PTEs will need to be verified; use perfmon if you have Windows 2000 SP4 or above. Another way to see the maximums for the kernel pools is to use a kernel debugger such as livekd from sysinternals.com. They have instructions on the site for installing it. If there are 10,000 or fewer PTEs, it’s prudent to make a registry change to allocate memory from the other kernel pools to PTEs.
  • Within the Windows registry under the ORACLE key there is a “HOME” key where the windows size in bytes needs to be specified. Create a REG_SZ value called “AWE_WINDOW_MEMORY”. I have found a setting of 640MB will address 11GB of upper memory for buffer cache. That is real bytes, not the storage standard of 1000B=1KB, but where 1024B=1KB. So, 640MB = 671088640B for ease of use. Please use powers of 2 and be precise. This setting is not necessary because by default the window size will be 1GB and allow up to 16GB to be addressed. Remember that the window comes out of SGA and PGA lower memory, so to allow more concurrent users, setting this to the minimum for the desired size of upper memory to be addressed is recommended.
Oracle init settings:
  • The new “CACHE” parameters cannot be used. DB_BLOCK_BUFFERS must be used and this means that only systems using only the default block size will be candidates for this procedure. This precludes SGA_TARGET as well.
  • Set USE_INDIRECT_DATA_BUFFERS=TRUE
That’s it. Make these changes and you’re using upper memory for buffer cache and lower memory for the various pools and session sort areas. It is an iterative process to discover where the boundaries are for the window size and if you’re starving the system of free PTEs then it will be a problem for the OS. Locations and settings for PTEs can be found in the MSDN library for the registry. I set the PTEs to MAX, “ffffffff”, because the other pools are usually nowhere near their maximums.

If you need more information, contact Gary Codeluppi at 937-431-1026 x123 or visit the Ross Group Inc web site at www.rossgroupinc.com.
Comments (0)Add Comment
Write comment
 
 
smaller | bigger
 

busy
search | login