Consultor Eletrônico

INTRODUCTION:
=============
This Progress Technical Knowledge Base entry is intended to highlight
some undocumented system limitations encountered on the MicroSoft NT
& Win 2000 system and to give a brief explanation of how to remedy
these limits.

WHY YOU NEED TO KNOW THIS:
==========================
If you need to run the Progress database that exceeds the size of
few hundred gigabytes, then it may happen that your system hits
the limit of the PagedPoolSize of the Kernel that has the maximum
value of the 192 MB.
This limit has been determined by the Microsoft and it can be
changed through the registry settings even though Microsoft doesn't
recommend that. For detailed explanation of the PagedPoolSize
registry key and how can it be changed take a look at this KBase
reference list.

EXPLANATION:
============
NT treats all storage devices as if they were virtual RAM resources.
This lets virtual machines issue resource requests using the same
interface, communications methods, and system connections that are
already present for memory management. Using the RAM resource model,
storage device access is homogenized and isolated from the virtual
machines. All access to storage devices is managed by the I/O Manager
from the kernel?s Executive Services. The I/O Manager is responsible
for understanding all the communications requirements for interacting
with each storage device.
The NT operating system supports up to 4 GB of physical RAM.
Even without the physical existence of RAM, NT still offers 4 GB of
address space to every active process by using virtual machines to
isolate and protect processes. Each virtual machine is assigned a
unique 4-GB address space. The Virtual Memory Manager (VMM) from
the kernel?s Executive Services maps the address space from each
virtual machine.

What if an application were to request a file that was larger
than the total physical memory on the system, including the paging file? Even in this situation, the VMM has one more trick up its
sleeve: mapped file I/O. In mapped file I/O, the VMM maps virtual
memory addresses to a large disk file and loads only those
portions of the file to which the application requires access
at any given time. This lets the system use memory more
efficiently, while giving the application the access to the
file that it needs. Mapped file I/O is especially useful with
applications such as a database management system (DBMS) that
occasionally use extremely large data files.
However, the mapped file I/O approach might have one major drawback.
Since the translation between linear address space representing a
very large file demands a certain number of pages to be stored
in the Kernel swap area known as PagedPool, this could limit the
total size of simultaneously opened files under certain circumstances.
If your DBMS tries to open a file that is greater then available
RAM on your system, the VMM will use mapped file I/O, therefore
creating adequate number of address translation pages.
Page address translation is a logical to physical address mapping.
Some bits in the logical/linear address are used as an index in the
page table, which provides a logical to physical mapping for pages.
The page translation mechanism on Intel platforms has two levels,
with a structure called page table directory at the second level.
As the name suggests, a page table directory is an array of pointers to page tables.Some bits in the linear address are used as an index
in the page table directory to get the appropriate page table to be
used for address translation.
The page address translation mechanism on the Intel platform
requires two important data structures to be maintained by the
operating system, namely, the page table directory and the page
tables. A special register, CR3, points to the current page table
directory. This register is also called Page Directory Base Register
(PDBR). A page table directory is a 4096-byte page with 1024 entries
of 4 bytes each. Each entry in the page table directory points to a
page table. A page table is a 4096-byte page with 1024 entries of 4
bytes (32 bits) each. Each Page Table Entry (PTE) points to a
physical page. Since there are 1 million pages to be addressed,
out of the 32 bits in a PTE, 20 bits act as upper 20 bits
of physical address. The remaining 12 bits are used to maintain
attributes of the page.

E.g. if your system has 256 MB RAM and you have one extent of the
270 MB, VMM will use page table to locate physical address in that
file and therefore it will allocate at least 68 pages
(1 PDBR + 67 PT) that's 68 * 4K = 272 K.
If you have an extent of 2 GB size, VMM will use aprox. 2 MB for page
address translation. All of this space has to go in PagedPool Kernel
swap area which can grow up to the 192 MB. Therefore, with all large
extents that are bigger then your RAM, you will be limited to
approximately 190 GB database (or less).
Bear in mind that additional processes could also make the Kernel
to allocate space in the PagedPool area that can limit your DB size
even further. All of the registry hive and SAM database goes into
this area. So it is not advisable to have large registry hive
(lot of installed applications and services), nor to have the system
operate as PDC or BDC.

This limitation on Windows 2000 (Intel platform) for PagedPool
is 320-340 MB. If you use the -4GT switch with NT Enterprise
then both the Win NT and Win 2000 revert to limit of the 192 MB
in PagedPool area.

SOLUTION:
============
Make sure that the extents in your database are smaller than the
installed RAM on your Win NT / Win 2000. Then you will be able to
alleviate the limit imposed by PagedPoolSize and therefore to
run a database server for a very large database.

skraljev

Reference:
- Microsoft article Q126402 - PagedPoolSize and NonPagedPoolSize
Values in Windows NT
- Undocumented Windows NT By Dabak, Prasad / Phadke, Sandeep / Borate,
Milind ISBN: 0764545698