Kbase P55740: How to scope and define a multi-volume srt file for idxbuild
| Autor |
Progress Software Corporation - Progress |
| Acesso |
Público |
| Publicação |
09/04/2009 |
|
Status: Verified
GOAL:
How to create multivolume sort file?
GOAL:
How to create a multivolume srt file?
GOAL:
Overcoming the SRT Files Size limitation
GOAL:
How to define a multi-volume sort file for "idxbuild"
GOAL:
How to size and define a multi-volume srt file for idxbuild
GOAL:
How to create a multi-volume srt file for idxbuild
GOAL:
How to calculate the size of a multi-volume srt file?
GOAL:
What would a multi volume .srt file look like?
GOAL:
When do I need to define a multi volume sort file ?
GOAL:
How to address SYSTEM ERROR: I/O error in , ret file , addr . (290) during idxbuild
GOAL:
How to address Seek error 22, file <num>, addr -2147467264. (163) during idxbuild
FACT(s) (Environment):
All Supported Operating Systems
Progress 9.x
OpenEdge 10.x
OpenEdge Database Category: Maintenance
FIX:
When you perform the Index Rebuild utility and choose the Sort option, you might encounter space limitations that can cause the utility to terminate. These are some of the reasons that you might reach the limit:
The temporary sort file allocated to do the sort reaches the operating system limit for file size (2GB on most systems).
To overcome this limitation, simply create a file that contains specifications for the directories and the amount of space per directory that you want the SRT file to have access to during the Index Rebuild. The file that contains the specifications must be a text file, must have the same name as the database with an extension of .srt (dbname.srt), and must reside in the same directory as the .db file.
Please ensure that the multi-volume srt file is valid by:
Where:
1.) The naming format is [databasename].srt which has been created in the same directory that the database.db resides.
2.) That the space defined in the multi-volume srt file will be available on disk for the duration of the index build and the directories must have been created on the system. (eg; /usr1/sort/)
3.) That each directory defining the space allocated is less than 2GB in each instance.
4.) That there are no TABs used, use spaces instead
5.) That the trailing "/" for UNIX, or "\" for Windows is in place
6.) That there are no empty lines, not even at the end of the dbname.srt file.
In addition, the contents of the file must follow these conventions:
- List the directory and the amount of space that you want to allocate to the index rebuild sort on separate lines. Note, what listed are directory names not file names.
- The size that you specify in the dbname.srt directory specification is the maximum (in 1 Kbyte units) that the file can grow. Specifying 0 for any directory indicates that you want to allow unlimited growth.
- When you provide a file size that is larger than the maximum allowed by your operating system, Index Rebuild ignores the value you specified in the file and uses the maximum size allowed by the operating system.
-When you specify a nonzero file size that is less then the temporary block size (-TB), Index Rebuild ignores the value you specified in the file and uses the temporary block size (-TB) value.
- Separate the directories from the size by at least one blank space.
- Terminate the line with a slash (/), or backslash for windows, followed by end of line.
The Index Rebuild utility opens the files for each of the directories before it actually starts the sort process. As a result, one of the following messages is displayed for each file:
Temporary sort file at pathname used up to nK of disk space.
or:
Temporary sort file at:pathname will use the available disk space. (this message is displayed even if the .srt file was not found).
When the sort completes, the following message is displayed for each file:
Temporary sort file at pathname used nK of disk space. In some cases the message displays OK. This simply means that the sort took place completely in memory.
If Index Rebuild does not find a dbname.srt file, then by default, it uses the directory supplied by either the -T parameter or the current working directory.
The disk space needed for sorting can be larger than the database size in cases where running an idxbuild operation that has many indexes on char field(s), for example. This is because strings are not compacted in phase 1 of "idxbuild". Although disk space may not be a problem, the .srt .tmp files themselves may not exceed 2GB, hence the need for defining a multi-volume srt file for the operation. Using a file <dbname>.srt for "multi sort files" was first introduced in Progress Version 8.0B to overcome both the 2 GB limit of the sort file and space limits within directories.
As a rule of thumb, dependant on the effici.ency of indexing, the size of the multi-volume .srt file should be calculated as: 75% of database size.
In other words, databases that exceed 2GB in size should always have a multi-volume srt file defined when undertaking an "idxbuild ALL" operation.
A more specific calculation for .srt size is: Size of one index entry * the number of records in the file * 3
or running a "proutil -C idxanalys" report which will show the number of blocks in the database being used for indexes then multiplying by 2 (for sorting) then multiplied by the database blocksize (then round up)
A complete index build can then be run offline with the proutil command, the syntax for idxbuild remains the same:
$ proutil dbname -C idxbuild ALL -TB 24 -TM 32 -B 1024 -SG 45
At the prompt:
Do you have enough disk space for index sorting? (y
)
You must answer "y" for the multi-volume sort file to then be used by idxbuild. Answering "n" would make idxbuild use an entirely different algorithm, which does not rely on sort files at all.
Should the idxbuild abort because either it has hit the 2GB limit on the last sort file, or because it has filled the file system, or because you abort it knowing it is going to hit the 2GB limit, you can recover by creating a valid <dbname>.srt file and then re-run the idxbuild. In this case, you see the following:
Previous index rebuild was interrupted. This is a new try. (2748)
Can't find selected index list file .xb; assuming ALL. (2786)
Is this correct? (y
)
Choosing "y" will start the Index Rebuild, and the new .srt file will be used.
Example of a Multi Volume Sort File.
If there are 3 GB space in /usr1 and then enough space remaining in /usr2, a 5 GB sort file definition which uses 0.5 GB delimiters for each .srt file leaving the last one as variable would look as follows where the directories listed already exist:
512000 /usr1/sort1/
512000 /usr1/sort2/
512000 /usr1/sort3/
512000 /usr1/sort4/
512000 /usr1/sort5/
512000 /usr1/sort6/
512000 /usr2/sort1/
512000 /usr2/sort2/
512000 /usr2/sort3/
0 /usr2/sorta/
On Windows, the only difference would be the separator, eg:
512000 C:\tmps\sort1\
512000 C:\tmps\sort2\
512000 C:\tmps\sort3\
512000 C:\tmps\sort4\
512000 C:\tmps\sort5\
512000 C:\tmps\sort6\
512000 D:\tmps\sort7\
512000 D:\tmps\sort8\
512000 D:\tmps\sort9\
0 F:\tmps\
.