Consultor Eletrônico

Why .bi file grows larger than original bi with roll forward

In order to explain how transactions are written to a
.bi file and to the .ai file, and what may happen to a
.bi file when roll forwarding with an .ai file, the
following symbols and diagrams will be used to represent
notes, clusters, etc.:

There are 4 notes written to a .bi and .ai file for each
transaction. They are:

1. Begin note
2. Pre note (snapshot of record before change; can be big)
3. Post note (changes to record)
4. End note

We'll represent each note by Nn, where n represents either
note 1 (Begin), note 2 (pre), note 3 (post) or note 4 (end).

Let's represent a transaction with a Tn, where n is the
transaction number.

Boxes such as these will represent clusters:


+---------+---------+---------+---------+
| | | | |
|Cluster 1|Cluster 2|Cluster 3|Cluster 4|
+---------+---------+---------|---------+

Typically a pre note follows closely after a begin note,
and an endnote follows closely after a post note.

Therefore, in a multi-user multi-threaded environment,
you could see a cluster with the following information:

T1N1 T1N2 T2N1 T1N3 T2N2 T1N4

This would represent Transaction 1 with a begin and pre note,
Transaction 2 with a begin note, followed by Transaction 1's
post note, Transaction 2's pre note, then Transaction 1's end
note. Assuming this was all in one cluster, this would mean
Transaction 1 was complete, but Transaction 2 is still under
way. This cluster is not available for reuse until Transaction
2 is finished.

If you had just truncated the .bi file and were starting out
with a fresh .ai extent, the first four clusters would contain
the same information until the fourth cluster was full
(given that they were the same size). At this point, this
is where the contents of the .ai and the .bi file will begin
to look differently. In our previous example, once Transaction
2 was finished, the first cluster of the .bi file would be
available for re-use.

For a picture of this, let's assume we have the following 5
multi-threaded sessions taking place:

T1 | N1 N2 N3 N4
T2 | N1 N2 N3 N4
T3 | N1 N2 N3 N4
T4 | N1 N2 N3 N4
T5 |.. N3 N4
+-------------------------------------------------
time

The .bi file could look something like this:

---------+------------------------+------------------------+
T3N1 T5N3|T4N1 T5N4 T4N2 T4N3 T4N4|T3N2 T1N1 T2N1 T1N2 T2N2|
---------+------------------------+------------------------+
Cluster 5 Cluster 6

Note that Cluster 5 has an end note for Transaction 5, plus
the begin and end notes for Transaction 4. There are no other
"open" transactions residing in this cluster. Therefore, this
cluster is now available for reuse. At this point, we are ready
to write T1N3 to the .bi file. Rather than extend the .bi file
with another cluster, Progress will reuse cluster 5, and create
a new cluster 7 to hold the rest of the information. Therefore,
the clusters will look like this after we put the remaining
notes into the .bi file:

---------+-------------------------+------------------------+
T3N1 T5N3|T1N3 T1N4 T3N3 T3N4 T2N3 |T3N2 T1N1 T2N1 T1N2 T2N2|->
---------+-------------------------+------------------------+
Cluster 5 Cluster 6
+-------------
| T2N4...
+-------------
Cluster 7

However, the .ai file is a sequential file, and will not reuse
clusters. Therefore, the .ai file looks like this:

---------+------------------------+------------------------+
T3N1 T5N3|T4N1 T5N4 T4N2 T4N3 T4N4|T3N2 T1N1 T2N1 T1N2 T2N2|->
---------+------------------------+------------------------+
Cluster 5 Cluster 6
+-------------------------+------------------
|T1N3 T1N4 T3N3 T3N4 T2N3 | T2N4...
+-------------------------+------------------
Cluster 7 Cluster 8

Obviously at this point, the .bi and the .ai files are no longer
the same length. And the notes are in a different order
physically (but not logically). The .al file continues to grow
sequentially while the .bi file is constantly taking advantage
of the reusable clusters in the .bi file.

Also, it is important to note that the .bi file is not allowed
to reuse a cluster until it has aged at least 60 seconds.
(This is the default, and can be overridden with the -G
parameter, but please read up on this before using it so you
fully understand the implications first.) Since we have users
who will stop to answer a phone, get a cup of coffee, shuffle
papers, etc., the writing to the .bi file by users will not
necessarily be a constant stream. This aids in the aging of
clusters in the .bi file. These two factors: aging of clusters
and the rate at which the .bi file is being written to during
normal processing is important to remember for the journey
into roll-forwarding.

ROLL FORWARDING: THE AFFECT ON THE .BI FILE

Let's assume we now have stopped using this .ai file and are
ready to apply it to a previous backup. The .bi file during
this day's activity yielded a .bi file of approximately 8MB.
Will the .bi file that's created during roll forward create
a .bi file of the same size, smaller, or larger? It would
not be unlikely at all to have the .bi file created during
roll forward to be larger, and this is why:

Using the .ai file above, the .bi file notes the transactions
as they are applied to the database. They are applied in
rapid succession, as we are not talking hand-entered data
from our previous real-time experience. By the time cluster
6 is full, cluster 5 cannot be reused, as proper aging has not
occurred for Cluster 5. Therefore, the .bi file will be
extended, and the information will be written to cluster 7.

Another side effect on .bi size is that the notes in the .ai
file could be spread further apart for any given transaction.
For example, instead of having all of Transaction 2 roll forward
by Cluster 7, which is where the end note for this transaction
resides in the original .bi file, it won't be completed until
it reaches Cluster 8 in the .ai file. This again means further
.bi file growth.

These two important factors will affect the growth of the .bi
file during roll forward sessions. Additionally, please note
that having different cluster sizes for your .bi and .ai files
could also further impact how big the .bi will grow during roll
forward.

SUMMARY:

An infrequent occurrence of .bi file growth to many times the
size of the .ai file during roll forward recovery is not usually
an indicator of required tuning with respect to modification
of bi blocks, bi cluster, or database block sizes. Each roll
forward of the new ai file will cause the bi file to grow in very
different patterns as opposed to the previous ai roll forward.
This is due to the very differently recorded sequence of events.
The promon "activity" screen is the proper location for making
this type of determination.

Progress Software Technical Support Note # 17844