Status: Verified
GOAL:
How to create a hot backup to use with AI files
GOAL:
How to use After-imaging to create and maintain a hotspare database.
GOAL:
How to use after-imaging for standby database.
GOAL:
After-image disaster recovery plan.
FACT(s) (Environment):
All Supported Operating Systems
Progress 9.x
OpenEdge 10.x
FIX:
OverviewThe following is applicable where two machines are available, one for live production, and one for backup in case the live machine goes down due to hardware failure.
The idea is that a copy of the live database is initially restored on the backup machine (hotspare) and, through time, the copy is kept in sync with the live machine via after-imaging.
Setting up the live machine1) Database server shut down.
Starting after-imaging on a database can be done only while the database is stopped.
2) rfutil sports -C aimage end
To disable after-imaging if previously enabled.
3) proutil sports -C truncate bi
4) prostrct add sports addai.st
Assuming prior to this point the database had NO after-image extents, at this stage they are added (but left inactive). addai.st contains only the information regarding the structure and location of the new after-image extents. Please see Progress Solution 21460 for further details.
5) prostrct list sports
Verify the sports.st file with the new AI extents included.
6) probkup sports sports.bk
The Database is backed up at this point.
7) prorest sports sports.bk -vf
Fully verify backup before creating the hotspare database.
8) Transfer the backup (sports.bk) onto the backup machine and create the hotspare database.
This is detailed in section "Setting up the hotspare machine" below.
9) rfutil sports -C aimage begin
After-imaging enabled on live system.
At this point there will be a backed up database on the live machine and a new reference point for rolling forward the after-image extents, on the hotspare machine.
10) Database server started.
11) Via operating system utilities (for example: cron on UNIX, or Scheduled Tasks on Windows 2000), a script is fired every N minutes in order to forward any full after-image extents to the hotspare machine.
rfutil sports -C aimage extent full should be used to determine which after-image extent is full.
rfutil sports -C aimage extent empty should be used to mark the after-image extent as empty and ready for re-use, after it has been backed up.
How often this script should fire depends on the size of the after-image extents and the load of your system: basically you must avoid that all after-image extents become full at any time.
Since OpenEdge 10.1A, the database can be enabled to use the AI Archiver deamon to manage ai file switch/copy. Please refer to the OpenEdge Database Administration Documentation.
12) [Optional] Online backup of "live" database: probkup online sports sports.bk.
This will generate an after-image extent switch and the full AI extent can be rolled forward on the current hotspare database, or the next-full ai extent can be rolled forward against the prorest of this backup. This is also the method that can be employed to take a new hotspare baseline should the need arise in future. The ai script management set up in the previous step should take care of this case.
Setting up the hotspare machine:1) prorest sports sports.bk
Restore the database backup create in step 6 above. Ensure that a dbname.st file exists in the directory the dbname.db file will be restored to, in order that the correct pathnames are modified (if needed). Consider adding 'extra' extents per Storage Area for the hotspare database, so that when more space needs to be added to the production database, these are already in place for the hotspare thereby negating the need for a new hotspare baseline. In particular, the hotspare should have more bi file space than the production database as by nature, the bi file on the hotspare will be larger than it's equivalent on the production database. For history p.urposes you may consider including the log file for the production database.
2) Via operating system utilities, a script is fired every N minutes in order to check whether any after-image extent has been forwarded from the live machine. If so, roll forward to the hotspare database.
rfutil sports -C roll forward -a <aifilename>
3) No process except for the roll forward utility can access this database.
4) [Optional] Off-line backup of the "hotspare" database.
This should be done via probkup sports -norecover. Please note that the -norecover is an absolute requirement: without it, probkup would perform a roll back before backing up the database, thus invalidating the roll forward sequence.
Other considerations:a) Whatever disaster recovery plan that you decide to implement needs to be fully verified. Experience has proven that not testing or not documenting the after-image strategy put in place has detrimental (and costly) consequences.
b) In a production environment, an important consideration is when the structure of the live database will eventually need altering, that is, adding/removing after-image extents to the live database without breaking the after-imaging process. While adding a new after-image extent poses no problems, removing after-image extents from the live database is unfortunately not possible: in order to remove an after-image extent, after-imaging must first be disabled, therefore interrupting the chain of after-image files to be fed to the hotspare database. Similarly, data extents can only be added to but not removed from the two databases and before-image extents cannot be added to the hotspare database without first truncating the bi file (which is not supported as it can and does cause roll-forward to fail afterwards, in which case a new baseline of the hotspare will need to be established).
c) Progress does not support cross platform compatibility, so the same operating system should be used on both the live and hotspare machines. Please ensure that both machines are running exactly the same level of Progress (that include Service Pack levels).
d) The after image extents for the live database should not be placed on a remote drive. This is not supported for safety reasons, since writing across a network is likely to create unexpected behavior and can impact performance severely. The after image extents should be placed on the same machine as the live database, but on a different disk.
e) It is not advised to run after-imaging with only one AI extent. Furtermore, extents should be defined as FIXED length and in this way not only prevent hitting the 2GB limit (in versions prior to Progress 9.1C), but also control when an after-image extent switch occurs and how much disk space each extent uses.
Whether fixed or variable AI extents are employed, if a "disk space shortage" or a "no empty AI extent available" scenario arises, emergency maintenance will need to be performed, as Progress will force a shutdown. To prevent this from happening, the -aistall startup parameter should be considered if the database is in multi-user mode. Once -aistall is used, the database will then instead, be suspended, a message will be sent to the log file, the extent can be archived and marked as empty and the system will automatically switch to this extent as database activity resumes.
To estimate the amount of after image space needed, view the "BI Log Activity" using the promon utility. Measure the BI Bytes statistic toward the end of the period that you are considering using after-imaging. Typically 0.05% fewer bytes are written to the AI file. Always allow extra disk space for unanticipated growth.
f) In case the live machine goes down and you need to switch to the hotspare you have to consider that the hotspare will not have after-imagi.ng enabled. In the event the hotspare machine goes down as well, you risk losing all the work since the last backup. Therefore a disaster recovery plan must be set up for the now-live database.
Once the machine that went down is available again, you must set up both machines again, as described at the beginning of this Solution.
g) There are licensing issues to be considered: one license is needed for each machine where Progress is running. These are best discussed with your Account Manager.
h) Since OpenEdge 10.1C a new feature on Enterprise Databases called the "After-image Archive Manager" can be enabled on the database and manage the archving and emptying of FULL extents automatically. The Database Administration Guide and Reference has further detail on this feature..