Important Notice on Product Safety Elevated voltages are inevitably present at specific points in this electrical equipment. Non-observance of these conditions and the safety instructions can result in personal injury or in property damage. Only trained and qualified personnel may install and maintain the system.
Important Notice on Product Safety Elevated voltages are inevitably present at specific points in this electrical equipment. Non-observance of these conditions and the safety instructions can result in personal injury or in property damage. Only trained and qualified personnel may install and maintain the system.
Important Notice on Product Safety Elevated voltages are inevitably present at specific points in this electrical equipment. Non-observance of these conditions and the safety instructions can result in personal injury or in property damage. Only trained and qualified personnel may install and maintain the system.
Backup and Restore Management A50023-X4006-A210-6-7619 2 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 f Important Notice on Product Safety Elevated voltages are inevitably present at specific points in this electrical equipment. Some of the parts may also have elevated operating temperatures. Non-observance of these conditions and the safety instructions can result in personal injury or in prop- erty damage. Therefore, only trained and qualified personnel may install and maintain the system. The systemcomplies with the standard EN 60950 / IEC 60950. All equipment connected has to comply with the applicable safety standards. The same text in German: Wichtiger Hinweis zur Produktsicherheit In elektrischen Anlagen stehen zwangslufig bestimmte Teile der Gerte unter Spannung. Einige Teile knnen auch eine hohe Betriebstemperatur aufweisen. Eine Nichtbeachtung dieser Situation und der Warnungshinweise kann zu Krperverletzungen und Sachschden fhren. Deshalb wird vorausgesetzt, dass nur geschultes und qualifiziertes Personal die Anlagen installiert und wartet. Das System entspricht den Anforderungen der EN 60950 / IEC 60950. Angeschlossene Gerte mssen die zutreffenden Sicherheitsbestimmungen erfllen. Trademarks: All designations used in this document can be trademarks, the use of which by third-parties for their own purposes could violate the rights of their owners. Copyright (C) Siemens AG 2003-2003. Issued by the Information and Communication Networks Group Hofmannstrae 51 D-81359 Mnchen Technical modifications possible. Technical specifications and features are binding only insofar as they are specifically and expressly agreed upon in a written contract. A50023-X4006-A210-6-7619 3 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management This document consists of a total of 82 pages. All pages are issue 6. Contents 1 Backup & Restore Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1 Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2 General description of Backup and Restore management . . . . . . . . . . . . . . 8 1.3 Data to backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.3.1 Basic Backup&Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.3.2 Backing up Unix file systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.3 Backing up databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.4 Restoring user data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.5 Restoring databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.6 Backup & Restore procedure logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.3.7 Examples for backup- & restore configurations . . . . . . . . . . . . . . . . . . . . . 14 1.4 Description of installation from Backup & Restore management viewpoint 16 1.4.1 Location of files after RC installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.4.2 Installation of Backup & Restore Management (BRM) . . . . . . . . . . . . . . . . 16 1.4.2.1 Installation with internal backup server . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.4.2.2 Installation with external backup server . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.4.2.3 External backup server configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.5 Administration Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 1.5.1 Backup and Restore possibilities on the OMP . . . . . . . . . . . . . . . . . . . . . . 30 1.5.1.1 Graphics mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 1.5.1.2 Manual mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 1.5.1.3 Manual mode recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 1.5.1.4 Graphics mode recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 1.5.2 Backup and Restore on an OMT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.5.2.1 In graphics mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.5.2.2 In manual mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.5.2.3 Recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.5.3 Informix database backup and restore with BRM. . . . . . . . . . . . . . . . . . . . 32 1.5.3.1 In graphics mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.5.3.2 In manual mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.5.3.3 Recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.5.4 Oracle backup possibilities with Backup & Restore Management . . . . . . . 34 1.5.4.1 In graphics mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 1.5.4.2 In manual mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 1.5.4.3 Recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 1.5.5 Other backup possibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 1.5.6 About backup/restore logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 1.6 Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 1.6.1 Backup/Restore possibilities in maintenance . . . . . . . . . . . . . . . . . . . . . . . 36 1.6.1.1 Enable Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 1.6.1.2 Disable backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 1.6.1.3 Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 1.6.1.4 Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 1.6.1.5 Important backup and recovery options . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.7 Disaster data recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 1.7.1 Hardware preferences for Recover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 1.7.2 Perform disaster recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 1.7.3 Recover Solstice Backup indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 1.8 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2 Procedures (PROC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.1 Supplementary backup server installation . . . . . . . . . . . . . . . . . . . . . . . . . . 52 2.2 Supplementary backup client installation. . . . . . . . . . . . . . . . . . . . . . . . . . . 55 2.3 Activate OMP Level 0 Backup (Dump of System Data) . . . . . . . . . . . . . . . . 58 2.4 Activate OMT Level 0 Backup (Dump of System Data) . . . . . . . . . . . . . . . . 61 2.5 Activate OMP Level 0 Restore (System Data) . . . . . . . . . . . . . . . . . . . . . . . 63 2.6 Activate OMT Level 0 Restore (System Data) . . . . . . . . . . . . . . . . . . . . . . . 67 2.7 Activate Restore of Internal Disks (Disaster data Recovery) . . . . . . . . . . . . 70 2.8 Activate Restore of External Disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 2.9 Activate Restore of Informix DB (SEM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 2.10 Activate Restore of Oracle database in RC . . . . . . . . . . . . . . . . . . . . . . . . . 78 A50023-X4006-A210-6-7619 5 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management Illustrations Fig. 1.1 External backup server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Fig. 1.2 Data safeguarding - HW architecture example . . . . . . . . . . . . . . . . . . . 15 Fig. 1.3 Sequence for backup installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Fig. 1.4 Important backup and recovery options. . . . . . . . . . . . . . . . . . . . . . . . . 40 6 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Tables Tab. 1.1 Tape drives properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Tab. 1.2 Submitted jukebox types to the supported disk sizes . . . . . . . . . . . . . . . . 9 Tab. 1.3 Location of files after RC installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Tab. 1.4 Location of commands for nwadmin/nsradmin . . . . . . . . . . . . . . . . . . . . 21 Tab. 1.5 Available functions via graphical interface (nwadmin) . . . . . . . . . . . . . . 29 Tab. 1.6 Brm.cfg variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Tab. 1.7 Brm_srv_resource.cfg variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 A50023-X4006-A210-6-7619 7 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1 Backup & Restore Management 1.1 Architecture To backup and restore files you will need an internal and an external tape drive. For the Backup and Restore feature to work you will need to designate a backup server. This can be the OMP or an external backup server. The Server Module component must be installed on the designated backup server (OMP) or external backup server. The Client Module component must be installed on all OMPs that need to be man- aged for data backup. The Oracle Module component must be installed on the OMP where the Oracle RDBMS runs. The SW is automatically installed by the RC installation procedures. A license is required for the backup server and for each OMP + OMT as a client. You have got following SW modules: Solstice Backup Network Edition (SBU), Autochanger Software Module 1-8 slots/1-9 slots (SBU version 6.1.3), 1-16 slots, BusinessSuite Module for Oracle Unix client, Jukebox Module. The Solstice Backup Network Edition Version 5.1 permits a maximum of 10 client con- nections. For SBU Version 5.5.1 or higher the maximumnumber of permitted client con- nections is 4. i Up to now only single OMP installations are supported. i There is no released configuration where the OMT is used as a backup server. ! The Technical Stocklist is a constituent part of the Release Description that will be supplied and made available to the customer with each new Load Patch. Due to ongoing technical improvements, some of the specifications given in this manual may not correspond to those of the delivered components. Therefore, it is important to note that only the Technical Stocklist is binding for the delivered components. 8 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.2 General description of Backup and Restore management On the OMP as internal Backup Server (released Configuration) it's required to have a DDS- Drive without a Tape changer named as "internal Tape" (e.g. on SUNFIRE 4800 the Tape in the D240 Unit) and a Tape-Drive with a Tape changer Unit (see Tab. 1.1) named as "external Tape". The internal tape - the drive without Tape changer Unit - can be used only for purpose of Basic Backups (offline), accordingly it is not applicable for backup/restore operations, which will be executed with the help of Solstice Backup Utility. Reason: SBU tools don't support the handling of these drives. The tape drive with Tape changer Unit - alias jukebox - is maintained for on-line backup operations, so Basic Backups fulfillment with jukebox is not supported. For Basic Backups on the OMT there must be an internal Tape (DDS Type), in most cas- es this Drive is a Stand alone DDS3 - Tape Drive Unit. This Unit is connected to a build in SCSI- Controller on the OMT. The RC has got four external tape devices for backup management: external 12/24 GB 4 mm DDS-3 tape drive, StorEdge Autoloader L280 (with 8 slots 280/560 GB), StorEdge Autoloader L9 (with 9 slots 360/720 GB). StorEdge Autoloader L8-LTO Ultrium (with 8 slots 800/1200 GB). The four different tape drives offer different capacities and read/write performance. The amount of data to be backed up and the backup time available determine which tape drive is used. The most important device properties are shown below: These tape drives can compress the data before writing them onto the tape, effectively increasing the tape capacity. The compression ratio, however, depends largely on the kind of data. The less randomthe data, the better it can be compressed. A compression ratio of 1.5 is a good approximation. Drive type: DDS-3 StorEdge L280 StorEdge L9 StorEdge L8 Write speed: 1 MB/sec. or 3.6 GB/hour 3.5 MB/sec. (native 5) 6 MB/sec. 15 MB/sec Native capacity: 12 GB (on single tape cassette) 45 GB (on single tape cassette) 40 GB (on single tape cassette) 100 GB (on single tape casette) Compressed capacity: 108 GB (optimum value) 420 GB (optimum value) 720 GB (optimum value) 800 GB (optimum value) Number of slots: 6 8 9 8 Jukebox type: 4mm/4mm_xxGB dlt7000 dlt8000 LTO_Ultrium Note: Backs up 9 GB in 2.5 hours. (xx represents a numer- ic value - 4, 8, 12 or 20) Backs up 9 GB in less than 45 minutes. Backs up 21.6 GB in less than 60 min- utes. Tab. 1.1 Tape drives properties A50023-X4006-A210-6-7619 9 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management With the help of table Tab. 1.2 can be selected the submitted jukebox type by the cur- rent configuration. The OMP and OMT columns of the tabloe contain the size of internal (external disks in Gigabytes on the given machine by the conceded HW configuration. The capacity reveals the prospective size (in Gigabytes) of the backups in compressed mode in a week. In the last column are represented the suggested jukebox type to the selected configuration. In consideration of subsequent configuration changes we sug- gest always the largest supported jukebox type. The table mentioned above was calculated with 1 OMT (backup client) accordingly by more acting OMTs have to be taken account twice of OMTs sizes and attend to this value (by capacity column) have to find the most opposite jukebox type. OMP OMT Internal disk (GB) External disk (GB) Internal disk (GB) Capacity claim in a week Jukebox 4x36 20 400 L8/L9/L280 36 432 6x36 20 544 L8/L9 36 36 576 8x36 20 688 36 720 L8 12x36 20 976 36 1008 Tab. 1.2 Submitted jukebox types to the supported disk sizes 10 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 The tape drive type and the tape device path are automatically detected during activa- tion (which is part of the installation). If jukebox detection cannot find the tape drive type or device, it will terminate the run. If at the start of jukebox detection the media type is defined as a file in Installation & Update of RC Software (IUM) configuration file's general_environment section (in the OMC_JB_Devicetype variable), then backup will be made in the given file systempath, what is defined in the same section in OMC_JB_Device variable skipped the detection. When in the OMC_JB_Device variable(s) are defined several pathes separated by space, they will be used as a tape in the jukebox, namely all of them will be labeled to the target pool. The RC offers three backup and restore features: Basic backup&restore, Backup&restore for Unix le systems, Backup&restore for databases There are two possible backup and restore architectures: OMP as backup server (please refer to Fig. 1.2) External backup server (please refer to Fig. 1.1) i Amount of tapes to be used in jukeboxes: When jukebox detection was successful, all in the foregoing not labeled tapes - which are loaded in the slots - will be la- beled for the pool(s) of Radio Commander. For the completion of labeling in the jukebox there must be at least one free - not labeled - tape. Irrespectively of this fact you are advised to fill all available slots in jukebox with free tapes before labeling - except the last slot, which maintains the cleaning tape -, considering the large size of backup dataset. Dependent on the jukebox type you should label for the pool(s) of RC application as many tapes as possible: - DDS-3 StorEdge: 5 tapes for the goal of Radio Command- er backups + 1 cleaning tape, - L280 or L8 StorEdge: 7 tapes for the goal of Radio Com- mander backups + 1 cleaning tape, - L9 StorEdge: 8 tapes for the goal of Radio Commander backups + 1 cleaning tape, - by use of other jukeboxes: try to label so many tapes for Radio Commander, which can contain all backups, what will be done during a week in consideration of possible size of RC application and the capacity of a single tape in the juke- box. This possibility is valid only for manual configured exter- nal backup server. It is advisable to sustain permanently enough tapes for Radio Commander backups under work. A periodically renewal of the tapes should be considered too. A50023-X4006-A210-6-7619 11 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1.3 Data to backup The following components need to be backed up and will need to be restored if the sys- tem crashes: 1. Operating System data (OS data) Unix le system (UFS) data Low frequency data changes Extraordinary data changes Systemsoftware should be backed up after each installation or upgrade. This has to be started manually as the modications do not occur at regular intervals. 2. User data (3rd-party tools, OMP or OMT SWload, OMP or OMT conguration les such as the CFG omcconf.txt le or other statistics data). Unix le system (UFS) Low frequency data changes Ordinary data changes These data should be backed up every time a change is made, but they can be backed up regularly. 3. Database data and log data Unix le system (UFS) High frequency data changes Ordinary events data changes This data should be backed up regularly. The Oracle and Informix databases log all transactions in special log files that are backed up when they become full. If the data- bases need to be restored, these transaction logs can be used to redo all the transac- tions. This brings the databases up to date. All RCbackup data activities are ordinary activities and can be executed both in ONLINE mode and in OFFLINE mode. The ONLINE mode runs in a shared environment that reduces the load on system re- sources. 1.3.1 Basic Backup&Restore These simple OFFLINE backup procedures are based on an operating system com- mand and provide a snapshot of OMP/OMT data taken using the Unix command ufs- dump. It can be used to backup and restore OMP/OMT internal disks, OMP configuration data and the OMP external disk layout. This is a local function on each machine. The backup and restore activity has to be car- ried out locally on each machine. This feature can be used to prepare the disaster recovery tape. i The Operating system data backup should be done for each OMP and each OMT. 12 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.3.2 Backing up Unix file systems Periodic backups can be scheduled for the OMC (RC) UFS file system: scheduling backup for OMP user data (OMP SW load, conguration data and OS settings) scheduling backup for OMT user data (OMT SW load, conguration data and OS settings). 1.3.3 Backing up databases For Informix (SEM) RDB, the backup procedure consists of backing up data files belong- ing to dbspace, the logs and control files. For Oracle RDB, the backup procedure consists of backing up data files belonging to tablespace, the logs and control files. 1.3.4 Restoring user data The restore procedure can restore files from file systems, directories or disk partitions. All RC restore/recovery data activities are extraordinary actions and must be executed in OFFLINE mode, except: some les and directories may be restored if they are not relevant to the working RC or if the user has relocated them Important properties of file system restore using Solstice Backup Utility: The recover searches (browses) the client le index for a specied client and recov- ers les frombackup volumes to the specied client. Consequently the operator has the possibility to restore the data which were backed up from the local machine, or fromother machine, but he cant start a restore fromthe backup server (or fromoth- er client) to a given backup client, because restore is a local function of each ma- chine. Restore operation can overwrite the les, which are located on the disk - if it is nec- essary - and the restored backup set contains these les. But Solstice Backup doesn't overwrite les, which are located in the target directory of the restore, but the restored backup set doesn't contain entries with the same name. So it is not possible to loose data, which are created after the restored backup set. 1.3.5 Restoring databases All RC restore/recovery data activities are extraordinary actions, and during this actions Radio Commander must be in OFFLINE mode unless: Your database server has failed, in this case you can restore data using a warm re- store. A warm database restore can be carried out for data in non-critical storage spaces, but this method are not directly supported in Radio Commander. i For a successful Informix database backup you must be sure to have enough free disk space on the / (or /var - if it exists). Minimum is the current size of the Informix da- tabase. Free disk space of 50 MB should be sufficient. A50023-X4006-A210-6-7619 13 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management It is generally advisable to restore the entire database. If the operator can guarantee that only one or a few dbspaces need to be restored, it is possible to restore just those. Db- spaces/tablespaces are the smallest units that can be restored. During the restore, INFORMIX and ORACLE must be in OFFLINE mode. This means that the RC cannot run at the same time. Certainly the mode of databases will be checked and changed automatically during the restore. Please do not change this mode during database restore. For Informix you can check the mode as Informix or root user with following command: onstat -l 2>/dev/null If there is no output the database is in offline mode. Otherwise the database is online or in a interim state. For Oracle you can check the mode as Oracle user with following commands: a) ps -ef|grep ora_arc If there is no running process oracle is in offline mode. or b) sqlplus omcdbsys/omc_br60 select name, log_mode from v$database, quit The larger the amount of data to be restored, the longer the RC will not be available to other components in the RC system. Restoring the entire database is less risky than re- storing only parts of it. There is a lower risk of introducing inconsistencies between the restored dbspaces and those that are not restored. If it is not completely clear which parts of the database need to be restored, bearing in mind the unwanted side effects of restoring only parts of a database, a complete database restore should be considered. 1.3.6 Backup & Restore procedure logging The radio commander writes a logfile concerning all the important backup and restore activities. This logfile is in /nsr/applogs/ directory with name brmsr.log. There are the log messages about the following actions: all basic backups jukebox conguration, database backup and restore (Informix and Oracle too), backup enable\disable (Networker, Oracle, Informix). 14 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.3.7 Examples for backup- & restore configurations The following two figures shows you examples for External backup server and for Da- ta safeguarding. Fig. 1.1 External backup server i The optical drive is not yet released for the backup device! A50023-X4006-A210-6-7619 15 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management Fig. 1.2 Data safeguarding - HW architecture example 16 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.4 Description of installation from Backup & Restore man- agement viewpoint 1.4.1 Location of files after RC installation After installing the 3rd-party tool Installation & Update of RC Software (IUM), install the Backup & Restore Management (BRM). After successful BRM installation, the neces- sary files are placed in the following directories: (To make a backup you need to find the start script in the /usr/sbin/nsr/ directory. You can also find the recovery scripts in /usr/sbin/nsr.) 1.4.2 Installation of Backup & Restore Management (BRM) RC installation covers the BRM installation/activation too. The installation is controlled by the Installation & Update of RC Software (IUM) configuration file, named ium_configuration.cfg, that you can find in the /opt/omc/install directory. This file con- tains more variables for BRM activities. Before installation, you have to set the variable values. The BRM-specific variables in this configuration file are as follows (see example in Appendix): BACKUP_SERVER_TYPE: The most important BRM-specific variable. This variable gives the current backup server type. It can pick up the following values: "None" - You do not want to use the extended basic backup feature based on Solstice Backup Utility. This means you cannot use the (extended) BRM scripts. "Internal" - The OMP machine is the backup server (Networker server). The intbackupserver value for the machine_roles variable has to be written in the config file in the machine descriptor section. The given machine will be the backup server. "External" - The backup server is not an OMP or OMT but an external machine. Name Function RUNNING RUNNING/swl This contains all the scripts, programs (no specified start path required) and the BRM install files. /usr/sbin/nsr/ Scripts used by SBU for backup and restore /usr/bin/nsr/ Path of SBU binaries Tab. 1.3 Location of les after RC installation i Before installation ensure that the /opt/omc/install/ium _configuration.cfg file exists with valid backup parameters, and the ium_parse_ini.sh parser file in the same directory. i There is no released configuration where the OMT is used as a backup server. A50023-X4006-A210-6-7619 17 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management The jukebox and all devices needed for backup or restore activities are connected to this machine. These variables are the basis for the RC backup installation (and all backup/restore ac- tivity) and consequently, modification is not possible after installation. The backup installation runs using the following sequence (please refer to Fig. 1.3): i Important: The BRM installation runs using the BACKUP_ SERVER_TYPE and so this value is most important. NSRServer The machine name of backup server. The Solstice Backup server will be installed on this machine and the NSR resourc- es activated. OMC_Poolname This pool stores all the RC-specific backup data such as all the OMP, OMT, INFORMIX databases and the Oracle back- up, except for bootstrap information and the default backup. OMC_Bootpoolname This pool stores the bootstrap information and client indexes. Currently the pool set is the same as that used for data. JB_name This variable is the name of the Jukebox. This parameter is not used, if the backup device is defined as a file. OMC_JB_Device Jukebox-specific data for connection to the Networker serv- er. This gives the path name for the jukebox execution files, if jukebox detection cannot find a valid path to the jukebox de- vice during installation. The default value is /dev/rmt/1cbn. If the specified type of device is a file, several file system pathes can be defined separated with space as backup/re- store target device. This will be used as the volumes of the jukebox. Otherwise it will be overwriten during backup server configuration, when the value is not equal with the detected jukebox device. OMC_JB_Devicetype For further details see Chapter 1.2. If it is necessary it will be overwritten automatically during backup server configuration. Precondition: It's value is not specified as "file". OMC_Inactivitytime Specifies the number of minutes of inactivity before the con- clusion is reached that a client machine is (presumably) per- manently hung. Access attempts will be stopped after "n" minutes. Default value: 30. Modification not needed. i If volumes are loaded in jukebox, unload the mounted volumes fromthe jukebox device before installation, because Solstice Backup isn't able to find the mounted tapes during inventory and labeling, accordingly the loaded tapes won't be labeled. 18 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Fig. 1.3 Sequence for backup installation The figure shows the important steps of BRM installation. The brm_srv_jbconfig.sh script configures the jukebox and creates the jukebox resource, Networker. Without this, the jukebox cannot be handled. The brm_srv_resource.sh creates the NSR resources that are needed to create backups. The jukebox slots or file devices are then labeled. i Although this installation creates an automatically mechanism, the backup/restore sys- tem cannot repair any kind of failures (for example: tape I/O error, etc.). Accordingly it will be more practical to get up a backup preserving way, which can ensure the system recover, when the tapes - and/or any backups on it - in jukebox will be damaged, or will be inefficient. Proposal: take off the tapes from the jukebox once in two weeks and replace them with other tapes. So you retain always 2 tape sets to restore beside the tape set of jukebox. But this way is only a proposal; please consider the local conditions and expectations. A50023-X4006-A210-6-7619 19 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1.4.2.1 Installation with internal backup server This is a short description of what happens when the BACKUP_SERVER_TYPE is set to internal. (This is carried out by the RC installation.) If the backup is server internal, the installation process checks the jukebox type. If it is a 4 mm, then the kernel must be modified and the machine rebooted. If any other juke- box type is connected to the backup server (dlt7000, dlt8000 or LTO_Ultrium), kernel modification is necessary only for better using of jukebox capacity and performance. The following installation steps are the same for all jukebox types. If the jukebox re- source does not exist, brm_srv_jbconfig.sh runs which configures the jukebox. Finally, brm_srv_resource.sh runs, which creates all the Networker resources (schedules, groups, clients, pools and labels). This uses the ium_configuration file. (If you modify the name of the ium_configuration file, you have to set this filename in the brm_functions.sh (in /usr/sbin/nsr directory) in the BRM_CFG_FILE variable. All BRM scripts will read the re-named config file.) 1.4.2.2 Installation with external backup server The Radio Commander supports the installation method of an external backup server: The operating systemand Solstice Backup Utility are already installed on the current machine and the necessary conguration settings will be executed manually using this document through the operator. The backup server will be installed during Radio Commander installation using RCebs package and this document only. This chapter contains the necessary infor- mation about this case. Precondition of the correct installation of the external backup server SW (RCebs): Installed Solaris 2.6/8.x operating system on the machine. As the newest SW ver- sion of Solaris 8.x will be distributed, we speak in further text only about Solaris 8. Installed Solstice backup server (SBU version 6.1.3, with valid license for Oracle module). Installed Jukebox to the machine. in the /opt/omc/install directory has to be copied: ium_parser_ini.sh ium_configuration.cfg (They have to be copied manually from the OMP or OMT machines, which can be found in the /opt/omc/install directory in the OMP/OMTs.) The ium_conguration.cfg le consist: information about the backup clients (List of the OMP/OMTs machine with their IP addresses, which will act as the back- up clients, and consist of information what kind of data can be found on the backup clients, e.g. oracle database.) the relevant settings for the conguration of Solstice backup server (See details in the "Description about the setting in the ium_conguration.cfg.) One of the important setting is, that the BACKUP_SERVER_TYPE has to set to ex- ternal. Following activities are carried out during installation of the RCebs SW. (ebs = external backup server): 20 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Conguration of the jukebox, if necessary Conguration of the backup server and his client, which means the creation of the resources (on the external backup server). The empty jukebox slots are then labeled for RC usage. Description about the setting in the ium_configuration.cfg The installation of the Rcebs is controlled by the Installation & Update of RC Software (IUM) configuration file, named ium_configuration.cfg, which has to be located in the /opt/omc/install directory. This file contains more variables for BRM activities. Before in- stallation, you have to set the variable values. The Backup & Restore Management-spe- cific variables in this configuration file are as follows (see example in Appendix): BACKUP_SERVER_TYPE: The most important BRM-specific variable. The variable gives the current backup server type. It can pick up the following values: "External": The backup server is not running locally on this machine. This is the used configuration for backup clients. "Internal": The backup server is running locally. The jukebox and all devices needed for backup or restore activities are connected to this machine. 1.4.2.3 External backup server configuration There are two ways to execute external backup server configuration: Using the predened EBS package method, which will be run automatically during installation. In this case, after installation of the OS and Solstice, the third-party backup tool will be named /usr/sbin/nsr/brm_srv_resource.sh script, which will make the conguration utilize the conguration le data, as specied at the start of instal- lation. The script will create all necessary backup resources, and label the available free volumes in the jukebox to the specied pools. This method is particularly helpful if the backup server wasn't installed previously, and it is designed principally to back up/restore Radio Commander software only. Manual conguration of external backup server. When all resources will create a trough operator, and the jukebox labeling is a task for him. More exibility than the backup resources can be specied, but it requires knowledge of Solstice Backup or Legato Networker. Because the configuration is run automatically by EBSpackage installation, only manual configuration of the external backup server will be defined here, which is based on the same steps as brm_srv_resource.sh script, and therefore the EBS package. Configuration steps The configuration can be executed with the help of nsradmin as a command-line tool, or nwadmin as a graphical interface of the SBU/Networker. The description will be present in the administration method in the nwadmin tool. The location of these commands is as follows: i Important: The BRM installation runs using the BACKUP_SERVER_TYPE and so this value is most important. A50023-X4006-A210-6-7619 21 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management The resource names and data can be changed by all resources, but these changes have to be executed consequently, because any difference can cause partial backup errors. Preconditions: Operating system installed Solstice Backup/Legato Networker 3rd party tool installed Machine name and IP address entries in /etc/hosts le, which will be used as backup clients Roles of backup clients in Radio Commander software Administrator list update The administration rights of Radio Commander user to the backup server can be set here. Users who can make changes on the backup server from the backup client side can also be specified here. To carry out this step, select the "Server update" item from the "Server" menu. Select "Administrators:", and specify the necessary users in the following format: <user>@<hostname> For example: root@ssrv16sd. In this case, the root user of machine ssrv16sd can modify resources, and start/stop backups, when the backup client is installed on ssrv16sd. It is practical to admit all roots of backup clients to the administrator list. Create OMP resources Creation of the following resources for using OMP backups is recommended: 1. Click on "Schedules" in the "Customize" menu. 2. The "Schedules" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR schedule name: OMPtimer period: week action: incr full incr incr incr full incr 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: OMPtimer_Browse period: days Solstice Backup Networker nwadmin /usr/bin/nsr /usr/bin nsradmin /usr/sbin/nsr /usr/sbin Tab. 1.4 Location of commands for nwadmin/nsradmin 22 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: OMPtimer_Retention period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Groups" in the "Customize" menu. 2. The "Groups" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR group name: OMPtimer autostart: Enabled start time: 0:30 4. Left mouse click on the Apply button. 1. Click on "Client setup" in the "Clients" menu. 2. The "Clients" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR client name: <hostname> schedule: OMPtimer browse policy: OMPtimer_Browse retention policy: OMPtimer_Retention group: OMPtimer save set: /nsr/applogs backup command: saveomp 4. Left mouse click on the Apply button. If there are two or more OMPs which are backup clients in the system, a separate client resource must be created for all clients. Creating Informix database resources The creation of the following resources for using Informix backups is recommended: 1. Click on "Schedules" in the "Customize" menu. 2. The "Schedules" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR schedule name: MIStimer period: week A50023-X4006-A210-6-7619 23 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management action: full full full full full full full 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: MIStimer_Browse period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: MIStimer_Retention period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Groups" in the "Customize" menu. 2. The "Groups" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR group name: MIStimer autostart: Enabled start time: 21:00 4. Left mouse click on the Apply button. 1. Click on "Client setup" in the "Clients" menu. 2. The "Clients" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR client name: <hostname> schedule: MIStimer browse policy: MIStimer_Browse retention policy: MIStimer_Retention group: MIStimer save set: /nsr/applogs backup command: savemis 4. Left mouse click on the Apply button. If in the system there are two or more OMPs which are backup clients and there are In- formix databases installed, a separate client resource must be created for all clients. 24 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Creating Oracle database resources The creation of the following resources for using Oracle backups is recommended: 1. Click on "Schedules" in the "Customize" menu. 2. The "Schedules" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR schedule name: Oracletimer period: week action: full skip skip skip full skip skip 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: Oracletimer_Browse period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: Oracletimer_Retention period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Groups" in the "Customize" menu. 2. The "Groups" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR group name: Oracletimer autostart: Enabled start time: 0:30 4. Left mouse click on the Apply button. 1. Click on "Client setup" in the "Clients" menu. 2. The "Clients" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR client A50023-X4006-A210-6-7619 25 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management name: <hostname> schedule: Oracletimer browse policy: Oracletimer_Browse retention policy: Oracletimer_Retention group: Oracletimer save set: /nsr/applogs backup command: saveoracle 4. Left mouse click on the Apply button. If in the system there are two or more OMPs which are backup clients and Oracle is in- stalled, a separate client resource must be created for all clients. Creating Oracle archive log resources The creation of the following resources for using Oracle archive log backups is recom- mended: 1. Click on "Schedules" in the "Customize" menu. 2. The "Schedules" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR schedule name: Oraclelogtimer period: week action: full full full full full full full 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: Oraclelogtimer_Browse period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: Oraclelogtimer_Retention period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Groups" in the "Customize" menu. 2. The "Groups" window will be displayed. Now click on the Create button. 3. Fill in the following elds: 26 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 type: NSR group name: Oraclelogtimer autostart: Enabled start time: 8:00 interval: 4:00 4. Left mouse click on the Apply button. 1. Click on "Client setup" in the "Clients" menu. 2. The "Clients" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR client name: <hostname> schedule: Oraclelogtimer browse policy: Oraclelogtimer_Browse retention policy: Oraclelogtimer_Retention group: Oraclelogtimer save set: /nsr/applogs backup command: saveoralog 4. Left mouse click on the Apply button. If in the system there are two or more OMPs which are backup clients and Oracle is in- stalled, a separate client resource must be created for all clients. Create OMT resources Creation of the following resources for using OMT backups is recommended when there is a machine in the system which will be used as a backup client is installed as OMT: 1. Click on "Schedules" in the "Customize" menu. 2. The "Schedules" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR schedule name: OMTPtimer period: week action: incr full incr incr incr full incr 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: OMTtimer_Browse period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Policies" in the "Customize" menu. A50023-X4006-A210-6-7619 27 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 2. The "Policies" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR policy name: OMTtimer_Retention period: days number of periods: 6 4. Left mouse click on the Apply button. 1. Click on "Groups" in the "Customize" menu. 2. The "Groups" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR group name: OMTtimer autostart: Enabled start time: 0:30 4. Left mouse click on the Apply button. 1. Click on "Client setup" in the "Clients" menu. 2. The "Clients" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR client name: <hostname> schedule: OMTtimer browse policy: OMTtimer_Browse retention policy: OMTtimer_Retention group: OMTtimer save set: /nsr/applogs backup command: saveomt 4. Left mouse click on the Apply button. If in the systemthere are more machines which are installed as OMT and backup clients, a separate client resource must be created for all machines, which will use the saveomt script in the backup command argument. Resource connection to the jukebox If the above-mentioned resources exists, the operator has to make a connection be- tween these resources and the jukebox. For this purpose, a pool resource and its label must be selected, and it must contain all the above-mentioned groups, if the jukebox and device resource already exist. If the pool is not available for this purpose, it is recom- mended that one is created using the RC pool settings, which are as follows: 1. Click on "Label templates" in the "Customize" menu. 2. The "Label templates" window will be displayed. Now click on the Create button. 3. Fill in the following elds: name: OMCplus type: NSR label elds: OMCplus,001-999 28 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 separator: . 4. Left mouse click on the Apply button. 1. Click on "Pools" in the "Media" menu. 2. The "Pools" window will be displayed. Now click on the Create button. 3. Fill in the following elds: type: NSR pool name: OMCplus pool type: Backup label template: OMCplus groups: OMPtimer, OMTtimer, MIStimer, Oracletimer, Oraclelogtimer 4. Left mouse click on the Apply button. When the backup resources are created, tape labeling must be effected as follows: 1. Select Mount from the Media menu, and left mouse click on it. 2. The Jukebox Mounting window is displayed, which contains any information about tapes which are placed in the slots of current jukebox. If a star is displayed in the volume column, then the current slot has not yet been labeled, but it does contain a tape and may be relabeled. If the Pool column is empty, the tape in the given slot was labeled, but this information has been removed from the media database, and it is therefore available for re-labeling. If the whole row by the slot is empty, the slot is currently empty. 3. When the slots for labeling are selected, click on the Cancel button. 4. Then select "Label" from the Media menu using the left mouse click. 5. Enter a slot number into the "First slot:" eld, and repeat it in the "Last slot:" eld. 6. From Pool click on the pool, where the current volume must be labeled 7. Click on OK This procedure can be repeated several times. Once labeling is finished, the external backup server configuration is ready. A50023-X4006-A210-6-7619 29 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1.5 Administration Guide The backup activities can run in 2 different modes: automatically, as a scheduled job manually, on demand when initiated by the user The most widely used backup, the scheduled backup. If you have successfully installed BRM, backup runs automatically at the specified time, defined in the schedule resources (for further details see chapter 1.8). If you want to check or modify these settings, you can do it fromthe SBU administration program (nwadmin). This backup & restore commands can be called from RC GUI. The commands (backupAdmin, recoverOmp, recocerOmt) can be launched via RC Re- gion, RC Sum panel. a) Double click in the RC Sum icon to open the RC Sum panel. b) In the RC Sum panel right-click on the RC icon to choose the context menu. c) Choose Backup & Restore Management. All BRM-commands will be displayed. d) Choose either backupAdmin, recoverOmp or recoverOmt. backupAdmin: The command allows to start the Solstice Backup Administration on OMT. recoverOmp: The command allows to start the Solstice Backup Recovery on OMP. recoverOmt: The command allows to start the Solstice Backup Recovery on OMT. BackupAdmin (nwadmin applications) can be used to modify the BRMsettings and con- trol the backup activities. The BRMjob can also be controlled fromthe terminal window. Consequently, BRM jobs can be controlled from: the RC GUI using backupAdmin the terminal using the ium_maintenance.sh script (see maintenance chapter 1.6). The following functions are available via the graphical interface (nwadmin): Panel Activity Write /Session Controls the current job status and its opera- tion. - /Server/Group Control Show the current group status. Starts or stops the given groups. Clients/Client setup Sets backup client types. Group/Schedule Sets the start time of automatic backup. Tab. 1.5 Available functions via graphical interface (nwad- min) 30 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.5.1 Backup and Restore possibilities on the OMP On the OMP, the scheduled backup is used. The backup process can also be started using backupAdmin or ium_maintenance.sh (or in standard UNIX mode). 1.5.1.1 Graphics mode To start the backup from graphical user interface environment, complete the following steps: 1. Run the BackupAdmin command from the RC GUI, RC Region panel, RC icon or directly using /usr/bin/nsr/nwadmin. 2. Choose the "Group control". 3. Select the OMPtimer entry from the window. 4. Click on the start button. If the group is started, all OMP clients will be saved. The group backups the internal and external disks of all OMP clients. But from graphics mode there are no possibilities to start only OMP internal/external disk backup. If you want to cancel the save process, click on the stop button. You can monitor the backup process in this window. 1.5.1.2 Manual mode Manual mode is controlled using ium_maintenance (see chapter 1.6 on Mainte- nance) It is possible to start the backup using a simple Unix command. The OMP backup can be started manually using the saveomp script. You should enter the following command in a Unix shell /usr/sbin/nsr/saveomp The saveomp has got two parameters: internal, stores the entire OMP internal disk external, stores the entire OMP external disk. Without input of parameters the command will save the internal and external disk of OMP (functionality of OMPtimer group). The backup will be run on the current OMP machine. If you want to back up a different client, you need to use the nwadmin command in graphics mode or telnet to the other client and run the saveomp command from there. saveomp stores the entire OMP internal disk. saveompsw only stores the RC SWload and configuration data. This can only be run from maintenance and not as scheduled job. 1.5.1.3 Manual mode recovery If OMP files or directories are lost or if the disk crashes and the relevant files have been backed up, they can be recovered by running the recomp.sh script. The script has four switches: force, internal, external and -m. A50023-X4006-A210-6-7619 31 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 'Force' means that during the restore all files are overwritten, without this switch none are overwritten (default functionality). 'Internal' restores the internal disk and 'external' restores the ufs part of the external disk. When these parameters are not used, the script will restore both: the internal and the external disks. '-m' gives the possibility to control the recover manually. This possibility is very helpful, when the last OMP backup went wrong, or on other reason the use of an older backup is necessary. In this case the script will show the available OMP backups and the oper- ator can select the sufficient backup set(s) from a menu. If an error occurred during recover, the script will show following line: Error during recover process (Code: <error>). Continue [y/n]: By selecting "y" the script will use automatically the manual control possibility, and shows a list of the available backup sets, which can be used for recover. "n" is used to exit the script. 1.5.1.4 Graphics mode recovery Naturally, in graphics mode you can only restore those files which do not affect the op- eration of the machine. This must be done by the administrator. 1. Run the backupOmp command from RC GUI, RC Region panel, RC icon or direct using the /usr/bin/nsr/nwrecover. 2. Choose the recovered client from Change/Client... ordinary. 3. Select the les or directories that you want to recover. 4. Click on the Start button. 5. Click on the OK button in the new window. 'Recover' will run. The recover process only happens on the current machine. i It is intended to change the GUI commands backupOmp in recoverOMP and back- upOmt in recoverOmt. 32 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.5.2 Backup and Restore on an OMT The OMT backup and restore options are similar to the OMP. The differences are: 1.5.2.1 In graphics mode After starting the backupAdmin command on the RC GUI, the backup group/schedule name is OMTtimer. 1.5.2.2 In manual mode The Unix backup script is /usr/sbin/nsr/saveomt and saveompsw. The backup path can be entered into the saveomt.cfg file in the same directory. 1.5.2.3 Recovery The recover script in Unix is recomt.sh, which has 2 switches: force and -m. 'Force' means that during the restore all files are overwritten, because in default case none files are overwritten. '-m' gives the possibility to control the recover manually, which can be helpful, when the operator doesn't want to use the last backup set for restore (for example: it is damaged, etc). In that case the operator can choose the corresponding save set from a manual of available backup sets. If an error occurred during restore - independent from current control mode -, the script will show following line: Error during recover process (Code: <error>). Continue [y/n]: By selecting "y" the script will use automatically the manual control possibility, and shows a list of the available backup sets, which can be used for restore. Press "n" to quit without successfully restore. 1.5.3 Informix database backup and restore with BRM The basic backup and restore options are the same with Informix as they are with OMP. Differences are as follows: 1.5.3.1 In graphics mode After the BackupAdmin command is started on the RC GUI, the backup group/schedule name is MIStimer. 1.5.3.2 In manual mode The Unix backup script is the /usr/sbin/nsr/savemis. 1.5.3.3 Recovery The recover script in Unix is /usr/sbin/nsr/brm_mis_resinformix.sh. A50023-X4006-A210-6-7619 33 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management The script has only one parameter: -m. Without this parameter the script will restore the Informix database fromthe last backup. Use the -mparameter, if you want to restore the database from another backup. In this case the script will show all available database backup, and give the possibility to choose the necessary backup. 34 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.5.4 Oracle backup possibilities with Backup & Restore Management 1.5.4.1 In graphics mode After the BackupAdmin command is started on the RC GUI, two backups are possible. The first is a full database backup with archived log files. This is run from Oracletimer. The second is only a backup of the archived logs without any other database data. This is run from OracleLogtimer. 1.5.4.2 In manual mode The full database backup script, with archived logs, is /usr/sbin/nsr/saveoracle and the script for the archived log backup is saveoralog in /usr/sbin/nsr/ path. If saveoralog is starting and an Oracle full backup was not executed up to now, the run of the saveoracle script will make a full backup. 1.5.4.3 Recovery The Oracle recovery can be run using the /usr/sbin/nsr/brm_ora_restore.sh script. This script contains the same rules as the OMP recover script. The brm_ora_restore.sh will shut down the database and then re-open. The brm_ora_restore.sh script can restore by SCN number, log sequence or a speci- fied time. These possibilities can be specified using the following parameters: The parameterized run is only executed, if automatically restore was not successfully, or the operator want to use an older backup to restore, because a tape error occurred, or the last backup content data are partly incomplete or undesirable. Without any parameters (default), the script runs with the parameter "-s", and it is the automatically method of Oracle database restore, which ask only about the overwrite so- lution, and restores the last known consistent state of Oracle. After Oracle restore a full Oracle backup is necessary (otherwise the archived log back- up makes no sense). Please refer to chapter 2.10 for further information. i The log backup runs automatically every 4 hours, but the full database backup is only scheduled for twice a week. s SCN based restore l log sequence based restore t time based restore u user defined SCN restore (from maintenance not available) m manual run, can be used with other parameters. i If during Oracle restore the Oracle backup activities produce error code(s), so you should not make backups! A50023-X4006-A210-6-7619 35 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1.5.5 Other backup possibilities It is possible to run other backup activities from the Unix shell. /usr/sbin/nsr/savepred.sh lists all other backup possibilities that are based on available *.cfg files. Backup paths can be defined in these *.cfg files. 1.5.6 About backup/restore logging The most important activities write their messages into a log file. The place of this log is the /nsr/applogs/ directory. The messages are readable in the brmsr.log file. The format of a log is the following: ---------------------------------------- Send: filename Date: day-month-year hour:min:sec Message: message text Error messages (if relevant) ---------------------------------------- Example: ---------------------------------------- Send: saveoracle Date: 08-10-02 10:09:02 Message: Oracle DB files backup: Ready. (Id: 1) Archived log(s) status: 1 closed. Archived log(s) backup: Ready. (Seq: 25) ---------------------------------------- i The words (text) with inclined character depend on the current settings and scriptnam- es. 36 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.6 Maintenance The ium_maintenance.sh script (in the running directory) provides a menu, where the operator can choose and perform Radio Commander specified activities. From this script you have got the possibility to start backup and restore processes, or to establish backup/restore settings. For further information refer to Fig. 1.4 which gives you an overview on the most impor- tant backup and recovery options. 1.6.1 Backup/Restore possibilities in maintenance The backup/restore activities are placed under Backup and Restore Management item in the maintenance. From this item you can start the following backup/restore processes: You can enable/disable the following backup activities: 1.6.1.1 Enable Backup These steps are made during Radio Commander activation, accordingly they have to be used if an error occurs during installation, or if the backup is to be installed later. Preconditions: Valid NSR Server Radio Commander must be down (without backup server change) Enable all backups - create all backup client resources on the SBU server, and en- able their backup starts automatically. In addition it makes the necessary settings to make Oracle database backups, but for this operation the Radio Commander have to be stopped. You can enable Oracle database backup, when the necessary back- up client (in Oracletimer group) is already created (see Congure Backup server). Congure NSR server - Apply it only if backup is enabled. It creates all Networker resources and labels the available empty slots. This is a useful option if an error oc- curs during installation, or the backup is to be installed later. Change Backup server - This option changes the NSRServer value in conguration le, modies the Oracle settings, and restarts the backup processes. This operation is on-line, accordingly it is usable under RC system and Oracle database running. 1.6.1.2 Disable backup Preconditions: Valid NSR server Radio Commander must be down (without backup server change) Disable all backups - Removes all backup client resources on the SBU server, and disable their backup starts automatically. Furthermore it can remove all settings, which are necessary to make Oracle backup/restore (offline operation, accordingly RC has to be stopped). A50023-X4006-A210-6-7619 37 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1.6.1.3 Backup From here can you start the available backup operations. Preconditions: Valid NSR server Enable Oracle backups OMP - Basic Backup These items are usable only in single user mode. Create Emergency Tape - Level 0 backup using omposbackup script. This script used the ufsdump command to backup the operating system.The using of this script is the same as for ompbasbackup, but it backups only the / partition. Backup the OMP internal disk - Level 0 backup using ompbasbackup script. It can save the /,/export and all available partitions to the tape. Details you can nd in chapter 2.3. - Menus of Basic Backup / OMP Backup the OMP internal disk - Run the OMP internal disk backup script, namely /usr/sbin/nsr/saveomp using the internal switch. Details are available in 1.5.1.2 chapter. Backup the OMP external disk (ufs part) - OMP external disk backup, using saveomp script with external parameter (Chapter 1.5.1.2). Backup the OMP SW Load and configuration data - Starting the saveompsw script (Chapter 1.5.1.2) OMT These items are usable only in single user mode. Create Emergency Tape - Start the omtosbackup backup, and backup -using ufs- dump command- the / partition of OMT. Backup the OMT internal disk - Details can you nd in chapter 2.4. - Menus of Basic Backup / OMT Backup the OMT internal disk - Run the OMT internal disk backup script /usr/sbin/nsr/saveomt (Chapter 1.5.2.2). Backup the OMT SW Load and configuration data - Start the OMT SW load backup script, namely /usr/sbin/nsr/saveomtsw (Chapter 1.5.2.2). Informix Backup Informix database - Run the Informix database backup script. Description is available in chapter 1.5.3.2. Oracle Backup the complete Oracle database - It backups all Oracle datales, close the cur- rent redo log, and save all available archived logs, and at the end backup the current control le. Backup the archived Oracle logs - It makes a backup set, which contains the ar- chived redo logs (before it closes the current redo log) and the current control le. The datales will be not saved. 38 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Predefined backups To detailed description see chapter 1.5.5. 1.6.1.4 Restore Preconditions: Valid NSR server Radio Commander must be down Available backups on tapes of jukebox External disk configuration Create the external disk (this activity configures the external disk during installation). OMP Restore OMP internal disk - Start the recomp.sh script with the switch force to re- store the OMP internal disk. Accordingly the script overwrites the current content of the internal disk, when there exist the restored le, and restore the external disk too. Details can you nd in chapter 1.5.1.3. Restore OMP external disk (ufs part) - Runs the recomp.sh external force script (chapter 1.5.1.3). It can restore only the external disk, and it overwrites the external, when the le already exists on the disk. Restore OMP SWLoad and conguration data - Starts the recompsw.sh script to re- store the SW load on the OMP machine. OMT Restore OMT internal disk - Restores the internal disk of OMT. It uses last available backups of the current client. It can restore only the current OMT machine. If you want to restore another OMT client, please use the telnet command, and start the restore from the given backup client (OMT). Restore OMT SWLoad and conguration data - It can restore the SWload of OMT machine using the recomtsw.sh script. The same conventions are effective as for OMT internal disk restore. Informix Restore Informix database - Runs the Informix database restore script. Description is available in chapter 2.9. Oracle Recover Oracle database - Recovers the Oracle database to the last consistence state. It uses the bmr_ora_restore.sh script without parameter. For details please read the chapter 2.10. Advanced restore possibility - These itemis helpful, when the last consistent state of Oracle is not usable, because the user removes data's, which are necessary, but the last backup didn't contain it, or the Oracle restore wasn't successfully. Details are available in chapter 2.10. - Time-based recover: Start brm_ora_restore.sh with the switches -m -t. - SCN-based recover: Start brm_ora_restore.sh with the switches -m -s. A50023-X4006-A210-6-7619 39 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management - Log sequence-based recover: Start brm_ora_restore with the switches -m -l. Create empty database - Creates an empty Oracle database in the same way as for OMP activation. 40 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.6.1.5 Important backup and recovery options Fig. 1.4 Important backup and recovery options A50023-X4006-A210-6-7619 41 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1.7 Disaster data recovery 1.7.1 Hardware preferences for Recover If hardware becomes damaged or destroyed, use the following selections to install and configure your system hardware correctly: Ensure that the replacement disk is as large or larger than the original disk. Try to use the same controller, driver, and SCSI ID used prior to the disaster. Use the same computer name, host ID, TCP/IP host name and DNS domain as by original machine. Internal Tape and/or Jukebox connected to the local machine (if backup device is specied as UFS le system, jukebox connection is not used). Available tape with a valid backup. This selection applies to the case, the recover will happen to the original hardware, or to another machine. If the old and new hosts ID aren't identical, contact local SUN service or local staff. 1.7.2 Perform disaster recovery Precondition: In addition to the jukebox -connected to the backup server (OMP as internal or external), a (internal or external) tape is necessary to carry out a complete disaster data recovery. The jukebox is only relevant in case the OMP is used as backup server. If an external backup server is installed by the customer this backup server must fulfill the configurations rules. The first step in preparing for a disaster is to create a "disaster tape". This disaster tape contains the operating system, the Solstice Backup binaries (situated on the root partition) and the SBUresources (indexes). This can be done using the basic backup feature (backup to tape). A backup should be made after every installation and after main changes to the system such as making or increasing partitions. Create disaster tape (contains OS, SBU binaries and SBU indexes) /usr/sbin/nsr/omposbackup and a configuration file that defines which partitions are to be saved /usr/sbin/nsr/omposbackup_fslist i If the recovery happened to other machine, it is most important to use the same host ID, because all used licenses are depends from the host ID (It fixed on the Clock-Board of the machine). i For level 0 backup Stand Alone Tape unit DDS-3 single tape can be used for Sun E420R, Sun Blade 100, 150, 1000, 2000, Sun Ultra 10, 60. 42 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 There is an option to back up the entire internal disk using basic backup features: /usr/sbin/nsr/ompbasbackup /usr/sbin/nsr/ompbasbackup_fslist and to restore the entire internal disk: /usr/sbin/nsr/omposrestore then /usr/sbin/nsr/ompbasrestore) If disaster strikes (internal disk crash) the system can be restored with the following steps: - First step: restore the operating system + SBU configurations/index files: Prerequisite: The correct partition and ufs file system on the internal disk. Type the following: 1. boot cdrom -s 2. cd /tmp 3. mt -f /dev/rmt/0mbn rewind 4. tar xvf /dev/rmt/0mbn 5. cd ./backup 6. ./omposrestore This step creates again all existing partitions - all directories and files will be lost from the disk -, and creates a working OS with SBU. - Then: Layout re-creation (only in case of disk change). Please refer to chapter 1.6 Mainte- nance. restore the ufs part of the internal disk: /usr/sbin/nsr/omprestore This step involves restoring the SBU indexes and the ufs part of the internal disk. More precisely the activities depend on the role-list of the machine. If BACKUP_SERVER-TYPE=none, then the basic backup feature was called: /usr/sbin/nsr/ompbasrestore If BACKUP_SERVER-TYPE=internal, and the given machine is the backup server, then the SBU indexes are restored /usr/sbin/nsr/DisasterRec.sh after that the ufs part of the internal disk was restored. A50023-X4006-A210-6-7619 43 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management /usr/sbin/nsr/recomp.sh force If this machine is not a backup server, then only the ufs part is restored. Using ium_maintenance.sh the next step has to be performed manually: - restore the ufs part of the external disk (see details above, can be restored using maintenance) /usr/sbin/nsr/recomp.sh force external - restore the Informix database (see details above, and can be restored using main- tenance) - restore the Oracle database (see details above, and can be restored using mainte- nance) The same sequences can also be used for OMT: - backup and restore the OS and SBU: /usr/sbin/nsr/omtosbackup /usr/sbin/nsr/omtosbackup_fslist /usr/sbin/nsr/omtosrestore /usr/sbin/nsr/omtrestore - backup and restore the entire internal disk: /usr/sbin/nsr/omtbasbackup /usr/sbin/nsr/omtbasbackup_fslist /usr/sbin/nsr/omtbasrestore 44 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.7.3 Recover Solstice Backup indexes The Solstice Backup Utility (SBU) stores all information about the previous backups in a special database, which is named Media Database. The Media Database and the re- sources - which are generated during installation - are placed only on the backup server. Both compose the index of SBU. Example: Unexpected power loss is a frequent occurrence and can damage the Media Database. Then backup server processes cannot start. If the Media Database is damaged, the backup server cannot start, and it will be neces- sary to restore SBU indexes. How can you recognize the Media Database failure? The backup server process will start automatically on machine start-up, if SBU server packages are installed. Use the following if you want to check if the backup server process is running: If SBU is working properly the following output will be displayed: If the Media Database is damaged only the nsrexecd processes will be displayed. The output of the backup server is very similar to the output of the backup client, because the nsrexecd processes are running only on backup clients (SBU clients). Check if the current machine is really an installed backup server: It is suggested the SBU index restore to execute as root user (this method only restores the SBU indexes, it is not a complete machine restore) in the following way: 1. The SBU index restore can only function when the backup server is active. But with- out Media Database the backup server cannot start. Therefore, stop the running SBU processes: /usr/sbin/nsr/nsr_shutdown -a -q 2. Delete the Media Database: rm -r /nsr/mm/* 3. Re-start SBU server again with the script: /etc/init.d/networker start 4. Load the last used tape - which contains a bootstrap record - into the jukebox drive. Start the following script: /usr/sbin/nsr/DisasterRec.sh ps -ef | grep nsr root 343 1 0 Jun 14 ? 0:03 /usr/sbin/nsr/nsrexecd root 346 343 0 Jun 14 ? 0:03 /usr/sbin/nsr/nsrexecd root 347 1 0 Jun 14 ? 0:17 /usr/sbin/nsr/nsrd root 515 347 0 Jun 14 ? 0:03 /usr/sbin/nsr/nsrmmdbd root 2921 347 0 Jun 14 ? 4:02 /usr/sbin/nsr/nsrmmd -n 1 root 575 347 0 Jun 14 ? 0:01 /usr/sbin/nsr/nsrindexd pkginfo | grep sbu When the output contains ve packages, the machine is a working backup server. When the output contains ve packages, the machine is a working backup server. A50023-X4006-A210-6-7619 45 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management This script will restore the last state of the Media Database and the resources that ex- isted at the time, when the bootstrap record was saved. It is also possible to use an older bootstrap record to restore the SBU index if you execute the DisasterRec.sh script with -m parameter. For further information about running DisasterRec.sh please see chapter 2.5 on pages 63f. 46 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1.8 Appendix Before installing the OMP/OMT package, you must edit or update the following config- uration files: ium_configuration.cfg ium_configuration.cfg is the config file, which contents all necessary settings for in- stallation, found on all machines. Radio Commander uses this file to configure the back- up/restore activities. In this file you can set the following backup/restore specified variables: An example for BRM variables from the ium_configuration.sh: brm.cfg variables Role Modify BACKUP_SERVER_TYP E Possible values: None Internal External No NSRServer The Networker server name. No OMC_Poolname This pool stores all the RC-specific backup data such as all the OMP, OMT, INFORMIX databases and the Oracle backup, except for bootstrap information and the default backup. Yes OMN_Bootpoolname This pool stores the bootstrap information. If you set it as the default, any manual saves that have no target pool, will be saved to this pool. This may avert potential failures. The default value for this is Bootstrap. Yes JB_name This variable is the name of the Jukebox. Before installation, this variable takes its val- ue from the IUM configuration file. No OMC_JB_Device Jukebox-specific data for connection to the Networker server. This gives the path name for the jukebox execution files, if jukebox de- tection cannot find a valid path or type. The default value is /dev/rmt/1cbn. Yes OMC_JB_Devicetype Jukebox type. Further details see chapter 1.2. Yes OMC_Inactivitytime Specifies the number of minutes of inactivity before the conclusion is reached that a client machine is (presumably) permanently hung. Access attempts will be stopped after "n" minutes. Default value: 30 Tab. 1.6 Brm.cfg variables A50023-X4006-A210-6-7619 47 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management This variable is the basis for the RC backup installation (and all backup/restore activity) and consequently, modification is not possible after installation. brm_srv_resource.cfg is used during installation. The entries are used for creating the Networker resource. It contains the following variables: [general_network] . . NSRServer=ssrv01sd # Name of Solstice Backup Server for BRM and IUM . . [general_environment] . . BACKUP_SERVER_TYPE=in- ternal # possible values: # none => Backup and restore is not used # internal => RC internal solution # external => RC external solution # _EV, _DEF: none JB_name=JB_OMCplus # Name of Jukebox for BRM OMC_JB_Device=/dev/rmt/1cbn # Default Jukebox device for BRM or directory names separated with comma if backup device is defined as "file" OMC_JB_Devicetype=dlt7000 # "dlt7000" for DLT7000 L280 Autoloader "4mm" for 4 mm DDS-3 Autoloader "dlt8000" for DLT8000 L9 Autoloader "LTO_Ultrium" for L8 LTO Ultrium Autoloader "file" for UFS files system OMC_Poolname=OMCplus # _VUP, Name of RC backup pools for BRM OMC_Bootpoolname=OMCplus # _VUP, Name of bootstrap pools for BRM OMC_Inactivitytime=30 # Unavailable backup kill time in minutes for BRM NSRResource=/nsr # where SBU indexes are stored Variable Function [...] The exercise of the machine type. For example: OMP or OMT. IMPORTANT: This name must not include spaces. If you have 3 series in this format, you will create 3 machine types, not including the Networker clients as they are cre- ated using the IUM configuration file. IUM_IDENTIFIER IUM ID for the machine type in the IUM config file (ma- chine role). It does not need editing or updating. Tab. 1.7 Brm_srv_resource.cfg variables 48 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Once you have made the necessary modifications, you can begin with the installation. TIMER_NAME Timer resource names in Networker for the given machine types. No space available. EXCEPTION The roles from IUM config file, what are in the machine roles, need not create the client, although the IUM_IDENTIFIER enabled that. TIMER_SET Here you can set 3 timers. The first string may be 7, 30 or 31 characters long. If this string contains the 7 characters, this represents a week and 30 or 31 characters represent a month. This character can be: f-full, i-incremental, [1..9] - n.level, s-skip. The second string is the appointment time (e.g. 15:00) to start the backup on a given day. The third string is option- al. If you set a value (in hh:mm format), the backup will be run several times a day at the interval specified. For ex- ample: 8:00 - runs 3 times a day, every 8 hours. See the following example: "TIMER_SET fiiifii 03:00 6:00" A backup will be started every week on Sunday and Thursday at 3:00, 9:00, 15:00 and 21:00. CLIENT_BUCOMMAND Backup command file or script name in the current ma- chine type. This script must be in the path: /usr/sbin/nsr and the script name must begin with "save" or "nsr". CLIENT_SAVE_SETS The path that you want save. It can be an apparent name too. You can enter an executable command enclosed in "%" and without spaces BROWSER_TIME Browser time for online file index entries. Browser time shows you the successful backups that can be recovered. There are two options: The first is a number and the sec- ond an interval (days, weeks, months, years). For exam- ple: BROWSER_TIME 2 weeks - shows the file index for the last 2 weeks. RETENTION_TIME Retention time for media database entries. It has the same format as BROWSER_TIME. Variable Function Tab. 1.7 Brm_srv_resource.cfg variables A50023-X4006-A210-6-7619 49 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management Example for part of brm_srv_resource.cfg: . . [INFORMIX] IUM_IDENTIFIER MISbss MISnmc TIMER_NAME MIStimer TIMER_SET fsssfss 19:00 CLIENT_BUCOMMAND savemis CLIENT_SAVE_SETS /nsr/applogs BROWSER_TIME 4 weeks RETENTION_TIME 5 weeks . . . 50 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 A50023-X4006-A210-6-7619 51 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 2 Procedures (PROC) This chapter contains following procedures: 2.1 - Supplementary backup server installation 2.2 - Supplementary backup client installation 2.3 - Activate OMP Level 0 Backup (Dump of System Data) 2.4 - Activate OMT Level 0 Backup (Dump of System Data) 2.5 - Activate OMP Level 0 Restore (System Data) 2.6 - Activate OMT Level 0 Restore (System Data) 2.7 - Activate Restore of Internal Disks (Disaster data Recovery) 2.8 - Activate Restore of External Disks 2.9 - Activate Restore of Informix DB (SEM) 2.10 - Activate Restore of Oracle database RC 52 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 2.1 Supplementary backup server installation The backup server installation is supplementary if the backup server packages were not installed during machine installation but it later becomes necessary to use a backup server. As a result, the installation of backup server packages is necessary. Preconditions: Operating system installed on the machine OMP/OMT is down, if Radio Commander was installed on it Radio Commander installation CDs are available Installation is carried out as follows: A50023-X4006-A210-6-7619 53 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1 Login as root user to the machine This is necessary for installation of a backup server 2 Edit the /opt/omc/install/ium_conguration.cfg le using an editor, for example: bvi /opt/omc/install/ium_conguration.cfg and set the NSRServer variable in the general_network section: [general_network] . NSRServer=<hostname> . . . . . . . . . . . . #hostname of backup server machine . In the general_environment section, you have to set the following variable be- fore installation: [general_environment] . BACKUP_SERVER_TYPE=external JB_name=<jukebox name> . . . . . . . . . . . . . . . . #default: JB_OMCplus OMC_JB_Device=<jukebox device> . . . . . . . . #default: /dev/rmt/1cbn OMC_JB_Devicetype=<jukebox type> . . . . . . #possible: 4mm_12GB, dlt7000, dlt8000.LTO_Ultrium OMC_Bootpoolname=OMCplus . . . . . . . . . . . #name of bootstrap pool OMC_Poolname=OMCplus . . . . . . . . . . . . . . . #name of RC pool NSRResource=/nsr . . . . . . . . . . . . . . . . . . . . . . #directory to installation 3 Insert a CD ROM Insert CD#6 into the CD ROM on the installed backup client.. 4 Run the following commands cd /var/RC/install b./install_tool.sh -s /cdrom/cdrom0 -i backup@6.1 And if the package LGTOnmo isn't installed yet, select: b./install_tool.sh -s /cdrom/cdrom0 -i sol_ora@6.1 The execution of the following steps is necessary only for the internal backup server, which is already installed as OMP. For the external backup server, con- tinue the execution by "External backup server conguration", because this part contains only the conguration steps of internal backup servers. 5 Select a script Select the following script from current RUNNING directory: b./brm_srv_resource.sh 54 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 to create the NSR resources, and label the tapes in the jukebox. 6 Select a script Select the bium_maintenance.sh script from the current RUNNING directory, and select item < 2 > Backup and Restore Management 7 Select a item Select item < 1 > Backup Enable 8 Enabling Press 1 <enter> to select the "Enable all backup" item When the last activity has nished, the backup server installation will be ready, and the OMP/OMT can be restarted. END A50023-X4006-A210-6-7619 55 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 2.2 Supplementary backup client installation The backup client installation is supplementary if the machine is already installed as a part of the Radio Commander, namely as OMP or OMT, but during installation the ma- chine wasn't installed as a backup client, but it is necessary to make an on-line backup of the machine. Preconditions: The operation system is installed on the current machine The backup server is installed, congured and running during installation OMP/OMT have to be down Installation CDs are available Installation is carried out as follows: 56 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1 Login as root user to the machine This is necessary for installation of a backup client 2 Edit the /opt/omc/install/ium_conguration.cfg le using an editor, for example: bvi /opt/omc/install/ium_conguration.cfg and set the NSRServer variable in the general_network section: [general_network] . NSRServer=<hostname> . . . . . . . . . . . . . #hostname of backup server machine . In the general_environment section, you have to set the following variable be- fore installation: [general_environment] . BACKUP_SERVER_TYPE=external JB_name=<jukebox name> . . . . . . . . . . . . . . . . #default: JB_OMCplus OMC_JB_Device=<jukebox device> . . . . . . . . #default: /dev/rmt/1cbn OMC_JB_Devicetype=<jukebox type> . . . . . . #possible: 4mm_12GB, dlt7000, dlt8000, LTO_Ultrium OMC_Bootpoolname=OMCplus . . . . . . . . . . . #name of bootstrap pool OMC_Poolname=OMCplus . . . . . . . . . . . . . . . #name of RC pool NSRResource=/nsr . . . . . . . . . . . . . . . . . . . . . . #directory to installation 3 Login to the backup server Login to the backup server as root user, or another user who is authorized to modify backup resources. 4 Select the nwadmin tool Select the nwadmin tool (the location of this tool is described in the "Congura- tion steps" chapter 1.4.2.3) and register the root user of the new backup client in the administrator list (details in the "Administrator list update" section 1.4.2.3 - External backup server conguration). 5 Exit and logout Exit the nwadmin tool and logout from backup server. 6 Insert a CD Insert the CD#6 into the CD ROM on the installed backup client. 7 Selection of a command A50023-X4006-A210-6-7619 57 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management Select the following command: cd /var/RC/install b./install_tool.sh -s /cdrom/cdrom0 -i backup_cli@6.1 8 If the backup client is an OMP If the backup client is an OMP, install the Legato Networker module for Oracle package, because it is necessary to make on-line Oracle backups: b./install_tool.sh -s /cdrom/cdrom0 -i sol_ora@6.1 9 Selection of a script Select the bium_maintenance.sh script from the current RUNNING directory, and select item < 2 > Backup and Restore Management 10 Selection of a item Select item < 1 > Backup Enable 11 Enabling Press 1 <enter> to select the "Enable all backup" item When the last activity has nished, the backup client installation will be ready, and the OMP/OMT can be restarted. END 58 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 2.3 Activate OMP Level 0 Backup (Dump of System Data) Scope This procedure holds for OMP in: single mirrored redundant configuration. The backup procedure is able to manage the following: Operating System Patches (Operating System and Products) OMC's third-party products OMC application Disk Layout Structure Pay attention to restore the tape produced by level 0 backup on an OMP with exactly the same hardware configuration as the backed-up one. The tape produced with Level 0 backup does not contain any database or configuration data. Only internal file systems (UFS) are dumped. This backup activity has to be done after the installation and always after main changes in the system (e.g. making new/bigger partitions). The file systems managed by this procedure are: / /export Information about dumped file systems and progress status is logged into: /nsr/applogs/brmsr.log. Prerequisites Proceeding with the backup of the OMP requires the following: CD ROM: Solaris Installation (Radio Commander installation CD #1) OMP console DAT drive DAT tape(s) (the number depends on the amount of disk space to be dumped) A50023-X4006-A210-6-7619 59 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1 OMP Shutdown If OMP is currently running then log in as omc_plus At the login prompt enter: bomc omp stop <return> 2 System Shutdown Log in as root at the login prompt, then enter: bhalt <return> or: Note: halt stops the system at once; shutdown y g30 i0 gives you a warn- ing, here: 30 secs before shutdown bshutdown y g30 i0 <return> then on the monitor level at the ok prompt enter: bboot s <return> after which the following message is displayed: Type ctrl-d to proceed with normal startup (or give root password for system maintenance): Type the root password to enter in single user mode. To mount on the partitions of the internal disk, enter: bmountall -l <return> 3 Backup Activation Move the current directory to the directory containing the OMP setup distribu- tion. Typically: bcd /usr/sbin/nsr <return> Insert a tape into the tape drive and wait for the ready LED on the tape drives panel. Type: b./ompbasbackup <return> At this point several prompts will scroll on the console. From this point on user intervention is not required. This operation requires approx 20 min. (It depends on the amount of the disk space to be dumped). Information about dumped le systems and progress status is logged into /nsr/applogs/brmsr.log. At the end a message Dump process terminated. is shown. You can thus resume a multiuser level typing: 60 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 binit 6 <return> Continuing with the normal operating state (multi user mode). Note: You can also type <CONTROL+D> to exit from single user mode. If one tape does not have enough space to contain all system data, a new tape is requested by the procedure itself. END A50023-X4006-A210-6-7619 61 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 2.4 Activate OMT Level 0 Backup (Dump of System Data) Scope The backup procedure is able to manage the following: Operating System Patches (Operating System and Products) OMC's third-party products OMC application Disk Layout Structure Pay attention to restore the tape produced by level 0 backup on an OMT with exactly the same hardware configuration as the backed-up one. Only internal file systems (UFS) are dumped. This backup activity has to be done after the installation and always after main changes in the system (e.g. making new/bigger partitions). The file systems managed by this procedure are: / Information about dumped file systems and progress status are logged into: /nsr/applogs/brmsr.log. Prerequisites Proceeding with the backup of the OMT requires the following: CD ROM: Solaris Installation (Radio Commander installation CD #1) OMP console DAT drive DAT tape(s) (the number depends on the amount of disk space to be dumped) 1 OMT Shutdown If OMT is currently running then log in as omc_plus At the login prompt enter: bomc omt stop <return> 62 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 2 System Shutdown Log in as "root" at the login prompt, then enter: bhalt <return> or: Note: halt stops the system at once; shutdown y g30 i0 gives you a warn- ing, here: 30 secs before shutdown bshutdown y g30 i0 <return> then on the monitor level at the ok prompt enter: bboot s <return> after which the following message is displayed: Type ctrl-d to proceed with normal startup (or give root password for system maintenance): Type the "root" password to enter in single user mode. To mount on the partitions of the internal disk, enter: bmountall -l <return> 3 Backup Activation Move the current directory to the directory containing the OMP setup distribu- tion. Typically: bcd /usr/sbin/nsr <return> Insert a tape into the drive and wait for the ready LED on the tape drives panel. Type: b./omtbasbackup <return> At this point several prompts will scroll on the console. From this point on user intervention is not required. This operation requires approx 20 min. (It depends on the amount of the disk space to be dumped). Information about dumped le systems and progress status is logged into /nsr/applogs/brmsr.log. At the end a message Dump process terminated. is shown. You can thus resume a multiuser level typing: binit 6 <return> Continuing with the normal operating state (multi user mode). Note: You can also type <CONTROL+D> to exit from single user mode. If one tape does not have enough space to contain all system data, a new tape is requested by the procedure itself. END A50023-X4006-A210-6-7619 63 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 2.5 Activate OMP Level 0 Restore (System Data) Scope This procedure holds for OMP in: single mirrored redundant configuration. The restore procedure is able to manage the following: Operating System Patches (Operating System and Products) OMC's third-party products OMC application Disk Layout Structure Make sure to restore the tape produced by level 0 backup on an OMP with exactly the same hardware configuration as the dumped one. The tape produced with Level 0 back- up does not contain any database or configuration data. Only internal file systems (UFS) are dumped, including OMP system identification data. The file systems managed by this procedure are: / /export The OMP Level 0 Restore by default restores the whole internal disk, with the content backed-up at backup time. This implies that to complete a backup procedure to the lat- est state (e.g. in disaster case - internal disk crash) you have to Apply: Activate SBURestore (OMCDatabase and Configuration) to restore the internal disk to the latest state, and restore the OMC databases (Oracle, Informix) together with restore and rein- itialize the external disk, when it is needed. Prerequisites Proceeding with the restoration of the OMP requires the following: CD ROM: Solaris Installation (Radio Commander installation CD #1) OMP console CD-ROM drive DAT drive backup DAT tape(s) 64 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 1 OMP Shutdown If OMP is currently running then log in as omc_plus At the login prompt enter: bomc omp stop <return> 2 System Shutdown Log in as "root" at the login prompt, then enter: bhalt <return> or: Note: halt stops the system at once; shutdown y g30 i0 gives you a warn- ing, here: 30 secs before shutdown bshutdown y g30 i0 <return> 3 Operating system boot Insert the Solaris CD ROM Installation (Radio Commander installation CD #1) into the drive, then type: bboot cdrom -s <return> This will bring up a basic operating system in single user mode. 4 Procedure download At the end of the boot process at the shell prompt type: bcd /tmp <return> Insert the rst tape containing the backup and then type: bmt f /dev/rmt/0mbn rewind <return> btar xvf /dev/rmt/0mbn <return> 5 Procedure activation Type: bcd ./backup <return> b./omposrestore <return> i Following lines up to next information with text The procedure will proceed... are only valid for BR7.0 and not valid for UMR2.7. A50023-X4006-A210-6-7619 65 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management The script will check user entitlements in the rst step (single user mode) and the device state. When all preconditions are present on the machine, the follow- ing message will be displayed: Restoring host <machine>(omp) Then the script will display the name of the current restorable partition and a list of the available disks. The logical device name will be displayed, for example c0t1d0sN, and the disk can be selected from the menu, where the current parti- tion has to be restored. The script will repeat the question after displaying the message "Invalid selection" in the event of a wrong selection. If the input con- tains no value (it was only <enter>), the script will select the rst item automati- cally. Once a successful selection is made , the restoration of the current partition will be started. The representative turnout of disk selection menu: Available disks for / where original was c0t10d0s1: 1.) c0t0d0sN (Board: 0, Slot: N/A) 2.) c0t1d0sN (Board: 0, Slot: N/A) Select target disk [def: 1]: i Text starting from here is valid for BR7.0 and or UMR2.7. 66 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 The procedure will proceed for a time period depending on the hardware used without any further users intervention. The above-mentioned selection has to be executed before all partition restorations and after the end of previous parti- tion restoration. Only the unspent partition will be displayed in the menu , so it is impossible to restore more partitions contained on the same partition. At the end of such process the system reboot automatically, resuming the system pre- viously dumped. The CD-ROM must be left in the drive for all time required fromthe restore stage. At the end of the reboot the OMP run controls are disabled. In order to complete the process apply the following procedure: Activate Restore of Internal Disks (Disaster data recovery) in case of using the extended backup feature. Please refer to chapter 2.7. or: only when you dont use the extended backup feature based on the Solstice Backup Utility is there then a possibility to restore the remaining part of the in- ternal disk with basic backup feature. To do this type: binit -s bcd /usr/sbin/nsr/backup b./ompbasrestore END A50023-X4006-A210-6-7619 67 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 2.6 Activate OMT Level 0 Restore (System Data) Scope The restore procedure is able to manage the following: Operating System Patches (Operating System and Products) OMC's third-party products OMC application Make sure to restore the tape produced by level 0 backup on an OMT with exactly the same hardware configuration as the dumped one. Only internal file system (UFS) are dumped, including OMT system identification data. The OMT Level 0 Restore by default restores the whole internal disk with the content backed-up at backup time. This implies that to complete a backup procedure to the lat- est state (e.g. in disaster case - internal disk crash) you have to Apply: Activate SBU Restore to restore the internal disk to the latest state. Prerequisites Proceeding with the restoration of the OMT requires the following: CD-ROM: Solaris Installation (Radio Commander installation CD #1) OMT console CD-ROM drive DAT drive backup DAT tape(s) 1 OMT Shutdown If OMP is currently running then log in as omc_plus At the login prompt enter: bomc omt stop <return> 2 System Shutdown Log in as "root" at the login prompt, then enter: bhalt <return> or: Note: halt stops the system at once; shutdown y g30 i0 gives you a warn- ing, here: 30 secs before shutdown bshutdown y g30 i0 <return> 68 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 3 Operating system boot Insert the Solaris Installation CD-ROM (Radio Commander installation CD #1) into the drive, then type: bboot cdrom -s <return> This will bring up a basic operating system in single user mode. 4 Procedure download At the end of the boot process at the shell prompt type: bcd /tmp <return> Insert the rst tape containing the backup and then type: bmt f /dev/rmt/0mbn rewind <return> btar xvf /dev/rmt/0mbn <return> 5 Procedure activation Type: bcd ./backup <return> b./omtosrestore <return> The script will check user entitlements in the rst step (single user mode) and the device state. When all preconditions are present on the machine, the follow- ing message will be displayed: Restoring host <machine>(omp) Then the script will display the name of the current restorable partition and a list of the available disks. The logical device name will be displayed, for example c0t1d0sN, and the disk can be selected from the menu, where the current parti- tion has to be restored. The script will repeat the question after displaying the message "Invalid selection" in the event of a wrong selection. If the input con- tains no value (it was only <enter>), the script will select the rst item automati- cally. Once a successful selection is made , the restoration of the current partition will be started. The representative turnout of disk selection menu: Available disks for / where original was c0t10d0s1: 1.) c0t0d0sN (Board: 0, Slot: N/A) 2.) c0t1d0sN (Board: 0, Slot: N/A) Select target disk [def: 1]: i Following lines up to next information with text The procedure will proceed... are only valid for BR7.0 and not valid for UMR2.7. i Text starting from here is valid for BR7.0 and or UMR2.7. A50023-X4006-A210-6-7619 69 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management The procedure will proceed for a time period depending on the hardware used without any further users intervention. The above-mentioned selection has to be executed before all partition restorations and after the end of previous parti- tion restoration. Only the unspent partition will be displayed in the menu , so it is impossible to restore more partitions contained on the same partition. At the end of such process the system reboot automatically, resuming the system pre- viously dumped. The CD-ROM must be left into the drive for all time required from the re- store stage. At the end of the reboot the OMT run controls are disabled. In order to complete the process apply the following procedure: Activate Restore OMT internal disk (Disaster data recovery) in case of us- ing the extended backup feature. Please refer to chapter 2.7. or: there is a possibility to restore with basic backup feature the remaining part of the internal disk. To do this type: bcd /usr/sbin/nsr/backup b./omtbasrestore After the execution of the script le "./omtosrestore" the system reboots auto- matically! It is necessary to execute the script in the single user mode. To re- store the remaining part of the internal disks. To do this type: binit -s bcd /usr/sbin/nsr/backup b./omtbasrestore END 70 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 2.7 Activate Restore of Internal Disks (Disaster data Recov- ery) Scope All data contained on internal disks have been lost or the internal disk has been dam- aged. Before you start with this procedure Activate OMP Level 0 Restore (System Data) must have been excecuted. Please refer to chapter 2.5. In case of OMT you have got to start with procedure Activate OMT Level 0 Restore (System Data) must have been excecuted. Please refer to chapter 2.6. After the Level 0 Restore there is a working OS and Solstice Backup Utility. Under Activate of Internal Disk the content of the internal disk has to be restored, which means the restore all partitions to their latest backed-up status from backup tapes present in the Auto-loader device. It uses bootstrap and index information saved by Sol- stice Backup onto tape). The operator must make sure he uses the right tape containing the latest index informa- tion during the recover procedure (generally the last one). During Solstice index re-build- ing the recover may request a previous index tape to be loaded: follows the instructions reported later. If the recover procedure fails, it can be re-run after having solved the problemthat arose. Keep open a window on an OMP with the nwadmin program (Solstice Backup Pro- gram). The hardware configuration of OMP must not be changed from the last valid backup. Prerequisites System after Activate OMP Level 0 Restore (System Data). Please refer to chapter 2.5. In case of OMT you have got to start with procedure Activate OMT Level 0 Restore (System Data) must have been excecuted. Please refer to chapter 2.6. A50023-X4006-A210-6-7619 71 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 1 Restore Activation Log in as root at the login prompt, then type the following commands: bcd /usr/sbin/nsr/backup <return> b./omprestore <return> Note: in case of OMT machine, instead of omprestore, please start: b./omtrestore In both scripts 3 parameter are available : '-nd': Using this parameter the Level 0 backup/restore does not happen before running. In this case the scripts use the ium_conguration.cfg le, to dene the backup server type. In other case the scripts apply to the conguration, which was backed up by level 0 backup. '-nf': You use this parameter during restore for not overwriting the les/directo- ries, which are already existing on the disk. When this parameter wasn't typed, the script overwrites the les, which are existing in the used backup set, and on the disk too. '-mc': Using this parameter a list of the available backup sets will be represent- ed, which can be used for restore, and the operator can select the wanted back- up set for recover. Only if the OMP is a backup server you do get the message: Press <enter> to start recover... or Load a tape into the drive and press <enter> ... or In the rst case the drive contains a mounted tape, in the second case the drive is empty. Before you press <enter> please check the drive, and insert the tape into the drive, which is usable for restore (and contains bootstrap records). But in both cases mount the tape from a selected slot manually using of jukebox console. If the correct tape is in the drive - it is recommend to choose the last tape of backup -, press <enter> to continue the restore. i Jukebox console is the controller interface of the jukebox hardware. It is found in the front side of jukebox. Here are placed the status lights, the control buttons (namely: pre- vious, next, enter, cancel), and a small LCDdisplay, which presents the current jukebox operations. i Please use the same tapes - if it possible - in the same slot, as they were placed during backup. The jukebox must contain all tapes, which are neccessary to the recover, oth- erwise the recover leads to an error, and stops the run. 72 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 After the check of available device(s) it will scan the corresponding device for available bootstraps. The scan will be started at the end of tape; it will proceed to the rst record(s), which are placed on the device. First it will scan only the last 11 records, and if there are not bootstrap records, it continues the scan up to the part, where bootstraps can be detected. If the script was started with "-mc" parameter, after the successfully scan it will present the detected bootstraps in the following format: Detected bootstrap records: 1.) 4/02/03 7:20 (id: 2324645377) 2.) 4/02/03 9:45 (id: 2324652180) 3.) Detect more Choose an item [def: 2]: With the help of "Detect more" point the scan can be continued to further (old- er) bootstraps. This item will be not presented, when all bootstraps were scanned already, which were on the device. If no selection happened (only <en- ter> was pressed), the script will use the last bootstrap record to recover the media database. When the scanner couldn't nd bootstraps it will present the error: Bootstrap not found on device /dev/rmt/0cbn (no record). Continue [y/n]: With selection of "n" you quit the recover. Before positive answer the target tape can be changed. When backup device are specied as UFS le system, this function is not avail- able. The script will scan all devices in the order of last writing time, and use the rst, where bootstrap records are placed. If neither from the devices contains bootstrap records, the restore will be terminated with following error: Device not found with bootstrap record. After the selection of bootstrap record - when manual mode was not enabled ("- mc" was not given in command line) the selected record will be automatically the last - the script will start the media database recover automatically, with the message: Recover media database (id: <bootstrap identier>) It performs the following actions: 1. Solstice indexes re-build (if Solstice Backup Installation is Full) and in every case the next step is the restoring the internal disk 2. Restore of the whole internal disk Note: If omprestore is running you can see only the most important errors which can abort the restore process. You can check the status of the restore process with the nwadmin program. If, during this activity, a message appears in the Pending messages window on the Solstice Backup Program requesting you to load a previous index volume, then proceed as follows: in the rst sub-window select the logical device which the autoloader is con- nected to A50023-X4006-A210-6-7619 73 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management from toolbar select Unmount change index tape, loading the requested one click on OK then, if a new request appears, start from the beginning. 2 Post Actions After restoring the internal disk the following step has to be carried out: 1. If only the internal disk is damaged: 2. If only the external disk is damaged: Important: When the disaster situation (internal disk crash) has been eliminated, including the above mentioned two post actions, in the Solstice backup utility the dened groups should be active once again: b/usr/bin/nsr/nwadmin then choose from the nwadmin main menu: Customize then choose from the Customize pull down menu: Groups There for all OMC dened groups, as MIStimer OMPtimer OMTtimer OraclaLogtimer Oracletimer set the Autostart attribute to Enabled. END i Activate Restore of Informix DB (SEM). Please refer to chapter 2.9. i Activate Restore of External Disks. Please refer to chapter 2.8. 74 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 2.8 Activate Restore of External Disks Scope If all data contained on internal disks have been lost or the internal disk has been dam- aged, you must perform as a first step the procedure Activate OMP Level 0 Restore (System Data), and as a second step the procedure Activate Restore of Internal Disk. After the Level 0 Restore there is a working OS and Solstice Backup Utility (SBU). If OS and SBUare running, but some data are lost, you must performonly the procedure Activate Restore of Internal Disk (refer to chapter 2.7). In the first step, when the external disk is damaged, the external disk has to be recon- figured, then the UFS part of the external disk has to be restored, which means the re- store of the /commonArea - ufs partitions to their latest backed-up status from backup tapes present in the auto-loader device. If the recover procedure fails, it can be re-run after having solved the problemthat arose. Keep open a window on an OMP with the nwadmin program (Solstice Backup Pro- gram). The hardware configuration of OMP must not be changed from the last valid backup. Prerequisites Internal disk is working correctly or the "Activate Restore of Internal Disk" - details in chapter 2.7 - was executed successfully on the system. This means you have a working internal disk with OS and OMC SW Load. 1 Maintenance Log in as root at the login prompt, then type the following commands: change directory to the actual RUNNING directory if OMP e.g.: bcd /export/local/OMC/SW/OMP/RUNNING <return>> b./ium_maintenance.sh <return> A50023-X4006-A210-6-7619 75 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 2 Re-conguration of external disk If only the external disk is damaged and the external disk has to be re-cong- ured: choose External disk Management, type b1 (External disk Management) <return> then choose (Create Layout) , type b2 (Create layout - even if layout has been already congured) <return> Outputs: (examples) Saving the structure of the directory: /commonArea/home Into the le . Saving the structure of the directory: /commonArea/OMC Into the le . Calling EDMs script edm_dlm.sh with parameter force Press Return/Enter to continue Menue appears again Continue with procedure 3 Activate Restore of UFS part of External Disks or fromthe Maintenance main menu choose Backup and Restore Management, type: b2 (Backup and Restore Management) ) <return> then choose Restore, type: b4 (Restore ) <return> then choose External disk conguration, type b1 (External disk conguration) <return> 3 Activate Restore of UFS part of External Disks from the Maintenance main menu choose Backup and Restore Management, type: b2 (Backup and Restore Management) ) <return> then choose Restore, type: b4 (Restore ) <return> then choose OMP, type 76 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 b2 (OMP) <return> then choose Restore OMP external disk (ufs part), type: b2 (Restore OMP external disk (ufs part)) <return> 4 Restore of Oracle database to external disk from the Maintenance main menu choose Backup and Restore Management, type: b2 (Backup and Restore Management) ) <return> then choose Restore, type: b4 (Restore ) <return> then choose Oracle, type b5 (Oracle) <return> then typically choose Recover Oracle database, type: b1 (Recover Oracle database) <return> END i This step is the same as in procedure 2.10, Activate Restore of Oracle database in RC. So please refer to the more detailed description in chapter 2.10. A50023-X4006-A210-6-7619 77 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 2.9 Activate Restore of Informix DB (SEM) Scope It restores the Informix database, which is a database of SEM, and also the NMC data- base. The main feature of the procedure "Activate Restore of Informix DB (SEM)" is to use it, if problems occur with Informix DB and a media recover is required, but you can also execute this procedure without a problem. After execution of this task the Informix database is recovered. Prerequisites 1. shutting down RC Application processes with omc omp/omt stop command 2. All OMT sessions must be closed 3. When the internal disk is damaged, Activate Restore of Internal Disk is neces- sary. Please refer to 2.7. 1 Maintenance Log in as root at the login prompt, then type the following commands: change directory to the current RUNNING directory if OMP e.g.: bcd /export/local/OMC/SW/OMP/RUNNING <return> b./ium_maintenance.sh <return> 2 Activate Restore of Informix database from the Maintenance main menu choose Backup and Restore Management, type: b2 (Backup and Restore Management) ) <return> then choose Restore, type: b4 (Restore ) <return> then choose Informix, type b4 (Informix) <return> then choose Recover Informix database, type: b1 (Recover Informix database) <return> END 78 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 2.10 Activate Restore of Oracle database in RC Scope It restores only Oracle databases. OMC database is restored and recovered. It can be activated only if OMC DB has problems and a media recover is required. Please refer to chapter 1.5.4.3 for further information. Prerequisites 1. Stop the RC Application with omc omp/omt stop 2. All OMT sessions must be closed 3. When the external disk is damaged, Activate Restore of External Disk, chapter 2.8 is necessary 1 Maintenance Log in as root at the login prompt, then type the following commands: change directory to the actual RUNNING directory if OMP e.g.: bcd /export/local/OMC/SW/OMP/RUNNING <return> b./ium_maintenance.sh <return> 2 Activate Restore of Oracle database from the Maintenance main menu choose Backup and Restore Management, type: b2 (Backup and Restore Management) <return> then choose Restore, type: b4 (Restore) <return> then choose Oracle, type: b5 (Oracle) <return> then choose Recover Oracle database, type: b1 (Recover Oracle database) <return> A50023-X4006-A210-6-7619 79 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management 3 Troubleshooting a), as well as restoring and recovering the whole Oracle DB, there is the possi- bility to create an empty Oracle database. To activate it, choose Create empty database, type b(Create empty database) <return> b), furthermore, there is an advanced restore possibility, which currently re- stores and recovers the database to the latest available status of the database based on either in time, SCN, or log sequence number. Choose Advanced re- store possibility, type b(Advanced restore possibility) <return> and then either choose Time-based recover, type: b(Time-based recover) <return> or choose SCN-based recover, type: b(SCN-based recover) <return> or choose log sequence-based recover, type: b (log sequence-based recover) <return> 4 Description of Oracle recovery run 1.) Check the Oracle database status, the state of the Radio Commander - it must be stopped-, and the available full backups too. During this activity the script shuts down the database and re-starts it. Depend- ed on Oracle status after re-start, it will display the database status and prompt the operator to overwrite it: Database: Complete Overwrite parameters [y/n]: (Note: conrm it with y) This message will appear if all database les (control, redo log, data) are avail- able, and there are no differences found during the consistency check. Answer- ing "n" will end the run. Database: Incomplete Overwrite parameters [y/n]: In this case only the parameter, conguration, and control le are available, but some data les may get lost, or an error may occur with the consistency check. The script continues, but with a no answer it won't overwrite the control le. Database: Incomplete It waits for a no answer. If the Oracle control le is not found, it will start the pa- rameter and control le restore. . 2.) Depending on the answer (or if necessary) it overwrites the parameter, con- guration and control le. However, using the "-m" parameter will prompt which backup you want to use: Restore to date [mm/dd/yy]: Please type the date whose database content you want to restore - in the format 80 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 month/day/year separated with "/"-, for example: 10/15/02. It shows the date 15th of October 2002. If you insert no date and press <Enter>, it will use the last date on which an Oracle backup was made. You can then see the backups which were made on this date: Available controlle backups: 1.) 00:01:06 (id: 1 on OMCplus.001) 2.) 04:00:11 (id: 1 on OMCplus.001) 3.) 08:01:07 (id: 2 on OMCplus.002) 4.) 12:00:04 (id: 3 on OMCplus.002) Choose an item [def: 4]: In this case it presents the 4th backup, which was issued on 10/15/02. It shows only the backup time (hour/min/sec format) and the volume where this backup was placed. Choose an number to continue (for example: 2). If you give no an- swer and press <Enter>, it will use the last backup of the given day to restore. After that it will overwrite the aforementioned les. Without parameters this activity runs automatically to the last available Oracle backup date. If the controlle restoration fails or an error occured during an automatic run, the script will automatically use the last succesful controlle backup. In manual execution, the script will present the menu again, but without showing the save set, where the restore wasn't complete, and it iterates the premised ac- tivity for the successful restore. All aborted restore attempts are logged into the /nsr/aplogs/brmsr.logle. If it cannot nd a valid backup, with which a restoration is possible, it will terminate execution with the message: All omc parameter saveset recover failed. If parameter and controlle restoration was succesful, the following message will be displayed: Ctrlle: Ready . 3.) Check any environment settings and backup. The script selects the last com- plete full backup set, checks the link on the external disk (if data les place was there before Oracle crash) and the mounted database using the restored control le. At the end of this step this message is displayed: Checking: Ready In this case the third backup set was the last, and it was made on 12th October 2002. . 4.) Restore the last complete full backup set.The script overwrites all data les without exception. During the restore it displays the current restored data les. The end of the operation is indicated with the message: Restoring datales (set: 2): Ready Here the operator can show the identier of the used full backup set. If the re- store failed, the script starts automatically the restore of an older successfully A50023-X4006-A210-6-7619 81 Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Backup and Restore Management Oracle backup set until the complete restore of Oracle datales. If all full backup sets are damaged or failed, the script terminated the work with this message: All omc datale set recover failed. The detailed description of occurred errors are available in /nsr/applogs/brm- sr.log le. . 5.) Archived log backup restore. Depending on the parameter used it can work automatically (no parameters are given), or manually ("-m" parameter). In the second case the available archived log les are displayed after the message: Available logle backups: The display format depends on the parameters used: -s) item number.) last SCN number in the log (backup date) -l) item number.) Logle name (backup date) -t) item number.) Backup date (logle name) Info: The backup date is presented in month/day/year format. After it you have chosen the item number, the script will restore the log les. When no item number is given, it will use the last detected archived log data. By using "-u" (only for users with mastery of Oracle) the machine will present the log le in this format: Item number.) last SCN number in the log (logle name) In this case the operator has to enter an SCN number. With using this data's the script will recover the archived logs. During recover it shows the current recovered archived log, and at the end of the operation sends this message: Recover archive log: Ready The program restores the archived logs one after the other, and if an error oc- curred during restore, it will stop the restore run, and tries to use only the ar- chived logs to roll forward operation, which are already restored to the disk. The detailed report of developed errors is available in /nsr/applogs/brmsr.log. . 6.) In the last step the script will have opened the database, because during re- store the database is in "mount" state. If the operation was successful, the fol- lowing result will be displayed: Database status: Opened If the restore process was successfully terminated, it will send this row to the output: Process terminated. Result of automatically (not parameterized) Oracle restore -successful-: Database omc restore operation in progress Database: CompleteOverwrite [y/n]: y Ctrlle: Ready 82 A50023-X4006-A210-6-7619 Backup and Restore Management Emergency Cases Radio Commander BR 7.0 / UMR 3.0 Checking: Ready Restoring datales (set:4) : Ready Restore archived log: Ready Database status: Opened Process terminated. Output of parameterized (./brm_ora_restore.sh -m -s) Oracle restore -success- ful-: Database omc restore operation in progress Database: CompleteOverwrite [y/n]: y Available controlle backups: 1.) 00:01:06 (id: 1 on OMCplus.001) 2.) 04:00:11 (id: 2 on OMCplus.001) 3.) 08:01:07 (id: 2 on OMCplus.002) 4.) 12:00:04 (id: 3 on OMCplus.002) Choose an item [def: 4]: 3 Ctrlle: Ready Checking: Ready Restoring datales (set: 2): Ready Available logle backups: 1.) 234566 (10/12/02) 2.) 235982 (10/13/02) 3.) 242001 (10/13/02) Choose an item [def: 3]: 2 Restore archived log: Ready Database status: Opened Process terminated. END