PACK YOUR STATS, WERE GOING TUNING!

Richard Basile, Collective Technologies

INTRODUCTION
To measure how well your Oracle database system is tuned and identify where improvements can be made, you need a tool that will help gather and measure performance statistics. The Statspack utility is such a tool. Historically, the DBA has used the BSTAT/ESTAT tool provided by Oracle in gathering performance statistics. However, the tool was limited in breadth and the report cryptic to interpret. Introduced in Oracle 8i release 2 (8.1.6) and enhanced in release 3 (8.1.7), the Statspack utility builds on the strengths and addresses many of the weaknesses of the old BSTAT/ESTAT utility. When driving on a long trip, you might measure the amount of gas in your tank and the odometer reading at both the start and end of your trip. By calculating the delta, you can know miles traveled, total fuel consumption and miles per gallon. In much the same way, the BSTAT/ESTAT utility reads specific performance statistics at two points in time, and these readings are compared to measure system performance. Yet it is not enough to measure miles traveled, total fuel consumption and miles per gallon. We also needed to measure miles per hour, tire tread wear, and whether our kids would make it to the next rest stop! The BSTAT/ESTAT utility is limited. Disappointed with the limited scope and cryptic report of the BSTAT/ESTAT utility, many have blazed their own trail, writing their own reports. We wanted more information in a simple, readable format. We wanted ratios calculated and conditions summarized. Show us the high-resource SQL! Our pride was measured not only in time (database uptime, recovery time, query response time) but also in building a better mousetrap - writing a better performance script! Yet even our own methods were limited. We took a single snapshot in time and had nothing to compare it with. Our statistics measured were aggregates, not deltas (since database startup rather than a measured window). Even though we did a good job in building our mousetrap, we suspected a few mice got away.

A BETTER MOUSETRAP
Like the BSTAT/ESTAT report, Statspack collects and reports information about database performance. However, Statspack collects more data, such as high resource SQL, calculates performance metrics such as Buffer Cache Hit Ratio and Memory Sort Percentage, and summarizes data in an easy to read report. Statspack allows you to take a snapshot of performance statistics and store the data for later comparison. Each snapshot is a reference to a point-in-time. Comparing any two snapshots allows you to quantify database performance over that window of time, much the way BSTAT/ESTAT captured a window of time. Because Statspack stores performance data in the database, you can make historical comparisons whenever necessary, at whatever time interval necessary and perform trend analysis. Trend analysis helps you to answer the question, "What has changed?"
Paper #

Statspack data collection is easily automated using a job scheduler such as cron in UNIX or the dbms_job package provided by Oracle. Because Statspack separates data collection and report generation, you might schedule data collection every hour on the hour and generate reports whenever needed.

ANATOMY OF A UTILITY
The Statspack utility is included with the 8.1.6 and 8.1.7 releases. Though unsupported, patches included with the 8.1.6 release allow you to run Statspack in any Oracle 8.x release prior to 8.1.6. You may download either version of Statspack from the Oracle at http://otn.oracle.com. Click on Internet DBA, Performance and Scalability, Software. The Statspack utility includes a collection of SQL, PL/SQL and SQL*Plus scripts and the perfstat schema. The perfstat schema includes the perfstat user, tables, constraints, indexes, a sequence and the STATSPACK package. Read-only privileges are granted to the perfstat user which allow queries against the V$ dynamic performance views. The following is a listing of the scripts and files included with each release of Statspack. Though the examples that follow are based on the 8.1.7 release, you can easily identify which files to use for the 8.1.6 release. 8.1.7 spdoc.txt spcreate.sql spreport.sql spauto.sql spuexp.par spurge.sql sptrunc.sql spup816.sql spdrop.sql spcusr.sql spctab.sql spcpkg.sql spdtab.sql spdusr.sql n/a n/a 8.1.6 statspack.doc statscre.sql statsrep.sql statsauto.sql statsuexp.par n/a n/a n/a statsdrp.sql statscusr.sql statsctab.sql statspack.sql statsdtab.sql statsdusr.sql statscbps.sql statsrep80.sql Description Documentation Installs Statspack Generates the Statspack report Sets up scheduled snapshot collection using dbms_job Parameter file for exporting the perfstat schema Purge a range of snapshots Truncate the perfstat tables Upgrade the perfstat schema from 8.1.6 to 8.1.7 Uninstalls Statspack Called by the install script; creates perfstat user Called by the install script; creates perfstat tables Called by the install script; creates statspack package Called by the uninstall script; drops perfstat tables Called by the uninstall script; drops perfstat user Creates the view v$buffer_pool_statistics; to be used when running Statspack in Oracle 8.0 through 8.1.5 Generates the Statspack report; to be used when running Statspack in Oracle 8.0
Files included in Statspack

For Oracle versions 8.0.x, you are limited to using the 8.1.6 version of Statspack. For Oracle 8.1.x, you may use either release of Statspack. To install Statspack in a pre-8.1.6 database, you must first run the statscbps.sql script (included only with the 8.1.6 release), connected as INTERNAL, before continuing with normal Statspack installation. The statscbps.sql script will create the dynamic performance view v$buffer_pool_statistics.

Paper #

When running Statspack in an 8.0 release, you will use the statsrep80.sql script rather than the statsrep.sql script to generate reports. This is because of feature differences between the 8.0 and 8.1 releases. Each snapshot is dependent on a snaphost identifier (snap_id), a database identifier (dbid) and an instance number (instance_number). The snapshot identifier is a unique id automatically generated with each snapshot using a sequence. The instance number creates a relationship between the snapshot and the instance from which it was generated in the case of a Parallel Server environment.

INSTALLATION
Storage requirements vary depending on how often snapshots are collected. The storage requirement when you install Statspack is about 35m for 8.1.6 and 45m for 8.1.7. The scripts required to install Statspack are located in $ORACLE_HOME/rdbms/admin on UNIX and %ORACLE_HOME%\rdbms\admin on NT. If you downloaded Statspack, copy the files to this location before proceeding. During installation, you will be prompted for the default and temporary tablespaces to be assigned to the perfstat user. Do not use the SYSTEM tablespace for either the default or temporary tablespace assignments. I suggest you separate the perfstat schema from the application schemas. In a typical environment, you might specify TOOLS as the default tablespace assignment and TEMP as the temporary tablespace assignment. The install process will create the dbms_shared_pool and dbms_job packages. The dbms_shared_pool package will allow you to pin the statspack package in memory, and the dbms_job package will allow you to schedule a job that will automate the periodic collection of STATSPACK data. The scripts for both of these packages are located in $ORACLE_HOME/rdbms/admin on UNIX, and %ORACLE_HOME%\rdbms\admin on NT. To install Statspack, you should either make ORACLE_HOME/rdbms/admin your current directory or specify the path when calling the script as follows: UNIX:
SQL> @?/rdbms/admin/spcreate

NT:
SQL> @%ORACLE_HOME%\rdbms\admin\spcreate

Statspack must be installed from SQL*Plus. The install scripts use features available only in SQL*Plus that will cause errors in Server Manager. Do not use Server Manager as errors will result in a failed installation.

STEPS FOR INSTALLING STATSPACK

1. Start SQL*Plus and connect internal
$ sqlplus nolog SQL> connect internal

Paper #

2. If installing Statspack in any Oracle 8.x version prior to 8.1.6, call statscbps.sql before continuing with normal 8.1.6 installation.
SQL> @?/rdbms/admin/statscbps

3. Install Statspack. Use statscre.sql for 8.1.6, spcreate.sql for 8.1.7.

SQL> @?/rdbms/admin/spcreate

You will be prompted for the default and temporary tablespace assignments. 4. Change the password for the perfstat user:
SQL> alter user perfstat identified by <password>;

5. Check the log files for errors.

ADDITIONAL CONSIDERATIONS
Certain statistics are more meaningful when the timed statistics option is turned on for your instance. Because the timed statistics option creates some minor overhead, there is some debate as to whether or not this option should be used. I recommend using timed statistics. In an environment that is well tuned and operating at peak performance, the overhead incurred is negligible. When performance becomes a problem, you cannot afford to be without the information that timed statistics provides. To turn on timed statistics, set the timed_statistics parameter in your init.ora file to true.
timed_statistics=true

You will not have to restart the instance. Instead, issue the following command to turn timed statistics on dynamically.
SQL> alter system set timed_statistics = true;

You may also need to set job_queue_processes if you are going to use dbms_job to schedule automatic snapshot collection. See the section Snapshots for more details. Though you only need to install Statspack once, if you are using Oracle Parallel Server you will need to set these parameters for each instance in the cluster. You should also set the instance_number parameter in the init.ora file so as to retain a consistent instance number for each instance between instance startups. Setting the instance number will make it easier to compare performance statistics over time for a single instance.

SNAPSHOTS
To manually take a snapshot, login to SQL*Plus as the perfstat user and execute the statspack.snap procedure.

Paper #

SQL> exec statspack.snap

However, to take advantage of the fullness of Statspack, you will want to schedule automatic snapshots using a job scheduler. The dbms_job package allows you to do this. To use dbms_job, set the parameter job_queue_processes to a value greater than 0 in the init.ora file.
job_queue_processes = 1

This parameter specifies the number of job queue background processes to start. The default setting is 0 (zero), and the range is between 0 and 36. The process is identified as SNPn, where n is 0-9, A-Z. You will need to set this parameter for each instance where you intend to use dbms_job to schedule automatic snapshot collection. Note that this change will not take effect until the next database shutdown/startup. Next, you will want to create the job that executes snapshot collection. The script spauto.sql will create this job to run every hour on the hour. The following is the output from this action.
SQL> @?/rdbms/admin/spauto PL/SQL procedure successfully completed. Job number for automated statistics collection for this instance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Note that this job number is needed when modifying or removing the job: JOBNO ---------1 Job queue process ~~~~~~~~~~~~~~~~ Below is the current setting of the job_queue_processes init.ora parameter - the value for this parameter must be greater than 0 to use automatic statistics gathering: NAME TYPE VALUE -------------------------------- ------- -----------------------------job_queue_processes integer 1 Next scheduled run ~~~~~~~~~~~~~~~~~ The next scheduled run for this job is: JOB NEXT_DATE NEXT_SEC ---------- ---------- -------1 02/05/2001 22:00:00 Output from spauto.sql

Note that in a Parallel Server environment you will want to collect statistics for each instance. Collecting statistics for a specific instance can only be accomplished by logging into that instance. If using dbms_job to schedule snapshot collection, you will need to login to each instance and create a job for that instance.
Paper #

You can easily change the frequency of the job using the dbms_job.interval procedure. Use the job number for the first parameter. The second parameter sets the schedule. The trick is to divide one day by the number of intervals you wish to schedule the job to run. If you wanted the job to run every four hours, you would consider that there are six four-hour intervals in a day. Hence, you would divide one day by six intervals as in the following example.
SQL> exec dbms_job.interval(1, 'SYSDATE+1/6');

To remove a job, use the dbms_job.remove procedure. This procedure takes the job number as the argument.
SQL> exec dbms_job.interval(1);

SNAPSHOT CONFIGURATION
There are three levels of data collection: 0, 5 and 10. Level 0 collects general performance statistics. Level 5 collects these same statistics plus high resource SQL statements. Level 10 additionally collects child-latch data. Because level 10 is resource intensive, it may take longer to collect data and should only be used when asked to do so by Oracle Support. Level 5 is the default. High resource SQL is relative to the thresholds you configure. These thresholds are number of executions (default 100), disk reads (1000), parse calls (1000), buffer gets (10000), size of sharable memory (1m), and version count of the SQL statement (20). When collecting a snapshot at level 5 or 10, any SQL statement that exceeds any one of the thresholds will be captured. You can modify the snapshot level and SQL thresholds while taking a snapshot or independent of data collection. The statspack.snap procedure accepts parameters that will modify the snapshot level and SQL thresholds while taking a snapshot. The statspack.modify_statspack_parameter procedure modifies the snapshot level and SQL thresholds without taking a snapshot. The appendix lists the parameters for the snap and modify_statspack_parameter procedures for both 8.1.6 and 8.1.7. The following illustrates how to change the thresholds for executions, disk reads, parse calls and buffer gets independent of taking a snapshot using the modify_statspack_parameter procedure.
SQL> exec statspack.modify_statspack_parameter( i_executions_th => 1000 , i_disk_reads_th => 10000 , i_parse_calls_th => 10000 , i_buffer_gets_th => 100000 );

The lions share of performance issues can be traced back to the application. However, collecting SQL statistics can be space intensive. It is a good idea to keep a baseline of SQL statistics and take new snapshots at level 5 on a regularly scheduled basis. For example, I like to take level 0 snapshots on an hourly basis and two level 5 snapshots on Monday or Friday, depending on the activity in the database. Raising the thresholds on SQL statistics collection is another option to consider in reducing the amount of SQL statistics collected. It should be noted that the 8.1.7 version of Statspack collects SQL statistics more efficiently and therefore is less space intensive than the 8.1.6 version. Also, the 8.1.7 version includes a purge utility not available with 8.1.6. You must decide on an approach that is best for your system.

THE STATSPACK REPORT

Are you ready? Lets find out how well your instance is running! From SQL*Plus, login to your instance as the perfstat user and run the report script. If you are using the 8.1.6 Statspack release in an 8.0 instance, use the statsrep80.sql script.
Paper #

Otherwise, use the report script included with the release of Statspack you are using. The report script for the 8.1.6 release of Statspack is statsrep.sql; for 8.1.7 use spreport.sql. 8.0
SQL> @?/rdbms/admin/statsrep80

8.1.6
SQL> @?/rdbms/admin/statsrep

8.1.7
SQL> @?/rdbms/admin/spreport

You will be given a listing of available snapshots on which to base your report. You will then be prompted for the beginning snapshot id, the ending snapshot id and the name of the report output file. Remember that the beginning and ending snapshots must have both been taken during the same uptime period. If two snapshots are separated by a database shutdown and startup, the report will be invalid.

PURGING SNAPSHOTS
Purging snapshots is only supported in the 8.1.7 release of Statspack. The script sppurge.sql will purge a range of snapshots. The script sptrunc.sql will truncate all snapshots. Login to SQL*Plus as the perfstat user to proceed. When purging data, you will: be prompted for a range of snapshots to purge have the opportunity to change your mind by issuing a rollback

In contrast, truncating snapshots is complete and immediate. Now is a good time to say this. Whether purging data from Statspack or some other application, always backup your data before you proceed! In this exercise, a simple export will do the job.

UPGRADING STATSPACK
You have installed the 8.1.6 release of Statspack and wish to upgrade to the 8.1.7 release. Upgrading the perfstat schema is not supported. However, the 8.1.7 release does include the script spup816.sql as a convenience to upgrade your 8.1.6 perfstat schema to an 8.1.7 model. Snapshots collected by the 8.1.6 release may not be compatible with snapshots taken by the 8.1.7 release. However, I have not experienced any inconsistencies to date. Just as when purging snapshots, it is a good idea to backup your data before you perform an upgrade!

APPENDIX THE STATSPACK PACKGE

There are two procedures in the statspack package that you will use: snap and modify_statspack_parameters. The parameters to these two procedures that you will want to learn are listed in the following table.
Paper #

Parameter i_snap_level i_session_id i_ucomment i_executions_th i_parse_calls_th i_disk_reads_th i_buffer_gets_th i_sharable_mem_th i_version_count_th i_modify_parameter

Values 0, 5, 10 sid# Text >=0 >=0 >=0 >=0 >=0 >=0 true, false

Default 5 0 Null 100 1000 1000 10000 1048576 20 see note

Description Snapshot Level For capturing session specific statistics Comment to be stored with Snapshot SQL Threshold: number of executions SQL Threshold: number of parse calls SQL Threshold: number of disk reads SQL Threshold: number of buffer gets SQL Threshold: consumption of sharable memory SQL Threshold: versions of a SQL statement Save parameter changes

The defaults specified refer to the value set by Statspack at time of installation. However, the default for i_modify_parameter is false in the snap procedure, always true in the modify_statspack_parameters procedure. If you have modified a parameter, you do not need to specify that parameter again when making future modifications. Both procedures are defined with additional parameters that presently have no effect. The i_sharable_mem_th and i_version_count_th parameters are new in the 8.1.7 release.

Paper #