Gathering Statistics for the Oracle Optimizer

OracleWhen you execute a query in an Oracle database, Oracle has to decide how to retrieve that data. That’s where the query optimizer steps in. The query optimizer makes decisions like which indexes to use, weather to perform a full table scan etc. based on the tables, columns, calculations and joins in a query. Oracle can do this quite efficiently with the cost-based optimizer, but it is important that there be accurate statistics available for it to use.

A short history

Originally the query optimizer worked on a static set of around 20 rules. These rules would be applied regardless of the size and type of data in a table.

Oracle 7 introduced the cost-based optimizer which can make more intelligent optimization decisions. By analyzing pre-gathered statistics on database objects the cost-based optimizer estimates the “cost” of processing several possible execution plans. The cost-based optimizer then chooses the cheapest execution plan and the database executes the plan.

In Oracle Database 7 through 9i either the cost-based or rule-based optimizer could be used. The rule-based optimizer is no longer included in Oracle 10g.

How can I tell which optimizer mode I am using

If you are running Database 10g or later, you are using the cost-based optimizer.

If you are running Database 7 through 9i you should check the optimizer_mode parameter.

SQL> show parameter optimizer_mode

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
optimizer_mode string CHOOSE

RULE means your database is using the old rule-based optimizer. The good news is you do not have to gather statistics on your data. The bad news is your queries probably aren’t running as well as they could.

CHOOSE was introduced as a stop-gap between rule- and cost-based optimizers. If there are no statistics available, the rule-based optimizer will be used; however if statistics are available the query optimizer will default to cost-based mode.

FIRST_ROWS or ALL_ROWS will force your database to use the cost-based optimizer regardless of statistics, so you’d better gather them.

How can I gather statistics for my index, table, schema, database, etc.?

The DBMS_STATS package is used to gather statistics for the cost based optimizer. Historically the ANALYZE command would perform similar operations

Here are a few popular examples. Of course you should always consult the documentation for your Oracle distribution before using a new command.

These can be run through SQL*Plus, but you will probably want to automate them for more active databases. There are more options that I have chosen to show here, but these should be a good start.

DBMS_STATS.GATHER_DATABASE_STATS

This will analyze statistics for your entire database. It is likely to take quite a while (hours) and generally should not be necessary, but if you want to analyze the whole database this will do it with one command.

EXECUTE DBMS_STATS.GATHER_SCHEMA_STATS(ownname => 'JEMMONS')

This will analyze statistics for everything owned by the user ‘JEMMONS’. It is important to put the username in single quotes and all capitol letters. You can exclude the parameter and parentheses to analyze the current user’s statistics.

EXECUTE DBMS_STATS.GATHER_TABLE_STATS(ownname => 'JEMMONS', tabname => 'ENROLLMENT_DATA')

This will gather statistics for a specific table and all its indexes. This may be a good idea on tables which change drastically on a regular basis.

EXECUTE DBMS_STATS.GATHER_INDEX_STATS(ownname => 'JEMMONS', indname => 'ENROLLMENT_CRSE_NUMB')

This will gather statistics on a specific index. If there is a need to drop and rebuild an index you could use this to re-analyze the index after rebuild.

The GATHER AUTO option can and should be added to the commands above after initial analysis. This will cause only objects with missing stats or more than 10% changed since last analysis (via insert, update or delete) to be analyzed. The resulting command should look something like the following:

EXECUTE DBMS_STATS.GATHER_DATABASE_STATS(options => 'GATHER AUTO')

How often should I gather statistics?

This is a question I cannot answer. I have schemas which do not change often that I may analyze once a month, others that I will gather new statistics on once a day. The GATHER AUTO option should be used to automatically gather missing and stale statistics rather than re-analyzing everything; however, sometimes there may be an advantage to re-analyzing an entire schema or database.

oracle, database, database administration, dba, database tuning

Internet Traffic Report

Internet Traffic ReportImagine having a quick-and-dirty overview of network health on a worldwide basis. That’s exactly what the Internet Traffic Report offers.

When working at WebCT I frequently used this site to help diagnose why University A in City B could not reach our servers while other people could. Now as Plymouth State University starts reaching a wider audience, both nationally and globally, I can see this becoming a useful tool here as well.

Like any other tool, Internet Traffic Report is not going to solve your network problems. It should be considered just another tool in the toolbox. From their FAQ:

Q: How does this relate to me?
A: Your Internet surfing safari may be smooth today, but perhaps you can’t reach Yahoo or a few web sites in Europe. This web site will tell you if those regions of the Internet are currently slowed down. By checking the Internet Traffic Report, you can determine if your problems are global or local.

networking, internet, bandwidth, troubleshooting, network, network administration

Top Oracle Blogs

OracleIn no particular order, here are some of the other folks blogging good stuff about Oracle. Most are more up on current topics than I am since I mostly write how-to stuff so I can find it when I need it.

Rittman.net – Mark Rittman’s Oracle Weblog is a fantastic resource with a focus on data warehousing and business intelligence. One of the best sites by an individual.

PeteFinnigan.com – Pete Finnigan has a lot of security related info. A great resource!

AskTom.Oracle.com – While Ask Tom is not really a blog, there is a wealth of information here on most Oracle topics.

OracleDoug.Blogspot.com – Doug has a bit more casual site and, like me, talks about tech and non-tech topics.

OracleToday.Blogspot.com – Not a frequent poster, but some great nuts-and-bolts SQL and PL/SQL stuff.

ThinkOracle.Blogspot.com – Another blog with a potpouri of Oracle topics.

TKyte.Blogspot.com – The Tom Kyte Blog. Tom works for Oracle (see AskTom.Oracle.com above) and some of his Oracle stuff and peripheral interests and opinions spill over into here.

More are out there. Feel free to comment with your favorites and also check out some of my favorite Oracle web resources

oracle, dba, database, database administrator, rdbms, database security, data warehouse

SQL – Finding the Last Weekday in a Month

OracleSidney V. commented on my post Performing Math on Oracle Dates that he was looking for a way to calculate the last weekday of a month. After some research I concluded that Oracle does not provide a function for this, so I wrote one.

I started with the last_day Oracle function to find the last day of the month. I’m using December 2005 as an example as the last day of the month was a Saturday.

SELECT last_day(to_date('12/2005','MM/YYYY')) FROM dual;

LAST_DAY(
---------
31-DEC-05

This gives us the last day of the month regardless of the day of the week. Now we rewind from that 7 days by subtracting 7 from the date so we can look at the last week in detail.

SELECT last_day(to_date('12/2005','MM/YYYY')) - 7 FROM dual;

LAST_DAY(
---------
24-DEC-05

This lets us start 7 days before the end of the month. Now we can look at this last week in detail. To see the last Monday of the month we use the next_day function to go forward from the last_day() - 7.

SELECT
next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Monday') AS day
FROM dual;

DAY
---------
26-DEC-05

Using this we could find the last Monday, Tuesday, etc. of the month. In this case we want to find the last weekday (Monday through Friday) of the month. We can accomplish this with a UNION of several of these statements.

SELECT day
FROM (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Monday') AS day FROM dual)
UNION (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Tuesday') AS day FROM dual)
UNION (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Wednesday') AS day FROM dual)
UNION (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Thursday') AS day FROM dual)
UNION (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Friday') AS day FROM dual);

DAY
---------
26-DEC-05
27-DEC-05
28-DEC-05
29-DEC-05
30-DEC-05

So we know when the last Monday, Tuesday, Wednesday, Thursday, and Friday of this month was, but which was latest? For that we’ll apply the SQL MAX function to the results of the above union. To do this we put the union above in the FROM part of the query.

SELECT MAX (day)
FROM ((SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Monday') AS day FROM dual)
UNION (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Tuesday') AS day FROM dual)
UNION (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Wednesday') AS day FROM dual)
UNION (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Thursday') AS day FROM dual)
UNION (SELECT next_day((last_day(to_date('12/2005','MM/YYYY')) - 7), 'Friday') AS day FROM dual));

MAX(DAY)
---------
30-DEC-05

Now we have the date of the last weekday of the month. The Oracle to_char function can be applied to this result to get the date in a different format.

SELECT to_char(MAX (day), 'DAY, MM/DD')
FROM ((SELECT next_day((lASt_day(to_date('12/2005','MM/YYYY')) - 7), 'Monday') AS day FROM dual)
UNION (SELECT next_day((lASt_day(to_date('12/2005','MM/YYYY')) - 7), 'Tuesday') AS day FROM dual)
UNION (SELECT next_day((lASt_day(to_date('12/2005','MM/YYYY')) - 7), 'Wednesday') AS day FROM dual)
UNION (SELECT next_day((lASt_day(to_date('12/2005','MM/YYYY')) - 7), 'Thursday') AS day FROM dual)
UNION (SELECT next_day((lASt_day(to_date('12/2005','MM/YYYY')) - 7), 'Friday') AS day FROM dual))
;

TO_CHAR(MAX(DAY)
----------------
FRIDAY , 12/30

Now let’s try this logic on another date to make sure it’s sound. We’ll try today (Jan, 6 2006).

SELECT to_char(MAX (day), 'DAY, MM/DD')
FROM (
(SELECT next_day((last_day(sysdate) - 7), 'Monday') AS day FROM dual)
UNION (SELECT next_day((last_day(sysdate) - 7), 'Tuesday') AS day FROM dual)
UNION (SELECT next_day((last_day(sysdate) - 7), 'Wednesday') AS day FROM dual)
UNION (SELECT next_day((last_day(sysdate) - 7), 'Thursday') AS day FROM dual)
UNION (SELECT next_day((last_day(sysdate) - 7), 'Friday') AS day FROM dual)
)
;

TO_CHAR(MAX(DAY)
----------------
TUESDAY , 01/31

So the logic works, but this would be a lot of typing if you wanted to use it, so let’s make it into a function so we can use it anywhere in the database.

CREATE OR REPLACE FUNCTION last_weekday(month_check DATE DEFAULT sysdate)
RETURN date
IS
last_weekday_date DATE;
BEGIN
SELECT MAX(day) INTO last_weekday_date
FROM ((SELECT next_day((last_day(month_check) - 7), 'Monday') AS day FROM dual)
UNION (SELECT next_day((last_day(month_check) - 7), 'Tuesday') AS day FROM dual)
UNION (SELECT next_day((last_day(month_check) - 7), 'Wednesday') AS day FROM dual)
UNION (SELECT next_day((last_day(month_check) - 7), 'Thursday') AS day FROM dual)
UNION (SELECT next_day((last_day(month_check) - 7), 'Friday') AS day FROM dual));
RETURN last_weekday_date;
END;
/

Note: If you want other users to be able to use this function you have to grant them execute on it. Creating a synonym for it may also be usefull.

Now let's test drive the function we created. We assigned a default of sysdate so if we call it without any parameters it should return the last weekday of the current month.

SELECT last_weekday FROM dual;

LAST_WEEKDAY_DATE
---------
31-JAN-06

If we pass this function a date as a parameter it will return the last weekday of that month.

SELECT last_weekday(to_date('12/2005','MM/YYYY')) FROM dual;

LAST_WEEK
---------
30-DEC-05

Thanks Sidney for the challenge!

UPDATE: Oraboy writes with this algorithm. His code is much shorter than mine and likely more efficient. Thanks Oraboy!

With date_v as
(select last_day(to_date('&MM-&DD-&yyyyy','MM-DD-YYYY')) mydate from dual)
select max(date_v.mydate-(i-1)) last_weekday
from date_v,(select level i from dual connect by level < =3) where to_char(date_v.mydate-(i-1),'Dy') not in ('Sat','Sun')

oracle, sql, dba, database administration, database development