A Bounty Bar from 1980 together with it's modern counterpart

There’s more. My phone keeps insisting that I can’t spell and arbitrarily replacing words when I’m writing an SMS.
“You’re just the breast” took a bit of explaining to a rather skeptical Deb.
Having finally persuaded my phone that I really don’t require it’s assistance when composing a short missive, I have now turned my attention to SQLDeveloper.

Whilst most modern IDEs have those ever-so-helpful code completion features, I do sometimes find that they get in the way, especially when you know what you want to type. Pause for a second or two, or start going back to change something and that SQLDeveloper feels compelled to interrupt with some “helpful” suggestion of what I might want to put.

To put an end to these annoying interruptions, simply go to the Tools menu and select Preferences…

In the tree on the left-hand side, expand the Code Editor node and click on Completion Insight.

Uncheck the boxes marked Enable Completion Auto-Popup in SQLWorksheet and Enable Completion Auto-Popup in PL/SQL Editor.

No need to re-start, the changes should take effect as soon as you next start typing.

Now, if you should happen to require some assistance, you can simply press CTRL + SPACE and SQLDeveloper will present you a variation on the theme of “helpful suggestion”.

Whilst I’m being a grumpy old man, what is it with all this spam ? All these unsolicited offers of Viagra. I mean, how do they know ?!

I thought she “kneeded” cheering up, but to date, my attempts at lightening the mood, seem only to have given her the “kneedle”.

Sitting quietly, whilst Deb is wandering through various furniture store websites, I had cause to reflect on Oracle’s own version of Nested Tables.
These were introduced way back in Oracle 8, when Oracle confidently predicted that the Object-Relational Database was the way of the future.
Imagine if they were just bringing this feature out now. You can picture it. Larry would have spent months making disparaging remarks about IKEA’s occasional table range, before unveiling his own version, which was better, cheaper and more efficient.

Whilst you’re never going to be able to rest your pint on one, a Nested Table in Oracle may be useful on occasion.

The Application Error Log

In many Oracle database applications you will find a utility that’s used for debugging issues.
It consists of :

• a table to hold the error log information
• a procedure to write to the table
• a large amount of logging data

The structure of the table will probably be something like this :

CREATE TABLE error_logs(
    username VARCHAR2(30),
    datetime TIMESTAMP,
    program_name VARCHAR2(30),
    parameters VARCHAR2(4000),
    message VARCHAR2(4000))
/

And the procedure is likely to be a variation on theme of :

CREATE OR REPLACE PROCEDURE log_error_pr (
    i_prog_name VARCHAR2, i_params VARCHAR2, i_err VARCHAR2) AS

    PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
    INSERT INTO error_logs(
        username, datetime, program_name,
        parameters, message)
    VALUES(
        USER, SYSTIMESTAMP, i_prog_name, 
        i_params, i_err);
    COMMIT;
END log_error_pr;
/ 

At this point, it’s worth considering the parameters column. Yes, it is a VARCHAR2(4000) and whilst this may, in exceptional circumstances, prove to be too small to hold the requisite data, on the whole it should be OK.
Unfortunately, getting the required data ( in this case the parameter names and values) into the required format to go into the table requires a fair bit of messing about.

CREATE OR REPLACE PROCEDURE trouble_pr(
    i_name IN VARCHAR2, i_payday IN VARCHAR2,
    i_been_to_ikea IN VARCHAR2)
     AS
    --
    -- This procedure only exists to cause trouble
    -- It starts off having just returned from a well-known 
    -- furniture store
    --
    l_msg VARCHAR2(4000);
    e_bits_missing EXCEPTION;
    l_params error_logs.parameters%TYPE;
BEGIN
    --
    -- here it comes...
    --
    l_msg := 'Back to the shop';
    RAISE e_bits_missing;
EXCEPTION
    WHEN OTHERS THEN
        l_params :=
            'i_name : '||i_name
            ||' i_payday : '||i_payday
            ||' i_been_to_ikea : '||i_been_to_ikea;
        log_error_pr( 
            i_prog_name => 'TROUBLE_PR',
            i_params => l_params,
            i_err => l_msg);
        RAISE_APPLICATION_ERROR(-20000, l_msg);
END trouble_pr;
/

Of course, you can have a peek in the data dictionary to get the parameter names by means of DBA_ARGUMENTS. You could then use this information in the error logging procedure meaning that you only had to build a string of parameter values in the order in which they appear in the program. However, looking at the parameter column, especially when there are a large number of parameters passed into the program in question, can get a bit tedious.

Let’s take a different approach which, for the want of a better gag, we’ll call the IKEA method.

Object Relational thingys

The steps for this are :

• create a database type to hold the data
• create another type – essentially an arary of the first type
• create the table with a column defined as the array type
• enter and retrieve the data using the new column

The base type essentially defines a record. The record consists of two fields – the parameter name and the parameter value :

CREATE or REPLACE TYPE paramlist_typ AS OBJECT( 
  param_name VARCHAR2(30), 
  param_val VARCHAR2(4000)) 
/  

Now to create an array of the first type. This will enable us to use the base type as a Nested Table :

CREATE or REPLACE TYPE paramlist_tab_typ AS TABLE OF paramlist_typ 
/

Next up, the table :

CREATE TABLE error_logs( 
    username VARCHAR2(30), 
    datetime TIMESTAMP, 
    program_name VARCHAR2(30), 
    parameters paramlist_tab_typ, 
    message VARCHAR2(4000)) 
    NESTED TABLE parameters STORE AS paramlist 
/

Insert and Select on a Nested Table

Probably a good idea to go through a simple insert statement at this point to see how this nested table thing works :

INSERT INTO error_logs( 
    username, datetime, program_name, 
    parameters, message) 
VALUES( 
    USER, SYSTIMESTAMP, 'TEST', 
    paramlist_tab_typ( 
        paramlist_typ('PARAM1', 'Someval'), 
        paramlist_typ('PARAM2', TO_CHAR(SYSDATE, 'DD-MON-RR HH24:MI')), 
        paramlist_typ('PARAM3', 123456)), 
    'Some error message') 
/

If we want to select from the nested table, we can do :

SQL> select parameters 
  2  from error_logs 
  3  where program_name = 'TEST'; 

PARAMETERS(PARAM_NAME, PARAM_VAL) 
-------------------------------------------------------------------------------- 
PARAMLIST_TAB_TYP(PARAMLIST_TYP('PARAM1', 'Someval'), PARAMLIST_TYP('PARAM2', '0 
7-JAN-12 20:44'), PARAMLIST_TYP('PARAM3', '123456')) 

Hmmm, not the most elegant output. By casting the parameters column in the error_logs table to a TABLE, we can flatten out the nested table, which looks quite a bit better :

SQL> SELECT pl.param_name, pl.param_val 
  2  FROM error_logs err, TABLE( err.parameters) pl 
  3  WHERE err.program_name = 'TEST'; 

PARAM_NAME	     PARAM_VAL 
-------------------- -------------------- 
PARAM1		     Someval 
PARAM2		     07-JAN-12 20:44 
PARAM3		     123456 

SQL> 

Now we’ve got that sorted, we can see how this translates into our error logging utility.
The procedure to log the errors first :

CREATE OR REPLACE PROCEDURE log_error_pr ( 
    i_prog_name VARCHAR2, 
    i_params paramlist_tab_typ, 
    i_err VARCHAR2) AS 

    PRAGMA AUTONOMOUS_TRANSACTION; 
BEGIN 
    INSERT INTO error_logs( 
        username, datetime, program_name, 
        parameters, message) 
    VALUES( 
        USER, SYSTIMESTAMP, i_prog_name, 
        i_params, i_err); 
    COMMIT; 
END log_error_pr; 
/ 

And now for a procedure to call it :

CREATE OR REPLACE PROCEDURE more_trouble_pr( 
    i_name VARCHAR2, 
    i_somedate DATE, 
    i_some_num NUMBER) IS 

    l_msg VARCHAR2(4000); 
    e_bits_missing EXCEPTION; 
    
BEGIN 
    l_msg := 'Where did I leave that lump hammer ?'; 
    RAISE e_bits_missing; 
EXCEPTION 
    WHEN OTHERS THEN 
        log_error_pr( 
            i_prog_name => 'MORE_TROUBLE_PR', 
            -- 
            -- Might be a good idea to do type conversions in 
            -- the program calling the error logging routine 
            -- as at this point we know, for example, what the 
            -- date format should be 
            --           
            i_params => paramlist_tab_typ( 
                paramlist_typ( 'i_name', i_name), 
                paramlist_typ( 'i_somedate', TO_CHAR(i_somedate, 'DD-MON-YYYY')), 
                paramlist_typ( 'i_some_num', TO_CHAR(i_some_num,9999)) 
            ), 
            i_err => l_msg); 
        RAISE_APPLICATION_ERROR(-20000, l_msg); 
END; 
/

And now to run the procedure :

SQL> exec more_trouble_pr('MIKE', TO_DATE('03012012','DDMMRRRR'), 1234) 
BEGIN more_trouble_pr('MIKE', TO_DATE('03012012','DDMMRRRR'), 1234); END; 

* 
ERROR at line 1: 
ORA-20000: Where did I leave that lump hammer ? 
ORA-06512: at "MIKE.MORE_TROUBLE_PR", line 28 
ORA-06512: at line 1 


SQL> 

Now let’s have a look at the entry in the error_logs table :

SQL> SELECT pl.param_name, pl.param_val 
  2  FROM error_logs err, TABLE(parameters) pl 
  3  WHERE err.program_name = 'MORE_TROUBLE_PR'; 

PARAM_NAME	     PARAM_VAL 
-------------------- -------------------- 
i_name		     MIKE 
i_somedate	     03-JAN-2012 
i_some_num	      1234 

I could go on, but Deb has just forbade me from making any more “kneedless” puns on pain of having to live in the shed.

I love going to the Conference. You get the chance to see lots of great presentations about all sorts of things in the Oracle world.
Takeaways from this year? Well, apart from the stress-ball and the cuddly Rhino ( yes, we did have a wander through the exhibition hall as well), I learned quite a bit about Application Express.

Just in case they’re struggling for an angle for APEX in the Oracle marketing department, how about :
“Application Express – Forms 3.0 for the Internet Age”

I suppose I’d better do some explaining fairly quickly before I am taken to task by any APEX aficionados who happen to be reading.

Back in the good old days, when I still had hair, Forms 3 was the character based interface for the Oracle database. A major advance on Forms 2.3, you were able to code actual PL/SQL right into the triggers. Of course, everything ran on the server back then. Forms, the database ( we don’t talk about SQL*Reportwriter…ever !)

APEX has certain similarities to it’s ancestor. The code is stored in the database itself and you can write PL/SQL in it. Of course, it is also “web-aware”. It could easily be thought of as a UI for SQL and PL/SQL…without all that mucking about with Java.

Enough of this Oracle Tech naval gazing. The point of this post is that, if you’ve downloaded Oracle 11g XE, you will have APEX4.0 included. Due to the tiresome reluctance of software vendors to use major release numbers, you may have been under the misapprehension that APEX 4.1 was just a minor tweak. The truth is a rather different.

APEX is maturing rapidly. So, if you’re running XE 11g on a Debian OS ( or even 10g XE), you may very well be interested in getting the latest version of APEX to have a play with…

NOTE – I ran this installation on 11g XE running on Mint.
I’ve tried to highlight any differences you may get when installing on 10gXE, but I haven’t actually done the installation on this database version.

Pre-Installation Checks

There are a few bits and pieces to check here :

• shared pool size
• Oracle XDB version
• PL/SQL Web Toolkit version

Shared Pool Size

The shared pool size needs to be at least 100 MB. Shouln’t be a problem. In SQL*Plus you can check this by :

show parameter shared_pool_size

Alternatively, you can just run this query :

SELECT value
FROM v$parameter
WHERE name='shared_pool_size';

VALUE

--------------------------------------------------------------------------------

0

Hmmm, not quite so straight forward then.

When this parameter is set to 0 then Oracle automatically determines the size of the shared pool.
If you’re running on a 64-bit OS then this should ( according to the documentation) equate to 128 MB. If you’re on a 32-bit OS however, this will default to 64MB.

If you need to change this parameter, then you can simply :

ALTER SYSTEM set shared_pool_size=100M;

If you do need to make this change then it’s probably best to stop and re-start the database at this point to make sure that the additional memory is utilized.

Oracle XDB

The Oracle XDB should be there by default on XE.
Just to check :

SELECT comp_name, version, status
FROM dba_registry
WHERE comp_id = 'XDB';

Provided you get a row back from this query where the status is VALID, you’re good to go.

PL/SQL Web Toolkit

You need to have version 10.1.2.0.6 or later. Once again, this should already be there on XE. If you want to check then :

SELECT owa_util.get_version
FROM dual;

On 11gXE you should get version 10.1.2.0.8.
On 10g XE you’ll probably be on version 10.1.2.0.4.

The APEX 4.1 download does come with files to install version 10.1.2.0.6 and you should be able to run this upgrade by means of the owainst.sql script which will be in the apex/owa directory after you’ve unzipped the download.
If you are going to do this, then I’d suggest you do a full backup of the database beforehand because (a) it’s just good practice and (b) I’ve no idea if this will do what it claims without error on 10gXE.

In fact, even if everything is fine at this point, it’s still a good idea to backup your database before you run the installation… unless you don’t mind losing what you have on there already.

Download Apex 4.1

Next step is to download the Apex4.1 zip file from OTN.

NOTE – if I’m starting to bore you at this point and you’re tempted to follow the “official” installation instructions, the ones for XE are a lot more concise and less confusing than the full-blown APEX installation guide ( which is here).

Once you’ve downloaded the file – apex_4.1_en.zip to give it it’s full name ( I’ve just downloaded the English Language version), you need to copy it somewhere sensible.

In my case, I did the following :

cp apex_4.1_en.zip /u01/app/oracle

Wherever you choose to drop the file, you need to make sure that there are no spaces in the path. Otherwise, the script to install the icons isn’t very happy and you end up with a blank screen…er..so I’ve heard. I mean, I’ve got this friend who tried it. No obviously, I wasn’t dumb enough to do that myself. Ahem.

Anyway, next step is to unzip the file. To save lots of messing about with file permissions and such like, I just switched to the oracle user :

cd /u01/app/oracle
sudo su oracle
sudo unzip apex_4.1_en.zip
cd apex

That last cd was to the newly created apex directory. From this point on, this is where we’ll be running the apex upgrade scripts themselves.
There is one more bit of admin before we get to that, but as the rest of this stuff happens on the database, we can connect now.

At last – the installation

You need to connect to Oracle as SYS as SYSDBA. If you’re the oracle user on the OS then – after making sure that your $ORACLE_SID is set to XE, you should simply be able to do :

sqlplus / as sysdba

If you want to be a bit more boring and conventional ( and haven’t su’d to oracle) then :

sqlplus sys as sysdba@xe

Now, whenever I’m in a multi-database environment, I do like to make sure that I’ve connected to the database I think I have. This makes it so much easier to avoid those hilarious situations where you mistakenly drop a production database or similar.
Anyway, if you’re feeling equally paranoid then :

SELECT name FROM v$database;

This will either set your mind at ease ( in my case by returning ‘XE’) or cause you to quit the SQL session with unseemly haste.

Once we’re on the correct database as SYS then we first need to unlock the APEX_PUBLIC_USER:

ALTER USER apex_public_user UNLOCK;

Next, we need to find the default and temporary tablespaces for the APEX user in the database ( it should be SYSAUX and TEMP respectively) :

SELECT default_tablespace, temporary_tablespace
FROM dba_users
WHERE username = 'APEX_040000';

NOTE – if you’re running this on 10g XE with APEX 2 installed ( the version that 10gXE shipped with) then the username will be FLOWS_020100. There will also be a FLOWS_FILES user and you will need to make a note of the default tablespace for this user.

Now we can get on with running the scripts.
First up is apexins.sql, which takes three arguments :

• default tablespace of the APEX user (usually SYSAUX)
• the same as the first argument, unless you have the FLOWS_FILES user, in which case it’s the default tablespace of the FLOWS_FILES user( this is also SYSAUX by default)
• the APEX user’s temporary tablespace (usually TEMP)
• a virtual directory for the APEX images ( I used “/i/” like it says in the installation notes)

So, assuming your setup is like the one I’m using :

@apexins.sql SYSAUX SYSAUX TEMP /i/

You’ll be pleased to know that this script creates a logfile in the apex directory with a name in the format installYYYY-MM-DD HH24:MI:SS.log.
You’ll be slightly less pleased to know that it’s a bit big (over 700K). Yes, it doesn’t sound that huge…until you have to wade through it.
Anyway, this script takes a little while to run, especially on my asthmatic, ageing laptop with a whopping 1GB RAM and 2GHz Celeron Processor.

Eventually, I got the cheery message :
“Now beginning upgrade. This will take several minutes”

This did cause me to wonder vaguely what on earth the script had been doing for the previous 25 minutes.
Finally, the script completed and logged me off.

Now to load the images – by means of apxldimg.sql.
This script accepts one parameter – the directory you unzipped apex into ( in my case, /u01/app/oracle).

Reconnect to the database as SYS as SYSDBA and run the script

@apxldimg.sql /u01/app/oracle

The final step is to set the password for the APEX ADMIN user :

@apxchpwd.sql

Enter the new password when prompted (it’s hidden so you won’t see anything when you type it in).
You’ll be asked to reset the password as soon as you use it for the first time so it doesn’t need to conform to the databases password complexity rules at this stage.

Post Installation steps

If you now click on the link on your desktop to the database start page, you may get a bit of a shock.
Instead of the start page, you’ll get the rather unfriendly “Error – ERR-1014 Application not found”.

Don’t worry, you haven’t missed a step ( necessarily). We’ll fix this in a minute.
For now, point your browser at http://localhost:8080/apex/apex_admin.

You will now be presented with a login screen.
Enter admin as the username and the password you’ve just set for the admin user.

At this point, you’ll be asked to set a new password, which will have to conform to the password complexity rules ( unless you’ve disabled them) – i.e. at least one uppercase, and one punctuation character. The password will also need to be a minimum of 8 characters.

Once the password change has been processed, you’ll be asked to connect again using the new password.

You should now see :

Looks like Christmas has come early

Fixing the desktop link

When you install 11gXE, a link to the APEX home page is automatically created on your desktop.
The link calls a script called gettingstarted.sh in $ORACLE_HOME/config/scripts.

In order to fix the link, you need to edit this file.

The line :

/usr/bin/$ihttp://localhost:8080/apex/f?p=4950

needs to change to :

/usr/bin/$ihttp://localhost:8080/apex/

Deb is starting to fret that we haven’t quite got enough alcohol in the house to see us over the festive period, so it looks like I’m going to have to venture out into the cold winter evening and brave the crush at the all-night supermarket. Christmas – it’s for the kids really.

• Go through the package conversion process
• Install the database using steps applicable both to Mint and Ubuntu ( and any other Debian based distro)
• Apply some finishing touches so that the menu items work as intended

Along the way, we’ll find out just why Oracle can’t speak English (and lots of other languages), where Mint has hidden the .bashrc, and how Aliens can be friendly.

Because I’m trying to cover both distros in this post, the installation process will be done entirely on the command line. Don’t worry, it’s not as bad as it sounds.

But first…a small morsel of Linux history. Debian, the distro upon which both Mint and Ubuntu are based, was named after a Deb. I had to mention that as this will cause my beloved to think that I’m writing about her ( again), and thus give me enough time to finish writing this !

What are we waiting for then ? Let’s get going.

Before you install – counting the bits

In fact, the experience of constructing flat-packed kitchen units will stand me in good stead for this installation. This is because, whilst XE is available for both 32 and 64-bit windows, Oracle have seen fit to release only a 64-bit version for Linux.
This immediately presents me with a bit of a problem because all of my Linux installs are 32-bit.
If you’re not sure whether your current installation is 32 or 64-bit, simply open a terminal and issue the command :

uname -i

If the output of this command is x86_64, the you’re good to go. If it’s i686, then you’re going to need to install a 64-bit version of your OS before you go any further.
For my part, I’ve taken one of my machines and installed a 64-bit version of Linux Mint 11.

Getting the required Linux packages

We need two packages for this installation – libaio1, which Oracle uses, and alien, which will allow us to convert Oracle’s rpm package to a Debian package, which we can then install.
Oracle’s supported Linux distros are based on Red Hat so there is no Debian package available at the time of writing. Some kind soul may eventually build one ( as happened for 10gXE), but for now, we’ll just have to make do and mend.
So, open Terminal and type :

sudo apt-get install alien libaio1

The next step is to get the Oracle package itself. Rather than go to a repository, we need to go to the web page. Oh, how Windows.
Anyway, point your browser to the 11g Express Edition Download Page. and select Oracle Express Edition 11g Release 2 for Linux x64.
If you haven’t already got an account, you’ll have to create one, but it doesn’t cost anything and you’re unlikely to get loads of spam filling up your inbox as a result.
Save the file ( the default of the Downloads directory is as good as any) and away you go.
Congratulations, you are now the proud owner of a 301 MB file called oracle-xe-11.2.0.1.0.x86_64.rpm.zip

RPM to DEB…with nothing up your sleeves

The next step is to unzip the downloaded file :

unzip oracle-xe-11.2.0.1.0.x86_64.rpm.zip

The output for this command should be something like :

Archive:  oracle-xe-11.2.0-1.0.x86_64.rpm.zip 
   creating: Disk1/ 
   creating: Disk1/upgrade/ 
  inflating: Disk1/upgrade/gen_inst.sql  
   creating: Disk1/response/ 
  inflating: Disk1/response/xe.rsp   
  inflating: Disk1/oracle-xe-11.2.0-1.0.x86_64.rpm  

The unzipped rpm is in the Disk1 sub-directory. Now all we need to do is to use alien to magically transform the ugly rpm into a beautiful deb. Look, she’s standing behind me right now, so I had to write that.

cd Disk1
sudo alien --scripts oracle-xe-11.2.0-1.0.x86_64.rpm

It’ll probably take a while, but eventually, you should get the following message :

oracle-xe_11.2.0-2_amd64.deb generated

Re-arranging the furniture

Before we play with our shiny new deb file, we need to do a bit of sorting out.
Whilst Red Hat and Debian are both Linux, there are some variations in where stuff is kept. We need to make sure that these bits and pieces are accessible when we’re running the installation.
I know this because I blithely ignored any of these precautions before I ran this the first time with, as they say, hilarious results.
So, the following steps are definitely required if your flat-packed database is to be transformed into an all-singing all-dancing Oracle Instance.

chkconfig

For this particular tip, I’m indebted to a certain Dude who mentioned this in his installation guide for the XE11 beta on Ubuntu.

First off, check to see if you’ve got a chkconfig file under the /sbin directory.
If not, then you need to create the chkconfig file as follows :

cd /sbin
vi chkconfig

The contents of the file ( taken from the above link, complete with comments) should be :

#!/bin/bash
# Oracle 11gR2 XE installer chkconfig hack for Debian by Dude
file=/etc/init.d/oracle-xe
if [[ ! `tail -n1 $file | grep INIT` ]]; then
   echo >> $file
   echo '### BEGIN INIT INFO' >> $file
   echo '# Provides:             OracleXE' >> $file
   echo '# Required-Start:       $remote_fs $syslog' >> $file
   echo '# Required-Stop:        $remote_fs $syslog' >> $file
   echo '# Default-Start:        2 3 4 5' >> $file
   echo '# Default-Stop:         0 1 6' >> $file
   echo '# Short-Description:    Oracle 11g Express Edition' >> $file
   echo '### END INIT INFO' >> $file
fi
update-rc.d oracle-xe defaults 80 01

Now set the appropriate permissions on the file :

sudo chmod 755 chkconfig

When you list the file, you should see…

ls -l chkconfig
-rwxr-xr-x 1 root root 611 2011-11-20 15:12 chkconfig

Offer a silent prayer of thanks to the inestimable Mr Dude ( well, I assume it’s Mr from the name), and move on to…

Awk

But we’ve already got awk. I mean, this is Linux. Awk is always there. Except, apparently, when it’s somewhere else.
In this case, Red Hat keeps awk under /bin whereas Debian puts it under /usr/bin.
There is, fortunately, a simple fix – just create a symbolic link to /usr/bin/awk in /bin :

sudo ln -s /usr/bin/awk /bin/awk

If you check, you should now see an entry for awk in the /bin directory :

cd /bin
ls -l awk
lrwxrwxrwx 1 root root 12 2011-11-20 15:49 awk -> /usr/bin/awk

There is one more minor glitch to take care of, but to do this, we need to do the installation first.

Installing Oracle

For this bit, it’s probably a good idea to be connected as root.

sudo su -

If the $ prompt has suddenly become a # you can congratulate yourself on having been bestowed God-like powers. You are now root.
A point to note here is that su – takes you to the home directory of the user you’re switching to ( in this case /root), so you need to cd back to the directory containing the .deb file ( in my case /home/mike/Downloads/Disk1).
NOTE – stay connected as root for these next few steps. I’ll let you know when it’s safe to re-join us other mere mortals.
OK, deep breath…

dpkg --install ./oracle-xe_11.2.0-2_amd64.deb 

At this point you should get output similar to the following :

Selecting previously deselected package oracle-xe. 
(Reading database ... 141714 files and directories currently installed.) 
Unpacking oracle-xe (from ./oracle-xe_11.2.0-2_amd64.deb) ... 
Setting up oracle-xe (11.2.0-2) ... 
Executing post-install steps... 

 Adding system startup for /etc/init.d/oracle-xe ... 
   /etc/rc0.d/K01oracle-xe -> ../init.d/oracle-xe 
   /etc/rc1.d/K01oracle-xe -> ../init.d/oracle-xe 
   /etc/rc6.d/K01oracle-xe -> ../init.d/oracle-xe 
   /etc/rc2.d/S80oracle-xe -> ../init.d/oracle-xe 
   /etc/rc3.d/S80oracle-xe -> ../init.d/oracle-xe 
   /etc/rc4.d/S80oracle-xe -> ../init.d/oracle-xe 
   /etc/rc5.d/S80oracle-xe -> ../init.d/oracle-xe 

You must run '/etc/init.d/oracle-xe configure' as the root user to configure the database. 

Processing triggers for ureadahead ... 
ureadahead will be reprofiled on next reboot 
Processing triggers for bamfdaemon ... 
Rebuilding /usr/share/applications/bamf.index... 
Processing triggers for desktop-file-utils ... 
Processing triggers for python-gmenu ... 
Rebuilding /usr/share/applications/desktop.en_GB.utf8.cache... 
Processing triggers for libc-bin ... 
ldconfig deferred processing now taking place 
Processing triggers for python-support ... 

If you want to start over

From this point on, if, by some chance, you get to a point where you’ve got an intractable error, or realise you’ve missed a step, or simply decide that you really are better off with MySQL after all, you can uninstall Oracle with the following steps :

 /etc/init.d/oracle-xe stop 

Now remove the package…

dpkg --purge oracle-xe 

And finally, tidy-up…

rm -r /u01/app 
rm /etc/default/oracle-xe 
update-rc.d -f oracle-xe remove 

Assuming you’re still raring to go, before you run the final configuration, it’s time for that other bit of tidying up I mentioned earlier.

Lock

the /etc/init.d/oracle-xe that has just been created will be looking for a directory called /var/lock/subsys. On Debian distros this is located in /var/lock.

grep /var/lock/subsys /etc/init.d/oracle-xe

You should see this :

	touch /var/lock/subsys/listener 
	touch /var/lock/subsys/oracle-xe 
			touch /var/lock/subsys/listener 
		touch /var/lock/subsys/oracle-xe 
    if [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/listener 
	rm -f /var/lock/subsys/oracle-xe 

Easiest way to fix this is :

sed -i 's,/var/lock/subsys,/var/lock,' /etc/init.d/oracle-xe

Now to when you check again…

grep /var/lock/ /etc/init.d/oracle-xe

	touch /var/lock/listener 
	touch /var/lock/oracle-xe 
			touch /var/lock/listener 
		touch /var/lock/oracle-xe 
    if [ $RETVAL -eq 0 ] && rm -f /var/lock/listener

Configure Oracle

Now, finally, we’re ready to roll. When you run the configuration, you’ll be prompted for

The HTTP Port for Application Express (default is 8080)
The port for the listener (default is 1521)
A password for SYS and SYSTEM
Whether you want XE to start at boot time

Where appropriate, you can just press Enter to accept the default.

/etc/init.d/oracle-xe configure

Oracle Database 11g Express Edition Configuration 
------------------------------------------------- 
This will configure on-boot properties of Oracle Database 11g Express 
Edition.  The following questions will determine whether the database should 
be starting upon system boot, the ports it will use, and the passwords that 
will be used for database accounts.  Press <Enter> to accept the defaults. 
Ctrl-C will abort. 

Specify the HTTP port that will be used for Oracle Application Express [8080]: 

Specify a port that will be used for the database listener [1521]: 

Specify a password to be used for database accounts.  Note that the same 
password will be used for SYS and SYSTEM.  Oracle recommends the use of 
different passwords for each database account.  This can be done after 
initial configuration: 
Confirm the password: 

Do you want Oracle Database 11g Express Edition to be started on boot (y/n) [y]:n 

Starting Oracle Net Listener...Done 
Configuring database...Done 
Starting Oracle Database 11g Express Edition instance...Done 
Installation completed successfully.

You’ll notice that I decided not to have XE startup on boot. I can make any number of technical justifications for this, but Deb just gives me a withering look and tells me it’s because I’m a control freak.

If you are similarly afflicted, do not despair. Once the installation is complete, you will have an option on the desktop menu to allow you to start the database ( assuming you’re condition is not acute and you prefer to start the database from the command line).
NOTE – OK, that’s quite enough of the power trip. You can disconnect from the root account now.

Anyway, the last thing we need to do is setup the environment variables and then we can get on to the database.

Look, Oracle even provides a handy script to do this for us – /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh. I mean, what could possibly go wrong…

Help ! Oracle can’t speak Welsh

It’s English isn’t any better… y posiblimente, no hablo Espanol.
I should probably explain this from the beginning.
When I run the script, I get…

sh /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh 
export: 3: KINGDOM.AL32UTF8: bad variable name 

Time to take a closer look.

cat /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh
export ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe 
export ORACLE_SID=XE 
export NLS_LANG=`$ORACLE_HOME/bin/nls_lang.sh` 
export PATH=$ORACLE_HOME/bin:$PATH 

The offending line is the NLS_LANG setting, for which another script is called – nls.lang.sh in the same directory.
The script builds the NLS_LANG value by identifying the appropriate character set and then doing a lookup on locale.
You can follow through what it’s doing as follows :

locale charmap
UTF-8
echo $LANG
en_GB.UTF-8

In the script, the locale charmap output ( UTF-8 in my case) gets resolved to AL32UTF8.
The $LANG value then gets transformed as follows :

echo $LANG |sed 's/[.@].*$//'
en_GB

case $locale in 
…
en_gb) nlslang='ENGLISH_UNITED KINGDOM';;
…
esac

# construct the NLS_LANG 
# 
NLS_LANG=${nlslang}.${charset} 

echo $NLS_LANG 

Can you spot where the problem might be ?
Yes, because it’s outputting a string with a space in it but not enclosing it in quotes, instead of returning ENGLISH_UNITED KINGDOM.AL32UTF8, it returns simply KINGDOM.AL32UTF8.

Having looked through the script, you may well encounter this problem if your $LANG starts with any of the following :

af_ZA (South Africa) – Note to Wayne – you don’t have this problem in Australia
ar_AE (United Arab Emirates)
ar_SA (Saudi Arabia)
cs_CZ (Czech Republic)
cy_GB(Wales) – just in case someone was getting smug
en_BW(Botswana)
en_HK( Hong Kong)
en_NZ( New Zealand)
en_ZA(South Africa)
en_ZW(Zimbabwe)
es_AR( Argentina)
es_BO(Bolivia)
es_CL(Chile)
es_CO (Colombia) – yes German, even you
es_CR(Costa Rica)
es_DO (Dominican Republic)
es_EC(Ecuador)
es_GT(Guatemala)
es_HN(Honduras)
es_MX(Mexico)
es_NI(Nicaragua)
es_PA(Panama)
es_PE(Peru)
es_PR(Puerto Rico)
es_SV(El Salvador)
es_US( US Spanish)
es_UY(Uruguay)
es_VE(Venezuela)
gv_GB(Isle of Man)
kw_GB( Cornish)
mi_NZ(Maori New Zealand)
nl_NL( The Netherlands)
pt_BR(Brazil)
sr_YU( Serbian)
zh_CN(China)
zn_HK( Hong Kong – Chinese)
zh_TW(Taiwan)

Given that this file is always run on the machine you are on, the simplest thing to do is to just edit the oracle_env.sh script and hard-code your NLS_LANG setting ( putting it in quotes)….

export ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe 
export ORACLE_SID=XE 
export NLS_LANG='ENGLISH_UNITED KINGDOM.AL32UTF8' 
export PATH=$ORACLE_HOME/bin:$PATH 

If you now re-run the script, all the environment variables should now be set :

 sh /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh 
echo $ORACLE_HOME 
/u01/app/oracle/product/11.2.0/xe 
echo $ORACLE_SID 
XE 
echo $NLS_LANG 
ENGLISH_UNITED KINGDOM.AL32UTF8 
echo $PATH 
/u01/app/oracle/product/11.2.0/xe/bin:/u01/app/oracle/product/11.2.0/xe/bin:...

At this point, you can go running off to play with your lovely new database. However, it’s probably worth doing a bit of housekeeping first.
Deb’s eyes have just lit up at the mention of housekeeping and she’s started dropping hints about me running the hoover round so I’d better make this quick.

Set environment variables on startup

Rather than having to run the oracle_env.sh script every time you want to play with Oracle on the command line ( and to ensure that the desktop menu items work), it’s a good idea to just call the script from your .bashrc. You’ll probably want to add it to the oracle user’s .bashrc as well.
At this point, Mint users ( at least, the newer ones like me) will be wondering why they can’t find a .bashrc in their home directory.

Apparently, Mint abandoned individual users having their own .bashrc as standard some years ago and instead, simply uses the system-wide file which is at /etc/bash.basrc

Ubuntu users do have their own .bashrc in their home directory but, as with Mint, it executes after the /etc/bash.bashrc

Rather than mess around with multiple .bashrc changes for both myself and oracle, I’ll just add the following to the end of /etc/bash.bashrc

. /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh

If you’re an independent minded soul, and do go down the route of setting up a .bashrc in your own home directory, make sure that it’s executable before you start another terminal session :

chmod a+x .bashrc 

What’s on the Menu ?

NOTE – as stated previously, I’m doing this on Mint. More to the point, I’m still on a Gnome 2.3 desktop. The steps here should be pretty much the same on Unity/Gnome 3/KDE/ Xfce but I can’t make any promises.

Whilst you’ve been doing all of that hacking around in terminal, there have been some changes to your Applications menu. Also, your desktop has acquired a new icon with the snappy name oraclexe-gettingstarted.desktop

There are a couple of things you now need to do to before you can go happily clicking on these options.

The desktop icon

Right-click the file on the desktop and select the Permissions tab.
Now check the box to “Allow executing file as program”.
Close the window.

You will notice that the icon has transformed into the familiar Oracle beehive and is now called
Get Started With Oracle Database 11g Express Edition.

Getting the Menu Options to work

Before you take this invitation, there’s one more bit of tidying up, for which you’ll have to return to the command line. Well, you could do this via tools in the Administration menu but I’m trying to keep things generic here.
So, once more unto the terminal dear friends, we need to make both ourselves and oracle members of the DBA group that was created as part of the installation :

sudo usermod -a -G dba oracle 
sudo usermod -a -G dba mike 

Just to check everything has gone according to plan :

sudo grep dba /etc/group 
dba:x:1001:oracle,mike 

Red and Green should not be seen...unless you're installing Oracle on Mint

Getting onto the database

Finally, with the database up and running, click on the Getting Started icon.
After a bit of thought on the part of your browser (well, Oracle actually), you should see something like this :

Ooohh, that looks a bit different. But it does prove it's working

Speeding up startup

Once the novelty wears off, you may begin to tire of the fact that Oracle seems to take an inordinate amount of time to start, and you’re never quite sure when it has actually started.
The reasons for this are connected with the order in which XE starts various components. If you want to tweak this ( and add a desktop notification to boot), head over to this post.

Acknowledgement

There’s a pretty good tutorial on XE installation by Alexander Penev here, which I used as a starting point. If you have any issues with these steps, it may well be worth a look.

I’d love to stay for a while, but domestic drudgery calls.

ALTER SESSION SET NLS_DATE_FORMAT='DDMMRR'

Time for a quick example.

Let’s say we have a table called birthdays :

CREATE TABLE birthdays(
    person VARCHAR2(50),
    dob DATE)
/

Now, when I come to populate this table with all those dates I simply mustn’t forget, do I can just do this …

ALTER SESSION SET NLS_DATE_FORMAT='DDMMRR'
/

INSERT INTO birthdays( person, dob)
VALUES( 'LARRY', '170844')
/

INSERT INTO birthdays( person, dob)
VALUES( 'STEVE B', '240356')
/

INSERT INTO birthdays( person, dob)
VALUES( 'LEO', '180953')
/

That was easy, look ….

SELECT person, TO_CHAR(dob, 'DD-MON-YYYY')
FROM birthdays
/
PERSON			       TO_CHAR(DOB,'DD-MON-YYYY')
------------------------------ ---------------------------------
LARRY			       17-AUG-2044
STEVE B 		       24-MAR-1956
LEO			       18-SEP-1953

Hmm, unless Oracle have acquired some Silicon Valley start-up that has IP rights on the fountain of youth (or possibly time travel), I think that may not have gone totally according to plan.

A better option may have been :

ALTER SESSION SET NLS_DATE_FORMAT='DDMMYYYY'
/

INSERT INTO birthdays( person, dob)
VALUES( 'LARRY', '17081944')
/

INSERT INTO birthdays( person, dob)
VALUES( 'STEVE B', '24031956')
/

INSERT INTO birthdays( person, dob)
VALUES( 'LEO', '18091953')
/

Now when we run the query we get :

SELECT person, TO_CHAR( dob, 'DD-MON-YYYY')
FROM birthdays;

PERSON			       TO_CHAR(DOB,'DD-MON-YYYY')
------------------------------ ---------------------------------
LARRY			       17-AUG-1944
STEVE B 		       24-MAR-1956
LEO			       18-SEP-1953

Most of the time, the RR format works fine. However, if you’re messing around with dates before the 50th year of a century, you may well get problems like this.
If the year number is less than 50, RR will assume that you’re talking about the current century. If it’s 50 or more, it’ll assume that you’re talking about the last century.
Not an issue you’re likely to come across every day I grant you, but useful to know if you’re trying to suck up to billionaire tech tycoons.

On the whole though, I’d rather not have to see Sunday morning from that early on. So, if there is, for example, something that needs to run on my database on a Sunday morning, I’d rather the database just did it without my intervention.

What I plan to do here is :

1. set up a scheduler job
2. explore the ways in which we can control whether a class of job runs on a given database
3. stop jobs running on database startup
4. teach the scheduler how to tell the time – especially in terms of daylight saving

Creating a scheduler job

There is a really good and comprehensive guide on the myriad options available for Oracle’s database scheduler over at Oracle Base.

Here, I’m going to keep things quick and simple.
I’ve got a table that looks something like this :

CREATE TABLE reminders(
    message VARCHAR2(4000),
    msg_ts TIMESTAMP)
/

For some reason I can’t think of right now, I need to insert a row into this table at 8.30 every Sunday morning.
Rather than me having to drag my weary bones all the way to the keyboard, I can simply get the database to remember to do it. So, as a user with CREATE JOB privilege :

DECLARE
    l_job_action VARCHAR2(500) := 
        'BEGIN '
            ||'INSERT INTO reminders( message, msg_ts) '
            ||'VALUES(''What time do you call this ?'', SYSTIMESTAMP); '
            ||'END;';
    
BEGIN
    DBMS_SCHEDULER.CREATE_JOB(
        job_name => 'wakeup_sun',
        job_type => 'PLSQL_BLOCK',
        job_action => l_job_action,
        start_date => SYSDATE,
        repeat_interval => 'freq=weekly; byday=SUN; byhour=8; byminute=30',
        end_date => NULL,
        enabled => TRUE,
        comments => 'Do something automatically on Sunday morning');
END;
/

Notice anything odd about the START_TIME parameter ? Neither did I…at first. We’ll come back to this a little later. In the meantime, if we want to test that our job will actually do what we need it to, we can do a test run :

BEGIN
    DBMS_SCHEDULER.RUN_JOB(job_name => 'wakeup_sun');
END;
/

PL/SQL procedure successfully completed.

SQL> SELECT * 
  2  FROM reminders;

MESSAGE
------------------------------
MSG_TS
---------------------------------------------------------------------------
What time do you call this ?
10-NOV-11 20.16.09.044987

We could even tidy things up a bit by creating a stored procedure to do the work and then just calling that from the scheduler job.

CREATE OR REPLACE PROCEDURE sunday_am_pr IS
BEGIN
    INSERT INTO reminders( message, msg_ts)
    VALUES( 'Now we know that DBA stands for Does Bugger All !', SYSTIMESTAMP);
END sunday_am_pr;
/

And now the job to call it …

BEGIN
    DBMS_SCHEDULER.CREATE_JOB(
        job_name => 'sunday_am',
        job_type => 'STORED_PROCEDURE',
        job_action => 'sunday_am_pr',
        start_date => SYSDATE,
        repeat_interval => 'freq=weekly; byday=SUN; byhour=8; byminute=30',
        end_date => NULL,
        enabled => TRUE,
        comments => 'Do something automatically on Sunday morning');
END;
/

The scheduler can’t tell the time

Let’s have a look at those jobs we just created in a little more detail :

SQL> SELECT TO_CHAR(start_date, 'DD-MON-RR HH24:MI:SS TZR')
  2  FROM dba_scheduler_jobs
  3  WHERE owner = USER
  4  AND job_name = 'SUNDAY_AM';

TO_CHAR(START_DATE,'DD-MON-RRHH24:MI:SSTZR')
--------------------------------------------------------------------------------
10-NOV-11 20:22:55 +00:00

SQL> 

Hmmm, the start time doesn’t seem to have an offset to account for daylight savings. So, rather than using SYSDATE as the START_TIME when defining a job, maybe we should be using SYSTIMESTAMP ?
Well, yes and no. SYSTIMESTAMP will work fine for jobs that have run intervals of less than 24 hours. However, for jobs such as this one, it will blithely ignore changes in daylight saving. So, when, in my case, we move from the optimistically named British Summer Time ( BST) back to GMT, the job will run an hour later than before.

If we’re going to get the job to run at the correct time, we need to be explicit.
Just in case you’ve been typing along whilst reading this post and are now busy cursing me as you’ll have to go back and drop and re-create any jobs you’ve already set up, this script should set things right :

set serveroutput on
DECLARE
    CURSOR c_jobs IS
        SELECT job_name, start_date
        FROM dba_scheduler_jobs
        WHERE owner = USER;
BEGIN
    FOR r_jobs IN c_jobs LOOP
        DBMS_SCHEDULER.SET_ATTRIBUTE(
            name => r_jobs.job_name,
            attribute => 'start_date',
            value => TO_TIMESTAMP_TZ(
                TO_CHAR(r_jobs.start_date, 'DDMMRR HH24:MI:SS')
                    ||'EUROPE/LONDON', 'DDMMRR HH24:MI:SS TZR'));
        DBMS_OUTPUT.PUT_LINE('Job '||r_jobs.job_name||' altered.');
    END LOOP;
END;
/

After running this script, if we re-run the same query we get…

SQL> select TO_CHAR(start_date, 'DD-MON-RR HH24:MI:SS TZR')
  2  FROM dba_scheduler_jobs
  3  WHERE job_name = 'SUNDAY_AM'
  4  /

TO_CHAR(START_DATE,'DD-MON-RRHH24:MI:SSTZR')
--------------------------------------------------------------------------------
10-NOV-11 20:22:55 EUROPE/LONDON

Help, my database is spamming me

So, you have your scheduled jobs running happily in the database. You probably have a few sending e-mails to you as the DBA to let you know all is well, or if there is a problem.
What happens when you refresh your test database with a copy of Live ?

Yep, as soon as you open the instance, the scheduler checks to see if any jobs should have run. If the current system time is later than the next_run_time, then they’ll kick off immediately. Even if the first thing you do after opening the database is to disable these jobs, your inbox is still going to be flooded with spam.

In the good old days of dbms_job, you could avoid this simply by setting the job_queue_processes parameter to 0. This solution has now finally found it’s way into 11gR2 as well. However, if you’re currently stuck somewhere between 10g and 11gR1 then you’ll need an alternative.
The workaround below is a quick n dirty version of this infinitely more elegant solution proposed by Dan Morgan.

Getting classy

The first thing we need to do is create a service. For our specific purposes, you can think of a service as….well…a thingy.

BEGIN
    DBMS_SERVICE.CREATE_SERVICE( 
        service_name => 'not_so_fast', 
        network_name => 'some_random_string');
END;
/

Next, we create a job class and associate it with the service…

BEGIN
    DBMS_SCHEDULER.CREATE_JOB_CLASS(
        job_class_name => 'only_on_live',
        service => 'not_so_fast');
END;
/

Finally, we assign our job to the job class

set serveroutput on
DECLARE
    CURSOR c_jobs IS
        SELECT job_name
        FROM dba_scheduler_jobs
        WHERE owner = USER;
BEGIN
    FOR r_jobs IN c_jobs LOOP
        DBMS_SCHEDULER.SET_ATTRIBUTE(
            name => r_jobs.job_name,
            attribute => 'job_class',
            value => 'only_on_live');
        DBMS_OUTPUT.PUT_LINE('Job '||r_jobs.job_name||' added to ONLY_ON_LIVE class.');
    END LOOP;
END;
/

NOTE – if you subsequently create any jobs that are assigned to a class owned by someone else then you need to GRANT EXECUTE ANY CLASS to the job owner. Otherwise, when you run the job you’ll get :

ORA-27486: insufficient privileges
ORA-06512: at "SYS.DBMS_ISCHED", line 2751
ORA-06512: at "SYS.DBMS_SCHEDULER", line 1794
ORA-06512: at line 2

In order to enable or disable jobs in a specified class, all we need to do is enable or disable the service with which the class is associated.

BEGIN
    DBMS_SERVICE.START_SERVICE( service_name => 'not_so_fast');
END;
/

You can see what services are running on your database by querying v$active_services…

SQL> SELECT name
  2  FROM v$active_services 
  3  /

NAME
------------------------------
not_so_fast
XEXDB
XE
SYS$BACKGROUND
SYS$USERS

If the service is not running, then none of the jobs associated with the class that is, in turn, associated with the service, will run.
So how do we stop the spam ? Dan’s solution used a simple config file based on the same format as oratab. Mine is a bit more basic, but does have the advantage of meaning that you have one less thing to remember when cloning your database.
Here, we’re going to use a database startup trigger :

CREATE OR REPLACE TRIGGER start_service_trg
    AFTER STARTUP ON DATABASE 
DECLARE
    lc_live_name CONSTANT v$database.name%TYPE := 'XE';
    l_db_name v$database.name%TYPE;
BEGIN
    SELECT name INTO l_db_name FROM v$database;
    IF l_db_name = lc_live_name THEN
        DBMS_SERVICE.START_SERVICE( service_name => 'not_so_fast');
    END IF;
END;
/

Simple really, only start the service if we’re running on production. When I start a test database (in this case, where the database name is not XE, the trigger will not enable the services ( and therefore the jobs) thus keeping my inbox spam free.
Regular readers may be wondering how I got through this entire explanation without any Monty Python references.

Take it easy, I’m just recovering

What about those times when you’re bringing the production database back up after some unforeseen downtime. The last thing you want is a load of potentially long-running jobs to kick off and hog all of the available database resources.
If you haven’t used the Services method above, you’re going to have to find an alternative solution…

STARTUP NOMOUNT
ALTER SYSTEM ENABLE RESTRICTED SESSION;

Once you open the database, restricted session is still enabled and the jobs don’t run.
You can then disable them at your leisure before…

ALTER SYSTEM DISABLE RESTRICTED SESSION;

Once you open the database, restricted session is still enabled and the jobs don’t run.
You can then disable them at your leisure before…

ALTER SYSTEM DISABLE RESTRICTED SESSION;

Deb has been sulking most of the way through this post. In fact, she’s not been entirely happy ever since the French knocked out Wales in the Semis. Oh well, only another 4 years till the next World Cup. I mean, how long can a sulk last ?

What do you think this is, a race track ?

… and now you know what it’s like to be a programmer, who has channeled raw inspiration through his or her dancing fingers to produce a thing of beauty and elegance…only to run into the QA person pointing out that the commas are in the wrong place according to page 823, paragraph 2 sub-section e of The Coding Standards.

Often measured in weight rather than the number of pages, Coding Standards documents are often outdated, arbitrary and just plain wrong.
On the other hand, their absence can cause much heartache, not least to those poor souls in support who are trying to maintain code where the Agilista philosophy of Code over Documentation has been taken to the ultimate extreme.

What follows is an attempt to make sense of the Coding Standards conundrum.
I’ll look at what I think a Coding Standards document should contain, and what it shouldn’t.
Then I’ll give some suggestions as to standards for Oracle SQL and PL/SQL which you can either embrace or throw rocks at, depending on your preference.
Before all of that however, I feel the need for some serious catharsis…

Wacky Coding Standards

What follows is a select list of standards that I’ve been required to work with at various places. Some of them are out of date, some just plain arbitrary. Others are what happens when you take the word of “experts” without checking it out for yourself. As you can imagine, this particular section has required quite a bit of restraint on my part.

The ever-changing alias

“When joining two or more tables in a SQL query, the first table must have an alias of ‘a’, the second ‘b’, and so forth.”

So, if we use the HR schema as an example, the following would fall foul of the standards…

SELECT emp.first_name, emp.last_name
FROM employees emp, departments dept
WHERE emp.department_id = dept.department_id
AND dept.department_id = 60
/

What we must do to conform to this stricture is :

SELECT a.first_name, a.last_name
FROM employees a, departments b
WHERE a.department_id = b.department_id
AND b.department_id = 60
/

You can just imagine how exciting it is when a table alias can be any arbitrary letter from a to z and it’s different in every query.

Implicit Cursors are bad, mkay?

“Thou shalt not use an implicit cursor!”

This one was quite popular for quite a long time. This was almost entirely based on the opinion of the great and the good of PL/SQL programming ( you know who you are).
It’s surprising how few people actually thought about this in any depth.
The premise behind this prohibition was that an implicit cursor always executes an extra fetch after retrieving the first row. This extra fetch, the argument went, was inefficient. Using an explicit cursor would result in only one fetch.

What happens when an implicit cursor finds a row when performing that second fetch ? You get a TOO_MANY_ROWS error. If you’re using an implicit cursor, it can be reasonably assumed that you want to fetch one, and only one, row. If there’s more than one row being returned, you probably want to know about it.

To be fair to the experts in question, one of the main health-warnings they issue is “don’t take my word for it”.

Look Mum, less typing

This standard was born out of the ban on implicit cursors.
“When fetching a single row from an explicit cursor, use a FOR LOOP to reduce the amount of code written…”

CURSOR c_deptno IS 
    SELECT department_id 
    FROM departments 
    WHERE department_name = 'IT'; 

BEGIN 
    ... 
    FOR r_deptno IN c_deptno LOOP 
	    l_deptno := r_deptno.department_id; 
	    EXIT; 
    END LOOP; 
    ... 

None of that OPEN FETCH CLOSE malarkey, oh no. For us it was the simple and oh so easy to read construct you see above.
Of course, those unfamiliar with our coding standards often did a double-take when they saw this sort of thing, but they soon got used to it. After all, less typing = less work, right ? I mean, what could possibly go wrong ? Um…

It’s C Jim, but not as we know it

This last one isn’t an Oracle example, it comes from a C coding standards document.

“When incrementing a variable by 1, don’t use the ++ operator. Instead use var = var + 1 as this is more readable”.

The author of this particular gem was later cornered by the entire Development Team and forced to read aloud, extracts from “The C Programming Language” by Kernighan and Ritchie.

I’m sure that anyone who has been programming professionally for any length of time will have their own list of favourites. But how does this help move the discussion on ?
Sure, it gives us some insight into the problems with Standards, but how are we going to solve them ?

This brings us on to the “ideal” Coding Standards document, or at least, my ideal Coding Standards document.
At this point, you may well wonder what right I have to lay down the Coding Standards ? The only answer I can think of is that I’m the egomaniac at the keyboard right now. So there.

To my mind, Coding Standards need to be :-

• concise
• concerned exclusively with Standards, not Best Practice
• concerned with a single language or language family (e.g. SQL and PL/SQL)

The Secret is in the Scope

The first thing to do is to put the Standards Document on a diet.

Best Practices change between releases. They are not standards, to my mind.
Do you remember when you used SELECT COUNT(rowid) instead of SELECT COUNT(*) ?
Back in Oracle 7, the former was more efficient. However, since Oracle 8, the latter statement now works in exactly the same way.

Best Practices belong in a version-specific Best Practices document. Standards should be rather more immutable.

Rather than being buried in a comprehensive Coding Standards document for all of the technologies being used on site, each language or language-family, should have it’s own document.

I have a feeling that my final point on our Standards Diet may be a touch controversial… Standards do not have to specify code layout in any detail…

Code Layout Standards

There always seems to be reams written about how to indent and align code, not to mention the old favourite of where to put your commas.
There are historical reasons for this.
Once upon a time, Oracle would determine whether a statement was cached by doing a straight character-by-character comparison. Consistent use of whitespace was therefore important.
Since around 9i, it takes a slightly different approach.
All non-literals in the statement are converted to lowercase and whitespace is stripped prior to the comparison.

Therefore, I’d contend that all you need to say in the Standards is that code should be indented consistently and be readable.
Provided peer reviews are a feature of your development process, this should be more than enough to ensure that the code that goes forward to production is fit for purpose in terms of maintainability. This is, after all, the primary purpose of Coding Standards.

If you really are an obsessive/compulsive, you can always make use of whichever IDE happens to be in favour and stipulate that code should be run through it’s formatter prior to check in. Yes, all of them can be configured so you can get your commas wherever you want them.

SQL and PL/SQL Coding Standards

If you’ve read this blog for any length of time, you’ll have noticed that my coding standards have evolved over time. One of the advantages of being my own QA department is that I can adopt/change/drop standards as I come across better ways of doing stuff.

PL/SQL is essentially, ADA with SQL extensions. As so much of PL/SQL relies on SQL, it makes sense to combine the standards for both in one place, at least, if you’re exclusively an Oracle shop.

The standards themselves should begin with a disclaimer along the lines of :

“These standards should be adhered to, unless there is a sensible reason to deviate from them.”

This get-out clause should mean that we won’t get lumbered with following a rule which is no longer sensible/ was wrong to start with.
One example that springs to mind…it would seem sensible to have a standard stipulating that all calls to stored program units from within PL/SQL should be by reference, right ?
Sounds like a great idea…until you get to DBMS_OUTPUT.PUT_LINE, for example.
Do you really want to be lumbered with …

DBMS_OUTPUT.PUT_LINE( a => 'Some line to output');

What now follows are the standards I use because I’m comfortable with them and it’s what I’m used to. There’s no reason to believe that you’ll be struck by lightning if, for example, you put your SQL keywords in lowercase.
Anyway, rocks at the ready…

Object naming

In general terms, I prefer the use of suffixes rather than prefixes. This is mainly because my heart sinks every time I look at a list of, say triggers, in an IDE and find that they all start with TRG_.
By using a suffix, you’ll still be able to easily identify the different object types and have a nice usable alphabetical listing.

Another point to note – object names are stored in uppercase in the data dictionary. Therefore, separating words in object names is best done with underscores ( e.g. order_items).
Using Camel Case is problematic in this context as the word boundaries will be lost in the data dictionary ( e.g. OrderItems will be stored as ORDERITEMS).

Tables

Table names should be meaningful. They should also be plural, unless you’re creating a table to hold only a single row.
EMPLOYEES ( because we’ve got rows for more than one) rather than EMPLOYEE or EMP.
Generally speaking, we don’t need a suffix for tables. The exceptions are :
Global Temporary Tables should have a suffix of GTT
Permanent segments used to hold temporary data should have a suffix of TMP.
Comments should be added for all columns in a table.
The comment should specify :

• Whether the column is part of a Primary or Foreign Key.
• If it is part of a Foreign Key, what the parent table/column is.
• If it is a Synthetic Key generated from a sequence, the name of the sequence.
• If there is a check constraint a description of the allowable values rather than just the values themselves.

Primary Key, Foreign Key and Check constraints should be named with appropriate table aliases :

• Primary Key – alias_pk
• Foreign Key – child alias_parent alias_fk
• Check constraint alias_column_name_ck

Where an index is created to support a Foreign Key relationship, it’s named after the Foreign Key constraint with a _IDX suffix. Otherwise, it should start with the alias of the table and end with _IDX. What you put in the middle is more or less up to you.

As an example :

CREATE TABLE contracts ( 
    contract_id NUMBER CONSTRAINT con_pk PRIMARY KEY, 
    client_id NUMBER CONSTRAINT con_cli_fk REFERENCES clients(id), 
    status_ind VARCHAR2(1) CONSTRAINT con_status_ck 
        CHECK (status_ind IN ('L', 'D', 'F')) 
) 
/ 


CREATE INDEX con_cli_fk_idx ON contracts( client_id) 
/ 

COMMENT ON COLUMN contracts.contract_id IS 
    'Synthetic Key, populated from sequence CON_ID_SEQ' 
/ 

COMMENT ON COLUMN contracts.client_id IS 
    'FK to clients.id' 
/ 

COMMENT ON COLUMN contracts.status_ind IS 
    'Valid values are L(ive), D(ead) and F(rozen)' 
/ 

Stored Program Units

As Oracle supplied packages are essentially extensions to the PL/SQL language, it makes sense to treat them in the same way as the core language keywords. In my case, I put SQL and PL/SQL keywords in uppercase.

I use the following suffixes :

• function – _fn
• materialized_view _mv
• package – _pkg
• procedure _pr
• sequence – seq
• trigger _trg
• view _vw

For SQL statements, each clause begins on a new line unless the entire statement can fit comfortably onto one line ( I use a limit of 80 characters as a rule of thumb on this one).

For PL/SQL, indentation must be consistent throughout the program.

Triggers

The naming convention I use for DML Triggers is :
table alias_firing timing_DML action(s)_level ( if statement)_trg.

The firing timing is either B(efore) or A(fter).
The DML actions are I(nsert), U(pdate) or D(elete)
Most DML triggers tend to be row-level. If I have a statement Level then I’ll use S.

Lets say we have a table to contain the names and teams of F1 drivers. For reasons best known to the Application Designer, the synthetic key on this table needs to increment by 1 each time a record is added, with no gaps. Yes, I know that without using a sequence we could get the same id being used twice if the latest record in the table is deleted and then another record inserted. Look, this is just something I knocked together to illustrate the trigger naming thing. I’m not really trying to hammer home the point that standards and best practice don’t mix.
Anyhow, say the table looks something like this :

SQL> desc drivers 
 Name					   Null?    Type 
 ----------------------------------------- -------- ---------------------------- 
 ID					   NOT NULL NUMBER 
 FIRST_NAME				   NOT NULL VARCHAR2(100) 
 LAST_NAME				   NOT NULL VARCHAR2(100) 
 TEAM					   NOT NULL VARCHAR2(100) 
 CREATED_BY				   NOT NULL VARCHAR2(30) 
 CREATION_DATE				   NOT NULL DATE 
 MODIFIED_BY					    VARCHAR2(30) 
 MODIFIED_DATE					    DATE 

SQL> 

Oh look, complete with old-style auditing columns. I wonder what we’d need a trigger for…

CREATE OR REPLACE TRIGGER drv_biu_trg 
    BEFORE INSERT OR UPDATE on drivers 
    FOR EACH ROW 
DECLARE 
    l_next_id drivers.id%TYPE; 
BEGIN 
    IF INSERTING THEN 
        SELECT NVL( MAX( id), 0) + 1 INTO l_next_id FROM drivers; 
        :new.id := l_next_id; 
        :new.created_by := USER; 
        :new.creation_date := SYSDATE; 
    ELSIF UPDATING THEN 
        :new.modified_by := USER; 
        :new.modified_date := SYSDATE; 
    END IF; 
END; 
/

There you go – the trigger name is the table alias followed by the timing ( BEFORE in this case), followed by the DML actions the trigger fires on (Insert, Update) and the appropriate suffix.
You’ll also notice that the code is (a) beautifully formatted and (b) not worth reading.

Variables

Variables should have anchored declarations where they are :

• always assigned a value directly from a database column
• is used in a query predicate as a comparison against a database column
• is used in a DML statement to assign a value to a database column

NOTE – I’ve deliberately not gone into details about scalar declarations because they tend to be version specific. For example, it may be more efficient to use BINARY_INTEGER, PLS_INTEGER or NATURAL_INTEGER depending on which database version you’re running.
Of course, by the time Oracle 12 comes out, it’s entirely possible that they’ll all be synonyms for the same thing. In 10g PLS_INTEGER and BINARY_INTEGER already are.

Variable Names

I use the following prefixes should for variable names :

• local variable – l_
• Global variable – g_
• Constant – gc_ for a global constant,
• local constant – lc_
• Type – typ_
• record – rec_
• Table of records – tbl_
• cursor – c_
• ref cursor – rc_

Parameters are prefixed as follows

• In parameter – i_
• Out parameter o_
• In/Out parameter – io_

Comments

To my mind, comments, when taken with the code, should allow you to work out what is going on.
A header comment should be included to explain the program’s purpose and, if it’s quite involved, to give a brief overview of the functionality it implements.

What does all this have to do with Cats ?

Yes, trying to control a bunch of computer programmers is like trying to herd cats.
My take on this is, if at first you don’t succeed…don’t bother.
By providing a simple concise framework for standards and then allowing code readability to be policed by means of peer review, everyone benefits from a degree of coding consistency from easy to follow rules.
The rules themselves do not get in the way of the actual process of writing code.
Of course, it may well mean that your Peer Review sessions get quite a bit more interesting but hey, we all need a bit of excitement now and then. Just stand back and let the fur fly !
In the meantime, I’m off to take cover from all those rocks that suddenly seem to be flying in my direction.

Unless you’re English, I guess some explanation may be called for at this point. So, at the risk of getting all anthropological…
Brownie Points are awarded by females to their mate for certain actions. When enough brownie points have been accrued, the male of the species can have these converted into a Pass.

The Pass can be used for a night out with the lads, at the footie, or whatever other pursuit is of interest.

This system is instinctively understood by females, although, due to the sudden and unexpected fluctuations in value, less so by the males.

You will often hear conversations such as :

“We’re going to Luton Saturday, their playing Mansfield, you reckon you can get a pass ?”
“Sorry, we’re at the Garden Centre on Saturday, need to earn some brownie points.”

This system goes under many different guises but is essentially the same the world over.
Lately however, it has become apparent that males are having to work harder for their brownie points than previously. The suspicion is that this is related to the retail price of women’s shoes.

What a mess. Where can you put your hard earned savings to ensure an index-linked return ?
Well, if you really want to know, you’ll have to read the Financial Times. Variables in PL/SQL, however, can be indexed-linked to database columns by the simple expedient of an anchored declaration.

Your Flexible Friend

Imagine we had to keep track of the brownie point values in Oracle. Let’s start with a table :

CREATE TABLE brownie_points( 
    activity VARCHAR2(20), 
    points  NUMBER(3)) 
/

You may also have a bit of code knocking around somewhere to display all of the activities for which you can gain points :

set serveroutput on 
DECLARE 
    l_activity VARCHAR2(20); 

    CURSOR c_activities IS 
        SELECT activity 
        FROM brownie_points 
        WHERE points > 0; 
BEGIN 
    FOR r_activities IN c_activities LOOP 
        -- 
        -- Slightly contrived for the purposes of this example 
        -- 
        l_activity := r_activities.activity; 
        DBMS_OUTPUT.PUT_LINE( l_activity); 
    END LOOP; 
END; 
/ 

Notice anything amiss with the l_activity variable ? Well, it’s declared as a VARCHAR2(20) same as the column from which we’ll be selecting data into it. No problem so far.
But what happens if, to take a completely random example, your better-half decides to get involved. Yes, Deb decided that 20 characters just isn’t long-enough to describe the various activities for which points may be awarded. As a result we have to amend the table as follows :

ALTER TABLE brownie_points MODIFY activity VARCHAR2(30)
/

Oh, and our new Business Analyst has had some input on the data we need to hold in this table…

INSERT INTO brownie_points( 
    activity, points)
VALUES(
    'Flowers ( all-night garage)', 2);

INSERT INTO brownie_points( 
    activity, points)
VALUES(
    'Flowers ( from florist)', 15);

INSERT INTO brownie_points( 
    activity, points)
VALUES(
    'Flowers( delivered)', 50);

INSERT INTO brownie_points( 
    activity, points)
VALUES(
    'Flowers( delivered to work)', 250);

INSERT INTO brownie_points( 
    activity, points)
VALUES(
    'Dinner( take away)', 5);

INSERT INTO brownie_points( 
    activity, points)
VALUES(
    'Dinner( restaurant)', 25);
    
INSERT INTO brownie_points( 
    activity, points)
VALUES(
    'Leaving the loo seat up', -10);
    
INSERT INTO brownie_points( 
    activity, points)
VALUES(
    'Late home from pub', -15);

The upshot of all of this is that when we next run our code, we get a nasty surprise…

ERROR at line 1: 
ORA-06502: PL/SQL: numeric or value error: character string buffer too small 
ORA-06512: at line 13 

If only I hadn’t acted like a rogue trader and tried to shave a bit off the typing, this problem would never have occurred. If only I’d used an anchored declaration…

set serveroutput on 
DECLARE 
    l_activity brownie_points.activity%TYPE;

    CURSOR c_activities IS 
        SELECT activity 
        FROM brownie_points 
        WHERE points > 0; 
BEGIN 
    FOR r_activities IN c_activities LOOP 
        -- 
        -- Slightly contrived for the purposes of this example 
        -- 
        l_activity := r_activities.activity; 
        DBMS_OUTPUT.PUT_LINE( l_activity); 
    END LOOP; 
END; 
/

Running this, we’d get :

Flowers ( all-night garage) 
Flowers ( from florist) 
Flowers( delivered) 
Flowers( delivered to work) 
Dinner( take away) 
Dinner( restaurant) 

Stored Program Parameters

You can also use anchored declarations for parameters in Stored Program Units.
Say we had a procedure to add records to the table…

CREATE OR REPLACE PROCEDURE add_activity_pr (
    i_activity brownie_points.activity%TYPE,
    i_points brownie_points.points%TYPE) IS

BEGIN
    INSERT INTO brownie_points(
        activity, points)
    VALUES( i_activity, i_points);
END;
/

Not only do we have a procedure that will never be susceptible to changes to the columns on which it relies, we also have what is, essentially, self-documenting code.
The extra typing you have to do to define the parameters is offset by the reduction in the amount of typing you need to do for the comments.

A bit less typing

You can also use anchored declarations to save you typing. Let’s say you wanted to pull out all the stops…

set serveroutput on 
DECLARE 
    rec_activity brownie_points%ROWTYPE; 

    CURSOR c_activities IS 
        SELECT activity, points 
        FROM brownie_points bp 
        WHERE points = (SELECT MAX(bp1.points) FROM brownie_points bp1); 
BEGIN 
    OPEN c_activities; 
    FETCH c_activities INTO rec_activity; 
    CLOSE c_activities; 
    DBMS_OUTPUT.PUT_LINE('If you want a Pass then '||rec_activity.activity||' is worth '||rec_activity.points||' points.'); 
END; 
/ 

Save this as all_stops.sql and …

SQL> @all_stops.sql 
If you want a pass then Flowers( delivered to work) is worth 250 points. 

PL/SQL procedure successfully completed. 

SQL>

Look at that. Instead of declaring two variables, we just need to declare one as %ROWTYPE for the table. Of course, we could go further. If we just want a cursor specifically to hold the values returned by the cursor, we can use the cursor itself for the basis of the variable declaration…

set serveroutput on 
DECLARE 
    CURSOR c_activities IS 
        SELECT activity, 
            CASE 
                WHEN points < 0 THEN 'Not a good idea' 
                ELSE 'This could work' 
            END as advice
        FROM brownie_points bp;

    rec_activity c_activities%ROWTYPE;          

BEGIN 
    OPEN c_activities;
    FETCH c_activities INTO rec_activity;
    WHILE c_activities%FOUND LOOP
        DBMS_OUTPUT.PUT_LINE( rec_activity.activity||' - '||rec_activity.advice);
        FETCH c_activities INTO rec_activity;
    END LOOP;
    CLOSE c_activities;
END;
/ 

Running this we get …

SQL> @advice.sql 
Flowers ( from florist) - This could work 
Flowers( delivered) - This could work 
Flowers( delivered to work) - This could work 
Dinner( take away) - This could work 
Dinner( restaurant) - This could work 
Leaving the loo seat up - Not a good idea 
Late home from pub - Not a good idea 
Flowers ( all-night garage) - This could work  

PL/SQL procedure successfully completed. 

SQL> 

A word of warning here – if you do want to use this construct, then you need to make sure that you declare the variable after the cursor. Otherwise, you’ll get something like:

ERROR at line 2: 
ORA-06550: line 2, column 18: 
PLS-00320: the declaration of the type of this expression is incomplete or 
malformed 
ORA-06550: line 2, column 18: 
PL/SQL: Item ignored 
ORA-06550: line 12, column 33: 
PLS-00320: the declaration of the type of this expression is incomplete or 
malformed 
ORA-06550: line 12, column 9: 
PL/SQL: SQL Statement ignored 
ORA-06550: line 13, column 31: 
PLS-00320: the declaration of the type of this expression is incomplete or 
malformed 
ORA-06550: line 13, column 9: 
PL/SQL: Statement ignored 

Anchored declarations are incredibly useful. As well as future-proofing, they serve to make code more readable and can even save you typing in the long-run.

NOTE – Brownie points can go down as well as up. The author accepts no liability for the consequences of acting on any of the relationship advice contained in this blog.

Ah those heady days when the world was young…and I first discovered ROWNUM.
That pseudo column was one I used often when investigating the structure and data within tables. It was also, the source of some confusion.

Let’s take an example in keeping with Simon’s adopted home…

CREATE TABLE hockey_teams( 
    club_id NUMBER, 
    team_name VARCHAR2(50)) 
/ 

INSERT INTO hockey_teams( club_id, team_name) 
VALUES( 1, 'TORONTO MAPLE LEAFS') 
/ 

INSERT INTO hockey_teams( club_id, team_name) 
VALUES( 2, 'MONTREAL CANADIENS') 
/ 

INSERT INTO hockey_teams( club_id, team_name) 
VALUES(3, 'OTTOWA SENATORS') 
/ 

INSERT INTO hockey_teams( club_id, team_name) 
VALUES(4, 'WINNIPEG JETS') 
/ 

INSERT INTO hockey_teams( club_id, team_name) 
VALUES(5, 'CALGARY FLAMES') 
/ 

INSERT INTO hockey_teams( club_id, team_name) 
VALUES(6, 'EDMONTON OILERS') 
/ 

INSERT INTO hockey_teams( club_id, team_name) 
VALUES(7, 'VANCOUVER CANUCKS') 
/

If I just wanted to have a look at a few rows of the table ( remember, this was in the days before IDEs that return only the first few rows of a query) …

SELECT *
FROM hockey_teams
WHERE rownum <= 5
/

Now let’s list them in reverse-alphabetical order …

SQL>  SELECT ROWNUM, team_name 
  2  FROM hockey_teams 
  3* ORDER BY team_name DESC 
SQL> / 

ROWNUM TEAM_NAME 
------ -------------------- 
     4 WINNIPEG JETS 
     7 VANCOUVER CANUCKS 
     1 TORONTO MAPLE LEAFS 
     3 OTTOWA SENATORS 
     2 MONTREAL CANADIENS 
     6 EDMONTON OILERS 
     5 CALGARY FLAMES 

7 rows selected. 

SQL> 

Here we can see the “catch” with relying on ROWNUM. It’s applied to the result set of the query before the ORDER BY. If we want to associate ROWNUM to the result set in order we’ll have to do something like :

SQL> SELECT ROWNUM, team_name 
  2  FROM ( 
  3	 SELECT team_name 
  4	 FROM hockey_teams 
  5*	 ORDER BY team_name DESC) 
SQL> / 

ROWNUM TEAM_NAME 
------ -------------------- 
     1 WINNIPEG JETS 
     2 VANCOUVER CANUCKS 
     3 TORONTO MAPLE LEAFS 
     4 OTTOWA SENATORS 
     5 MONTREAL CANADIENS 
     6 EDMONTON OILERS 
     7 CALGARY FLAMES 

7 rows selected. 

SQL> 

Since the days of Oracle 6, analytical functions have come on leaps and bounds. Let’s face it, since those days, dinosaurs have become extinct and this new-fangled interweb thing has taken over the planet.
In the intervening years, ROWNUM has all but disappeared from my SQL vocabulary.
Recently however, I’ve come across a couple of instances where this venerable pseudo-column has come in handy.

Sort of Random

Let’s imagine for a moment that I’m planning my next trip to Canada.
I want to catch some NHL action whilst I’m over there but can’t decide which teams I want to see.
I want to pick two teams to see, more or less at random, but I don’t want to drop into PL/SQL…

SQL> SELECT 
  2    CASE MOD( ROWNUM, 3) 
  3	 WHEN 0 THEN 'Going to see ' 
  4	 ELSE 'Not going to see ' 
  5    END 
  6    || team_name 
  7* FROM hockey_teams 
SQL> / 

CASEMOD(ROWNUM,3)WHEN0THEN'GOINGTOSEE'ELSE'NOTGOINGTOSEE'END||TEAM_NAME 
-------------------------------------------------------------------------------- 
Not going to see TORONTO MAPLE LEAFS 
Not going to see MONTREAL CANADIENS 
Going to see OTTOWA SENATORS 
Not going to see WINNIPEG JETS 
Not going to see CALGARY FLAMES 
Going to see EDMONTON OILERS 
Not going to see VANCOUVER CANUCKS 

7 rows selected. 

SQL> 

OK, so it’s not that random. However, if we take another example, the value of ROWNUM becomes a bit more apparent.

Say I want to test some new functionality on the HR schema – one of the example schemas that ships with the Oracle database.
Here, I want to account for employees that are paid either weekly or monthly.
I’ve written some fancy Payroll routine to identify and process all of the appropriate employees at the appropriate time. This imaginary routine relies on a flag set on a new column in the employees table.
The value of the colum must be either ‘W’ for weekly or ‘M’ for monthly.
Most employees will be paid Monthly, but I do want some set to Weekly to make sure I test both possibilities. At this point, it doesn’t really matter which records are assigned which value.

If you want to play along…

CREATE TABLE hr_employees AS SELECT * FROM hr.employees
/

ALTER TABLE hr_employees ADD ( pay_frequency VARCHAR2(1))
/

We should now have a table that looks like this :

SQL> desc hr_employees; 
 Name					   Null?    Type 
 ----------------------------------------- -------- ---------------------------- 
 EMPLOYEE_ID					    NUMBER(6) 
 FIRST_NAME					    VARCHAR2(20) 
 LAST_NAME				   NOT NULL VARCHAR2(25) 
 EMAIL					   NOT NULL VARCHAR2(25) 
 PHONE_NUMBER					    VARCHAR2(20) 
 HIRE_DATE				   NOT NULL DATE 
 JOB_ID 				   NOT NULL VARCHAR2(10) 
 SALARY 					    NUMBER(8,2) 
 COMMISSION_PCT 				    NUMBER(2,2) 
 MANAGER_ID					    NUMBER(6) 
 DEPARTMENT_ID					    NUMBER(4) 
 PAY_FREQUENCY					    VARCHAR2(1) 

SQL> 

The table contains 107 records. Now we want to set the PAY_FREQUENCY. We want to have around 10% of the rows to be set to ‘W’ and the rest to ‘M’.
Now, we could mess around with PL/SQL and it’s fancy cursor for loops…or we could just do this…

UPDATE hr_employees 
SET pay_frequency = 
    CASE MOD( ROWNUM, 10) 
        WHEN 0 THEN 'W' 
        ELSE 'M' 
    END 
/

Once we’ve run this, we can then see the effect…

SQL> SELECT pay_frequency, COUNT( employee_id) 
  2  FROM hr_employees 
  3  GROUP BY pay_frequency 
  4  / 

PAY COUNT(EMPLOYEE_ID) 
--- ------------------ 
W		    10 
M		    97 

SQL>

Alert Log as an External Table

For the other example of ROWNUMs continued usefulness, I’m going to use an external table. Yes, I know what you’re thinking after my last post, when you’ve got a hammer, then everything looks like a nail.

Consider the DBA. First thing in the morning, the sugar rush from all that Maple Syrup has long since worn off and the caffeine from that first cup of coffee is still negotiating it’s way from the mug to his or her brain.
Whilst in this state of chemical imbalance, our DBA needs to make sure that all is well after last night’s batch runs.
Part of this morning ritual may well involve wading through the alert log.
Now wouldn’t it make things just that little bit easier if we could get Oracle to do the thinking for us and just flag up any relevant alert log entries ?

Well, we know we can take a flat-file and treat it as an external table. Couldn’t we do the same with the alert.log ?
Er, Yep.

CREATE OR REPLACE DIRECTORY bdump AS 
    '/usr/lib/oracle/xe/app/oracle/admin/XE/bdump' 
/

Now to create our external table :

CREATE TABLE alert_log_ext( 
	line VARCHAR2(4000)) 
	ORGANIZATION EXTERNAL 
	( 
		TYPE oracle_loader 
		DEFAULT DIRECTORY bdump 
		ACCESS PARAMETERS( 
			RECORDS DELIMITED BY NEWLINE 
			FIELDS TERMINATED BY '~' 
			MISSING FIELD VALUES ARE NULL 
			( 
				line CHAR(4000) 
			) 
		) 
			LOCATION('alert_XE.log') 
	) 
	REJECT LIMIT UNLIMITED 
/

And do a select…

SELECT * 
FROM alert_log_ext
/
…
LINE 
-------------------------------------------------------------------------------- 
Sun Sep 18 13:23:23 2011 
Completed: ALTER DATABASE OPEN 

39243 rows selected. 

Lovely, we have the alert log in the database….except there’s no key so just querying the bit you want is still a bit fiddly. On the plus side, the rows seem to be read in in correct order by default ( i.e. the order in which they are read from the file) so the result set from the query will be in the correct order, even though no ORDER BY clause is specified.
NOTE – that last bit is supposition on my part. That’s the way it appears to be working, but I haven’t found anything in the Oracle documentation to confirm it.

But where does ROWNUM fit into all this, I hear you ask ? Well, if you want to query the alert log for a specific time window, you can do something like this :

SELECT entries.line_no, entries.line 
FROM( 
	SELECT rownum as line_no, line 
	FROM alert_log_ext 
	) entries, 
	( 
	SELECT MIN(line_no) as line_no 
	FROM ( SELECT rownum as line_no, line FROM alert_log_ext) 
	WHERE line LIKE TO_CHAR(SYSDATE -1, 'Dy Mon DD')||'%' 
	) yesterday 
WHERE entries.line_no >= yesterday.line_no 
/

That’s better. We can now just return the rows of the alert log that have been written since the start of yesterday. However, we may well want to rely on something a bit more precise that just yesterday’s date.

At this point, it’s probably worth looking at the incredibly useful KSDWRT.
Gesundheit ! I hear you cry. Thanks, but no, I haven’t just sneezed, I’m referring to DBMS_SYSTEM.KSDWRT.

Despite having a name that looks like you’ve just leaned on the keyboard, this procedure does have the incredibly useful ability to write a custom message to the alert log…

BEGIN 
DBMS_SYSTEM.KSDWRT(2, 'Oooh, a message'); 
END; 
/

Now, if we look in the alert log for this message…

SELECT line_no, line 
FROM( 
    SELECT ROWNUM as line_no, line 
    FROM alert_log_ext) 
WHERE line LIKE '%Oooh, a message' 
/ 

   LINE_NO LINE 
---------- ---------------------------------------- 
     38989 Oooh, a message 

With an appropriate message, we can narrow down our query to just the bit of the alert log that we’re interested in.

If we return to our bleary-eyed DBA, we could help out by simply writing a message to the alert log at the start of the overnight batch window.

DECLARE 
    l_msg VARCHAR2(30); 
BEGIN 
    l_msg := 'Batch Window Start - '||TO_CHAR(SYSDATE, 'DDMMRR'); 
    DBMS_SYSTEM.KSDWRT(2, l_msg); 
END; 
/

To make sure I’ve now got something after the message in the alert log, I’ve bounced the database.
Now let’s see what entries we have in the alert log after our message :

SELECT entries.line 
FROM( 
	SELECT ROWNUM as line_no, line 
	FROM alert_log_ext 
	) entries 
WHERE entries.line_no >= 
    (SELECT line_no 
     FROM ( 
        SELECT ROWNUM as line_no, line 
        FROM alert_log_ext) 
    WHERE line LIKE '%Batch Window Start - '||TO_CHAR(SYSDATE, 'DDMMRR')||'%') 
/

If the batch window starts before midnight, you’ll want to amend the line :

WHERE line LIKE '%Batch Window Start - '||TO_CHAR(SYSDATE, 'DDMMRR')||'%')

to

WHERE line LIKE '%Batch Window Start - '||TO_CHAR(SYSDATE -1, 'DDMMRR')||'%')

Simon and I have both come a long way since those heady days. He turned into Ewan McGregor and I …well…turned into a Dark Lord of the Sith apparently.

The Anti-Kyte on holiday...the costume didn't catch on

What are they?

External tables are a mechanism by which you can load flat file data into your database, or by which you can export data from your database.

Hold on, what about SQLLoader for data uploads ? And what exactly is wrong with Data Pump for exports ? Come to that, why not just use UTL_FILE for reading/writing files ?
Well, these are all pertinent questions.
In fact External tables make use of the SQL*Loader API for loading data and the Datapump API for exporting data.
They are essentially another means of achieving what you can do with all of these tools. Which one you choose will no doubt depend on what you want to accomplish.
The big difference is that you can use External Tables to query data not stored in the database wheras the other tools listed here are more concerned with getting data into the database first.
As I want to knock out a blog post about External Tables, well, you can guess which one I’ll be using here.
Our starting point is a table that contains all of the places I’m going to visit, together with the dates I’ll be there :

CREATE TABLE holiday_destinations(
    city_name VARCHAR2(30),
    flight_time DATE,
    arrival_time DATE)
/

INSERT INTO holiday_destinations(
    city_name, flight_time, arrival_time)
VALUES('HALIFAX', TO_DATE('20-08-11 12:35', 'DD-MM-RR HH24:MI'),
    TO_DATE('20-08-11 15:15', 'DD-MM-RR HH24:MI'))
/

INSERT INTO holiday_destinations(
    city_name, flight_time, arrival_time)
VALUES('MONTREAL', TO_DATE('22-08-11 12:25', 'DD-MM-RR HH24:MI'),
    TO_DATE('22-08-11 12:58', 'DD-MM-RR HH24:MI'))
/

INSERT INTO holiday_destinations(
    city_name, flight_time, arrival_time)
VALUES('TORONTO', TO_DATE('24-08-11 11:00', 'DD-MM-RR HH24:MI'),
    TO_DATE('24-08-11 12:15', 'DD-MM-RR HH24:MI'))
/

INSERT INTO holiday_destinations(
    city_name, flight_time, arrival_time)
VALUES('OTTOWA', TO_DATE('27-08-11 11:10', 'DD-MM-RR HH24:MI'),
    TO_DATE('27-08-11 12:10', 'DD-MM-RR HH24:MI'))
/

The data we want to upload relates to the flights between these cities. We’re going to put these into a csv file :

HALIFAX, 20-AUG-2011, 22-AUG-2011
MONTREAL, 22-AUG-2011, 24-AUG-2011
TORONTO, 24-AUG-2011, 27-AUG-2011
NIAGARA FALLS, 25-AUG-2011, 25-AUG-2011
OTTOWA, 27-AUG-2011, 30-AUG-2011

Setting up a directory

The first step is to tell the database where to look for the data we’re going to load into our external table. This involves defining a directory object to point to the os directory where the file is going to reside.
This is exactly the same step that you need to take when reading/writing files using UTL_FILE so, in best Blue Peter tradition, here’s one I made earlier :

cd /usr/lib/oracle/xe/app/oracle/admin/XE
sudo su oracle
mkdir ext_tabs

Next, copy the .dat file to this directory

sudo cp $HOME/itinerary.dat /usr/lib/oracle/xe/app/oracle/admin/XE/ext_tabs/.

Now, in the database, set up a directory

CREATE DIRECTORY myfiles AS
    '/usr/lib/oracle/xe/app/oracle/admin/XE/ext_tabs'
/

The owner of the directory object should already have READ and WRITE privileges.

The External Table (Part 1) – load

We need to tell the database how to hold the data in the csv file. This is where we create the external table

CREATE TABLE itinerary_ext(
    location VARCHAR2(30),
    from_date DATE,
    leave_date DATE)
    ORGANIZATION EXTERNAL
    (
        type oracle_loader
        default directory myfiles
        access parameters ( 
            records delimited by newline
            fields terminated by ','
            missing field values are NULL
            reject rows with all null fields( location, from_date, leave_date)
        )
        location( 'itinerary.dat')
    )
/

The table type can be either oracle_loader or oracle_datapump. We’re using the first option here as we want to read data in.

The External Table Upload

If we just want to query the data, then we now simply have to issue a SELECT statement

SELECT *
FROM itinerary_ext
/

If you get an error like this :

ERROR at line 1: 
ORA-29913: error in executing ODCIEXTTABLEOPEN callout 
ORA-29400: data cartridge error 
KUP-04063: unable to open log file ITINERARY_EXT_1718.log 
OS error Permission denied 
ORA-06512: at "SYS.ORACLE_LOADER", line 19 

then it’s more than likely a permissions issue on the os. Check that the oracle owner (usually oracle) has write permissions on the directory (at the os level rather than in the database) where the csv file is located.

Assuming all is working as expected, we can use this table in any valid query :

SELECT hd.city_name, ie.from_date, ie.leave_date 
FROM holiday_destinations hd, itinerary_ext ie 
WHERE ie.location = hd.city_name 
/

OK, so there are some limitations, you can’t specify indexes on the external table for example, and direct DML is also not going to work.
If we want to play with the data inside the database, we’re going to have to load it into a permanent segment…or are we ?
Say you had some external data that you wanted to manipulate in some way and dump the output back onto the OS without the overhead of a permanent segment such as a holding table ? Well, this is an option using only external tables…provided you want to dump the results into another Oracle Database. This is where the datapump type comes in.

External Table (Part 2) – Dump

Instead of creating a permanent table, where going to create a Global Temporary Table :

CREATE GLOBAL TEMPORARY TABLE holiday_itinerary_gtt(
    city_name VARCHAR2(30),
    from_date DATE,
    leave_date DATE,
    flight_time DATE,
    arrival_time DATE)
    ON COMMIT PRESERVE ROWS
/

OK, so the structure IS permanently held in the Data Dictionary, but the data itself is transient and lives exclusively in the PGA.

Next, we’re going to need a procedure to do the transformation and dump the results out onto the os. This will require us to create an external table on the fly…

CREATE OR REPLACE PROCEDURE export_flights_pr AS
    l_dummy PLS_INTEGER;
    l_exists BOOLEAN;
    CURSOR c_tab_exists IS
        SELECT 1
        FROM user_tables
        WHERE table_name = 'EXP_ITINERARY';
    
    l_out_tab VARCHAR2(4000) :=
        'CREATE TABLE exp_itinerary ORGANIZATION EXTERNAL '
        ||'( TYPE oracle_datapump '
        ||'  DEFAULT DIRECTORY myfiles '
        ||q'[  LOCATION( 'flights.dmp')]'
        ||') AS SELECT * FROM holiday_itinerary_gtt';  
BEGIN
    --
    -- Check to see if our external table currently exists.
    -- If so, we need to drop it
    --
    OPEN c_tab_exists;
    FETCH c_tab_exists INTO l_dummy;
    l_exists := c_tab_exists%FOUND;
    CLOSE c_tab_exists;
    
    IF l_exists THEN
        EXECUTE IMMEDIATE 'DROP TABLE exp_itinerary';
    END IF;
    --
    -- Now populate the GTT
    --    
    INSERT INTO holiday_itinerary_gtt( 
        city_name, from_date, leave_date,
        flight_time, arrival_time)
    SELECT ie.location, ie.from_date, ie.leave_date,
        hd.flight_time, hd.arrival_time
    FROM itinerary_ext ie, holiday_destinations hd
    WHERE ie.location = hd.city_name;
    --
    -- Now create an external table on the fly to dump out the results
    --
    EXECUTE IMMEDIATE l_out_tab;
END;
/

For the sake of clarity, the code for creating the external table is reproduced here :

CREATE TABLE exp_itinerary ORGANIZATION EXTERNAL( 
    TYPE oracle_datapump 
    DEFAULT DIRECTORY myfiles 
    LOCATION( 'flights.dmp') 
    ) 
    AS SELECT * FROM holiday_itinerary_gtt 
/

Just run the procedure…

exec export_flights_pr

You should now have a file called flights.dmp in your myfiles directory.

 ls -l flights.dmp 
-rw-r----- 1 oracle dba 12288 2011-08-25 02:56 flights.dmp 

So now we have a dump file, should be simple enough to import into the target database, shouldn’t it ? Not using datapump import, apparently, which gives you an ORA-31619 error complaining that the dump file wasn’t created by a datapump export. Honestly, there’s no pleasing some database utilities !

Sigh, I suppose we’ll just have to create another external table on the target database. This time, however, we specify the type as datapump…

CREATE table imp_flights (
    city_name VARCHAR2(30), 
    from_date DATE, 
    leave_date DATE,
    flight_time DATE, 
    arrival_time DATE)
    ORGANIZATION EXTERNAL
    (
        TYPE oracle_datapump
        DEFAULT DIRECTORY myfiles
        LOCATION('flights.dmp')
    )
/

Now it should all work as expected…

SELECT * FROM imp_flights
/

Right, sun cream, bear repellent, directions to the Moulson Brewery. I’m all set.