New Dog, Old Tricks – how to save yourself some typing with sed

We have a new addition to our household –

Teddy


Cute and fluffy he may be, but he’s got to earn his keep. He can start making himself useful by helping me with this post.

It begins one Friday afternoon when an urgent request lands on my desk with a large splat.

The requirement is that some csv files be uploaded into the Oracle 11g Datbasae serving the UAT environment to facilitate some testing.
There are around 20 files, each with a slightly different set of attributes.
The files are currently sitting on the on the Red Hat Linux Server hosting the database.
I have sufficient OS permissions on the server to move them to a directory that has a corresponding database object in the UAT instance.
Nevertheless, the thought of having to knock out 20-odd external tables to read these files might leave me feeling a bit like this…


Fortunately, a certain Lee E. McMahon had the foresight to predict the potential risk to my weekend and wrote the Stream Editor (sed) program

Continue reading

Advertisements

The Ping of Mild Annoyance Attack and other Linux Adventures

Sometimes, it’s the simple questions that are the most difficult to answer.
For example, how many votes does it take to get an MP elected to the UK Parliament ?
The answer actually ranges from around 20,000 to several million depending on which party said MP is standing for.
Yes, our singular electoral system has had another outing. As usual, one of the main parties has managed to win a majority of seats despite getting rather less than half of the votes cast ( in this case 37%).

Also, as has become traditional, they have claimed to have “a clear instruction from the British People”.
Whenever I hear this, can’t help feeling that the “instruction” is something along the lines of “don’t let the door hit you on the way out”.

Offering some respite from the mind-bending mathematics that is a UK General Election, I’ve recently had to ask a couple of – apparently – simple questions with regard to Linux… Continue reading

Oracle XE 11g – Getting APEX to start when your database does

They say patience is a virtue. It’s one that I often get to exercise, through no fault of my own.
Usually trains are involved. Well, I say involved, what I mean is…er…late.
I know, I do go on about trains. It’s a peculiarly British trait.
This may be because the highest train fares in Europe somehow don’t quite add up to the finest train service.
We can debate the benefits of British Trains later – let’s face it we’ll have plenty of time whilst we’re waiting for one to turn up. For now, I want to concentrate on avoiding any further drain on my badly tried patience by persuading APEX that it should be available as soon as my Oracle XE database is…
Continue reading

Tracing for fun and (tk)profit

If you ever wanted proof that time is relative, just consider The Good Old Days.
Depending on your age, nationality, personal preferences etc, that time could be when rationing finally ended; or when Trevor Brooking won the Cup for West Ham with a “bullet” header; or possibly when Joe Carter hit a three-run homer to seal back-to-back World Series for the Blue Jays.
Alternatively, it could be when you were able to get on to the database server and use tkprof to analyse those tricky database performance issues.

In these days of siloed IT Departments, Oracle trace files, nevermind the tkprof utility are out of the reach of many developers.
The database server itself is the preserve of Unix Admins and DBAs, groups which, with good reason, are a bit reluctant to allow anyone else access to the Server at the OS level.

Which is a pity. Sometimes there is just no substitute for getting into the nitty gritty of exactly what is happening inside a given session.

For those of you who miss The Good Old Days of tkprof, what follows is an exploration of how to access both trace files and even the tkprof utility itself without leaving the comfort of your database.
I’ll go through a quick recap of :

  • how to generate a trace file for a session
  • using tkprof to make sense of it all

Then, coming bang up to date :

  • viewing a trace file using an external table – and why you might want to
  • Using a preprocessor to generate tkprof output
  • implementing a multi-user solution for tkprof

Continue reading

Oracle External Table Pre-processing – Soccer Super-Powers and Trojan Horses

The end of the European Football season is coming into view.
In some leagues the battle for the title, or against relegation is reaching a peak of intensity.
Nails are being bitten throughout the continent…unless you are a fan of one of those teams who are running away with their League – Bayern Munich, Juventus, Celtic…Luton Town.
In their fifth season since relegation from the Football League to the Conference, Luton are sitting pretty in the sole automatic promotion place.
Simon is desparately attempting to balance his “lucky” Christmas-cracker moustache until promotion is mathematically certain. Personally, I think that this is taking the concept of keeping a stiff upper-lip to extremes.

"I'll shave it off when we're definitely up !"

“I’ll shave it off when we’re definitely up !”

With the aid of a recent Conference League Table, I’m going to explore the Preprocessor feature of External Tables.
We’ll start with a simple example of how data in an External Table can be processed via a shell script at runtime before the results are then presented to the database user.
We’ll then demonstrate that there are exceptions to the rule that “Simple is Best” by driving a coach and Trojan Horses through the security hole we’ve just opened up.
Finally, in desperation, we’ll have a read of the manual and implement a more secure version of our application.

So, without further ado… Continue reading

Generating an md5sum on Oracle Database LOBs – or how to organise your Holiday Snaps

Following the trend in these straightened times, Deb and I decided to stay at home this year rather than going away on holiday.
I say “decided”, but this was really more due to the fact we were terribly grown up and bought a house last year.
As a result, the only recreation we could afford was a walk around the garden…whilst pushing a lawn-mower.
In an attempt to recall happier times, I’ve had a look back of some of the photos from our last proper holiday, in Canada.
As well as providing some happy memories, this also gives me the opportunity to explore how to compare an operating system file ( such as a jpeg) with a LOB held in the database.
Continue reading

Windows batch scripting and learning to love German footballers

Windows scripting – the computing equivalent of the German football team.
I suppose I should qualify that statement, if only to avoid irate comments from any German readers.
The German National side has always been admired rather than loved.

Yes, they have been more successful than most, and have produced more than their share of great players (Beckenbauer,
Rummenigge, Matthaus…add you’re own favourites here). Yet they are rarely cheered by the neutral.
This lack of popularity probably has quite a bit to do with the fact that, in major finals, Germany seem to be condemned to be cast as the bad guys against the forces of footballing light (the Total Football of Cruyff’s Netherlands in 1974), or the plucky underdog (Czechoslovakia in 1976, Denmark in 1992).

The footballing triumph regarded by Germans above all others is Das Wunder von Berne.
The 1954 World Cup Final was meant to be a coronation of one of the great teams in history. The Hungarians of Ferenc Puskas came into the final on the back of an unbeaten run going back 4 years, including an 8-3 thrashing of West Germany in the group stages of the tournament.

After eight minutes of the final, all was going to plan with Hungary already 2-0 up. What followed was one of the great comebacks – and great upsets – in the history of the game. Rahn completed the miracle with his second of the match, and the winner, with six minutes left.
Sepp Herberger, Fritz Walter and Helmut Rahn are the German equivalent of Alf Ramsey, Bobby Moore and Geoff Hurst. Outside of Germany, they remain largely unknown.
Incidentally, apologies for that 1966 reference, but unlike Germans (Italians, Spaniards…) we English have only that single triumph, or the odd glorious failure to look back on.

So, back to Windows batch scripting. It’s widely used, but next to the richness and variety of it’s siblings in the unix world, it appears hideously limited. However, there are times where it is simply unavoidable.

What follows are some basic examples of

  • Accepting user input
  • Using variables
  • a simple for loop ( because there is no other kind )
  • branching
  • interaction with an Oracle database

At this point, I have to say that if you are on a Unix/Linux system, or have access to Cygwin, there are far better ways of working with your database.

For any remaining poor unfortunates… Continue reading