Frits Hoogland Weblog

Most DBA’s would say ‘NO’ without hesitation. I dare you to take another look.

What is this blog about?
This blog is about DBA’s who run databases on linux. Linux sysadmins are target audience too; this blog is about the stop/start system on linux, and to be more precise: about the specific implementation of the stop/start system on redhat and it’s derivatives (Oracle’s enterprise linux for example, but also centos)

What about it?
The redhat stop/start system looks and feels like the traditional unix system V stop/start system found on most unixes. Correct usage of the redhat stop/start system is in fact quite different from that.

The traditional unix system V stop/start uses an executable script in the init.d directory (often /etc/init.d or /etc/rc.d/init.d) and uses a symbolic link in the runlevel directories (rc1.d, rc2.d, rc3.d, etc.) from the script in init.d, but prefixing the name it with ‘S’ (for start) or ‘K’ (for kill, thus stop) and a two digit number to indicate the order for starting or stopping.

The redhat stop/start system works with an executable script in the init.d directory (/etc/rc.d/init.d). The correct way to enable stop/start is using the chkconfig command:
chkconfig --add yourscript
This command check if the script yourscript is present in /etc/rc.d/init.d, then reads a line looking like this in the stop/start script:
#chkconfig: 2345 80 05
This tells chkconfig that the script should run in runlevels 2,3,4 and 5 with order number 80 and stopped (killed) with order number 05.
To check if chkconfig did understand your script, use:
chkconfig --list
Further reading about chkconfig:
man chkconfig

Alright, but what about crashing the database?
A lesser known fact is that the redhat stop/start system uses a file with the same name as the stop/start script in /var/lock/subsys to determine if the script is already started (so it doesn’t get started if it’s already started in a previous runlevel) and needs to be created in the start part, but more importantly it determines if the script needs to execute the stop procedure by checking the existence of this file in /var/lock/subsys. As part of the stop procedure the file needs to be removed, so the stop procedure isn’t run again in another runlevel.

In many custom made scripts this file doesn’t get created in /var/lock/subsys in the start procedure, so the stop procedure actually is never executed, because the script is already stopped from the perspective of the stop/start system. This typically means that the oracle stop/start script does a perfect job starting the database, but actually never stops the database, which means that during a shutdown or a reboot, the database actually crashes.

Sadly, I see very much scripts forgetting this (even ones from respectable sources, like redhat; see: Redhat summit beginner’s guide to running oracle on RHEL, for example, there are much more). Check your stop/start script today!

I received an email from Oracle with an invitation for the Oracle ACE program! I must admit I am proud of the recognition from “the industry” for my community efforts. See my ACE profile here.

For this recognition I must thank Doug Burns, but also all other friends who helped me solving problems, help with presentations, or helping me in other aspects. Thank you! Last but not least my employer, VX Company..

A paper written as a companion for my presentation ‘Oracle security done right’, which will be presented at the Slovenian International Oracle User Group (SIOUG), and at the UKOUG has been put on technet in the security section: http://www.oracle.com/technology/deploy/security/index.html

If you are interested in enhancing security on linux/unix based systems and making DBA usage audit-able, fetch the paper. Of course, any comments are appreciated!

Various people have written about how you can add commandline history to sqlplus on Linux (and probably unix too?)

If someone is using Redhat Enterprise Linux, or one of the clones (Oracle Enterprise Linux, Centos), and are using yum (and I can see only very limited reasons for not using it; if you have to administer multiple machines, setting up a yum server will be a real timesaver, if the number is very small, use the internet repositories directly), add the EPEL repository: EPEL.

EPEL contains packages not present in the package collection of RHEL. One of these packages is rlwrap, which can give SQL*Plus commandline history. Installing rlwrap with EPEL is as simple as:

yum install rlwrap

Please notice this gives the SQL*Plus commandline some of the properties you have in the bash shell (when sqlplus is invoked with rlwrap of course: rlwrap sqlplus username/password@database). This means you can also search the history with the key combination CTRL + R.

Another property of the bash shell is autocompletion with the TAB key (something I use extensively). It would be extremely handy to have TAB autocompletion for Oracle keywords and a list of choices (if it can’t autocomplete because several options exist) with TAB - TAB (TAB two times).

Johannes Gritsch has written a little script to overcome some rlwrap difficulties and lists with autocompletion for keywords and data dictionary: http://www.linuxification.at/rlwrap_ext.html.en

Please note the tar needs to be extracted in ‘/usr/share/rlwrap’ when using the rlwrap version of EPEL, instead of the path listed in the readme. This path also needs to be modified in the ’sql+’ script.