Wiki’s: The agile way to document

May 28th, 2008

Wiki’s (see wikipedia for a description) are a nice and easy way to document.
Because of the agility it’s easier to maintain documentation. I often encounter documentation in word, which has aged and is not accurate anymore. Sometimes changes occure which just do not get documented, sometimes it is documented, but in the version of the one who did the changes, which is not uploaded back to the document location. Or in another document location.

Traditionally, for a wiki you need a webserver, an application server and storage (either in a database or just on the filesystem). In some cases (think tomcat, jboss) the application server can act as a webserver also.

But what if you want a personal wiki (just to put your own (installation|modification)logs, scripts, links, and other data), and you want wiki flexibility, but not the difficulty of starting webservers etc.?

The maximal minimalistic approach is here: Wiki on a Stick, a wiki that consists of one self-modifying XHTML file.

How an ASM diskgroup is found by the database.

May 26th, 2008

The short answer is simple: this is done by the ocssd (Oracle Cluster Support Services daemon).

But how does that work more indepth?

The ocssd is needed for ASM. We all know that, because the database create assistant tells us that when we make an ASM instance. The ocssd is started from the inittab; the ocssd installation procedure ($ORACLE_HOME/bin/localconfig add) adds an entry to inittab:
h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1

This entry tells us the following information:
-/etc/init.d/init.cssd is run in runlevels 3 and 5 (35)
-if init.cssd dies, init will start it up again (respawn)

The place of the init.cssd script is a little confusing; the directory /etc/init.d suggests it's a script which is used in the chkconfig start and stop scripts, which is not true, it is started via init/inittab.

After the ocssd is being started, it opens up a number of filedescriptors:
-some message files:
-- $ORACLE_HOME/srvm/mesg/procus.msb
-- $ORACLE_HOME/css/mesg/clssus.msb
-- $ORACLE_HOME/has/mesg/clsdus.msb
-some logging:
-- $ORACLE_HOME/log/hostname/cssd/cssdOUT.log (3 times)
– $ORACLE_HOME/log/hostname/cssd/ocssd.log
– $ORACLE_HOME/log/hostname/alertpmoracle.log
-the cluster registry disk, which can be file on the local (non-cluster) filesystem because the ocssd is the only process/daemon which uses it:
– $ORACLE_HOME/cdata/localhost/local.ocr
-some sockets on the local filesystem:
– /var/tmp/.oracle/shostnameDBG_CSSD
– /var/tmp/.oracle/sOracle_CSS_LclLstnr_localhost_0
– /var/tmp/.oracle/sOCSSD_LL_hostname_
– /var/tmp/.oracle/sOCSSD_LL_hostname_localhost
-and some network sockets:
– UDP [::1]:1027
– UDP [::1]:1028
– UDP [::1]:1029
– UDP [::1]:1030
– TCP 127.0.0.1:31144
– TCP 127.0.0.1:8986

(this information is found in /proc/pid/fd and using ‘lsof’)

When the ASM instance is started, it announces itself by the ocssd using the gmon background process:

First phase:
(this sequence of communications and actions is done for every process which contacts the ocssd;
the ‘authentication sequence’)
-Communication is initiated using the socket /var/tmp/.oracle/sOCSSD_LL_hostname_
This means a new private socket is created for the communication between ocssd and the ASM instance.
-A directory and file is created by the ocssd: $ORACLE_HOME/css/auth/directory
The directory name is 8 digit hexadecimal, the filename is also 8 digit hexadecimal.
-The ocssd communicates the path+filename through the socket
-Next the ocssd gets two messages through the socket
The second one triggers the ocssd to read the file with the hexadecimal name.
-The ocssd reads the file which consists of 4 bytes
-The ocssd reads /etc/passwd
-The ocssd sends a message and receives one
-The directory with the 8 digit hexadecimal name is removed
-The ocssd reads a message with the PID in it of the process which starts the ASM instance.

-The ocssd writes a message with its $ORACLE_HOME
-The ocssd reads a message
-The ocssd cleans up the private socket.

Second phase:
-The authentication sequence is done
-The ocssd reads a message with the diskgroup in it
This is the ASM instance announcing it’s diskgroup (one in my case).
-The ocssd writes a message with the diskgroup in it
-The socket is kept open

Third phase:
-The authentication sequence is done for the gmon process of the ASM instance
-The gmon process and the ocssd communicatie the following things:
–$ORACLE_HOME
–instance name (+ASM)
–diskgroup name(s); with and without ‘DG_’ in front of the diskgroup name
–’OSM_ALL’

At this point the ASM instance is running.

The database which uses ASM does roughly the same:
(please mind the ‘authentication sequence’ is done for every process for every communication to ocssd)
-The startup process asks ocssd about diskgroups, $ORACLE_HOME and instance name
This is done several times using several processes
-The first background process which asks ocssd about ASM is the ASMB process
-Then a process called ‘oracle_ASM instance name_asmb_database name‘ gets information
This process is not listed in the database instance in v$session (?). This process also looks at the registry disk.
It creates tracefiles in the trace directory of the ‘normal’ database.
-Then the checkpoint process
-Then the RBAL process.
First declaration of keyword ‘LOCAL’, presumably for declaration of local failgroup (?)
-Then a process which is gone after startup of the database
-Then dbw0
-Then lgwr
-Then server process of the sqlplus session which started the database
-Then mmon, smon, dbrm, fbda, q001
-Then 5 processes which do not exist anymore when the database opens

And now for something completely different…

May 18th, 2008

I know there are people out there who like Douglas Adam’s ‘The hitchhiker’s guide to the Galaxy’. Some of them actually did read the book, and the books that did follow it.

Also, some (old) people remember infocom. Infocom made ‘text adventures’ in the 80’s. Text adventures are computer games which present themselves as text (obviously), and are played by entering text. Probably superfluous, these are not made any longer, or at least are not that popular anymore.

Some of the old infocom games are released and playable online. One lesser known Infocom game is…The hitchhiker’s guide to the Galaxy!

You can play it (or get an idea about it) here.

I’ll get you a pint of lager if you can make it to the pub (in the game of course) without help of the walkthrough.

Journaling filesystems. What’s consistent?

May 15th, 2008

Recently I’ve had a discussion about filecorruptions / corruptions on the oracle (database) level, and what journalling filesystems do to protect that.

It seems that the general believe is journalling filesystems work the following way:
1. write is done to the filesystems’ “intent log”
2. write is done to the filesystem at the actual location
3. previous write action in the “intent log” is flagged as truly written

This way, when a disaster happens, the system just has to redo all writes not flagged as written to be consistent. In fact, that’s what I’ve been taught in HPUX system administration classes.

As you’ve probably guessed by now, that is not the case. In fact a thesis of Vijayan Prabhakaran describes how the journaling works for Ext3, ReiserFS, JFS, XFS and Windows NTFS.

The thesis investigated the capability of failure resolution of above described filesystems. To do so, it investigated the way the journaling works. This investigation shows the default setup of all these filesystems do only do journalling for the filesystem metadata, NOT for the data (!!!) Ext3 and ReiserFS can be configured to do journalling for data too, but that requires reconfiguration.

This means that after a crash, your filesystem itself is in a consistent state after online recovery, but the data inside your files might not be…

Does anyone know how journalling is done on filesystems on AIX, SUN, HPUX, True64 and 3rd party filesystems like Veritas?