Thus Spoke Herman

As you probably know, MySQL works by having a pluggable storage engine which handles the retrieval and storage of data for the mysql engine that sits on top of it. The nice thing about it is that there is an api and an example storage engine so you can write your own.

Within Yahoo!, there is an indexing product called Vespa, which is used to store structured, indexed data, which can be retrieved by webservice calls. Due to the problems with getting and aggregating data from disparate datasources, I thought it would be nice to create a storage engine which interfaces with this webservice to allow queries from within mysql for both local mysql tables and data held within this index. The following Jing movie is a demo of what the storage engine can currently do:

Storage Engine Demo

I had some difficulty in getting WireShark to recognise any of the interfaces on my machine. A quick net search led to a rather longer one, but revealed that my problems were because of permissions on the devices themselves for my user. I am sorry to the person who posted this result originally, I cannot remember which site I got it off:

sudo chown <user> /dev/bpf*

This allows wireshark to see the connections. It doesn’t require a wireshark restart either.

In my time trawling through various systems’s DB code, I see we are using “select *” in many places. This has to stop, and we need to specify the columns we want explicitly every time. The reasons for this are:

- A change to the table structure can break the code
- We end up getting all the columns from the DB when we almost certainly don’t need to. Some of the columns can be enormous, and it is a waster of DB and network resource in getting this data.
- A change to the query can add many more columns to be returned.
- It can make a difference to the query plan the DB uses, and can be many orders of magnitude faster to specify columns individually.

Using “select *” is lazy, suboptimal and a coding time bomb.

Just a quick one today…

I have an XP installation running in Parallels and have found some problems using ftp from within it.

FTP is a strange protocol in that it creates two way connections on a number of ports. Even using the FTP variants which initialise all the connections from the client side (passive) I was having trouble using it successfully from within Parallels. I tried host-based and shared networking. What did work was using Bridged-Mode and this allowed the dhcp server on my network to assign the address to the Parallels VM and this then allowed the ftp client to successfully connect to the ftp server and upload files. The other 2 modes do not allow the ftp client to complete the strangeness of ftp properly.

In previous OEM posts I have been dropping the Repository in a variety of ways. In a moment of Instruction Reading, something I rarely do because I love to conform to stereotypes, I found this little obvious-when-you-know beauty:

$ORACLE_HOME/sysman/admin/emdrep/bin/RepManager

[oracle@bbmos01 ~]$ $ORACLE_HOME/sysman/admin/emdrep/bin/RepManager $HOST $PORT $SID -action drop
Enter SYS user’s password :
Enter repository user name : SYSMAN
Enter repository user password :
Getting temporary tablespace from database…
Found temporary tablespace: TEMP
Checking SYS Credentials … OK.
Dropping the repository..
Quiescing DB … Done.
Checking for Repos User … Exists.
Repos User exists..
Clearing EM Contexts … OK.
Dropping EM users …
Done.
Dropping Repos User … Done.
Dropping Roles/Synonymns/Tablespaces … Done.
Unquiescing DB … Done.
Dropped Repository Successfully.

Ho hum. It can do it all.