Dealing with a really large database and keeping your web-based service scalable can be quite a challenge.

One thing you might want to consider in order to improve performance is to prevent Unix from updating atime (last access time) at every read/write.

This is done by adding the noatime flag when mounting the file system associated with the database and/or files being accessed alot.

Adding this flag will give you a tremendous boost in read and write capacity, so why not give it a try and see what happens?

Reference: Building Scalable Web Sites by Cal Henderson.

"We may, indeed, say that the hour of death is uncertain, but when we say this we think of that hour as situated in a vague and remote expanse of time; it does not occur to us that it can have any connection with the day that has already dawned and can mean that death may occur this very afternoon, so far from uncertain, this afternoon whose timetable, hour by hour, has been settled in advance. One insists on one's daily outing, so that in a month's time one will have had the necessary ration of fresh air; one has hesitated over which coat to take, which cabman to call; one is in the cab, the whole day lies before one, short because one must be back home early, as a friend is coming to see one; one hopes it will be as fine again tomorrow; and one has no suspicion that death, which has been advancing within one on another plane, has chosen precisely this particular day to make its appearance in a few minutes' time . . ." - Marcel Proust, The Guermantes Way.

I took the liberty to copy this quote used at the beginning of the latest novel by William Boyd called Restless, which I am currently reading, because I found the quote really insightful and appropriate.