You shut the door behind you, hang your keys on the wall, and expertly slip out of your shoes. You’re home, the birthday party is over, and what else is there to do but check the status of your servers.
Oh, no, disaster has struck.
df -m
reports your disk usage is at 100%. You feel the wave break over you as panic settles in.
But how?
The numbers from sudo du -scm /*
don’t add up to nearly enough.
You check lsof and see a billion instances of an SQLite3 database being held open, each taking up a measly seven megabytes. But together — together they accumulate to ruin.
You restart httpd, and suddenly there’s disk space for days.
But why were they being held open? Is there some bug in PHP or Apache where when SQLite3::close() isn’t explicitly called the file remains open?
An external library was opening the database, but never closing it. You extend their class and add a close() call to its destructor.
Unfortunately, you don’t get a chance to see if that prevents the issue from recurring, as you end up purging all reference to the library from your codebase.
And then everything swell.