Since mid-May I have maintained a DigitalOcean instance to download weather forecast data for a project. Although historical weather data — actual observation — is archived in a number of places, forecasts do not seem to be. The data from which they are computed is archived, but for this project my collaborator and I are interested in published forecasts. The only way to get them is to collect them as they actually appear, in the actual moment. The task is like the one familiar from daily life — keeping track of things one hears about just once, and making sure not to forget.
The download takes about two hours, and it’s convenient to do it by cron job on a server that I don’t have to worry about getting on line at a certain time of day. In daily life I do not have an equivalent routine — since I began to be aware of my poor memory at age twelve, I have tried to run internal “cron jobs” to improve my ability to remember things, but after all this time I still haven’t mastered whatever the API is.
On Tuesday morning I logged in to manage the data and found it incomplete — about 37% of the previous afternoon’s data had failed to load. The logs contained no explanation of why the download was truncated. I cleaned up and there has been no difficulty since then. Anyway, like the way I live with my own memory, the project is able to tolerate missing data, even consecutive days of it.
Today a message came from DigitalOcean. The outage was apparently serious and was caused by switch failure. The message was full of explanations and apologies, but I am mainly just glad that the data loss wasn’t my fault.
There was another message announcing a cash credit for my downtime. 1¢! Hooray! In ordinary life when I forget something important I should have done or take to long to recall something, no one apologizes or pays me anything — not even a penny.
[end]
Comments are enabled.