The
probability of a
power outage, where the loss in question occurs at your office, is directly proportional to the
necessity of having certain data from a
simulation and to the amount of time the simulation has been running.
Say, for example, my code is going to run for a couple of days, and you really only need the data for calibrating this algorithm you're writing. There will be the smallest fraction of a chance that the power is gonna go out. Now, say that it's going to run for a couple of days, but it's production data instead and it has to go out on Friday. A slightly greater chance that the power is going to go out. Say that now the code has to run for an entire month on the 32 node cluster, and that the data absolutely must go out at the end of the month. Yep, the power's going to go out on day 28 of the job and kill the disk your data is sitting on. Simple probability, really.