Everything2 is much bigger than Everything could ever hope to be.

There are now somewhere in the neighborhood of 520,000 records (nodes + writeups). Many of these are probably still less than 512 bytes total, and most are probably under 2KB, but a small fraction of them are much larger. Apparently writeups can go up to about 30KB; nodes only store titles, links to their writeups, and their softlinks but assuming 4 bytes per link, this can reach 5KB for a node with a long title, lots of writeups, and the limit of 100 softlinks; more, if more information is stored about softlinks. I guessing these long records bring the average to somewhere near 2KB, for a total of 1 GB to store Everything.

But all of E2 is english text. English text can be compressed with a ratio of about 1:2*. That means that all of E2, if your calculations are correct, would fit in a single 500 mb file.

This means you could easily provide a monthly or yearly snapshot of E2 on cd-rom which would soothe all the people who node for the ages and worry about the end of everything.


*bzip2 compresses a fairly large node of 11k to 4.6k with the default settings.

Log in or register to write something here or to contact authors.