Mirroring
Lately I've noticed a lot of people running scripts to mirror my
site. I cannot stress this enough...
PLEASE DO THIS
Maybe that's not what you expected. I know a lot of people complain
and say "Oh, it's hard on my system, etc". Screw that. Grab it all.
I don't even have a robots.txt for this very reason. I want the
data to be out there. Just try to not beat up the machine
recklessly. Mirroring is one thing but hammering it is
another.
If you ever feel bad about it just remember that I promised that
I'd make the database open and a
still haven't. That should
make you really angry. :P
The serious reason of why I haven't done this is because I'm
looking to do a redesign of the database. Something that'll easily
support internationalization (I've been talking about it for a
while), scanning project management, and several other neat tricks.
Once that's done I'll open it all up so people can mess with
it.