Reorganize index file, add a format version field.

Upgrades to the new index format should be transparent.

The version field is 3, because 1 was the old textual index, 2 was the
pre-versioned format.

This also includes some efficiency improvements to index loading, by
not copying a hash and using a reference.