Bigfile

Bigfiles are archive files used to compose multiple files together, they support compression on some platforms.

File names

File names are only stored in the form of a hash, this allows a fast lookup with a binary search.

Earlier games used a custom hash function. Since Defiance, CRC-32 with a polynomial of 0x4C11DB7 is used.

Specialisation

Tomb Raider Legend introduced specialisation, this allowed storing multiple files with the same file path for different languages. Specialisation is also used for storing different files depending on settings such as Next Generation Content.

A bitwise operation is used for matching the right file.

  10000000000000000000000000000100 German, Next generation content
& 10000000000000000000000000011111 All languages, Next generation content
= 10000000000000000000000000000100

A small tool to visualize specialisation masks can be found here.

Sectors

Files in the bigfile are aligned by chunks of 2048, this matches the sector size of the CD-ROM. In Legend and later the position of a file will not be the offset in the file, but rather the sector.

auto position = offset << 11;

Alignment

Bigfiles can be split over multiple files, the physical file that a file is in can be calculated with the alignment.

auto position = offset << 11;
auto file = position / alignment;

sprintf(extension, "bigfile.%03d", file);

The alignment depends on the game.

Game Alignment
Legend/Anniversary 0x9600000
Underworld and later 0x7FF00000
Deus Ex Defined in the header

Compression

Some platforms use compression, such as Xbox 360 version of Legend. compressedLength will be larger than 0 in that case.

The Zlib compression algorithm is used to decompress these files.

Format

This section will give a quick overview of the format of the bigfile per game.

Soul Reaver 2

Soul Reaver 2 Legacy of Kain Defiance

struct BigFileEntry
{
    uint32_t fileLen;
    uint32_t filePos;
    uint32_t compressedLen;
}

struct BigFile
{
    uint16_t numFiles;
    uint16_t pad;  

    // List of filename hashes, the position in this array will match position in contents array
    uint32_t hashes[numFiles];

    BigFileEntry contents[numFiles];
}

Tomb Raider Legend

Tomb Raider Legend Tomb Raider Anniversary Tomb Raider Underworld Guardian of Light

struct ArchiveRecord
{
    uint32_t size;
    uint32_t offset;
    uint32_t specMask; // Specialisation mask
    int32_t compressedLength;
}

struct ArchiveFile
{
    uint32_t numRecords;
    
    // List of filename hashes, the position in this array will match position in records array
    uint32_t hashes[numRecords];
    
    // All file records
    ArchiveRecord records[numRecords];
}

Deus Ex Human Revolution

Deus Ex Human Revolution

struct ArchiveRecord
{
    uint32_t size;
    uint32_t offset;
    uint32_t specMask; // Specialisation mask
    int32_t compressedLength;
}

struct ArchiveFile
{
    uint32_t alignment; // The archive alignment
    char configName[64] // Config name/build dir such as pc-w, xenon-w

    uint32_t numRecords;
    
    // List of filename hashes, the position in this array will match position in records array
    uint32_t hashes[numRecords];
    
    // All file records
    ArchiveRecord records[numRecords];
}