Thread: Explanation and Simplification of the RuneScape Cache

Results 1 to 6 of 6
  1. #1 Explanation and Simplification of the RuneScape Cache 
    Registered Member
    Join Date
    Dec 2010
    Posts
    498
    Thanks given
    41
    Thanks received
    84
    Rep Power
    54
    Now that you've clicked on this thread since the name looked interesting, here's why I'm posting this:

    I've been looking around all over the forums for and in-depth explanation of how RuneScape's Cache works, I was unable to find anything remotely close. So, after reading a number of different Cache readers/writers I decided to make a post and explain it. Note: None of this terminology is official and perfect, I'm simply going off what seemed most common throughout all of the different implementations I've read to keep confusion to a minimum.

    The Cache is like a Text Book

    An easy way I find to think of the RuneScape Cache is like a Text Book, below is definitions and comparisons between the different parts of make up the RuneScape Cache and a Text Book.

    Index Files – Table of Contents

    File Name: main_file_cache.idx0 . . . main_file_cache.idx255

    Index files can be thought of as large Tables of Contents like you would see in a book. They contain many entries, Indexes, which tell where individual files are within the main Data File so that they can be retrieved and used. Note: there are multiple Index Files, each Index File only holds the Indexes for 1 type of file, you can tell which file type a Index File holds by the number at the end.


    Data Files – All of the Information

    File Name: main_file_cache.dat and main_file_cache.dat2

    Data Files can be thought of as the book from chapter 1 to the last chapter, they contain many different files all bonded together to form a single file. It is impossible to quickly find the correct file within the Cache without using the Indexes provided by Index Files.

    Indexes – Table of Contents Entries

    Indexes can be thought of as the individual entries of the Table of Contents, they are used to find where files start and where they end so that Sectors from the data file can be combined together form their original file.

    Information Type Description
    File Size Medium (3 Bytes) The size of the file that this Index points to.
    Sector ID Medium (3 Bytes) The first Sector of the file that this Index points to.

    Example Code
    Code:
        public static Index read(ByteBuffer buffer) {
    	// Read values
    	int fileSize = ByteBufferUtil.getMedium(buffer);
    	int sectorId = ByteBufferUtil.getMedium(buffer);
    
    	// Create and Return
    	return new Index(fileSize, sectorId);
        }
    
        public ByteBuffer toByteBuffer() {
    	// Allocate buffer
    	ByteBuffer buf = ByteBuffer.allocate(SIZE);
    
    	// Write values
    	ByteBufferUtil.putMedium(fileSize, buf);
    	ByteBufferUtil.putMedium(sectorId, buf);
    
    	// Flip and Return
    	return (ByteBuffer) buf.flip();
        }
    Sectors – Pages

    Sectors can be thought of as pages within the book. Each page is exactly the same size but can be more or less filled with information, if there’s empty space that space is left blank rather than cutting it out. Multiple pages form chapters much like how multiple Sectors form files.

    Information Type Description
    File ID Unsigned Short The file that this Sector belongs to.
    Chunk ID Unsigned Short Which chunk of the file the data of the Sector is.
    Sector ID Medium (3 Bytes) Which Sector of the data file this is.
    Type ID Unsigned Byte The type of file this Sector belongs to.
    Data 512 Bytes The raw data that this Section contains.

    Example Code
    Code:
        public static Sector read(ByteBuffer buf) {
    	// Read header
    	int file = (buf.getShort() & 0xffff);
    	int chunk = (buf.getShort() & 0xffff);
    	int id = ByteBufferUtil.getMedium(buf);
    	int type = (buf.get() & 0xff);
    
    	// Read data into temporary buffer
    	byte[] temp = new byte[DATA_SIZE];
    	buf.get(temp);
    
    	// Create and Return
    	return new Sector(id, file, type, chunk, ByteBuffer.wrap(temp));
        }
    
        public ByteBuffer toByteBuffer() {
    	// Allocate buffer
    	ByteBuffer buf = ByteBuffer.allocate(SIZE);
    	
    	// Write header
    	buf.putShort((short) file);
    	buf.putShort((short) chunk);
    	ByteBufferUtil.putMedium(id, buf);
    	buf.put((byte) type);
    	
    	// Write data
    	buf.put(data);
    
    	// Flip and Return
    	return (ByteBuffer) buf.flip();
        }
    Still Working on this! Feel free to ask questions or fix mistakes while I finish this!
    Reply With Quote  
     

  2. Thankful users:


  3. #2  


    Major's Avatar
    Join Date
    Jan 2011
    Posts
    2,997
    Thanks given
    1,293
    Thanks received
    3,556
    Rep Power
    5000
    Seems correct, although it's 'indices' not "indexes".
    Reply With Quote  
     

  4. #3  
    ???

    funkE's Avatar
    Join Date
    Feb 2008
    Posts
    2,612
    Thanks given
    255
    Thanks received
    989
    Rep Power
    1366
    Quote Originally Posted by Major View Post
    Seems correct, although it's 'indices' not "indexes".
    An index of indices.
    .
    Reply With Quote  
     

  5. Thankful user:


  6. #4  
    Registered Member
    Join Date
    Dec 2010
    Posts
    498
    Thanks given
    41
    Thanks received
    84
    Rep Power
    54
    Quote Originally Posted by Major View Post
    Seems correct, although it's 'indices' not "indexes".
    That was an intentional error, I wrote it that way to make it clear it was multiple "Index"s. Also I'll be finishing this up tomorrow, was double checking the differences between old and new revisions as far as Archives and CRCs.
    Reply With Quote  
     

  7. #5  


    Major's Avatar
    Join Date
    Jan 2011
    Posts
    2,997
    Thanks given
    1,293
    Thanks received
    3,556
    Rep Power
    5000
    Quote Originally Posted by H3ll Ruler View Post
    That was an intentional error, I wrote it that way to make it clear it was multiple "Index"s. Also I'll be finishing this up tomorrow, was double checking the differences between old and new revisions as far as Archives and CRCs.
    "Indices" means multiple index...

    Edit: Oh I see what you mean now, multiples of the 'Index' data structure. Should probably just call them 'Indices' as that's what they are, and just add a note below it.
    Reply With Quote  
     

  8. #6  
    q.q


    Join Date
    Dec 2010
    Posts
    6,519
    Thanks given
    1,072
    Thanks received
    3,535
    Rep Power
    4752
    nice explanation
    Reply With Quote  
     


Thread Information
Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)


User Tag List

Similar Threads

  1. Replies: 6
    Last Post: 08-22-2010, 04:28 AM
  2. How do i get rid of the runescape title?
    By redvader in forum Help
    Replies: 2
    Last Post: 11-29-2009, 05:42 AM
  3. Models from the Runescape Cache
    By Outcast in forum Models
    Replies: 53
    Last Post: 07-26-2009, 10:49 PM
  4. Replies: 6
    Last Post: 08-27-2007, 08:58 PM
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •