Larger Cache Sizes?

Could we have an option to increase the cache size? I'd rather give over 10 Gb to Second Life than have to wait for things to load all the time. Many games now-a-days are reaching the 5 - 10 Gb size and it would make sense to give us the option if we want it.

Does the caching system keep my most frequently visited areas in the cache so that they don't have to be re-loaded each time I visit them? If not it should as it would save a hell of a lot of data transfer for LL.

i heard that the cache loose efficiency over 1Gb

It has efficiency?

lol. at least it is better than a few versions ago. To me at least. The textures don't seem to re-download everytime I tp anymore.

Do for me. I was in grey goo land for a while after TPing back to where I was (i.e. second TP, first one took me away) earlier today.

I'm not sure about it loosing efficiency, but it for some reason simply can't handle more than that amount of information, regardless of how many files it actually is (you could have 2 massive files and it won't like it, or a thousand smaller files). Why is an absolute mystery, as even an extremely simple caching model should be incredibly easy to do. e.g you store all the files in the folder under their asset ID. It shouldn't matter how much space they take up.
In a separate file you store a list of these IDs, when opening SL you take all of these and put them into an easy to search structure (such as a tree structure), then whenever you find an asset that needs loading, you quickly look up the tree to see if you have it stored locally, if you do, try to open it locally. If you can't get the file to open, or the asset is not in your 'cache tree' then download it normally.

I've just got an extra hard-drive, giving me a lovely 320gb total space (striped because I often work with ridiculously huge image files), +my external drive for backing up the important stuff.
I'd quite happily devote 20gb to caching if it meant SL would load places I visit frequently a lot faster.

The cache system needs to be changed.

Here is how I would change it.

1. Let you set a base directory
2. Create a directory for the first x chars of the GUID
3. Create a directory for the next x chars of the GUID
4. Name files for the next x chars of the GUID
5. Then use the existing flat file system to encode items into one large file that would be limited to 1GB

There would be a lot of issues in that it would eventually slow down the computer by fragmenting the file system, but there are programs to correct that.

The cache should also store the last update time of each object!

Right now the client has to re-download any object that goes out of the draw distance since it no longer knows if it has updated. This is annoying me as I try to test my 500m long AutoBahn test road. As I move back and forth the road has to re-rez and any packet loss leaves a hole in the road.

I would suggest a communications command that would send just the GUID's and the last updated time to the client. I know the sim maintains a last updated time since it uses it for auto return.

The client should also keep the objects in a separate cache since they are smaller.

This is a foundation piece of SL and I just don't see SL growing if LL does not improve the cache system since there is simply not enough internet bandwidth to constantly redownload the same objects.

Edit:
Another thought would be to break up the sim into 64m cubes and have the client send in requests like “Send me any object that has updated since time x in cube x,y,z”

The asset server is a folder system.

The folder solution is actually just adding indexing to the search. It allows for limiting but as the number of levels increase in your index, the gains created by indexing diminish. Not to mention you must keep your tables sorted and keeping the table sorted is time consuming.

Another problem is that modern file systems have large blocks sizes. NTFSs default block size is 4kib. Most assets don't fall nicely on 4kib boundaries. Not to mention that NTFS keeps quite a bit of meta-data. By having the cache in a single file, LL can pack the assets to any block size and only have one instance of file system meta-data. While the file system might be capable of doing a better job at indexing the data; SL's current cache can store it more efficiantly.

On another note, the SL cache is limited to 4GB in size, as it uses 32-bit addresses & sizes. Because LL uses 32-bit addresses they most certainly have a smaller file system table (your 320 gib hd uses 48 bit addressing).

There are many ways around that 4GB limit, and that limit is just for a single file.

One possibility today would be to make a cache server. A proxy would sit in between the SL client and the SL server. The proxy detects the requests for textures, and it redirects them to a local cache server. The cache server could be a SQL server that stores the data.

This would make it possible to store the textures on a local server instead of under the SL client cache.

The SL client cache is pretty quick when it is set to around 200MB. The local cache server could be set to hold several gigabytes of textures.

I would only implement it if I could easily protect the images stored in the local cache server, so it hasn't be done.

Cache server:
Where is it physically? LL? If so it already exists: The Asset Server. My house? If so, then I need another computer that is physically between my PC and the internet.

As I understand it the Sim does some local caching of asset and image data, but it is limited.

As I see it we are talking about caching 100's of GB of data and I agree that SL needs some sort of large cache file, but I think the files should be limited to 1 to 2 GB and the OS should be used past that point.

The bigger question is knowing when an object needs to be re-downloaded, since unlike textures and other items, objects do NOT get a new UUID on each update. This means that the object cache should be separate since it is different in that it has to use update times to decide when to re-request it from the sim. The sim of coarse would still re-send it automatically when it's updated but only to clients that are in the area.

Edit:
The client also needs to be able to ask “Give me a list of all UUIDS that still exist in this area”
The client would then use this list to trim the object cache list in the background to eliminate ghosting.

With a local cache server, I have two visionary suggestions.

First, if a business wanted to use SL for its virtual presence, it would not want to have to download textures to each client seperately from the the main servers. A business could set up a proxy to help prevent the extra bandwidth. The proxy would redirect commonly used textures to the local server.

Second, local cache servers could be enabled to share textures by P2P. This would be pratical if the textures are encoded to protect itellectual property.

I think P2P is a good idea for sharing textures between the clients and reducing the load on the LL servers. However some experimentation will be needed first to find out if people with slower connections slow down file transfers too much as this would have a detrimental effect.

Maybe storing the cache in a database would be a good idea, i.e. one that doesn't require extra software to be running such as MySQL.

If it was just a local cache with no database features beyond an index lookup, bittorrent style of P2P would work well. However, if database servers are setup, an active network of the servers, each being able to communicate with each other, would work well.

A local cache server that uses MySQL would be easiest to implement for the first attempt as a proof-of-concept.

Why is a simple folder structure not an option? I mean all it has to do is store a file, e.g, my texture's UUID is 1234567890, so the file is stored as:
1234567890.jpg in my cache folder, to see if the file is cached just look for the file, if it is there then load it, if not then download it. Since assets get a new UUID when updated, modify/creation date are not needed. For objects you would additionally check the modify/creation date with the modify date from the simulator.
Obviously objects wouldn't be stored in plain formats, but the idea is simple. There is some memory waste due to file-systems, but really with today's drives it is irrelevant, what we want is maximum cache size. Looking for a file is not an expensive task, especially if it means you avoid having to download the file from the simulator.

Then just give us options like maximum cache size and/or cache date. e.g, I could set my cache to 2gb on a smaller drive to keep it from filling my drive, or I could set to "older than a week" so that any asset I haven't viewed for a week is deleted.
This can be done by keeping an SQLite database that allows for fast searching of the files to determine which ones have to go.

The sooner someone develops a file system that’s essentially an extensible database the better, most people don't have MySQL installed but for those of us that do it would be nice to be able to use it for SL.

Well, the main issue is sorting through the files, not in having the files in the first place I think. This wouldn't be so much of an issue if SL used multi-threading better, since you could just create a separate thread when SL opens which looks through the cache for things to prune, and can be triggered whenever the cache gets too large (for those that don't care you can set a huge size).

Things in SL really don't change as often as LL thinks. If the client stored everything a person sees for the last month the amount of data transferred would be decreased a lot as people generally visit the same places most of the time.

Right now I am seeing that the problem is not the amount of data to be stored, but there seems to be no mechanism in SL where the client can find out what objects in the sim have updated since its last visit. The result is that you are constantly re-downloading the same objects.

Images on the other hand can be cached as they do not change without the ID# changing.

This is very apparent when trying to drive as the road objects take longer to load than the images and thus you get the road prims appearing fully textured.

--
The objects and images can’t just be stored in separate files since there is quite a bit of overhead for each file. Likewise, you can’t store everything into one large file.

Balance...

Just an FYI, the objects are not cached in the disk cache. Objects are cached by sim in seperate files. Object definitions are really small; only a few hundred bytes per object. So the average sim cache file is less then 1mb; though sometimes they get as large as 2mb. Objects are only downloaded as needed, so each sim's object cache grows and shrinks (which can lead to disk fragmentation).

Objects cannot be reliably cached by a proxy, they are too dynamic. Since they are so small, it's a non-issue.

LL does a pretty good job dealing with ghosting. It's a lot better then it use to be (i haven't had an object ghost in a long time).

I had one (two actually--in the same location) ghost the other week. It ghosted for me and one other person who was there at the time. An hour later he found it (where it was supposed to be) and tped me again so I could move it (what I'd been trying to do the first time).

Dynamism of tiny files is only an issue for people who can't think of a better way to organize the cached data for high-speed access. Yes, if you put all textures and objects in one big pile and dig through the whole pile every time you need something, it is going to be slow.

But a massive haystack is not the only way to store needles.

How many sims are there? Make a folder for each sim in the entire world, and put the objects for each sim in its respective folder. When a person teleports to a new location, the cache checks where they are going and preloads the huge mass of data for the sim into a RAM-disk for superfast loading.

The number of sims is huge but the number of sims any one person actually visits is small, so there could be thousands of sims in a few years but each person's cache would only end up holding data for perhaps 50 sims at most.

Sorting cache data by sim will lead to some duplication of objects, sounds, and textures across the various sim folders, but with the benefit of speed. And besides, both NTFS and linux support something known as hard-links where multiple file names in different locations can represent just one block of data, so the duplication of object file names across multiple sim folders does NOT necessarily mean more disk space used.


This could go further yet to reduce cache churn for people with a short view distance. Within each sim folder in the cache, subdivide the sim into 16 equal-sized parcels and cache the data per parcel, so that it loads the data in your immediate vicinity first.


So, cache optimization might work like this:

On entry to a new sim for the first time, the cache creates a folder for the sim, and a folder for th 64x64 patch you appear in, and writes all objects to the cache folder. For any other nearby sims and 64x64 patches within your view range, additional cache folders are created.

A separate idle process cleans up the cache to reduce data usage, either when you are offline or when the client detects you are AFK. A separate massive haystack list of all cached objects is kept, which tracks hard-links in the overall cache.

The newly cached sim folder is checked for objects already listed in the haystack, and the individual data files are deleted and replaced with hard-links to the same data listed in the haystack. New objects not listed are added to the haystack so they too can be used for hard-link generation later, and it finishes with a drive optimization to eliminate cache fragmentation.

Any existing sim/patch cache folder which has objects that have changed while using SL is marked as "dirty" when the new objects appear and written as individual files to the folder. Then during the idle process the dirty folders are scanned, the new objects either hard-linked or added to the haystack, and then the folder is marked "clean" again.


This would result in:
- superfast load times when appearing somewhere
- efficient object finding due to object name duplication in folders
- very small cache size from using object hard linking

Downside:
- needs offline or idle time to clean up and hard link the cache

The cache is already partially by sim. Most of the keys used to access objects are temporary per sim. It appears textures and agents keep the same key across sims. Any temporary key is obviously cached by sim.

The major slowdown of the cache is the processor usage of the client itself. I've ran tests and while the client runs there is a major impact on task switches and disk access. It even brings virtualization to a crawl (i.e. being able to run Linux and Windows on the same machine where each has psuedo-control of the hardware at seperate times).

The first obvious solution would be to move the cache off the processor the client runs on. This is where the proxy is involved.

I would expect to be able to keep the cache very small on the client and a large chache on the server where the proxy runs.

The solution sounds easy, but there is more to it than that - security for one.

30,000 objects * each object size = a lot of data to re-download every time I visit a sim.

Not everyone has a T3 internet connection.

Adding a method to cache non changing objects will be important in the long run as SL has a very nasty problem when you try to fly or drive. I really hate flying into a building and have it load all around me with me stuck inside of it.

Now, if LL does not cache objects, then maybe lib second life could. The only problem is that as I understand it there currently is not a good command to ask the sim which objects have changed.

I would rather see ghosts that disappear, than to see empty space that suddenly gets filled with objects.

Also remember that a lot of people use the same textures, so it would be more space efficient to have all the textures in one cache system

So…
1. Cache objects per sim, but implement "Get complete sim object list and last change time" request packet to remove ghosts.
2. Cache textures in a directory/file system with up to 2GB of textures per file.
3. Use a background process to update indexes and to limit the files from getting too big.
I would start splitting files when they reach 2gb since you do NOT want to hit the 4GB hard limit.

Edit: A proxy could cache the objects without the command by having a "updated" flag for each object in the cache. Upon entering the sim, the proxy would send all the objects to the client and then request them from the sim. After a delay it would then remove any object the sim does not send. This would improve caching, but would not reduce network traffic as the proxy would have to get the entire object and compare it to the one in it’s cache.

One of the things that I thought of re: common textures was to have all library textures kept on the client all the time (ie bundled with the program), and additional "texture packs" could be made available that you can drop-in as well and which then stay permanently cached, these would be compiled every so-often by looking at which textures are requested most frequently.

Another important one for say roads would be the ability to create "standard" objects. ie when you place an object it has not only its own key for the simulator, but also the UUID it was created from. If the object is changed in any significant way by the user or by script then the UUID is deleted.
What this allows is a "load it once" system, you load a road-piece once and you've loaded all road pieces of that type, the client only has to request the position and rotation information for the whole object. Or if the client doesn't handle it the simulator can say "Oh I've already sent them an object with UUID X, I'll just send the position and rotation for all others of that type". It will depend on how the client handles it all I guess, for example if it does:
- Connect to sim, request visible objects in range
- Object A received, got position, rotation and UUID
- Haven't received object with this UUID before
- - Request rest of info
This would allow builds that use standard building blocks to load quickly, especially if those blocks are linked sets, since you'd receive the whole linked set once, then afterwards just the position and rotation of any others of that type.
To avoid that kind of system above slowing down one-of-a-kind objects, it could maybe be a flag you check ie "Use UUID info" which is unchecked by default in an object's properties, so you'd set it in your inventory, then drag the objects out, optimising your builds as a result. Would allow you to build massive towers that take very little time to load at all, or massive grid-wide roads that load faster than the environments around them as you hurtle along on your motorbike/car/jet-skates