The idea is simply to distribute all heavily accessed data across all simulators that make up the grid, thus relegating the asset server (and possibly others) to the task of managing this data (ie, deciding which machines get which range of keys), as well as keeping the overall back-up of data.
Assuming keys were three digit numbers (e.g 123), and we had three simulators (over simplified
) it would be done something like:Machine A gets keys starting with a digit between 0 and 5
Machine B gets keys starting with a digit between 3 and 8
Machine C gets keys starting with a digit between 6 and 2
If I didn't make a mistake there you'll notice that each machine overlaps the next, so should a machine fail, then the two 'on either side of it' are able to provide the missing content. There may be even more robust systems though nothing super complex is needed, should two machines overlapping the same content fail then the asset server itself would still be there as a fall-back.
When an asset is added, the asset server routes it to the require machine(s), same with removing an asset.
The advantage of this system is that assets are no longer routed from a single source, so should overall be faster, especially if the capability for assets to stream from two or more machines is added (ie an asset starting with a 6 could be routed from machines B and C above). This distributes load tremendously and makes for a far more scalable system where the responsibility of the asset server is reduced to managing the location of assets, telling simulators when their assigned keys change so that assets can be moved around from simulator to simulator.
The only thought here is that simulators would possibly require additional hard-drives so that assets can be distributed. Potentially they could have fairly small extra drives to use as a cache for their share of assets, infrequently requested assets would still go via the asset server (causing them to become cached for later).
As a further note on performance:
While I say simulator in the above, it doesn't necessarily mean ALL simulators (ie every core of every machine), but could instead be each machine. This is likely more realistic, thus each machine - in addition to running four simulators - would run a light-weight, multi-core aware asset streaming program. This should 99% of the time remove issues of single simulators that are lagging a lot causing its assets to become painfully slow, as it is unlikely that all four simulators running on a given machine are ALL going to be suffering the same amount of lag. And as pointed out, if a machine is lagging too much anyway, then another other machine(s) with its assets could perhaps be used as a fallback if they are performing better.