Could LL adapt this logic to fight grid attacks?

OK, we are in the process of having a new building built for the M2 on our property in Sido. And today is the day when the old building was demolished to make way for the new M2 building.

So when the demolition took place, Katt returned the remaining objects that were on the property to their respective owners, and a large number of them were mine, because the old building was mine, etc.

Well, here's the kicker: I got the typical (and expected) string of IMs in my e-mail box saying that "Object '(name)' was returned to your inventory from Sido...' then I got something a bit different.

The next message I got read as follows:

The object 'Second Life' has sent you a message from Second Life: "Objects in Sido are being returned to you at an excessive rate. The IM notifications from this region to you are temporarily deactivated. Please check your inventory to view the returned objects."

OK, so I get that the system basically said, "Look, you're getting a crapload of stuff returned to you, for whatever reason. We're not going to tell you about every little brick and screw and shingle; just know that there's more than what we've already notified you about."

So what I am wondering is this:

Could there be a way to use similar logic to tell the system, "If more than xx number of objects are rezzed in xx time, intervene and stop the process"?

I'm not really talking about anything that would affect an ordinary sort of rezzing of objects; the reports on the last grid attack (the herpes balls) indicated that they were spawning at 200-300 per second! It would be very unlikely that that sort of thing would ever occur under normal circumstances. If I rez a huge house from my inventory, maybe it would rez that many prims in a second, but only once. Maybe the routine could say something like "If more than (example: 100) prims are rezzed in a sim for more than (3? 5? 10?) seconds running, activate no-build." If someone has a legitimate project that this interferes with, a Liason could always be called to turn it off temporarily.

Is this something that's at all feasible?

P2

Sounds good ta me!

Toodle-oo!

I had heard comments that they had already enabled throttling of rezzing, which is perhaps a slightly better solution than this, but anyway...

The problem with this, is that what looks like 200-300 prims/second to you as an attack, looks to someone else like a rez-foo of their new mansion. Or to someone else, a rezzing of their 'super, new, exciting' build in the sandbox, which they took to inventory last time they worked on it. Heck, a 100 prim limit will block a *lot* of pre-fabs from rezzing in one go.

Slowing down the rez-rate *IS* a better solution than just blanket blocking rezzing. Ok, eventually, the two will start to converge, but probably not until you're very close to trying to rez the entire sim's limit at once. A legitimate build will rez all its prims, and stop, an attack will continue to try and rez more and more, whether it is instant or not for the legimitate build is probably not that critical - and if you're rezzing several hundred prims at once, you're probably going to a see a lag spike anyway

from your description Phoenix,it sounds as if the only thing that the "system" is interfering with is the delivery of IMs. Any events that are occurring on the grid are happening regardless of "system's" interference.

Rather than limit, nerf, throttle, etc any LSL functions, I think LL needs to develop better back-end tools that allow them to refuse "service" to a creator's objects or an owner's objects (not my idea, it was in another thread..if I am misrepresenting it please correct me). I haveno idea how feasible that is, but it would be the most elegant solution I've heard so far.

So I would need Linden assistance everytime I add a forest to a sim? That doesn't sound good to me. And testing scripts ack! I had a sim's upload queue stacked over 7k with returns. The solution to attacks isn't to limit legimate usage it's to give lindens better tools to rapidly combat an attack once reported.

No, because you can still pass objects to other objects' inventories without actually rezzing them. You could still DOS the assett server, even without actually rezzing objects.

Only if rezing a forest involves sustained high-level rez-rates on the order of, say, 300+ objects per second for, say, 60+ seconds, if Ah read Phoenix`s post correctly. [Phoenix, did Ah?]Does adding a forest present that signature? Can anyone present that signature manually? Really, if ya can, Ah`d like ta know how!

Toodle-oo!

Edit: My numbers are different from his, but that is a matter of tweaking parameters, not a matter of method. I just happen to think that these numbers are less likely to accidentally grief innocent builders while still putting the breaks on run-away most self-replication.

P2, I believe one of the the problems with a grid attack is also the amounth of objects that are being returned and send a IM that they where. This might actually be already a fix to make the grid more resilient to grid attacks.

It should be mentioned that throttling / stopping rez *Prims* is not the way to go, but rather *objects*. So you can still rez a full-featured mansion with 500+ prims at once, the object count will be much lower. Attachments must be an exception to the rule as well. Additionally, the owner of the land / sim should not be afected by rez throttle in any way, IMO.

Just adding my thoughts here, in case the idea gets popular

That still doesn't solve the main issue though.

Sure, you can stop one person from flooding the sim with crap. What about 20 newbies? You can't do a global limit, what if it just happens that 20 people in a sandbox decide to start rezzing at once? And it's perfectly possible after a sandbox deletion.

IMO, a good solution would involve the ability to build a procedence tree, which would track the origin of every object. The griefer must originally create a prim, then replicate it lots of times somehow (passing to newbies for instance).

So, make it so that every object can be tracked back to the parent that created it. Every time a rez happens, increment the original parent's counter. Every time it's derezzed, decrement it. If it turns out the counter is growing at a high enough rate, send an automated message to a Linden, who would have a button to kill the whole tree with a keypress.

How does that sound?

The 60 seconds is a change fron the orginal suggestion of 3-10 seconds and I absolutely can present that signature for more than the original time suggestion. And do you seriously think I rez my forests MANUALLY?!?!?

It appears, at first, to be a good idea, but this has been gone over already in other threads. Machine guns, Rez-Foo, or simply a sim with multiple busy scripter/builders would be crippled.

A better measurement of sim performance is not how much is being rezzed, but ... sim performance - time dilation comes to mind immediately.

We should have regional monitors that cover clusters of a few dozen sims. Depending on the number of sims effected, the monitor could take appropriate action, ranging from turning off scripts automatically to shutting down the entire region as a grid-saving action.

Oh. How about multiple objects and an increasing burden on the server could trigger an alarm that called the Linden's attention and maybe even started blocking inter-sim object travel for that sim?

Maybe for an extended period of time, yes. A simple way to do this would be coloring sims orange on the map that can be turned on/off from map options, so that lindens could just see the map and problems.


I'm not aware of how the Lindens do the handoffs now; that may or may not require cooperation from the other sim(s) - and leans toward the regional monitor implementation.

Why wait for a Linden? Let any landowner turn this on or off on their own parcel! This is the first rez-limiting idea that seems to me to have potential without messing up legitimate scripts.

[X] Limit third-party object creation to [100] prims per second.
    [X] Disable third-party build for [30] minutes after limit reached.

There's no reason to limit the landowner.

There's no reason the landowner can't change it.

The default could be (say) 10% of the parcel's prim limit.

...but can y`all present the revised signature Ah proposed?Oops; Ah hadn`t thought o` that.

Toodle-oo!

Good, Darlin`. Hows about a hybrid scheme?

1. Use Phoenix`s idea, wi` ma parameters, wi` seperate counters for every resident rezzin` stuff in th` sim.
   
2. Use Phoenix`s idea, wi` even looser parameters ta maintain global counters.
   
3. If, and only if, Item #1 fails to stem the problem, Item #2 kicks in on th` whole sim.
   
4. If, and only if, Item #3, fails to stem the problem, globaly inhibit rezin` in th` whole sim.
   
5. When th` sim can handle returnin` ta normal operations, it does so automaticaly.
   
6. Both individual an` global counters would run up for rezzez an` could run down for de-rezzes.
   
7. All counters wound be based on objects, not prims!
   
8. All actions taken to reduce rezin` would also pop up a warnin` dialog on an appropriate Linden system console wi` an appropriate message an` th` options ta:
   1. Revert to any o` th` lesser restrictions [includin` normal operations] immediately.
      
   2. Advance to any o` th` greater restrictions [includin` preventin` imigration from outside th` sim, shuttin` th` sim down, an` restartin` it].
      
   3. Kill th` whole tree as described by Dale.
      
   
   
9. All affected users would get a warnin` pop-up explainin` what level of rez-stoppage was in force.
   

Whatd`y`all thinks? Is this bettah or worse than precedin` ideahs, or `bout th` same?

Toodle-oo!

Oh, I know that. I was just saying that if the system has the ability to recognize one type of action and take an action (notifying me about the IM cap), perhaps the same logic could also apply to the other situation.

P2

Stop wonderin`, Darlin; It can. It obviously hain`t, but it can.

In case anyone`s wonderin`, us programs, uh,er, programers, we knows thangs like that.

Toodle-oo!

We are thinking about implementing a rez throttle to fight the self-rezzing object problem. Zero Linden grabbed some object creation stats of the entire grid for a few days of mid April and I extracted the numbers and graphed them so we could see what kind of typical object rez rates (not prim rez rates) are happening and compare them to known problem events.

There were several high spikes that rose above those associated with known attacks, however it seems possible to tune a rez throttle to be especially sensitive to self-rezzing objects over machine guns and thereby reduce the collateral damage of such a system. The need for some sort of defensive system seems imperitive, but it is very likely that the many really fast rezzing objects in SL will be adversely affected should such a system be deployed.

Here's an interesting tidbit: It turns out that the top rez event from April 12-17 was some gun (or more likely a pile of guns) on the Teen Grid which churned out almost 800 bullets/sec. Of course, the vast majority of the data is below 20 objects/sec. Although 20 objects/sec seems high, there are (probably legit) bursty events higher than that every minute somewhere in the world.

and given Andrew's response, it looks like you were right on the money about how LL might handle the situation. Good post! I'm glad we got a response.

champie

I'd have sims track object handoffs to other sims and report the rate to a central machine (or cluster of machines) once a minute. If the handoff rate in a geographical area got really high (some # of standard deviations away from some baseline mean), that central machine could command the sims to tell it who the top five or so owners of these objects were. If the amount of handoffs for objects belonging to a certain resident were more than some number of standard deviations away from an acceptable mean, all sims within a certain radius could be commanded to "bounce" those objects rather than handing them over to the sim they were headed for, exactly the same as what happens to your av if you try to fly over a border where there isn't a sim to receive you.

This has the following benefits:

1. Throttling rezzing would slow the attack, but not stop it - you would eventually have gray goo all over the main grid. It might take a day rather than an hour, but you'd still have to clean it up, and the collapse would only be slower and more annoying. My idea would go about it differently, limiting the infection to a geographical area.
   
2. With this method, the infection would be contained very quickly.
   
3. The objects are contained rather than being returned, so as soon as the sim's parcels are all full, rezzing simply fails. You get a few sims in one geographic area that are rendered useless until LL can clean them up - and this can be triggered, since such a system would be proactive rather than reactive - instead the entire grid getting hosed.
   
4. Bouncing them rather than returning them means the asset server isn't hammered, which would be even worse since returning them creates more available slots for them to be rezzed, etc.

And the following disadvantages:

1. Baselines would have to be carefully calculated and updated from time to time to account for changing behaviors, although if the handoff rate is significantly higher during gray goo attacks, it shouldn't be too hard
2. Sims would have to keep track of owners of objects being handed off, which would require some memory and CPU time
3. Higher sensitivity to deliberately slow attacks would require more clever logic on the "watcher"
4. There would need to be a central machine or cluster of machines that watched this activity, and it would take time and money to develop it

Some interesting ideas have been presented here. Here are some more of my thoughts on the matter...

I don't like the idea of a central global throttle. The work required to implement a scalable system that can watch the entire grid scares me. I think the general consensus in the LL developer pool is that we don't want any more central servers.

It would be nice if we could nip the problem in the simulator before it becomes a global issue. A really slow virus that gradually permeates the grid without killing the sims as it goes is a different sort of problem. It is very likely that someone will notice it while they analyze the objects on their land and then complain to a Liaison at which point LL can deal with it.

Should an object hit the throttle threshold, it shouldn't be too difficult to thereafter prevent all subsequent 'suspicious' operations by that object and its army. That is, the throttle needn't be soft, and it could be given a non-trivial memory. It's region crossings could also be disabled on a local level, and neighbor simulators could exchange information about things that should be blocked.

Neither do I.Agreed.My proposal will do this.Agreed.Agreed.Cool.

Toodle-oo!

I'd rez one or two per sim and program them to start replicating exponentially at a set time