Welcome to the Second Life Forums Archive

These forums are CLOSED. Please visit the new forums HERE

did you read the blog recently...

Yosef Okelly
Mostly Harmless
Join date: 26 Aug 2007
Posts: 2,692
04-02-2008 14:36
From: poopmaster Oh
The problem is a combanation of two things 1) Massivly Huge Database 2) Poop


when they mix, well you know what happens....

The Massivly Huge Database hits the fan?
Kitty Barnett
Registered User
Join date: 10 May 2006
Posts: 5,586
04-02-2008 15:38
From: Keira Wells
I still wouldn't re-up. Instead of premiums, I think perhaps payment USED persons(Maybe a min amount too)?
That was actually the criteria they talked about that time: any resident that had a transaction with LL at one point or another.

That would include premium and someone who had bought L$ at one point or another (which L$ sellers have to do anyway too to get their LindeX clock ticking). It does exclude the "payment info on file" ones though.
Gabriele Graves
Always and Forever, FULL
Join date: 23 Apr 2007
Posts: 6,205
04-02-2008 16:45
From: Colette Meiji
Hard to know for sure. Force the Traffic and camperbots OFF the grid for a couple days and see what happens.


How do you do that? Only by painstakingly finding the Bot groups can you do that with any certainty. If you restrict logins to only Premium and Payment Info Used then a lot of people who fall into the following categories will be excluded too and may not come back: -

1. New people (as more people leave at one end of the scale we need fresh people). The new people of today contain the new Premium and customers of in-world business of tomorrow.

2. People who creative in-world and live completely off that creativity - spending their earned L$ in-world (I am not talking camper - more builders, artists and scripters).

3. People who for some reason and no fault of their own still have NPIOF status despite having given that info *and* bought currency to spend in-world.

I was NPIOF for over 3 months whilst I decided if I would stay - then I had PIOF and spent thousands of L$ over then next 6 months. I have only been a premium member for the last 3 and then only to own land on the mainland really - otherwise I would not have bothered.

If my access to the grid was severed because I had been lumped in with Bots - I might have left very upset and never have spent the money I have done in here - that reasoning has to be factored in to these things also.

So I don't think that is a good idea - maybe pushing Premium people to the head of a login queue is a reasonable way of distinguishing Premium accounts but cutting off access deliberately for some of these people in my opinion would be a mistake.
_____________________

Trout Rating: I'm giving you an 8.2 on the Troutchter Earth-Movement Slut Scale. You are an amazing, enchanting woman, and, when the situation calls for it, a slut of the very best sort. Congratulations and shame on you!
Colette Meiji
Registered User
Join date: 25 Mar 2005
Posts: 15,556
04-02-2008 16:47
From: Gabriele Graves
How do you do that? Only by painstakingly finding the Bot groups can you do that with any certainty. If you restrict logins to only Premium and Payment Info Used then a lot of people who fall into the following categories will be excluded too and may not come back: -


LOL its easier than you may think.

Just go to the highest traffic places on private ISlands and the Mainland and start /kick ing people
Gabriele Graves
Always and Forever, FULL
Join date: 23 Apr 2007
Posts: 6,205
04-02-2008 16:51
From: Colette Meiji
LOL its easier than you may think.

Just go to the highest traffic places on private ISlands and the Mainland and start /kick ing people


Ahh now that I can buy into - good point :)
_____________________

Trout Rating: I'm giving you an 8.2 on the Troutchter Earth-Movement Slut Scale. You are an amazing, enchanting woman, and, when the situation calls for it, a slut of the very best sort. Congratulations and shame on you!
Sling Trebuchet
Deleted User
Join date: 20 Jan 2007
Posts: 4,548
04-02-2008 16:53
From: Gabriele Graves
......

So I don't think that is a good idea - maybe pushing Premium people to the head of a login queue is a reasonable way of distinguishing Premium accounts but cutting off access deliberately for some of these people in my opinion would be a mistake.


Cutting people off is a bad move allright.

So is allowing everyone in and subjecting them to a broken SL.

There's no easy answer, but I think that continuing severe problems would be the worst state. It's destroying confidence and trust.

LL do want to push the limits, but I think that they should stick to Wednesdays for that.
Wednesday night is Beta night!!
Gabriele Graves
Always and Forever, FULL
Join date: 23 Apr 2007
Posts: 6,205
04-02-2008 17:06
From observing the past experiences I tend to think these types of failures (transactions, tp failures) are more to do with concurrency, failure of outside services that LL depend on, and/or hardware failures. I dont think that any of these are SL server software problems as such. So I find it hard to think how they might manage to keep these things to a Wednesday.

Unless you mean the Havok4 rollout? I would hope that has nothing to do with transactions and teleports. I guess it is possible that this rolling upgrade is stressing the servers and impairing their normal services but that should end when the upgrade is complete. I guess if they could have kept that just to a single day - they would have.

Didn't the current raft of problems start before the rollout though?
_____________________

Trout Rating: I'm giving you an 8.2 on the Troutchter Earth-Movement Slut Scale. You are an amazing, enchanting woman, and, when the situation calls for it, a slut of the very best sort. Congratulations and shame on you!
Argos Hawks
Eclectically Esoteric
Join date: 24 Jan 2007
Posts: 1,037
04-02-2008 17:09
It seems like the concurrency numbers grow in cycles. We get to a level where these problems are very common for a while at a certain concurrency level, and then when LL gets it worked out, the numbers shoot up for a while then level off again. Right now the level is between 55k and 65k. My guess is that this will be worked out over the next week or so, and once it's all actually fixed, concurrency will be on a steep rise up to 90-100K where it will level off, and we'll see another batch of trouble between 110-130k. If we start a forum pool for the first day of 100k concurrency, I pick June 30.
_____________________
Step 1: Create virtual world
Step 2: ???
Step 3: Profit
Nika Talaj
now you see her ...
Join date: 2 Jan 2007
Posts: 5,449
04-02-2008 17:15
So right now, we have on the blog, this:
From: someone
UPDATE 4:09 p.m. PST - kate] In world services are mostly available. There are still intermittent reports of stale transactions.
Aaaand this:

From: someone
The Dawning of a New Viewer: Second Life 1.19.1 Now Available!

Friends. Romans. Residents. After months in First Look and Release Candidate phases, we are pleased to announce that the Second Life 1.19.1 Viewer, featuring WindLight atmospheric lighting and gobs of other stability, performance, and feature enhancements, is now available for download! And so….
...
But what goodness does it contain?…
Generally, I don't feel this way ... but today? It's just insult to injury. Particularly as I know the shiny new viewer has more memory leaks than my creaky old Nicholaz.

Shiny. Bah.
.
HoneyBear Lilliehook
Owner, The Mall at Cherry
Join date: 18 Jun 2007
Posts: 4,500
04-02-2008 17:38
From: Sling Trebuchet
Premiums get priority, freebies join the login queue.


I'd prefer to see those with payment info on file getting priority.
_____________________
Virtual Freebies now has its own domain!
URL=http://virtualfreebiesblog.com

The Mall at Cherry Park - new vendors, new look!
Gabriele Graves
Always and Forever, FULL
Join date: 23 Apr 2007
Posts: 6,205
04-02-2008 17:42
From: HoneyBear Lilliehook
I'd prefer to see those with payment info on file getting priority.


So would I - give premiums more tier allowance for the money instead.
_____________________

Trout Rating: I'm giving you an 8.2 on the Troutchter Earth-Movement Slut Scale. You are an amazing, enchanting woman, and, when the situation calls for it, a slut of the very best sort. Congratulations and shame on you!
Nika Talaj
now you see her ...
Join date: 2 Jan 2007
Posts: 5,449
04-02-2008 17:56
From: HoneyBear Lilliehook
I'd prefer to see those with payment info on file getting priority.
If I were Philip and Robin, I think I'd stonewall on the question of preferential access to different classes of customers. I'm cynical, I guess, but deep in my pea brain I think I know how that discussion would go in Philip's staff meeting.

Robin: I have a proposal my people have worked up for tiered access based on account type. *presents something like what has been discussed on this thread*

CFO (chief financial officer): That's not the way to do it.

Philip: Oh?

CFO: Right. Preferential access goes to major accounts ... that's any Fortune 5000 corporate account, significantly sized nonprofits, strategic accounts (ESC, MOU), and individual accounts that have more than x sims. Everyone associated with those accounts, including their bots, get access. Then premiums. Then paid info on file. Then basics.

Philip & Robin: *Exchange unhappy looks*

CFO: Let me call Bill Gurley (board member) and get his thoughts.

Philip's vision for SL is very egalitarian, very net-neutral. I'm not sure how fond he would be of ANY preferential access.
.
Winter Ventura
Eclectic Randomness
Join date: 18 Jul 2006
Posts: 2,579
04-02-2008 18:07
From: Nika Talaj
Philip's vision for SL is very egalitarian, very net-neutral. I'm not sure how fond he would be of ANY preferential access.
.


Apparrently you haven't read the Blog recently...

http://blog.secondlife.com/2008/03/14/changing-my-job/

Right before all that "legal eagle" stuff started spamming the blog with "LL is a major corporation, not a community, and we will SUE you".. Phillip announced he was officially "Stepping down" as CEO.

I would not be surprised to learn (though I have heard nothing to suggest) that a meeting went down where Phillip said "if you do this, I will resign as CEO"... and they said "fine".

I'm not convinced that Phillip's "fondness" (or lack thereof) of any course of action carries much weight.
_____________________

● Inworld Store: http://slurl.eclectic-randomness.com
● Website: http://www.eclectic-randomness.com
● Twitter: @WinterVentura
Nika Talaj
now you see her ...
Join date: 2 Jan 2007
Posts: 5,449
04-02-2008 18:15
From: Winter Ventura
I would not be surprised to learn that a meeting went down where Phillip said "if you do this, I will resign as CEO"... and they said "fine".
Interesting thought. It could have played into the decision, (board seeing Philip as obstructionist to success with corporations), but I think the major factor there was his clear inability to grow the company beyond 250 people (small-company management style) and the grid beyond 55,000 concurrency (less than firm engineering management).

The VP of Technical Operations position (https://home.eease.com/recruit2/?id=30064) has been on LL's site for months now, and Philip has not succeeded in filling it. It's a tough job, requiring both engineering and operations leadership. The new CEO will bring someone in to fill it, or possibly split it into two separate jobs. At that point a decision may have to be made as to whether LL will put most of its mental energy into being a service provider (SL) or being a platform supplier. Interesting times coming.

More than one factor played into his decision, I'm sure.

.
Cocoanut Koala
Coco's Cottages
Join date: 7 Feb 2005
Posts: 7,903
04-02-2008 18:16
From: Nika Talaj
If I were Philip and Robin, I think I'd stonewall on the question of preferential access to different classes of customers. I'm cynical, I guess, but deep in my pea brain I think I know how that discussion would go in Philip's staff meeting.

Robin: I have a proposal my people have worked up for tiered access based on account type. *presents something like what has been discussed on this thread*

CFO (chief financial officer): That's not the way to do it.

Philip: Oh?

CFO: Right. Preferential access goes to major accounts ... that's any Fortune 5000 corporate account, significantly sized nonprofits, strategic accounts (ESC, MOU), and individual accounts that have more than x sims. Everyone associated with those accounts, including their bots, get access. Then premiums. Then paid info on file. Then basics.

Philip & Robin: *Exchange unhappy looks*

CFO: Let me call Bill Gurley (board member) and get his thoughts.

Philip's vision for SL is very egalitarian, very net-neutral. I'm not sure how fond he would be of ANY preferential access.
.

This throttling thing was LL's idea in the first place, though.

coco

Edited to add, after reading the other posts:

OK, then they don't do anything about throttling, since it isn't 100% wonderful for everybody.

And they don't do anything about bots either.

Fine. That is ALL FINE.

Of course, eventually you won't have much of anybody left, because the grid is so borked nobody can buy stuff reliably anyway.

Either way - I don't really care which way it goes.

Unless they come up with some other way to fix this, and assuming high concurrency is a primary culprit, then you have the option of making some kind of limitations somewhere, or just letting it all go to hell in a handbasket, just because people don't want any limitations.

I had an analogy for this a long time ago: Imagine a restaurant where anyone can come in and eat for free. If you pay, you can be seated at a table - that's the only difference.

Pretty soon everybody and his dog has heard of this restaurant and has jammed inside to eat for free.

There are so many people, sometimes you can't even get TO your table. Service is laughable, and you pay, but may or may not get your food.

Just how long is that going to last? Well, probably until the owners finally run out of money providing free (if unpredictable) food to the masses, because the ones who have been subsidizing it all by paying for tables have left for lack of service.

coco
_____________________
VALENTINE BOUTIQUE
at Coco's Cottages

http://slurl.com/secondlife/Rosieri/85/166/87
Gabriele Graves
Always and Forever, FULL
Join date: 23 Apr 2007
Posts: 6,205
04-02-2008 18:38
I think the flaw in your thinking coco is that the problems are caused by people who are eating for free. I very much think that the bot problem is caused by land owners - yes premium people or island owners. Most "free lunchers" are using exactly one account at a time for the most part I bet.
_____________________

Trout Rating: I'm giving you an 8.2 on the Troutchter Earth-Movement Slut Scale. You are an amazing, enchanting woman, and, when the situation calls for it, a slut of the very best sort. Congratulations and shame on you!
Argos Hawks
Eclectically Esoteric
Join date: 24 Jan 2007
Posts: 1,037
04-02-2008 20:20
From: Cocoanut Koala
I had an analogy for this a long time ago: Imagine a restaurant where anyone can come in and eat for free. If you pay, you can be seated at a table - that's the only difference.

Pretty soon everybody and his dog has heard of this restaurant and has jammed inside to eat for free.

There are so many people, sometimes you can't even get TO your table. Service is laughable, and you pay, but may or may not get your food.

Just how long is that going to last? Well, probably until the owners finally run out of money providing free (if unpredictable) food to the masses, because the ones who have been subsidizing it all by paying for tables have left for lack of service.


But in the restaurant that LL is running, all the food($L) that you get is because either you paid for it or someone else gave it to you. All the food is paid for by someone, just not necessarily the people that you see eating.
_____________________
Step 1: Create virtual world
Step 2: ???
Step 3: Profit
Sling Trebuchet
Deleted User
Join date: 20 Jan 2007
Posts: 4,548
Did you read the blog a year ago ...
04-03-2008 05:17
From: Gabriele Graves
From observing the past experiences I tend to think these types of failures (transactions, tp failures) are more to do with concurrency, failure of outside services that LL depend on, and/or hardware failures. I dont think that any of these are SL server software problems as such. So I find it hard to think how they might manage to keep these things to a Wednesday.

Unless you mean the Havok4 rollout? I would hope that has nothing to do with transactions and teleports. I guess it is possible that this rolling upgrade is stressing the servers and impairing their normal services but that should end when the upgrade is complete. I guess if they could have kept that just to a single day - they would have.

Didn't the current raft of problems start before the rollout though?



What I meant about Wednesdays is that it (for example) would be the day on which no login throttling occurred.
Concurrency could then climb without restriction, and we'd see SL break at whatever point.

Yes. The problems of high concurrency pre-dated Havok 4. New viewers etc.


Throttling logins was LL's proposal in the blog way back in early 2007.

From: http://blog.secondlife.com/2007/02/16/contingency-measures-to-ensure-service-as-second-life-grows/


Contingency Measures to Ensure Service as Second Life Grows
Friday, February 16th, 2007 at 3:04 PM by: Robin Linden

Since September concurrency rates have tripled, to a peak last week of over 34,000. While we love that so many people are enjoying Second Life, there have been some challenging moments in keeping up with the growth, resulting in the now somewhat infamous message “heavy load on the database”. When this happens it usually means that the demand for transmission of data between servers is outstripping the ability of the network to support it.

When the Grid is under stress, resulting in content loss and a generally poor experience, we would like to have an option less disruptive than bringing the whole Grid down. So we’ve developed a contingency plan to manage log-ins to the Grid when, in our judgment, the risk of content loss begins to outweigh the value of higher concurrency. Looking at the concurrency levels, it’s clear heaviest use is on the weekends.

When you open your log-in screen and see in the upper right hand corner Grid Status: Restricted, you’ll know that only those Second Life Residents who have transacted with Linden Lab either by being a premium account holder, owning land, or purchasing currency on the LindeX, will be able to log-in. Residents who are in Second Life when this occurs will only be affected if they log-out and want to return before the grid returns to normal status.

At the same time, new account registrations will be closed.

We hope that we won’t have to implement this contingency plan. As you know, we have regularly improved the Grid’s ability to manage higher concurrency rates, and have comfortably doubled our capacity in recent months. If we determine, however, that we do need to limit log-ins, there will be an announcement made in-world and also on the blog.

Scaling the Grid continues to be our highest priority, and over the next several months we’ll be working on the following:
- additional caching systems
- reducing write load on the central database by partitioning or removing data
- addressing critical bottlenecks
- deploying more internal ‘web services’.

Beyond that we are building ambitious plans for re-architecting the Grid, so that in the future we can realize the full potential of the Second Life Grid to support millions of concurrent users.

Screwtape Foulsbane
Registered User
Join date: 30 Dec 2007
Posts: 134
04-03-2008 05:40
LL could always ban camping...

..Ducks the flying produce.
_____________________

Silly & Sane, home of Mr. Pig and the Wearable Chair.
http://slurl.com/secondlife/Teal%20Island/88/210/25
Qie Niangao
Coin-operated
Join date: 24 May 2006
Posts: 7,138
04-03-2008 06:05
Right: Get rid of the Traffic metric as a means of influencing any form of Search ranking, and fake traffic will disappear from the grid. Much simpler than coding a throttle by account type.

But if there's still a problem after Traffic is gone (and there might be, because trafficbots don't do many transactions, and campers--aside from sorting Inventory--are pretty passive, too), throttling NPIOF doesn't seem all that unreasonable. I mean, the accounts are still free; how much hardship can that actually be? Especially in times like these, where LL has admitted that concurrency is the problem:
From: da Blog
Until the servers are in place, it is good practice to check the blog for updates, particularly during peak hours of 12:00 pm. - 4:00 p.m.
And I quite like an automated version of Colette's suggestion: when things get really bad, identify the highest traffic sims and randomly disconnect some clients; those with payment info on file can just log back in. (In theory a disconnect process could know which clients are NPIOF, but if the databases are already stressed, so maybe best not to exacerbate the issue.) It's less disruptive to them than a sim crash, and a *lot* less disruptive than having the whole grid grind to a halt daily.

(All this said, though: there's a system design flaw somewhere in how transactions are queued. It's not like the db can't tell it's under stress; there's no reason for flooding a queue with transactions that will timeout anyway.)
Dekka Raymaker
thinking very hard
Join date: 4 Feb 2007
Posts: 3,898
04-03-2008 06:08
however all said and done, having all those people on line at once indicates to LL that scalability needs improving.
Colette Meiji
Registered User
Join date: 25 Mar 2005
Posts: 15,556
04-03-2008 06:32
From: Qie Niangao
I quite like an automated version of Colette's suggestion: when things get really bad, identify the highest traffic sims and randomly disconnect some clients; those with payment info on file can just log back in.


Automatically Disconnect everyone who is logging on with more than one account per IP in high traffic areas.

Then you are more likely to get the bots.

Kinda like flushing them ...
Marcel Flatley
Sampireun Design
Join date: 29 Jul 2007
Posts: 2,032
04-03-2008 06:47
From: Colette Meiji
Automatically Disconnect everyone who is logging on with more than one account per IP in high traffic areas.

Then you are more likely to get the bots.

Kinda like flushing them ...

That is the last thing they will ever do. About every corporate account will use 1 outside ip number for all employees logging in. And those seem to be the users they want the most :-) Imagine the reaction of IBM if 1 of their employees can login during peak hours...

But it should not be so hard to find a system. My opinion still is Premium first, then payment on file, then the rest. As I stated before, I would even go for a system where no one can be a free member for longer then an evaluation period of 1 to 3 months. But... they want to show numbers, which results in the system crashing daily at the moment.

What I wonder by the way... I have 2 avatars logged in my shop. They use SLeek, and sit in my furniture. Yes, it is a way to get some traffic, and no it probably doesn't make me any better then bot users with 20 bots ;) But I like the fact that they show the poses in my furniture (and yes they do look better then an average newbie), and I like the fact it does generate some traffic.
But what I see, it that they hardly use any network traffic at all. So I highly doubt they put any stress on the system. One avatar moving around, constantly generates network traffic because they have to download textures and more. One avatar building things, puts a load on the asset server. But those 2 avatars who just sit and do their sitting animations, how on earth can they stress the system?

Marcel
_____________________
Colette Meiji
Registered User
Join date: 25 Mar 2005
Posts: 15,556
04-03-2008 06:55
From: Marcel Flatley
That is the last thing they will ever do. About every corporate account will use 1 outside ip number for all employees logging in. And those seem to be the users they want the most :-) Imagine the reaction of IBM if 1 of their employees can login during peak hours...


They could exempt Corporations from the "High traffic Area" criteria I am sure.

Most of them already are excluded from "high traffic" by virtue of being ghost towns.
Colette Meiji
Registered User
Join date: 25 Mar 2005
Posts: 15,556
04-03-2008 06:58
From: Marcel Flatley
But those 2 avatars who just sit and do their sitting animations, how on earth can they stress the system?


That would be a "We dont know"

Some people assume they don't stress the system. But theres really very little people can do to prove that.

However - even if they affect the system only fractionally compared to an "active" avatar, there is no justification for keeping a bot logged in when a Real Person resident cant Log On. None.

SO even if they have to log out 5 bots for every human that can log on, its still the right thing to do.
1 2 3 4 5