Second Life Forums Archive - Is Linden Lab's preview testing process inadequate?

Annah Zamboni

Banannah Annah

Join date: 2 Jun 2004

Posts: 1,022

10-31-2005 14:43

From: Doc Nielsen

At this point explanation is irrelevant.
What they need to do is fix it.
And take steps to ensure it doesn't happen again.
Like it or not SL IS a business, and businesses that repeat errors of this magnitude don't survive in the real world.

Yes its a business, and like a business sometimes your customers want explanations. They want to understand if this was a fluke or if your business model will continue to produce these kinds of 'results'.

Doc Nielsen

Fallen...

Join date: 13 Apr 2005

Posts: 1,059

10-31-2005 14:57

Hmmm, well *I* don't want 'explanations'. I just want my sim and a half or so working again.
If it happens again I'm history. In fact if they don't get it fixed this next update - I'm history. At the moment they are charging me substantial sums of money for a product I can't use!

_____________________

All very well for people to have a sig that exhorts you to 'be the change' - I wonder if it's ever occurred to them that they might be something that needs changing...?

Starax Statosky

Unregistered User

Join date: 23 Dec 2003

Posts: 1,099

Bugged

10-31-2005 15:02

* Shoots myself in the head *

Cristiano Midnight

Evil Snapshot Baron

Join date: 17 May 2003

Posts: 8,616

10-31-2005 15:43

* waves to Starax *

_____________________

Cristiano

ANOmations - huge selection of high quality, low priced animations all $100L or less.

~SLUniverse.com~ SL's oldest and largest community site, featuring Snapzilla image sharing, forums, and much more.

blaze Spinnaker

1/2 Serious

Join date: 12 Aug 2004

Posts: 5,898

10-31-2005 16:11

Nah, LL has more control than you think they do. It's not like they should go out of their way to tell everyone how they keep people from crashing the grid.

That'd be a tad silly.

That being said, they could do more. I suspect most of their techniques are one off and not particularly centralized.

I've proposed a centralized technique they could use, statistical sampling around points of failure .. Kelly said he thought it was an interesting idea. Hopefully that means they might actually do it or something similar.

I think the only real big mistake they're really making is this "big bang" approach to release, which Ian Linden admitted in his town hall that they recognize this is a mistake.

And, lets face it, there is a lot of reason to do a big bang approach. However, I think we all recognize it's just not going to work with SL where regression testing simply is not possible except on the main grid.

Neehai Zapata

Unofficial Parent

Join date: 8 Apr 2004

Posts: 1,970

10-31-2005 18:14

LL should hire me to manage their QA process, then I could get away from this bank.

_____________________

Unofficial moderator and proud dysfunctional parent to over 1000 bastard children.

DogSpot Boxer

vortex thruster

Join date: 23 Aug 2005

Posts: 671

10-31-2005 19:18

From: blaze Spinnaker

They should get rid of the preview grid entirely.

They should learn to rollout a feature to a percentage of the user base, and slowly roll it out to to more as it proves to be solid.

It's a bit more complex, but it's a scaleable and very proven development / qa methodology.

Yeah, the preview grid is a joke. If they really wanted it to be effective, they should have replicated the entire main grid and
let people move stuff between the two grids freely. That way
more people would be inclined to participate in the preview.

Reading this thread, I've come to realize that LL has a real problem with software development discipline and what appears to be little in the way of a cohesive and effective QA process.

_____________________

Dogspot Boxer
Charter Member Of The Socially Inept Club

Our Motto:

We may be inept, but at least we're social

Cristiano Midnight

Evil Snapshot Baron

Join date: 17 May 2003

Posts: 8,616

10-31-2005 19:39

From: DogSpot Boxer

Reading this thread, I've come to realize that LL has a real problem with software development discipline and what appears to be little in the way of a cohesive and effective QA process.

Based upon what I have seen recently, it seems Linden Lab is aware of this as well and is taking steps to change their development philosophies. Unfortunately, with a system as complex as SL, with the kind of legacy it has (meaning that each update cannot in any way break previous content), that process is much easier said than done. It is easy for people to armchair quarterback about how LL should be doing things. You can't compare SL as a one-one comparison to other large scale software projects in some ways. Ultimately, I think Linden Lab will find the right balance, but admittedly it has been painfully slow in coming. Something definitely has to give.

_____________________

Cristiano

ANOmations - huge selection of high quality, low priced animations all $100L or less.

~SLUniverse.com~ SL's oldest and largest community site, featuring Snapzilla image sharing, forums, and much more.

Khamon Fate

fategardens.net

Join date: 21 Nov 2003

Posts: 4,177

10-31-2005 20:11

From: Cristiano Midnight

Something definitely has to give.

Something what? Or what I mean to ask is "what are their options in terms of something giving?"

_____________________

Visit the Fate Gardens Website @ fategardens.net

DogSpot Boxer

vortex thruster

Join date: 23 Aug 2005

Posts: 671

10-31-2005 20:21

From: Cristiano Midnight

Based upon what I have seen recently, it seems Linden Lab is aware of this as well and is taking steps to change their development philosophies. Unfortunately, with a system as complex as SL, with the kind of legacy it has (meaning that each update cannot in any way break previous content), that process is much easier said than done. It is easy for people to armchair quarterback about how LL should be doing things. You can't compare SL as a one-one comparison to other large scale software projects in some ways. Ultimately, I think Linden Lab will find the right balance, but admittedly it has been painfully slow in coming. Something definitely has to give.

"in some ways" is correct. However, in most ways it is exactly the same as many large scale software projects. I worked as a senior
software technical lead with a major UNIX vendor and we had all of the same legacy compatibility issues to deal with.

I don't see LL's issues as being related to the technology as much as it is about a major lack of discipline in the methodology and processes area.

FWIW, I actually don't care about LL's software development process or QA process. SL is enertainment to me and I can find other entertainment if SL is down for a few days.

_____________________

Dogspot Boxer
Charter Member Of The Socially Inept Club

Our Motto:

We may be inept, but at least we're social

Chris Wilde

Custom User Title

Join date: 21 Jul 2004

Posts: 768

10-31-2005 20:39

There are alot of issues with some of these suggestions.

How do you replicate the entire grid? How about load replication? Sure these are possible, but you obviously cant replicate ALL the hardware and will have to simulate user load as well.

How do you 'slowly' push out the new client to different users? If there are problems with the new client is it due to interactions with older clients? How do you control who gets what client? What about server side changes? Do you change a few sims at a time? How do you track issues with a new sim vs different clients? What about backend server changes? Somethings you CANT change slowly over time.

Now you have to write code to deal with 2 versions. Not just the client side but backend as well.

It just seems people are suggesting taking an already complex system and wanting to multiply the complexity several times. I'd rather LL spend time figuring out (if they havent) how to replicate the grid and its load artificially than trying to replicate it physically and support multiple code versions.

Neehai Zapata

Unofficial Parent

Join date: 8 Apr 2004

Posts: 1,970

11-01-2005 03:07

Well who knew? LL is looking for a QA Manager!

From: someone

Linden Lab Research and Development
Quality Assurance Manager

As a QA Manager, this will be the biggest challenge you will ever take on: Fully testing, on a regular basis, the entire scope of Second Life, a digital world where 99.99% of the content is user created. You will be responsible for solving this ongoing challenge in order that Linden Lab continues to ship amazing new releases! Within this challenging role the Quality Assurance Manger will be responsible for designing automated systems and processes, driving tool selection and supervising testing components. You will develop, mentor and manage the Quality Assurance staff, including scheduling, performance reviews, team and individual development, and making ongoing staffing decisions. The ideal candidate will be passionate about building high quality solutions. This position reports to the Second Life Program Manager.

Responsibilities:

* Oversee the development of a test lab for Second Life
* Expand Quality Assurance processes including the selection of appropriate test tools
* Lead the QA team in developing tools to enable Second Life to test itself
* Mentor staff within Quality Assurance
* Plan and manage the reliability of Second Life in a 24 X 7 production environment
* Communicate and resolve issues between relevant parties
* Develop issue resolution escalation procedures
* Prepare staffing and operational plans for Quality Assurance

Qualifications:

* Software engineering experience with at least two years as a Quality Assurance manager or director, preferably in a similar industry
* Successful history of testing 24 X 7 mission critical applications
* Knowledge and preferably testing experience with online client/server products or live online games
* Experience implementing growth plans to handle product growth as well as continuous process improvements
* Expert knowledge of software development processes
* Experience selecting, deploying and using automated test tools such as but not limited to LoadRunner or WinRunner
* Experience providing both hands on solutions and supervision of projects
* Ability to analyze, develop and implement needs into workable solutions
* Excellent oral and written communication skills
* Ability to successfully interface cross-functionally with a wide range of personnel within the organization
* Customer Service oriented personality
* Authorized to work in the US
* Bachelors Degree in Computer Science, Computer Engineering, Management Information Systems, Electrical Engineering or Physics preferred

I could do all that!

_____________________

Unofficial moderator and proud dysfunctional parent to over 1000 bastard children.

Khamon Fate

fategardens.net

Join date: 21 Nov 2003

Posts: 4,177

11-01-2005 06:29

/clap you go Nee!

Can you afford the pay cut? Might be worth it to work in a cutting edge research facility. Flaws are flaws and have been blatently obvious for two years. The encouraging verbiage from Andrew indicates that they are formalizing their development process by establishing teams to focus on feature production and revision with a solid set of parameters to keep everything smooth as branches and twigs are grafted onto the base.

The fact that they've been misleading us into believing that they've operated this way all these years is not beside the point. But it isn't THE point. THE point is that they're going to begin functioning professionally. Granted it won't afford the fun, research laboratory environment they're used to; but they are an in production shop now offering a service to paying customers. So it is good to believe that they're going to start acting like it.

The other encouraging fact is that the customer service reps will no longer have to talk around ethereal feature schedules and such. They've been on the front line very bravely stalling us all these months because they've been consistently fed wholly unreliable information from the back room. Now they'll be able to treat us like adults and offer realistic projections or just say no, that won't happen.

_____________________

Visit the Fate Gardens Website @ fategardens.net

Rei Kuhr

Ground Repellant

Join date: 18 May 2005

Posts: 54

11-01-2005 07:15

From: Hiro Queso

The thing is Rei, *none* of the problems I am experiencing now (and these are major in your face problems) were present on any of my visits to the preview grid.

Doesnt change my point that the difference between the preview grid and the main grid was that the main grid has on average about 2000 people playing at once, whereas the preview in my sole estimation did not top 400. It might have been there but less noticable, due to either only one person having to load everything on that sim, or less stress upon the space and asset server.

From: Doc Nielsen

Well I can cause my sim, with only me in it, and no one in the adjoining sims, to drop to 4fps just by panning my camera.

I really think that had THAT sort of thing happened in the preview grid someone would have noticed.

Don't try and blame the previewers. Either the Preview grid wasn't in any way representative of the main grid, or (possibly and too!) the update was modified between last public preview release and the main grid release.

I can't accept any suggestion that the mess we are now in is the fault of the customers! In fact I find this sort of attitude extremely annoying. Development, testing and QC are LL's responsibility - NOT the customers.

Once again, read upwards. The preview grid's asset server and it's sims were not stressed as hard as the main grid is now. I'm not saying that the preview grid was the same as the main, but I am saying that the preview grid didnt recieve a proper stress testing. And furthermore, I dont know about you guys but I saw the beginnings of the problems on the preview grid about two days before it went live, in the form of textures taking forever to load. In fact, in the preview grid, some refused to load at all, or colors were wrong. For example, I was standing at the main platform in Lusk by the great tree, and the whole tree was white and untextured, and the vendors refused to show anything but grey to me. If the preview grid isnt indicative of the problems we were going to face on the main grid, then why on the day of release when everybody got onto the preview grid to wait for the main grid to come up were some of us facing the same problems that we are now having in 1.7?

Just because YOU didnt experience a bug doesnt mean it's out there.

All of those listed in the About SecondLife in the Help Menu of the client as helping test 1.7 are the ones here who should feel no guilt about this. I'm also not saying those not on the list should, but just think that the list could have been longer, and SL could have been less buggier.

Buster Peel

Spat the dummy.

Join date: 7 Feb 2005

Posts: 1,242

11-01-2005 10:13

The Linden SDLC (Software Development Life Cycle) is, in a word, amateurish.

They obviously don't include adequate load testing.

If a true load isn't available for load testing (is it ever?), then you have to simulate loads. Web sites do this all the time. While they may have managed to pile a bunch of av's into a sim, did they do it on the FINAL code? That load tests the sim, obviously, but what about the other servers? Was anything done to create a realistic load on all parts of the entire platform?

Trial-and-error is fine for your hobby. But I think a little more systematic approach is needed for SL.

Buster

Kurshie Muromachi

Primtastic!

Join date: 24 Apr 2005

Posts: 278

11-01-2005 11:26

From: Buster Peel

If a true load isn't available for load testing (is it ever?), then you have to simulate loads. Web sites do this all the time. While they may have managed to pile a bunch of av's into a sim, did they do it on the FINAL code? That load tests the sim, obviously, but what about the other servers? Was anything done to create a realistic load on all parts of the entire platform?

Yes, like simulating objects replicating into mass quantities. It may seem like a silly thing to do but not silly enough for someone to take advantage/abuse it.

Neehai Zapata

Unofficial Parent

Join date: 8 Apr 2004

Posts: 1,970

11-01-2005 18:45

From: someone

The Linden SDLC (Software Development Life Cycle) is, in a word, amateurish.

They obviously don't include adequate load testing.

If a true load isn't available for load testing (is it ever?), then you have to simulate loads. Web sites do this all the time. While they may have managed to pile a bunch of av's into a sim, did they do it on the FINAL code? That load tests the sim, obviously, but what about the other servers? Was anything done to create a realistic load on all parts of the entire platform?

Trial-and-error is fine for your hobby. But I think a little more systematic approach is needed for SL.

It would be interesting to see the benchmarks and performance test plans for SL.

I think it would be quite a challenge to create a solid QA process for LL.

I currently manage QA for a major North American bank's eCommerce activities.

We should start a thread defining the key performance metrics for SL.

_____________________

Unofficial moderator and proud dysfunctional parent to over 1000 bastard children.

Malachi Petunia

Gentle Miscreant

Join date: 21 Sep 2003

Posts: 3,414

11-01-2005 20:15

From: someone

We should start a thread defining the key performance metrics for SL.

Ooh, and that would probably be as effective as oh - just pulling an example out of thin air - the feature suggestion forum. Please excuse me now as there is a pot of water on the stove that needs to be watched.

_____________________

Khamon Fate

fategardens.net

Join date: 21 Nov 2003

Posts: 4,177

11-01-2005 20:39

From: Malachi Petunia

Ooh, and that would probably be as effective as oh - just pulling an example out of thin air - the feature suggestion forum. Please excuse me now as there is a pot of water on the stove that needs to be watched.

For a brief moment Khamon held his breath, knowing with the easy certainty of a master bricklayer that Mal was going to say "the feature voting page." He was right! He was wrong. His mind reeled with the justification of similarity for an eternity before he heard a voice exclaim "enough!" Was the voice his own?

_____________________

Visit the Fate Gardens Website @ fategardens.net

Neehai Zapata

Unofficial Parent

Join date: 8 Apr 2004

Posts: 1,970

11-02-2005 03:43

From: someone

Ooh, and that would probably be as effective as oh - just pulling an example out of thin air - the feature suggestion forum. Please excuse me now as there is a pot of water on the stove that needs to be watched.

I would actually find it very interesting.

Your mileage may vary. I do thank you for your constructive contribution to this issue.

_____________________

Unofficial moderator and proud dysfunctional parent to over 1000 bastard children.

Is Linden Lab's preview testing process inadequate?
Annah Zamboni Banannah Annah Join date: 2 Jun 2004 Posts: 1,022	10-31-2005 14:43 From: Doc Nielsen At this point explanation is irrelevant. What they need to do is fix it. And take steps to ensure it doesn't happen again. Like it or not SL IS a business, and businesses that repeat errors of this magnitude don't survive in the real world. Yes its a business, and like a business sometimes your customers want explanations. They want to understand if this was a fluke or if your business model will continue to produce these kinds of 'results'.
Doc Nielsen Fallen... Join date: 13 Apr 2005 Posts: 1,059	10-31-2005 14:57 Hmmm, well I don't want 'explanations'. I just want my sim and a half or so working again. If it happens again I'm history. In fact if they don't get it fixed this next update - I'm history. At the moment they are charging me substantial sums of money for a product I can't use! _____________________ All very well for people to have a sig that exhorts you to 'be the change' - I wonder if it's ever occurred to them that they might be something that needs changing...?
Starax Statosky Unregistered User Join date: 23 Dec 2003 Posts: 1,099	Bugged 10-31-2005 15:02 * Shoots myself in the head *
Cristiano Midnight Evil Snapshot Baron Join date: 17 May 2003 Posts: 8,616	10-31-2005 15:43 * waves to Starax * _____________________ Cristiano ANOmations - huge selection of high quality, low priced animations all $100L or less. ~SLUniverse.com~ SL's oldest and largest community site, featuring Snapzilla image sharing, forums, and much more.
blaze Spinnaker 1/2 Serious Join date: 12 Aug 2004 Posts: 5,898	10-31-2005 16:11 Nah, LL has more control than you think they do. It's not like they should go out of their way to tell everyone how they keep people from crashing the grid. That'd be a tad silly. That being said, they could do more. I suspect most of their techniques are one off and not particularly centralized. I've proposed a centralized technique they could use, statistical sampling around points of failure .. Kelly said he thought it was an interesting idea. Hopefully that means they might actually do it or something similar. I think the only real big mistake they're really making is this "big bang" approach to release, which Ian Linden admitted in his town hall that they recognize this is a mistake. And, lets face it, there is a lot of reason to do a big bang approach. However, I think we all recognize it's just not going to work with SL where regression testing simply is not possible except on the main grid.
Neehai Zapata Unofficial Parent Join date: 8 Apr 2004 Posts: 1,970	10-31-2005 18:14 LL should hire me to manage their QA process, then I could get away from this bank. _____________________ Unofficial moderator and proud dysfunctional parent to over 1000 bastard children.
DogSpot Boxer vortex thruster Join date: 23 Aug 2005 Posts: 671	10-31-2005 19:18 From: blaze Spinnaker They should get rid of the preview grid entirely. They should learn to rollout a feature to a percentage of the user base, and slowly roll it out to to more as it proves to be solid. It's a bit more complex, but it's a scaleable and very proven development / qa methodology. Yeah, the preview grid is a joke. If they really wanted it to be effective, they should have replicated the entire main grid and let people move stuff between the two grids freely. That way more people would be inclined to participate in the preview. Reading this thread, I've come to realize that LL has a real problem with software development discipline and what appears to be little in the way of a cohesive and effective QA process. _____________________ Dogspot Boxer Charter Member Of The Socially Inept Club Our Motto: We may be inept, but at least we're social
Cristiano Midnight Evil Snapshot Baron Join date: 17 May 2003 Posts: 8,616	10-31-2005 19:39 From: DogSpot Boxer Reading this thread, I've come to realize that LL has a real problem with software development discipline and what appears to be little in the way of a cohesive and effective QA process. Based upon what I have seen recently, it seems Linden Lab is aware of this as well and is taking steps to change their development philosophies. Unfortunately, with a system as complex as SL, with the kind of legacy it has (meaning that each update cannot in any way break previous content), that process is much easier said than done. It is easy for people to armchair quarterback about how LL should be doing things. You can't compare SL as a one-one comparison to other large scale software projects in some ways. Ultimately, I think Linden Lab will find the right balance, but admittedly it has been painfully slow in coming. Something definitely has to give. _____________________ Cristiano ANOmations - huge selection of high quality, low priced animations all $100L or less. ~SLUniverse.com~ SL's oldest and largest community site, featuring Snapzilla image sharing, forums, and much more.
Khamon Fate fategardens.net Join date: 21 Nov 2003 Posts: 4,177	10-31-2005 20:11 From: Cristiano Midnight Something definitely has to give. Something what? Or what I mean to ask is "what are their options in terms of something giving?" _____________________ Visit the Fate Gardens Website @ fategardens.net
DogSpot Boxer vortex thruster Join date: 23 Aug 2005 Posts: 671	10-31-2005 20:21 From: Cristiano Midnight Based upon what I have seen recently, it seems Linden Lab is aware of this as well and is taking steps to change their development philosophies. Unfortunately, with a system as complex as SL, with the kind of legacy it has (meaning that each update cannot in any way break previous content), that process is much easier said than done. It is easy for people to armchair quarterback about how LL should be doing things. You can't compare SL as a one-one comparison to other large scale software projects in some ways. Ultimately, I think Linden Lab will find the right balance, but admittedly it has been painfully slow in coming. Something definitely has to give. "in some ways" is correct. However, in most ways it is exactly the same as many large scale software projects. I worked as a senior software technical lead with a major UNIX vendor and we had all of the same legacy compatibility issues to deal with. I don't see LL's issues as being related to the technology as much as it is about a major lack of discipline in the methodology and processes area. FWIW, I actually don't care about LL's software development process or QA process. SL is enertainment to me and I can find other entertainment if SL is down for a few days. _____________________ Dogspot Boxer Charter Member Of The Socially Inept Club Our Motto: We may be inept, but at least we're social
Chris Wilde Custom User Title Join date: 21 Jul 2004 Posts: 768	10-31-2005 20:39 There are alot of issues with some of these suggestions. How do you replicate the entire grid? How about load replication? Sure these are possible, but you obviously cant replicate ALL the hardware and will have to simulate user load as well. How do you 'slowly' push out the new client to different users? If there are problems with the new client is it due to interactions with older clients? How do you control who gets what client? What about server side changes? Do you change a few sims at a time? How do you track issues with a new sim vs different clients? What about backend server changes? Somethings you CANT change slowly over time. Now you have to write code to deal with 2 versions. Not just the client side but backend as well. It just seems people are suggesting taking an already complex system and wanting to multiply the complexity several times. I'd rather LL spend time figuring out (if they havent) how to replicate the grid and its load artificially than trying to replicate it physically and support multiple code versions.
Neehai Zapata Unofficial Parent Join date: 8 Apr 2004 Posts: 1,970	11-01-2005 03:07 Well who knew? LL is looking for a QA Manager! From: someone Linden Lab Research and Development Quality Assurance Manager As a QA Manager, this will be the biggest challenge you will ever take on: Fully testing, on a regular basis, the entire scope of Second Life, a digital world where 99.99% of the content is user created. You will be responsible for solving this ongoing challenge in order that Linden Lab continues to ship amazing new releases! Within this challenging role the Quality Assurance Manger will be responsible for designing automated systems and processes, driving tool selection and supervising testing components. You will develop, mentor and manage the Quality Assurance staff, including scheduling, performance reviews, team and individual development, and making ongoing staffing decisions. The ideal candidate will be passionate about building high quality solutions. This position reports to the Second Life Program Manager. Responsibilities: * Oversee the development of a test lab for Second Life * Expand Quality Assurance processes including the selection of appropriate test tools * Lead the QA team in developing tools to enable Second Life to test itself * Mentor staff within Quality Assurance * Plan and manage the reliability of Second Life in a 24 X 7 production environment * Communicate and resolve issues between relevant parties * Develop issue resolution escalation procedures * Prepare staffing and operational plans for Quality Assurance Qualifications: * Software engineering experience with at least two years as a Quality Assurance manager or director, preferably in a similar industry * Successful history of testing 24 X 7 mission critical applications * Knowledge and preferably testing experience with online client/server products or live online games * Experience implementing growth plans to handle product growth as well as continuous process improvements * Expert knowledge of software development processes * Experience selecting, deploying and using automated test tools such as but not limited to LoadRunner or WinRunner * Experience providing both hands on solutions and supervision of projects * Ability to analyze, develop and implement needs into workable solutions * Excellent oral and written communication skills * Ability to successfully interface cross-functionally with a wide range of personnel within the organization * Customer Service oriented personality * Authorized to work in the US * Bachelors Degree in Computer Science, Computer Engineering, Management Information Systems, Electrical Engineering or Physics preferred I could do all that! _____________________ Unofficial moderator and proud dysfunctional parent to over 1000 bastard children.
Khamon Fate fategardens.net Join date: 21 Nov 2003 Posts: 4,177	11-01-2005 06:29 /clap you go Nee! Can you afford the pay cut? Might be worth it to work in a cutting edge research facility. Flaws are flaws and have been blatently obvious for two years. The encouraging verbiage from Andrew indicates that they are formalizing their development process by establishing teams to focus on feature production and revision with a solid set of parameters to keep everything smooth as branches and twigs are grafted onto the base. The fact that they've been misleading us into believing that they've operated this way all these years is not beside the point. But it isn't THE point. THE point is that they're going to begin functioning professionally. Granted it won't afford the fun, research laboratory environment they're used to; but they are an in production shop now offering a service to paying customers. So it is good to believe that they're going to start acting like it. The other encouraging fact is that the customer service reps will no longer have to talk around ethereal feature schedules and such. They've been on the front line very bravely stalling us all these months because they've been consistently fed wholly unreliable information from the back room. Now they'll be able to treat us like adults and offer realistic projections or just say no, that won't happen. _____________________ Visit the Fate Gardens Website @ fategardens.net
Rei Kuhr Ground Repellant Join date: 18 May 2005 Posts: 54	11-01-2005 07:15 From: Hiro Queso The thing is Rei, none of the problems I am experiencing now (and these are major in your face problems) were present on any of my visits to the preview grid. Doesnt change my point that the difference between the preview grid and the main grid was that the main grid has on average about 2000 people playing at once, whereas the preview in my sole estimation did not top 400. It might have been there but less noticable, due to either only one person having to load everything on that sim, or less stress upon the space and asset server. From: Doc Nielsen Well I can cause my sim, with only me in it, and no one in the adjoining sims, to drop to 4fps just by panning my camera. I really think that had THAT sort of thing happened in the preview grid someone would have noticed. Don't try and blame the previewers. Either the Preview grid wasn't in any way representative of the main grid, or (possibly and too!) the update was modified between last public preview release and the main grid release. I can't accept any suggestion that the mess we are now in is the fault of the customers! In fact I find this sort of attitude extremely annoying. Development, testing and QC are LL's responsibility - NOT the customers. Once again, read upwards. The preview grid's asset server and it's sims were not stressed as hard as the main grid is now. I'm not saying that the preview grid was the same as the main, but I am saying that the preview grid didnt recieve a proper stress testing. And furthermore, I dont know about you guys but I saw the beginnings of the problems on the preview grid about two days before it went live, in the form of textures taking forever to load. In fact, in the preview grid, some refused to load at all, or colors were wrong. For example, I was standing at the main platform in Lusk by the great tree, and the whole tree was white and untextured, and the vendors refused to show anything but grey to me. If the preview grid isnt indicative of the problems we were going to face on the main grid, then why on the day of release when everybody got onto the preview grid to wait for the main grid to come up were some of us facing the same problems that we are now having in 1.7? Just because YOU didnt experience a bug doesnt mean it's out there. All of those listed in the About SecondLife in the Help Menu of the client as helping test 1.7 are the ones here who should feel no guilt about this. I'm also not saying those not on the list should, but just think that the list could have been longer, and SL could have been less buggier.
Buster Peel Spat the dummy. Join date: 7 Feb 2005 Posts: 1,242	11-01-2005 10:13 The Linden SDLC (Software Development Life Cycle) is, in a word, amateurish. They obviously don't include adequate load testing. If a true load isn't available for load testing (is it ever?), then you have to simulate loads. Web sites do this all the time. While they may have managed to pile a bunch of av's into a sim, did they do it on the FINAL code? That load tests the sim, obviously, but what about the other servers? Was anything done to create a realistic load on all parts of the entire platform? Trial-and-error is fine for your hobby. But I think a little more systematic approach is needed for SL. Buster
Kurshie Muromachi Primtastic! Join date: 24 Apr 2005 Posts: 278	11-01-2005 11:26 From: Buster Peel If a true load isn't available for load testing (is it ever?), then you have to simulate loads. Web sites do this all the time. While they may have managed to pile a bunch of av's into a sim, did they do it on the FINAL code? That load tests the sim, obviously, but what about the other servers? Was anything done to create a realistic load on all parts of the entire platform? Yes, like simulating objects replicating into mass quantities. It may seem like a silly thing to do but not silly enough for someone to take advantage/abuse it.
Neehai Zapata Unofficial Parent Join date: 8 Apr 2004 Posts: 1,970	11-01-2005 18:45 From: someone The Linden SDLC (Software Development Life Cycle) is, in a word, amateurish. They obviously don't include adequate load testing. If a true load isn't available for load testing (is it ever?), then you have to simulate loads. Web sites do this all the time. While they may have managed to pile a bunch of av's into a sim, did they do it on the FINAL code? That load tests the sim, obviously, but what about the other servers? Was anything done to create a realistic load on all parts of the entire platform? Trial-and-error is fine for your hobby. But I think a little more systematic approach is needed for SL. It would be interesting to see the benchmarks and performance test plans for SL. I think it would be quite a challenge to create a solid QA process for LL. I currently manage QA for a major North American bank's eCommerce activities. We should start a thread defining the key performance metrics for SL. _____________________ Unofficial moderator and proud dysfunctional parent to over 1000 bastard children.
Malachi Petunia Gentle Miscreant Join date: 21 Sep 2003 Posts: 3,414	11-01-2005 20:15 From: someone We should start a thread defining the key performance metrics for SL. Ooh, and that would probably be as effective as oh - just pulling an example out of thin air - the feature suggestion forum. Please excuse me now as there is a pot of water on the stove that needs to be watched. _____________________
Khamon Fate fategardens.net Join date: 21 Nov 2003 Posts: 4,177	11-01-2005 20:39 From: Malachi Petunia Ooh, and that would probably be as effective as oh - just pulling an example out of thin air - the feature suggestion forum. Please excuse me now as there is a pot of water on the stove that needs to be watched. For a brief moment Khamon held his breath, knowing with the easy certainty of a master bricklayer that Mal was going to say "the feature voting page." He was right! He was wrong. His mind reeled with the justification of similarity for an eternity before he heard a voice exclaim "enough!" Was the voice his own? _____________________ Visit the Fate Gardens Website @ fategardens.net
Neehai Zapata Unofficial Parent Join date: 8 Apr 2004 Posts: 1,970	11-02-2005 03:43 From: someone Ooh, and that would probably be as effective as oh - just pulling an example out of thin air - the feature suggestion forum. Please excuse me now as there is a pot of water on the stove that needs to be watched. I would actually find it very interesting. Your mileage may vary. I do thank you for your constructive contribution to this issue. _____________________ Unofficial moderator and proud dysfunctional parent to over 1000 bastard children.

Welcome to the Second Life Forums Archive

Is Linden Lab's preview testing process inadequate?