|
Ricky Lucero
Registered User
Join date: 25 Jul 2006
Posts: 122
|
12-11-2006 10:19
Can any of the lindens explain to me more of the QA processes you have in place for testing new versions of SL? I was reading more of the employment postings, and noticed that in your development jobs it's stated "Must be able to QA own work", which to me says that you don't actually have any employees that are dedicated to doing QA on a new release. Being in software development for so long, this seems to be one of the most unorganized QA processes I've ever seen! There are critical bugs from versions that were released long before I ever became a resident, and they still exist today after more than 6 minor upgrades. Bugs that are small and so simple and easy to fix, are always overlooked by feature additions and changes, which then causes more bugs. The QA process adopted by LL just doesn't seem like one you would see in a corporate software development environment.
Which then leads me to my next thought which is, programmers can't QA their own work. I can tell you a million times that something works exactly as I wrote it, but I can also probably guarantee that there's at least one bug, no matter how many times I've told you it's bug free. It could be a hello world script with one line, and there's still a typo, but I told you it was bug free, and that I QA'd it on my own!
Residents and developers should not be relied upon for QA, and the perception given is that it's exactly who is doing ALL of the QA. Not that you have to give away any company secrets, but I'm curious as to what kind of QA is done on SL, because I truly believe that LL could release better versions, with the right QA team and management
Since I can't reply, Adding a couple edits:
-A developer making sure his own code works, is NOT QA. QA is done by someone who had no hand in developing the application. Of course I make sure my code works before I release it, but that just simply isn't real QA.
-Also, Test plans should not be written by developers. Developers should be letting QA know what is being done, and the QA team should be coming up wth the testing plans, based on requirements for the next build. Again, The developer doing QA work.
-Last: Bugs from residents SHOULD NEVER go to a developer directly! How freaking annoying, and what a WASTE OF TIME for that developer to be looking at bugs from residents that are probably false. QA should be looking at, and confirming/rejecting bugs entered by residents. These bugs should then be escalated to development as a confirmed bug. Takes a lot fo work off of the developers.
ok, just IMO.... TY for the great response!
And seriously, what's up with the critical bugs that were confirmed like 10 months ago, and still not fixed? Critical doesn't mean "fix it"?
|
|
Kelly Linden
Linden Developer
Join date: 29 Mar 2004
Posts: 896
|
12-11-2006 11:06
We do indeed have a QA department.
Our process is roughly thus:
- Project lead creates a branch from the trunk for a new project
- Developers working on that project check their code into that branch
- Developers write test plans for the code they wrote
- When the project is "finished", the test plans and the branch/revision is sent to QA
- QA tests the branch
- They run the given test plans
- They run related test plans
- They run 'smoke tests'
- They send bugs back to the Project Lead
Developers fix the bugs and send back to QA, rinse, repeat
The branch is merged back into trunk
Trunk goes to QA and bugs are sent back to the developers (of the many projects that are in at that point)
Developers fix the bugs and send back to QA, rinse, repeat
A "Beta" gets released (either viewer like the Focus Beta or full grid)
Residents and QA send bugs to developers who fix the bugs, rinse repeat
New version gets released to the main grid
This is a little basic, and a little bit idealized. Sometimes emergency bug fixes short circuit some of the steps above (such as public beta). There can be more merging and branching of code involved depending on the length of the project and what is happening in the trunk.
The summary is that we have a very complex system involving lots of hardware and lots of different software that all must work together, not to mention all the parts of each process internally. These systems are also being used in many, many different ways by many people; sometimes we just don't realize things are being done a certain way in world until we 'break' it.
Are we doing a good job of keeping the bugs out? No, not really. We do however have a QA department, and the process above is almost constantly under review and changing. I think we are doing a better job now than we have in the past. We are also dedicating a large portion of development towards fixing bugs, a lot of which includes future proofing against scaling issues and the like, and replacing core systems that were designed with what is now evident as faulty assumptions. These are often invisible to the user unless they break or we don't do it soon enough.
As a developer I QA my own work before sending it to QA. It doesn't make sense to just write code, compile and send off. I check to make sure it does what I think it does first before I send to QA. We do have a larger development team than we do a QA team and doing it any other way I don't really think makes sense. The ability to do this is probably what the job description refers to.
_____________________
- Kelly Linden
|
|
Kelly Linden
Linden Developer
Join date: 29 Mar 2004
Posts: 896
|
12-11-2006 12:36
As I said that was a simplified view. Bug reports from residents do not go straight to developers. They are read and repro'd by QA first, but the actual bugs found by the residents are fixed by developers.
Different definitions and different levels of critical, plus the 'behind the scenes' changes which are often far more critical than their hidden nature may imply. There are a large number of issues that are constantly being prioritized, and with a finite development team that means some issues can continually get filtered to the bottom, though we do try to prevent that when we can. Please do not take this as an invitation to post "what about bug X". Please submit bug reports for those when they happen.
No more comments within the thread please. Feel free to email me directly if you wish: [email]kelly@lindenlab.com[/email] or post further (seperate, distinct) questions here.
_____________________
- Kelly Linden
|