Save SL: Reduce Your Inventory!
|
|
Michael Bigwig
~VRML Aficionado~
Join date: 5 Dec 2005
Posts: 2,181
|
10-16-2007 06:44
Firstly, can we get a definitive answer as to whether inventory hording does actually create (or will create) performance issues? Let's get a straight answer here please.
Secondly, what in the name of Sam Elliot are some of you doing with over 5000 assets? That's just unnecessary. I guess if you give a RL pack-rat unlimited storage in a magic bag, they'll give OCD a new outlet.
It's certainly time for a cleaning on my part--I've never cleaned house once, and I've been around since '05. However, I'm still under the 1000 mark...go figure.
_____________________
~Michael Bigwig __________________________________________________Lead Designer, Glowbox Designs 
|
|
Colette Meiji
Registered User
Join date: 25 Mar 2005
Posts: 15,556
|
10-16-2007 06:46
Another item that youd have that is unique on the database is SNAPSHOTS.
Download the ones you want to keep but don't need in SL onto your hard drive.
Delete all snapshots you don't need.
I'm thinking snapshots would probably the the biggest number of database entries that the average user could affect.
10,000 people deleting 100 unique snap shots each from their inventories is 1,000,000 unique item database entires.
|
|
Brenda Connolly
Un United Avatar
Join date: 10 Jan 2007
Posts: 25,000
|
10-16-2007 06:54
From: Michael Bigwig Firstly, can we get a definitive answer as to whether inventory hording does actually create (or will create) performance issues? Let's get a straight answer here please.
Secondly, what in the name of Sam Elliot are some of you doing with over 5000 assets? That's just unnecessary. I guess if you give a RL pack-rat unlimited storage in a magic bag, they'll give OCD a new outlet.
It's certainly time for a cleaning on my part--I've never cleaned house once, and I've been around since '05. However, I'm still under the 1000 mark...go figure. Well, what can you get for the Android who has everything?
_____________________
Don't you ever try to look behind my eyes. You don't want to know what they have seen.
http://brenda-connolly.blogspot.com
|
|
Michael Bigwig
~VRML Aficionado~
Join date: 5 Dec 2005
Posts: 2,181
|
10-16-2007 06:54
From: Brenda Connolly Well, what can you get for the Android who has everything? I want to be a real boy!
_____________________
~Michael Bigwig __________________________________________________Lead Designer, Glowbox Designs 
|
|
Zaphod Kotobide
zOMGWTFPME!
Join date: 19 Oct 2006
Posts: 2,087
|
10-16-2007 07:03
Assets.. objects, textures, sounds, gestures, etc.. are actually NOT stored in a database. They are stored as files within a fault tolerant, distributed filesystem, fronted by http server clusters.
The MySQL systems only act as an "indexing" (and probably permissions, other meta data tracking) service for these distributed files. The files themselves, once located by SQL query, are transported via the http protocol, in the same way you would download a file from a website by right-click/save as, or the same way inline images are downloaded and displayed within a web page.
When we talk about asset system load, what we are really looking at is not storage load, but query load. Storage today is cheap, and almost infinitely scalable. A descriptor file for a given object, even a fairly complex one, is only a very small text file, and it generally takes fractions of a second for that file to transport to the simulator and instantiate in-world as the object it describes. The efficiencies intrinsic to this schema are what make it possible for the world to stream down to us, in close to real time. It could be better, but as it is, it's extraordinarily efficient.
So.. where does this leave us as far as bloated inventories are concerned? Every time we open our inventory folder, an SQL query is executed to populate the folder contents. Tens of thousands of inventory "items", times (potentially) tens of thousands of agents, simultaneously, and we can begin to understand the problem. Leave the inventory folder open, and it will need to periodically refresh the contents, through additional querying.
With read caching, I would suspect that the bottleneck isn't even so much with the queries themselves, but in the communications involved in transporting the *results* back to the client. Considering that this communication takes place within a VPN framework, there is network/processing overhead beyond the usual carrier overhead.
So.. while it is true that deleting an item from your inventory does not necessarily remove that asset from the file system, (there are garbage collection routines which remove orphaned files from the store) that is inconsequential to the general discussion about keeping lean and tidy inventories.
One very effective way to do this, without actually getting rid of stuff, is to simply box items up and store the box back in your inventory. Each entry inside that box will still consume a row (or more) in the MySQL database, but there will be no query run against them until the box is instantiated in-world and its contents folder opened.
It would be GREAT if maybe Ian could jump in here and talk about this a bit, or another database Linden, and specifically address the actual load issue being discussed here, and how substantial it really is (or isn't?). At any rate, keeping a lean inventory can certainly only improve things overall.
--Zaph
_____________________
From: Albert Einstein Problems cannot be solved at the same level of awareness that created them.
|
|
Pie Psaltery
runs w/scissors
Join date: 13 Jan 2004
Posts: 987
|
10-16-2007 07:05
To help keep my own inventory under control I tend to put full perms things (usually snapshots and textures I upload, altho I have done this for major builds as well) into notecards.
Just drag them from your inventory onto a notecard. It makes them much easier to paw thru when you are looking for something then finding a place you can rez a box to open.
|
|
Colette Meiji
Registered User
Join date: 25 Mar 2005
Posts: 15,556
|
10-16-2007 07:14
From: Zaphod Kotobide Assets.. objects, textures, sounds, gestures, etc.. are actually NOT stored in a database. They are stored as files within a fault tolerant, distributed filesystem, fronted by http server clusters.
The MySQL systems only act as an "indexing" (and probably permissions, other meta data tracking) service for these distributed files. The files themselves, once located by SQL query, are transported via the http protocol, in the same way you would download a file from a website by right-click/save as, or the same way inline images are downloaded and displayed within a web page.
When we talk about asset system load, what we are really looking at is not storage load, but query load. Storage today is cheap, and almost infinitely scalable. A descriptor file for a given object, even a fairly complex one, is only a very small text file, and it generally takes fractions of a second for that file to transport to the simulator and instantiate in-world as the object it describes. The efficiencies intrinsic to this schema are what make it possible for the world to stream down to us, in close to real time. It could be better, but as it is, it's extraordinarily efficient.
So.. where does this leave us as far as bloated inventories are concerned? Every time we open our inventory folder, an SQL query is executed to populate the folder contents. Tens of thousands of inventory "items", times (potentially) tens of thousands of agents, simultaneously, and we can begin to understand the problem. Leave the inventory folder open, and it will need to periodically refresh the contents, through additional querying.
With read caching, I would suspect that the bottleneck isn't even so much with the queries themselves, but in the communications involved in transporting the *results* back to the client. Considering that this communication takes place within a VPN framework, there is network/processing overhead beyond the usual carrier overhead.
So.. while it is true that deleting an item from your inventory does not necessarily remove that asset from the file system, (there are garbage collection routines which remove orphaned files from the store) that is inconsequential to the general discussion about keeping lean and tidy inventories.
One very effective way to do this, without actually getting rid of stuff, is to simply box items up and store the box back in your inventory. Each entry inside that box will still consume a row (or more) in the MySQL database, but there will be no query run against them until the box is instantiated in-world and its contents folder opened.
It would be GREAT if maybe Ian could jump in here and talk about this a bit, or another database Linden, and specifically address the actual load issue being discussed here, and how substantial it really is (or isn't?). At any rate, keeping a lean inventory can certainly only improve things overall.
--Zaph This is interesting. It wasn't what the techie types were saying the last half dozen times this idea came up. The unique item thing was given pretty strongly as the reason why the Lindens haven't bothered placing a cap on inventory items. However I was repeating what they had said. So it is possible I am wrong. ------------------------- So reducing the inventory might help if you are right about it? ------------------------- One thing though about boxing items. There were some issues, I believe Coca had, the last time this idea came up. In where the items she kept in boxes would disappear. I do not know if its related or not but there is a houskeeping program that goes and checks items to see if anyone anywhere has them in their inventory - if they don't it deletes them. If I remember right it doesn't "look" INSIDE boxes.
|
|
Raymond Figtree
Gone, avi, gone
Join date: 17 May 2006
Posts: 6,256
|
10-16-2007 07:25
From: Colette Meiji Another item that youd have that is unique on the database is SNAPSHOTS.
Download the ones you want to keep but don't need in SL onto your hard drive.
Delete all snapshots you don't need.
I'm thinking snapshots would probably the the biggest number of database entries that the average user could affect.
10,000 people deleting 100 unique snap shots each from their inventories is 1,000,000 unique item database entires. You are being most helpful. Thank you.
_____________________
Read or listen to some Eckhart Tolle. You won't regret it.
|
|
Atashi Toshihiko
Frequently Befuddled
Join date: 7 Dec 2006
Posts: 1,423
|
10-16-2007 07:25
I keep my inventory around 1,800 items. Sometimes it creeps up to 2,000 and then I get busy with deleting or putting stuff into storage prims. (FWIW I have never -- so far -- lost anything I put in a storage prim that I left rezzed in world. I have about a dozen such storage prims rezzed in the storage room.) However, in spite of having such a small inventory, I have still managed to have an awful lot of things get eaten by the servers. "That object is no longer in the database." "Landmark no longer in the database." Be it prims, landmarks, even cloths, I've had stuff vanish from the database. The past 3 weeks it has been an epidemic it seems.
I know when you get that message, that's supposed to trigger a mechanism to bring the missing item(s) back, so I'll usually try again the following day. Sometimes, maybe 20% of the time, it actually does bring back the missing item. Otherwise, it's another great SL feature to help keep inventory under control.
-Atashi
_____________________
Visit Atashi's Art and Oddities Store and the Waikiti Motor Works at beautiful Waikiti.
|
|
Kelly Kuiper
Registered User
Join date: 20 Nov 2006
Posts: 357
|
10-16-2007 07:26
Lessee what we've got in here... Dancing cow. Carboard box to put on head. Animation that makes me trot in one place while singing Tiptoe Through the Tulips. Something very rude that I'd forgotten about. A skirt that makes my bum look huuuuuuge. Nah. All essential stuff here. No way I can possibly delete any of this. 
|
|
Raymond Figtree
Gone, avi, gone
Join date: 17 May 2006
Posts: 6,256
|
10-16-2007 07:27
From: Zaphod Kotobide Assets.. objects, textures, sounds, gestures, etc.. are actually NOT stored in a database. They are stored as files within a fault tolerant, distributed filesystem, fronted by http server clusters.
The MySQL systems only act as an "indexing" (and probably permissions, other meta data tracking) service for these distributed files. The files themselves, once located by SQL query, are transported via the http protocol, in the same way you would download a file from a website by right-click/save as, or the same way inline images are downloaded and displayed within a web page.
When we talk about asset system load, what we are really looking at is not storage load, but query load. Storage today is cheap, and almost infinitely scalable. A descriptor file for a given object, even a fairly complex one, is only a very small text file, and it generally takes fractions of a second for that file to transport to the simulator and instantiate in-world as the object it describes. The efficiencies intrinsic to this schema are what make it possible for the world to stream down to us, in close to real time. It could be better, but as it is, it's extraordinarily efficient.
So.. where does this leave us as far as bloated inventories are concerned? Every time we open our inventory folder, an SQL query is executed to populate the folder contents. Tens of thousands of inventory "items", times (potentially) tens of thousands of agents, simultaneously, and we can begin to understand the problem. Leave the inventory folder open, and it will need to periodically refresh the contents, through additional querying.
With read caching, I would suspect that the bottleneck isn't even so much with the queries themselves, but in the communications involved in transporting the *results* back to the client. Considering that this communication takes place within a VPN framework, there is network/processing overhead beyond the usual carrier overhead.
So.. while it is true that deleting an item from your inventory does not necessarily remove that asset from the file system, (there are garbage collection routines which remove orphaned files from the store) that is inconsequential to the general discussion about keeping lean and tidy inventories.
One very effective way to do this, without actually getting rid of stuff, is to simply box items up and store the box back in your inventory. Each entry inside that box will still consume a row (or more) in the MySQL database, but there will be no query run against them until the box is instantiated in-world and its contents folder opened.
It would be GREAT if maybe Ian could jump in here and talk about this a bit, or another database Linden, and specifically address the actual load issue being discussed here, and how substantial it really is (or isn't?). At any rate, keeping a lean inventory can certainly only improve things overall.
--Zaph You are even MORE most helpful. Thank you!
_____________________
Read or listen to some Eckhart Tolle. You won't regret it.
|
|
Ann Launay
Neko-licious™
Join date: 8 Aug 2006
Posts: 7,893
|
10-16-2007 07:32
From: Michael Bigwig Secondly, what in the name of Sam Elliot are some of you doing with over 5000 assets? That's just unnecessary.
In your opinion. I find it quite necessary myself, and I'm neither a content creator nor a RL pack rat. *shrug*
_____________________
~Now Trout Re-Re-Re-Certified!~ From: someone I am bumping you to an 8.5 on the Official Trout Measuring Instrument of Sluttiness. You are an enigma - on the one hand a sweet, gentle, intelligent woman who we would like to wrap up in our arms and protect, and on the other, a temptress to whom we would like to do all sorts of unmentionable things.
Congratulations and shame on you! You are a bit of a slut.
|
|
Michael Bigwig
~VRML Aficionado~
Join date: 5 Dec 2005
Posts: 2,181
|
10-16-2007 07:36
From: Zaphod Kotobide Assets.. objects, textures, sounds, gestures, etc.. are actually NOT stored in a database. They are stored as files within a fault tolerant, distributed filesystem, fronted by http server clusters.
The MySQL systems only act as an "indexing" (and probably permissions, other meta data tracking) service for these distributed files. The files themselves, once located by SQL query, are transported via the http protocol, in the same way you would download a file from a website by right-click/save as, or the same way inline images are downloaded and displayed within a web page.
When we talk about asset system load, what we are really looking at is not storage load, but query load. Storage today is cheap, and almost infinitely scalable. A descriptor file for a given object, even a fairly complex one, is only a very small text file, and it generally takes fractions of a second for that file to transport to the simulator and instantiate in-world as the object it describes. The efficiencies intrinsic to this schema are what make it possible for the world to stream down to us, in close to real time. It could be better, but as it is, it's extraordinarily efficient.
So.. where does this leave us as far as bloated inventories are concerned? Every time we open our inventory folder, an SQL query is executed to populate the folder contents. Tens of thousands of inventory "items", times (potentially) tens of thousands of agents, simultaneously, and we can begin to understand the problem. Leave the inventory folder open, and it will need to periodically refresh the contents, through additional querying.
With read caching, I would suspect that the bottleneck isn't even so much with the queries themselves, but in the communications involved in transporting the *results* back to the client. Considering that this communication takes place within a VPN framework, there is network/processing overhead beyond the usual carrier overhead.
So.. while it is true that deleting an item from your inventory does not necessarily remove that asset from the file system, (there are garbage collection routines which remove orphaned files from the store) that is inconsequential to the general discussion about keeping lean and tidy inventories.
One very effective way to do this, without actually getting rid of stuff, is to simply box items up and store the box back in your inventory. Each entry inside that box will still consume a row (or more) in the MySQL database, but there will be no query run against them until the box is instantiated in-world and its contents folder opened.
It would be GREAT if maybe Ian could jump in here and talk about this a bit, or another database Linden, and specifically address the actual load issue being discussed here, and how substantial it really is (or isn't?). At any rate, keeping a lean inventory can certainly only improve things overall.
--Zaph Thank you, this was what I was looking for--a bit network/database techie for me...but nonetheless, was the answer we all should have been basing our posts on long-ago. Errrmm...assuming you're right of course. 
_____________________
~Michael Bigwig __________________________________________________Lead Designer, Glowbox Designs 
|
|
Nika Talaj
now you see her ...
Join date: 2 Jan 2007
Posts: 5,449
|
Thanks, Zaph, and a question
10-16-2007 07:42
Thanks for your post, Zaphod, nice to see more facts. I poked around on the SL wiki but couldn't find anything about asset handling architecture ... do you know, have the Lindens written any of this down anyplace that is accessible to us?
_____________________
.  To contact forum folks, join the inworld group "The Forum Cartel". New residents with questions about SL more than welcome! We has parties!  To contact forum scripters, join the inworld group "Scriptoratti" (thanks Void!). New scripter questions welcome!
|
|
Michael Bigwig
~VRML Aficionado~
Join date: 5 Dec 2005
Posts: 2,181
|
10-16-2007 07:53
From: Nika Talaj Thanks for your post, Zaphod, nice to see more facts. I poked around on the SL wiki but couldn't find anything about asset handling architecture ... do you know, have the Lindens written any of this down anyplace that is accessible to us? Zaphod appears well-informed--I equate that to him having two brains.
_____________________
~Michael Bigwig __________________________________________________Lead Designer, Glowbox Designs 
|
|
bilbo99 Emu
Garrett's No.1 fan
Join date: 27 Oct 2006
Posts: 3,468
|
10-16-2007 07:55
From: Kelly Kuiper Lessee what we've got in here... Dancing cow. Carboard box to put on head. Animation that makes me trot in one place while singing Tiptoe Through the Tulips. Something very rude that I'd forgotten about. A skirt that makes my bum look huuuuuuge. Nah. All essential stuff here. No way I can possibly delete any of this.  I blame the water supply .. the Severn gets very iffy north of Tewkesbury 
_____________________
Be polite .. that newbie could be your next ex-partner.
|
|
Zaphod Kotobide
zOMGWTFPME!
Join date: 19 Oct 2006
Posts: 2,087
|
10-16-2007 07:57
Some time ago, somebody at the lab actually sketched out the entire schema. It filled an entire wall, according to Zero. To my knowledge, there is no officially published data dictionary or schematic of the system as a whole. Also, I should note that my description is very high level, and is likely not *100%* accurate. Obviously, it is MUCH more sophisticated and complex than my own understanding extends. From: Nika Talaj Thanks for your post, Zaphod, nice to see more facts. I poked around on the SL wiki but couldn't find anything about asset handling architecture ... do you know, have the Lindens written any of this down anyplace that is accessible to us?
_____________________
From: Albert Einstein Problems cannot be solved at the same level of awareness that created them.
|
|
Michael Bigwig
~VRML Aficionado~
Join date: 5 Dec 2005
Posts: 2,181
|
10-16-2007 07:59
From: Ann Launay In your opinion. I find it quite necessary myself, and I'm neither a content creator nor a RL pack rat. *shrug* Ann--explain to me why (especially not being a content creator) you feel having thousands of assets to be 'NECESSARY?' The last word is key.
_____________________
~Michael Bigwig __________________________________________________Lead Designer, Glowbox Designs 
|
|
Nika Talaj
now you see her ...
Join date: 2 Jan 2007
Posts: 5,449
|
10-16-2007 08:02
*sigh* Michael. That which is necessary to one person may not be to another. Surely what Ann, or you, or I, thinks is necessary is none of anyone else's business. You are on the path to conflict here, why not just let it go?
If I had a roadkill AV, I would think it should be kept because it would be SO unique. A person whose whole gig in SL is to be attractive would think I was insane. Different strokes.
_____________________
.  To contact forum folks, join the inworld group "The Forum Cartel". New residents with questions about SL more than welcome! We has parties!  To contact forum scripters, join the inworld group "Scriptoratti" (thanks Void!). New scripter questions welcome!
|
|
Brenda Connolly
Un United Avatar
Join date: 10 Jan 2007
Posts: 25,000
|
10-16-2007 08:03
From: Michael Bigwig Ann--explain to me why (especially not being a content creator) you feel having thousands of assets to be 'NECESSARY?'
The last word is key. You aren't a woman, you just wouldn't understand.
_____________________
Don't you ever try to look behind my eyes. You don't want to know what they have seen.
http://brenda-connolly.blogspot.com
|
|
Colette Meiji
Registered User
Join date: 25 Mar 2005
Posts: 15,556
|
10-16-2007 08:11
From: Michael Bigwig Ann--explain to me why (especially not being a content creator) you feel having thousands of assets to be 'NECESSARY?'
The last word is key. Remember Imelda Marcos? Consider that FAR less women than men considered her 3000 pairs of shoes "excessive".
|
|
Michael Bigwig
~VRML Aficionado~
Join date: 5 Dec 2005
Posts: 2,181
|
10-16-2007 08:11
From: Nika Talaj *sigh* Michael. That which is necessary to one person may not be to another. Surely what Ann, or you, or I, thinks is necessary is none of anyone else's business. You are on the path to conflict here, why not just let it go?
If I had a roadkill AV, I would think it should be kept because it would be SO unique. A person whose whole gig in SL is to be attractive would think I was insane. Different strokes. I asked a legitimate question. If Ann wants to make a generalized statement like, 'it's necessary', then I should be able to inquire (passively) why she feels this way. I'm not trying to cause trouble--I think it would be a fascinating bit of information for me to absorb--the WHY someone finds inventory hording necessary. There are many reasons why I find it important to know... The word 'necessary' is a very strong one...which is why I wanted to understand her reasoning--you guys need to stop being so defensive. I mean well.
_____________________
~Michael Bigwig __________________________________________________Lead Designer, Glowbox Designs 
|
|
bilbo99 Emu
Garrett's No.1 fan
Join date: 27 Oct 2006
Posts: 3,468
|
10-16-2007 08:12
From: Michael Bigwig Ann--explain to me why (especially not being a content creator) you feel having thousands of assets to be 'NECESSARY?'
The last word is key. Oh Michael ..... dangerous .. very, very dangerous  A quick couple of lines on what my alt has taught me. When a guy goes out, he's out for fun. Moreover, to be the receiver of fun, amusement. OK, maybe a player/performer in that activity but one of the crowd .. the herd. Girls like the oneupmanship, to appear cooler and better than their peers. You must have seen parodies on the age old brides mother avoiding at all costs having the same dress as the grooms mother? For that effect, they need an arsenal of wide ranging, comprehensive components to launch that assault. I'm mebbe going to get a little singed here but heck Michael, most biological males share this view I'm sure 
_____________________
Be polite .. that newbie could be your next ex-partner.
|
|
Ann Launay
Neko-licious™
Join date: 8 Aug 2006
Posts: 7,893
|
10-16-2007 08:16
From: Michael Bigwig Ann--explain to me why (especially not being a content creator) you feel having thousands of assets to be 'NECESSARY?'
The last word is key. SL itself is not actually 'necessary.' However, if I'm going to play the game the way I want to play it, then my asset collection is indeed necessary to do so. It's a matter of opinion, like I said previously. Could I walk around with a couple changes of clothes, one set of hair, and a single skin? Sure, but it wouldn't be nearly as much fun for me. I like being able to change my av's appearance on a whim and the majority of my inventory reflects that. Yesterday, I was a lion and, briefly, a wyrmling. Today I'm back to Neko in a froufrou dress. Your priorities are obviously different, but that's the aspect of SL that *I* enjoy.
_____________________
~Now Trout Re-Re-Re-Certified!~ From: someone I am bumping you to an 8.5 on the Official Trout Measuring Instrument of Sluttiness. You are an enigma - on the one hand a sweet, gentle, intelligent woman who we would like to wrap up in our arms and protect, and on the other, a temptress to whom we would like to do all sorts of unmentionable things.
Congratulations and shame on you! You are a bit of a slut.
|
|
Michael Bigwig
~VRML Aficionado~
Join date: 5 Dec 2005
Posts: 2,181
|
10-16-2007 08:17
From: bilbo99 Emu Oh Michael ..... dangerous .. very, very dangerous  A quick couple of lines on what my alt has taught me. When a guy goes out, he's out for fun. Moreover, to be the receiver of fun, amusement. OK, maybe a player/performer in that activity but one of the crowd .. the herd. Girls like the oneupmanship, to appear cooler and better than their peers. You must have seen parodies on the age old brides mother avoiding at all costs having the same dress as the grooms mother? For that effect, they need an arsenal of wide ranging, comprehensive components to launch that assault. I'm mebbe going to get a little singed here but heck Michael, most biological males share this view I'm sure  We aren't talking about Sarah Jessica and her shoe fetish...we are talking about an overall hording of many different types of inventory assets. Everyone please relax, and put the guns down. Just because I want to better understand something or someone, doesn't mean I'm trolling. You guys are too quick to draw.
_____________________
~Michael Bigwig __________________________________________________Lead Designer, Glowbox Designs 
|