Any interest in a XML-RPC-based object tracking system?

During the development of the first major service offering of the SL Exchange I realized I have created a system that may be of interest to other developers, and I am just tossing this out there to see if my feeling is correct.

I have created a completely automated and self-correcting service where an external database keeps track of certain objects in Second Life. Current object data that is tracked and kept up to date is:
-Object name
-Status
-Last Update
-Region
-Coordinates
-Object type
-Hash key for secure and optional encrypted communications (security routines still in development)
-XMLRPC channel

On the LSL scripts side of things, the system is built on top of a OO-like standardized communications framework I have come up with that can (and should) handle communications for all scripts in the object or prim. This system handles all outbound email, in- and out-bound xmlrpc, and listens via a standardized protocol of tagged link messages between scripts.

Objects automatically initialize or update themselves with the server on rez, on "Object Tracking Module" script reset, and optionally on inventory change.

The server keeps track of objects by sending a "ping" to each one after 4 hours of inactivity ("inactivity" being how long since the last communication with that object). After 8 hours of disconnectivity (2 failed pings) an object's status is set as inactive and after 7 days of disconnectivity an object is deleted from the database outright at which time polling for that object stops. I still have yet to test this system with large numbers of objects, so I have not yet finalized my timeouts and such. Since I will be multithreading the server process that polls objects, I can probably get the system much closer to real-time.


It would be a fairly trivial task for me to create a web service or similar communications method to allow others to utilize this functionality for a fee. Is this something that you are interested in?

*Due to time constraints of SL Exchange development, this service will not be available for external use (outside SL Exchange) until at least a couple months from now.

Apotheus do you think latter that could be intergrated to a Map of sorts. Ie if you have all your objects intergrated into your system. Since the system keeping track of its location in world could there be a map developed where you could put up flags that indicate where you have items in world?

Yes this is definitely possible since all the required information is already in the database. I don't personally have much experience with programmatic image manipulation so I'm not sure if I would be the right person to create a mapping system. If I remember correctly I think Alpha Zaius has already done something like this for his vendors so it may be possible to integrate the two.

Kewlies, might have to get with you in a month or so.

Shadow

Woot thats an awesome idea Apotheus, great job!

I was playing with the idea of implementing a similar tracking ability into the piggy back rides, since people tend to leave them out constantly, but after reading your post, I think your service can take quite some work off my shoulders =)

Let me know when/if you'll decide to actually offer this service to the public.

Apotheus, that is crazy wierd... Just did similar setup myself, and have been doing the OOP'ish program modules for quite some time (and now really pays off with the network bit). I guess there is other nuts out there with the whole module programming concept in LSL.

Well yes, modules and code reuse makes life easier everywhere else, why not in LSL?

How much damage does this system do to the sim, computationally? If you use a seperate wait for each object, then you can easily hit a hundred active loops... and that probably cuts into sim computations pretty heavily. Have you run any tests?

-Ansi

Well actually...

Performance has been one of my primary concerns along with basic functionality and security. There are no timers, loops, listen's, sensors, or otherwise "active" elements whatsoever. Also all linkmessages (with the exception of the optional inventory change detection module) are sent only to the prim in which the scripts reside. I racked my brain for quite some time to figure out maximize performance because I easily envision thousands of these objects existing in a single sim for the SL Exchange project alone.

Each object will have an open XML-RPC channel, and I have not yet heard if these have any impact on sim FPS / run tasks (I really hope they don't). I will do some stress testing once the codebase is a little more mature (e.g., in a couple weeks). I am still working some kinks out at this time.

I'll post performance numbers in this thread once I have them.

I'm very impressed. It's good to see someone who thinks about these things.

-Ansi

Okay I've gotten everything running pretty stable and decided to use Moopf's sim (thanks Moopf!) as a torture victim, er performance test bed. My initial thought was correct.

0- 6300
32- 6200
64- 6300
128-
256- dipped to ~5000, then back to 6300
512- 4000-6000 (initial update running for 256 objects)
1024- 3500-6200 (initial update running for 512 objects)



Initial update can take as much as 10 minutes at which time the scripts are active while they check in with the server, update inventory, then the server runs its first update on those objects a few minutes later. After this, the update happens only during certain cirumstances and not all objects at once.

At the 1024 level I realized I may have borked the LL mail server by giving it 512 emails essentially in the timespan of about 2 seconds. I checked registrations and they stopped at 568, then a few minutes later started trickling in again so yep, I made it queue. There will never be a situation like this in production so I'm not particularly worried. The remaining emails took about 5 minutes to come through.

So, the fact is that the tracking system uses a minimal amount of resources. My server handled all transactions without breaking a sweat, which is excellent news. Also found out along the way that having an open XML-RPC channels does not affect run tasks / sim fps. Woo!

Very cool results Apotheus

Maybe put an llSleep(llFrand(60)) in your on_rez event, help smooth out the load so there's not this big spike?

Apoth, great job man. This endevour soundslike a great contribution to SL... I'd love to see it live in-world sometime!

I had an idea... do you think it would be possible to create a list of all the avatars on Second Life at one point in time using XML-RPC? (or at least the mainland)? If you could get a little sensor object in every single simulator, it would be pretty easy to create a DB that stored all the picked up names in different lists for different sims. Then, you could have a central object, where you could request the names of all the avatars in one simulator, and it could query that information from the database. Has anyone done this already? I think it'd be pretty cool : o )

I'm not sure if this is still active, but Ezhar Fairlight, author of Scite-ez editor, has been collecting some interesting SL statistics, including avatar names, birthdates, etc... on his website.

His plans were to provide a service that allows sending IMs from a webpage into SL and also to provide a Name2Key service to scripters at some point.

As I said, I'm not sure he is still collecting and maintaining this data, but its definitely worth a look

And sorry for going out of topic...

Sounds very useful, but wouldn't llName2Key be in conflict with the TOS? If not, I still think it would be an opportunity for annoyance. For example, with this database, you could script something to IM every single avatar in SL by key.

Hehe.. I just replied to your own idea, and Ezhar has no intentions to offer anonymous IM spamming as far as I know

Yeah, I was going to do this unfortunately I'm pretty sure LindenLabs will do this integrated into the Find interface fairly soon.