Questions posed by Cory re: communication

in this thread: /invalid_link.html



a) L$0 up to a certain rate-limited quantity and then some L$/KB to go over that. Option to select whether to go into pay-zone or just have transactions stall.
b) Objects should be able to communicate with each other and whether they're near each other or not should be transparent and of no concern to the developer.
c) It is better than email for what I want to do. Please don't use that other stuff, XML-RPC is designed specifically for B2B stuff and everything supports it and there are zillions of code examples.
d) what

Nope. That would be ideal.

Cory means (I assume) registering the receiving address before you can communicate with it.

b) How important is object-to-object communication within a simulator versus between objects in different simulators? How about neighboring simulators?

All three are important, but I'd give them different requirements for latency and throughput. In-sim needs low latency and high throughput so we can create interactive systems that span a whole sim. Cross-sim comms can be slower IMHO, as they typically won't be as interactive. If there have to be two different mechanisms to support both kinds of comms, I can live with it. As long as I can build both a sim-sized Pac-Man game and a doorbell that will beep me from home no matter where I am in SL, I'll be happy.

I agree with most of this, but I'll answer the questions myself as well anyway...



A) Definitly pay in L$, not everyone is able to pay in US$ for things, it may be better for you as a company for more profit, but its worse IMO for developers. Perhaps 1-10 L$ per KB over a set limit... I'm not really sure about this.

B) I'd say they'd both be equaly important, perhaps with objects in different simulators being *slightly* more important since we don't have many options for this currently, but they both are very important. Wouldn't Neighboring simulators count as different simulators?

C) RPC seems to be a pretty popular system, RSS would be neat to have, but RPC I think would be a good primary.

D) Even with cua's explination I don't fully understand this.

a) How much would you pay per KB (over some base limit) of data moving between the system and the real world? Would you prefer L$ or US$?

I'd pay $.01 per kilobyte if you wanted it in USD. I'd pay, I don't know, L$10-100 per kilobyte if we paid in Linden.


b) How important is object-to-object communication within a simulator versus between objects in different simulators? How about neighboring simulators?

It's important, but could be circumvented if we had XML-RPC or an API. Object-to-object communications, to me, are about four on a scale of one-to-ten strictly because the majority of things I would do with that kind of technology could be done, and done better, by accessing the web.

My opinion may change the more I read about Smalltalk and Erlang, and thus begin thinking about building software that works by distributing itself across several processes. However, for right now, as a LAMP developer, XML-RPC feels like a nice compromise.


c) Is outbound XML-RPC the best protocol for you? What about REST APIs or RSS?

XML-RPC is the best, yes. C#, Python, Perl, PHP, Visual Basic, Ruby, and just about every other language out there supports XML-RPC (either by itself or via SOAP). Using REST or RSS would be taking two steps backward at this point, since you already have XML-RPC half working.


d) Would having to whitelist outbound traffic bother you?

Do you mean: whitelist (WYT.list) v. To place a name, e-mail address, Web site address, or program on a list of items that are deemed spam- or virus-free.

It would really depend on the criteria for your list. I wouldn't mind putting my two or three web accounts/domains in a list, but at the same time, I would still want to be able to have an XML-RPC script/client on my home PC be able to touch Second Life. Before I can, personally, answer this, I'd need to know more.

thank you cory for answering a question and for asking for our input. and thank you huns for getting this thread up so quickly.

a) whether you mean amount of data per call or number of calls a second, i'd rather pay L$ over a base and the base per doesn't have to be very large.

b) inworld o2o is very important and should work by key regardless of location on the grid. we have to maintain the perspective that "the world" is a collection of servers communicating with each other just like any other set of servers on the net meaning that o2o could be accomplished through outside server calls. but i still hope that sl will become a viable w3b platform one day and it will make sense to have a proprietary, internal communication protocol built into lsl.

this is a scalability issue that, i'm sure, ll has already considered. objects residing in the same sim should communicate using a loopback address and the same addressing scheme they would use if they were in seperate sims.

c)use xml/rpc.

d)having a whitelist is fine by me.

a) I wouldn't pay at all, since it's most likely that I'd be using XML-RPC in projects for my own personal amusement, i.e., not in a way that would allow me to recoup the costs of using it.
b) I'd say it's fairly important.
c) I'm not familiar with REST, so I can't say, but I thought RSS was basically only for content syndication. In any case, I'd echo Jarod's sentiment that you're already halfway there with XML-RPC, so why start over now?
d) If it meant that I wouldn't have to pay for traffic, sure; otherwise, I guess I don't really care one way or the other. I'd probably only be using XML-RPC to communicate with my own server, anyway, so I don't think whatever whitelisting/registration mechanisms would be required would be too much of an inconvenience for me.

Some clarifications where needed:

a) Yes, all thoughts on L$ cost presume that the owner will be able to choose between paying L$ for traffic or paying L$ for traffic up to some weekly limit and then stopping the traffic.

b) Non-localized data flows are expensive in a decentralized system like SL. Therefore, it is much lower load on the system to communicate with an object in the same simulator than to do a system-wide lookup. In a similar fashion, neighboring sims are also cheaper to move data between. Thus, much like a), moving data between arbitrary objects in SL that aren't near each other may need to have a L$ cost associated with it (above some limit).

c) XML-RPC wouldn't go away. Protocols like REST or RSS would be in addition to it.

d) Yes, whitelist means that you associate legitimate outbound addresses with your account. Outbound XML-RPC (and/or others) not allowed to non-whitelisted addresses.

a) How much would you pay per KB (over some base limit) of data moving between the system and the real world? Would you prefer L$ or US$?

Not much, L$ only, and only for rate-limiting purposes. L$1-10 per KB max.

b) How important is object-to-object communication within a simulator versus between objects in different simulators? How about neighboring simulators?

In a general sense I think they are all equally important. Portable, avatar-worn devices for example are going to move around and not really be able to pin down as one of them day to day. It would not be good business to sell someone an object relying on communication but it breaks when they go to the sim the 'home base' is in. (within simulator communication)

c) Is outbound XML-RPC the best protocol for you? What about REST APIs or RSS?

Yes, as long as we get a full client. Do not limit the methods we can call! Do data type conversion automatically! I understand the need for it for inbound (as we are really not calling the script directly, but a proxy that generates events for the script).

d) Would having to whitelist outbound traffic bother you?

A bit. For self-written projects it would be acceptable to require that the endpoint implement a method (such as llAllowRemote() that simply returns 'true') in order to validate that I don't mind the calls. However for flexibility it would be lovely to call arbitrary endpoints; one of the best examples I can conjure up would be in-world blogging, posted via the MT blog APIs. In such cases there would not be the level of control needed to implement the custom method without resorting to a proxy.

Regarding Intra-Sim versus Inter-Sim communication, I think it's very important to have very low latency comms in SL even if it restricts us to using a single sim for our project.
Something like an LSL wrapper to standard unix FIFOs (named pipes) would be nice and easy to use

I think it would be better to account by hour than by week, so that if someone designs a system that becomes unexpectedly "popular," they won't find that 100% of transactions have stalled or been denied if they have to take a couple days away from SL.

I find this discussion interesting, but don't (yet) understand enough about XML-RPC to understand its full implications. Would someone mind explaining in child-speak to me what the dangers of not using a whitelisting system for communication? Can you give me some examples of how XML-RPC protocols might be misused if people didn't have to get outside addresses approved? Is this to protect the grid, or to ensure that LL can still control the direction that the SL experience takes?

L$1 per KB is far too high. A fully loaded email under this situation would cost L$4 each. RL cost per gig is around US$0.99 from most colo's - and much cheaper if you are buying from ISP's directly (which LL is doing). L$1 per MB would be a bit more justified.

Whitelisting would be perfect to prevent DoS attacks, and I'd prefer a protocol change - but instead of making things more complex and higher level, how about plain old HTTP with the ability to specify POSTVAR's in a list. (please dont predefine the format of the transaction)

Plain HTTP has some excellent features:
- You could implement XMLRPC over the top, if you had a cranial deficiency.
- SSL
- Simple, and implemented on everything known to man.
- Minimal overhead. (a fully loaded XMLRPC query in SL's format spends more than half of the data in just the XML parts)
- Lowers setup costs, dont need shell access to a machine to implement scripts using it.

-Adam

Example: I create some random toy that's the next big thing. I hide in it a secret little timebomb that, when it goes off, sends a bunch of bogus XML-RPC requests out. Everyone who has my object suddenly gets hit with a huge charge for outgoing communication that they didn't expect. Since the script is no-modify, they couldn't even tell the code would have done that. In the end, it's a pretty necessary protection so that I can't empty someone's bank account just by handing them an object and having them rez it.

In that vein, I totally would be OK with whitelisting. It'd be nice if it was really simple, something like llRequestPermissions is, so that non-techie users can handle it

I'm going to try and not be, as I have been called before, "hate filled and bile spewing" when I say this... Please get XML-RPC FULLY implemented before you do anything else. I understand the urge to add a lot of features, throw things against the wall to see what sticks, but please get one option fully working before adding more.

We -- the developers, the community, the sales people -- need faster searching capabilities, and the simplest way to do this, right now, is to be able to couple llListen() with an external, 3rd-party SQL query. RSS and REST will be nice, but for now XML-RPC will more than suffice.

Cory, thanks for replying to my post!

Before I answer the questions at the end of your reply, I'd like to clarify the question about the efficiency of listening on a high comm channel. In one of those two threads I linked, people said that it seemed not to matter which channel you listened on, whether or not it had any traffic, a listen on any channel affected the sim in the same manner. This was somewhat surprising to everyone that heard it, but the guess was that all chat is stored in the same queue, so every message had to at least
have its channel checked against every listen. Is that true, or were the tests flawed?

Now, the questions...

1. Quite willing to pay, within reason (less than 10L/Kb). Preferably, the first n Kb would be free.
   
2. I haven't done any projects that span multiple simulators. I know others have really wanted to be able to store things in one "server" script and access them from all over the grid. Neighboring sims doesn't seem particularly useful in light of wishing for full-grid communication. Within-sim communication is likewise overshadowed, and it's also less of an issue because we have methods to do that already via listens (albeit inefficiently).
   
   In the end, I'll take any one of these if I can get it, though the projects I've done so far would most benefit from fast in-sim communication.
   
3. nothing to add here
   
4. I can deal with whatever protocol you give me, although email doesn't really count. XML-RPC is probably best and sufficient because it's a nice buzz-word, so more sysadmins and free hosting sites will be likely to allow it.
   

Just a note, you can do a doorbell like this, I've already got one for my store. Your script can use a dataserver request to find out if you're online, and if so, send you an instant message.

HTTP as an option is okay, but XML-RPC really should be supported directly.

I don't know why you think XML-RPC is for someone with a "cranial deficiency." It lets all kinds of different systems talk to each other without all the going back and forth to try to invent Yet Another IPC Stack.

My thought on this subject really depends on if you are talking about paying and/or rate-limiting only for outbound or if you intend to implement it for inbound (the current XmlRpc implementation) as well. In the case of SL Exchange, there are a whole lot of small packets flying around to keep the website mostly in sync with what is happening in-world. I can deal with the current system as it is, but if a price is set on our current functionality as well, then the quality of the service may suffer.
I personally don't think there should be a distinction between location and inter-object communication. It should "Just Work."
Whatever is done, it should be made consistent. It doesn't make sense to have inbound communications via XmlRpc and outbound be something else (*cough* email), unless XmlRpc is deprecated and an entirely new system is implemented that handles both inbound and outbound.
Not even a little, but I think if you go to that extent, then we should have a [mostly] complete socket library at our disposal in LSL. The current "systems layered on systems layered on systems" approach to communications is about 2 degrees too redundant for my tastes. This became evident when I tried to make a bona fide client/server database app some months ago.

The quality of service at SLEx has already suffered immensely - thanks to you!!

And you're about 360*x2 redundant for my taste!!

Get over yourself dude!!


But hey, have a nice day in the mean time!

I agree. Even a seemingly small fee of L$1 per KB would pretty much kill large external database-driven services or at least make them prohibitively expensive.


I agree also. This is a perfect example of where keeping the system as simple, bare-bones, and open as possible will benefit the community at large. I would prefer raw socket access (as mentioned in my post above), but I would still be quite ecstatic if given plain HTTP.

Others can provide better feedback on where to set the rate. I would probably use anything that they would. However there need to be control over who is charged. Owner or creator.

Or even some third party, with permission (say a charge-to field, no-mod, in the object). For example, someone invents a really cool doo-dad which uses real-world communication. They sell wholesale to an investor, who decides to resell, but bundling the communication and server charges. The investor (neither the creator nor the eventual owner) should be able to get the bill for communication.

I don't want to have to worry about whether Object A and Object B are straddling a simulator boundary. If non-adjoining sim communication is more expensive, charge based on the SL distance between the objects.

In general: more respect for real-world physics, even when it's not that strongly related to the implementation cost.

No US$ charges, please!
I agree with the L$1/MB charge.

I agree with those who believe that the object-to-object communication interface should just work reguardless of where each object resides.

As long as you do NOT take away developer time from finishing XMLRPC to implement these alternate protocols. I dont want to be stuck with a half-baked implementation of RSS/REST while still stuck with a half-baked implementation of XMLRPC for another six months.

No.
==Chris