SL Protocol Reverse Engineering Team

Perhaps some comments on this project from Philip, Cory, Andrew, and other Linden staffers would be good, in addition to the one by Phoenix.

I was suprised by Phoenix's post as well, but given that the spec comes with the client and Phoenix's post, I suspect we can all safely assume that this is all done with LL's blessing.

That being said, this will create a lot of challenges for LL and the community. However, as the Chinese know so well, challenge is just another word for opportunitiy.

Sorry, but I think its going to take a little more than one Linden staff member saying on the forums "we wont come after you guns blazing if you reverse engineer our IP and you don't do evil with the information".

I have a mate who works in a supermarket, he's just told me its ok to take anything I want, does that provide me any protection when I get arrested for shoplifting?

This has to be an official policy decision, it has to be covered by the TOS. Anything less is simply DOA.

To those who can't read he didn't just say we won't come after you.



Huge difference between welcome and resigned to. You may not like this project and that's your perogative. But to claim LL doesn't simply shows your ignorance. Phoenix has said LL welcomes it. Philip has talked about the long term goal of going open source. And this discussion is being carried out in plain sight on Linden forums. If LL didn't like this project they wouldn't need you to say it for them.

ninjafoo: You bring up an interesting point. You friend isn't authorized to make such a statement.

LL has gone though a number of NeedToKnowInformation catastrophies, where information was leaked that shouldn't have been leaked (how do you think it became public knowledge that LSL is migrating to Mono?). Phoenix is one of the oldest Lindens; I'm sure Phoenix has been breifed on leaking information; and likewise has been breifed about keeping the party line. If Phoenix says it, it's most likely authoritative & authorized.

It appears that the RE team is running into a solid wall of mmu.nfp (minds made up. no facts please.)

There's no arguing with that. Save your energy.

I think its understandable that some people are worried, even if it is unfounded.

As someones already stated the protocol itself is nothing more than a description of how messages should be sent to the server.

Do you feel comfortable emailing people? Viewing webpages?

In both those cases the protocol is clearly defined, its open to all to see in the form of RFC's. The result of this is that there are a great many different clients and also servers. And in both cases issues with performance/hacking/etc generally only occur through human error, and usually only when that error has occured at the server end. A well crafted email server will reject email that doesnt conform to agreed protocols.

There are so many good things that can be done with the protocol in an open form. For example: /invalid_link.html
or even as someone has mentioned NPC's that dont need to use excessive amounts of in game scripting.
An IM system that would allow your character to be logged in and interfaced to an external IM program.
Or even be able to log on to a webpage thats linked directly to your character thats standing in a shop, see your most recent transactions, read and respond to IM's, generate traffic graphs based on people entering and leaving the store, cross reference with those that spent money and what they spent money on.

To a degree these things are already possible, but not without heavy amounts of scripting.

I believe LL have seen a way of bringing more people into SL and also greatly enhancing the second lifes of those who have already joined, and the only thing they have to worry about is making sure that there are no bugs hidden away that may or may not exist, and by working closely with the people that are giving their time to work through it, LL are going to get most of the hardwork done for them.

Again I can understand people worrying about this, but try and see what it might be able to do for you once its accomplished. There are almost infinite possiblilitys.

Actually, as far as I have seen in here, the folks on the RE team are about the only ones operating with all the facts. :-/

Will it lead to some people misappropriating assets? Probably. More than who do it already? It's possible. However, LL already has a mechanism in place to allow Residents to deal with the problem, despite the RE Team's efforts.

Regardless, if someone copies your stuff, you can more easily verify such actions, and report it as a Copyright violation, seeking redress against the offender.

After the reverse engineering is done, and programs are written to make use of the information, does Linden Labs have an obligation to keep all future protocol changes backward compatible? How much warning must they provide if they are making changes which might break one of these third-party products?

Do you have a notion of how much it would tie their hands to have to freeze the protocol spec?

The intentions of this RE team are not at issue; but once the information is discovered they won't be able to keep it to themselves (who could appreciate their true genius if they did?). Someone will use it to build a tool for circumventing in-world IP protection, and make it generally available.

Better start thinking of counter-measures now.

Or maybe IP is not an issue for you?

Oh, wait. We already know the answer to that. Never mind.

The tools for getting at inworld IP already exist; glIntercept and OGLE are but two - there's others too, like those that look inside the SL cache, etc.

The client library wont be dealing with asset downloads at the moment, it's just for simple tasks -- and the SL protocol is supplied alongside SL itself; so it wont break when LL changes it; at least not unless they do a complete rewrite (which they have indicated, and that's fine, let the library break)

I'll add to that - yes LL does need to look at countermeasures for IP theft; because there's a whole class of problems here which are already an issue now. The best route is the real world route - make it easier and more efficient to file DMCA claims against IP infringement.

Because everyone knows how well technological solutions to social problems work. Copyright (and other IP) infringement is a social problem, there's no silver bullet for a technological solution which does not drastically cripple how we use SL today.

Final point -- all the "reverse engineering" has been done. SL itself provided everything anyone would need to make a new client in the comms.dat file (no real reverse engineering required since it's all in a fairly plain-as-day format). Reuben/Eddy has already posted the details on how to open that file, and what's stored inside of it.

It's now just a matter of using that information to build a ground up client library.

Having the protocol is irrelevant to IP theft, as has already been stated objects and textures can be aquired from the opengl subsystem anyway and textures can be grabbed with screenshots needing very little expertise.

Perhaps it would make it easier, but then again what if someone decided to run a service where by you could register and update creations, that would use the created date as a reference, that could easily compare objects immediately to see if they were exactly the same and would allow visual object comparison quickly and easily.

Ultimately the same could be done for textures.

Perhaps that wouldnt stop the copying but at least it might lead to some sort of mediation and offer to a degree a proof of creation.

That would work well from the angle it would at least provide some 'proof' in regards to time of creation. Only problem might be, for legal recognition, the person running it may need to be some form of notary.

As SL becomes more and more like the world wide web, policy governing its content will also start following a similar path. Right now, you could rip an entire webpage by simply clicking File > Save Page As... in your webbrowser. The source code composing a webpage is publicly viewable by right clicking and selecting "View Source". Publishing that webpage on your server as your own is what's illegal. Copyright laws (as much as people hate them today) were created to prevent this kind of destructive republication. They will become increasingly more important as SL gets larger and more profitable. (and are what makes content stealing irrevelant in the context of protocol reverse-engineering)

Shooting this project down is a step backward IMO. The SL client is to the SL world is as a web browser is to the world-wide web. SL residents will benifet in the long term because of the opening of the protocol - take the Firefox vs. Internet Explorer browser war as an example. A new client that addresses the shortcomings of the current client is a goal we can now strive for due to the opening of the protocol.
==Chris

This app isn't even necessary since simply looking in the SL cache directory reveals all recently edited scripts (yours only) anyway (and recently played sounds and rezzed objects--that aren't necessarily yours either). http://secondlife.com/badgeo/wakka.php?wakka=fileformat for more info.

Including every word of yours, Dianne. Look in the mirror and see your own egotistical self. As with anything, this reverse-engineering can be used for "right" or "wrong". What's "right" to one person may be "wrong" to another person. It's all relative. If you don't want to be associated with this, move on.

While I don't want my content ripped off (like it was in Active Worlds), there are just some things you have to accept in life: people WILL copy your content whether you like it or not. You just have to be better than the copier and people will know who the TRUE artist is.

Psst, see the "Edit" button? Try it sometime...

Er, no. Only textures/images can be aquired from the OpenGL subsystem (i.e. GLIntercept). Objects are still safe. Sounds are already in the cache (as are objects but they're in a propietary format, unlike sounds).

OpenGL renders objects based on given dimensions and points. This means that objects dimensions have to be sent to the client to be rendered when an avatar is within viewing distance.

I dont imagine it is as easy to recover as textures, but surely there exists a way to recover that data from OpenGL?

Getting the textures is trivial, really really trivial.

Yus, I'll reword

I dont imagine recovering object data is as easy as recovering textures, but surely there exists a way to recover the object data from OpenGL?

I thought I had seen something like that

So the protocol as a meaningful library would simply give another way to achieve something thats already pretty simple anyway.

I think that the work this team are doing is going to allow some great services and programs to be created. I honestly think the good that can be done far outweighs the bad.

And as for LL changing the protocol in future releases, I would guess its guaranteed to happen, but any changes are likely to be trivial once the library has been created. I think that LL will happily work with people to keep a library up to date if services and software that are bringing LL revenue have been designed around it.

Since the protocol map is dynamically generated from a file each time Second Life or libsecondlife load, any time there is a protocol update you just grab the latest comm.dat from Second Life and feed it to libsecondlife. For a protocol update to break libsecondlife, it would require the addition or removal of adata type, a very significant number of features added or removed from the spec (this will throw off the keyword ordering), or a change in the comm.dat file format. Any of these changes would be a significant modification to both the client and servers, and Lindens are already letting us know of planned changes down the road so we can prepare. Luckily for our development efforts I think those major protocol changes are on the same roadmap as the Mono scripting backend and Havok 2 .