SL Protocol Reverse Engineering Team

There isn't a real proper forum for this, so apologies in advance since this is not directly related to LSL scripting but I'm hoping to capture the right target audience.

Right now there are less than a dozen people in Second Life who have working knowledge of some or most of the Second Life protocol and even fewer who have working implementations in code for portions of it. Linden support is increasing as the homebrew clients can pass their own identifier at the login stage, but getting any good official information comes down to catching the right person at the right time.

I've been putting in my own share of work on a reverse engineering endeavour lately, but there are a lot of different fields in each type of packet and different data storage types to solve, so I think putting more eyes on the problem will speed things up a lot. Also, I've seen people putting prices on protocol information for resale. It would be like someone discovering a secret LSL function and only people who pay the price of find it out themselves can improve their work with it.

A wiki has been started, and any contributions to it are covered under the GNU Free Documentation License. Any work you put in will not be hijacked and resold, it will become public knowledge. Right now it's mainly a sandbox for analysis and discussion, but I hope it will grow into an up-to-date spec sheet for the protocol, similar to what the LSL Wiki is for LSL (but better! hehe). You can be assigned full credit for all of your contributions or remain completely anonymous if you wish.

Right now the "team" consists of myself, a few people getting up to speed with the tools, and a few angel voices that are significantly further along than myself lending advice. If you want to contribute to the project send me an IM in game. One stipulation to working on the project is that any derivative work based on this research must be protected under the SL TOS section 4.1 and DMCA section 1201(f) exemptions that this research falls under. The work is difficult enough without wandering outside the legal safe zone.

Any chance of a link?

I was originally apprehensive to post the link since I'm still fairly new to being a sysop on a wiki and it would be a pain in the butt to play cleanup with free for all editing, but I'm learning a bit more how to revert changes and fix things up so here you go (if you make any edits please don't overwrite other people's work):

http://labs.highenergychemistry.com/slprotocol/

My questions are

What is the intended purpose of this reverse engineering?

What negative side-effects may there be by allowing peopel access to the raw data sent by SL?

Intended purpose is bring accessibility and build bridges to the Second Life world from places not imagined yet, for example web browsers and headless Linux servers. Once we're that far the possibilities are unlimited. There are already a couple of very cool working implementations that are improving the SL world as we speak, but I need permission from the authors before divulging any more details as to specifics.

As far as your second question, a quick clarification. We are not allowing people access to the raw data sent by SL; packet capturing libraries and the open architecture of computers do that. We're trying to document what is going on under the hood. If you are wondering whether bringing this knowledge to a wider audience will also bring existing exploits to a wider audience, it could be argued that exposing any existing exploits will speed up the process of getting things fixed, and make it more likely that a whitehat researcher will stumble upon an exploit instead of the typical Second Life cycle of "blackhat finds exploit, wreaks havoc, community erupts in anger, LL quickly tries to patch things up".

It's worth noting that this project is moving along very quickly. The entire exchange with the login.cgi script is decoded and implemented in code, and the login exchange with the sim server will arrive shortly meaning a fully independent SL client. Combine that with the decoding of some of more actions like teleport (sit was just finished), or the sim layout or land sales search packets (both on the wiki), and you can write some pretty interesting things.

(Shameless plug to try and recruit more minds)

Is this project legit, under section 4.2 of the TOS, or do I risk my second life membership by getting involved?

"You agree not to create or provide any server emulators or other software or other means that provide access to or use of the Service without the express written authorization of Linden Lab."

Very cool.

I know that if you manage to reverse-engineer the IM protocol, and script some kind of Jabber server or something to let one use normal IM clients, many people will be eternally grateful. People have been begging for an external IM and chat client for ages.

What would LL really lose by opening up the client (or at least the protocol)?

Since pretty much everything is server side, I can't see the harm in allowing other clients to exist.

Alternative clients could introduce bots into the system, that could be used both for Good and for Bad.

Bots could generate notecards for scripts -- some would consider this good, linden doesn't (otherwise they would not have made it clear that they don't want fully automated notecards.)

Bad uses include evil grid attacking systems that could hit all the private islands as well as the mainlands (a bot could be scripted to TP to all the islands, and drop stuff -- where right now it's a bit prohibitive to go TPing to all of them manually.)

Other "good" uses could include AV models, or "extras" in SL videos or RPGs...

Some other good uses include chat and IM gateways (although this may break the TOS, it needs to be investigated), automating the land market (input a buy order in to a bot with a variety of criteria and it buys land when a parcel meets the criteria), grid health monitoring from the web, cartography projects, and AI driven by a more powerful language than LSL, without the overhead of XML-RPC or HTTP requests.

Please be careful in your choice of what info you present.

Before you post anything put yourself in a criminal mindset and ask yourself what you can do with that information.

A wiki to help users is good, but it will do us no good if info that hackers and griefers can use gets handed to them on a silver platter.

Security through obscurity is not security at all.

If there are problems that can be exposed by publishing the protocol, then those problems can be exploited by anyone.

HTTPRequest and XML are both SL server services. They are slow. Let the LL servers look after the basic 3D, land management, building and financial aspects, and hopefully this will lead to everything else going client side. We gain faster sims (Less load on Server!), and faster client/alternate server/client interaction.

Maybe in future we build off world (not sure how yet), then the automated client will push the right SL server edit buttons and provide the SL server with a final build - less stress all round.

To be successful and be more than just a game for yuppies, SL needs to be much more professional in maintaining Sim uptimes, reducing lag, and fixing software bugs. They should provide the core, we should fill in the gaps with efficient clients.

Less core = easier to maintain security. And LL would in no way be responsible for what we do with our clients in the way of damage to each other and other web sites. One possible use would be for in world battles, with a second server forcing folk to stop cheating.

Projects like Gimp are already showing the way of the open source world.

Hopefully the Lindens will see it this way and support this project. If they don't they may not have a long term place in the 3d universe.

Just my $0.02 worth.

Cory Ondrejka talks in this http://www.lugradio.org/episodes/42 podcast about the fact that they have every intention of standardizing and opening up the protocol in the relatively near future, he seems very keen on the idea of people creating their own clients for a variety of purposes integrating IM systems provding roaming access from mobiles (cells ) )etc.

Maybe it would just be worth contacting him to see what he thinks of this venture, who knows he might even give some more info..

Be careful what you do, the idea of a open client is overall a good idea, it would enhace how we play the game as we know it. BUT with one good thing comes many problems. I can garentee, since SL is closed client, there are security holes that exist within the protocal that havent needed to be addressed since, after all its "closed" client. Also, knowing the protocal on the client side would thus enable (with a little work) one to create a "emulated" server. This is a very bad thing, for one, it will cause loss of membership and in turn loss of profit. This could be a bad financal move because no one likes $500,000+ loss of profit lawsuits. Just keep in mind, the line your walking is very fine.

Personally, I would much rather see the client remain closed. I like the fact that my subscription goes towards development rather than defense against malicious client design and combatting server emulation.

The road to hell is paved with good intentions.

The point that many people here are missing, that needs to be spelled out: THIS WILL HAPPEN ANYWAYS. People are already doing it, and have been doing it since the first day that LL's servers went live. By providing this information to everyone, it increases the chances that someone with good intentions can utilize the information in a way that is helpful in the whole, as well as providing LL with free security analysis at the same time.

In a world where weapons are banned, only criminals will have weapons.

By free security analysis, do you mean... "hey... the grid is offline... oooh, the asset server was formated... guess we better fix that" ...analysis?

I know, I know... that's rediculous. But hey... the llSetPayPrice bug is a perfect example of how an open-source client can be devestating to the SL community. Granted, scripting an error checking routine in the vendor using llSetPayPrice would eliminate the bug, but who would have known? And is that bug client side? Is it server side? The idea that something like that could be recreated with an open-source client is frightening. It's hard enough to catch all the bugs in the closed client... imagine trying to catch all of the exploits in an unlimited number of client versions.

We are not talking about a game like EQ2 where in-world money isn't directly associated with a USD cash value. We are talking about an interactive community where many members do actually earn a real income (however small or large) from participating. Openging up the code to any and all interested in manipulating that is in my opinion irrisponsible, reckless, and detrimental.

How much L$ / USD would be lost before someone realized a client was malicious and took steps?

No thanks.

If you want to provide free security analysis to LL for the SL client... then do that. Try to hack it and report the results to LL.

There's a big difference between somebody with the skill and commitment to do this kind of analysis (who's probably not interested in going into the prefab home or clothing business in SL) and a kid who downloads and installs a tool, points his camera at an object he'd like to sell, and rezzes a perfect copy. For one thing, there are likely to be hundreds of the latter.

Unfortunately, both let their excitement about doing something that sets them apart from others in SL (which is to say, their egos) get in the way of understanding the damage they would do to the economy.

But even if you can convince yourself that the availability of this information wouldn't change much, I wonder what would happen if someone built a useful tool that worked off the raw protocol (maybe even a better browser), sold five hundred or a thousand copies; and then Linden Lab changed the protocol so that the tool didn't work any more. There's a good chance that in the ensuing furor they would change the protocol back, making it that much harder to implement new features and performance enhancements.

My though it, please wait for Cory.

--Allie

I would much rather see a version checking protocol added to the session login to verify the version of the client, and deny access to modified versions of the client.

I have nothing against open-source software... when such a license is within the realm of proper context. SecondLife is not my idea of proper context.

Let me post and bold the sections that hold you back from completing or even starting this project.

----------------------------------------------------------------------------------------------------
4.2 You agree to use Second Life as provided, without unauthorized software or other means of access or use. You will not make unauthorized works from or conduct unauthorized distribution of the Linden Software.

Linden Lab has designed the Service to be experienced only as offered by Linden Lab at the Websites or partner websites. Linden Lab is not responsible for any aspect of the Service that is accessed or experienced using software or other means that are not provided by Linden Lab. You agree not to create or provide any server emulators or other software or other means that provide access to or use of the Service without the express written authorization of Linden Lab. You acknowledge that you do not have the right to create, publish, distribute, create derivative works from or use any software programs, utilities, applications, emulators or tools derived from or created for the Service, except that you may use the Linden Software to the extent expressly permitted by this Agreement. You are prohibited from taking any action that imposes an unreasonable or disproportionately large load on Linden Lab's infrastructure.
----------------------------------------------------------------------------------------------------

...all in all... BAD IDEA... but hey... its not my account... not to mention the accounts of all the users of your "client"... of course unless your get written autherization first

It's already there you should go check out the notes on the protocol wiki. Unfortunately with computers being general-purpose platforms and not consoles designed to play Second Life, it's very trivial to produce _exactly_ what Second Life sends to the server byte for byte. You can even trick the official client in to sending the login details to a local rogue client, which can terminate the official client and forward the login details to the server. Or it could keep the official client alive for the entire exchange, acting as a middle-man. MitM attacks are successful against TLS if you can convince Alice to connect to Eve instead of Bob, and vice versa and then start separate TLS exchanges with both Alice and Bob. Generally certificate authorities would protect against this but the official SL client could care less if the login server it connects to is even using SSL, most likely due to the code being using some library for it's transport layer, at least on the login.

Carlos: Thank you for the legal advice, but my counsel points out that contract law does not supercede federal law, and this research is still protected under the DMCA section 1201(f). Not to mention that clause of the contract is loosely enforced if at all. It would also ban Jeffrey Gomez's blender script or BVH importer if read with the same interpretation, and why have the Lindens been helping people create their third party clients for some time now?


Back on a brighter note, I hacked together a very simple C++ XML-RPC parser. It's actually just two functions, one to retrieve a string given a name and one to retrieve an integer given a name, but it's all you need to parse the login sequence. Combine that with manual assembly of the outbound XML and a lightweight transport library like OpenSSL (I'm using libcurl right now to speed up the prototyping) and you have a complete login.cgi handler. For the next step of connecting to the sim I downloaded an easy C++ socket wrapper, but when things get redesigned in to a library this will become direct asynchronous socket code (with #ifdef's to support each individual platform). The FreeBSD client is coming! Just imagine...

"You are standing in an open sim west of a white house, with a scripted front door. There is a low prim mailbox here."

Under United States law, reverse engineering a patented item can be infringement; however, if the artifact or process is protected by trade secrets instead of by a patent, then reverse engineering the artifact or process is lawful as long as the artifact or process is obtained legitimately. In fact, one common motivation of reverse engineering is to determine whether a competitor's product infringes on your patents.
Reference: http://en.wikipedia.org/wiki/Reverse_engineering