Welcome to the Second Life Forums Archive

These forums are CLOSED. Please visit the new forums HERE

Dynamic DNS service for HTTP-IN via Google App Engine

Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-03-2009 12:19
NOTE: While the original version of this code did not provide true DNS service, it has since been updated to do so. Please read the whole thread to understand the evolution of the code.

I had posted this to the library forum, but I'm not sure anyone gets around to approving those anymore, so here it is:

-----------

One of the big issues with the coming HTTP-IN is the fact that the URLs are dynamic and can change under many conditions. Many have balked at the requirements of having an external server to manage a Dynamic DNS service to keep track of these ever changing URLs. So here's a free solution that anyone can use, and expand with their needs.

First, download the files here: (i tried pasting them but the forum doesn't like the SQL-like commands)


http://home.comcast.net/~volfin/lsl-dns.zip


Now, what to do with these files:

Go to http://code.google.com/appengine/ to sign up for a free account.

Go here to install Python 2.5 (if you don't already have it)
http://www.python.org/download/releases/2.5.4/

download and install the App Engine SDK:
http://code.google.com/appengine/downloads.html
The SDK will be installed into a subdirectory called 'google_appengine'.

create a directory under the 'google_appengine' directory called 'lsl-dns'

Place the given dns.py and app.yaml files in this new directory.

Edit the given app.yaml file to the application name you specified when you signed up for the service:

application: YOUR-APP-NAME-HERE <-- Change this

From the 'google_appengine' directory (where the SDK was installed), run this command:

appcfg.py update lsl-dns/

This will install the application. You're done.


Once it's installed you can start using it:

http://yourappname.appspot.com/?type=add&name=[NAME]&url=Adds a new service named [NAME] with the HTTP-IN url [URL].
The URL must be converted to a string using llEscapeURL() first.
if the URL is added, returns the response 'Added', or 'Found' if the service
already exists.

http://yourappname.appspot.com/?type=remove&name=[NAME]
Removes the given service named [NAME].
Returns 'Removed' if successful, or 'None' if the service wasn't found.

http://yourappname.appspot.com/?type=update&name=[NAME]&url=[URL]Updates the service named [NAME] with the HTTP-IN url [URL].
The URL must be converted to a string using llEscapeURL() first.
If the URL is updated the response 'Updated' is returned.
If the service doesn't exist, a new service is added and 'Added' is returned.

http://yourappname.appspot.com/?type=retrieve&name=[NAME]
Retrieves the url of the given service named [NAME].
Returns the URL if the service is found, or 'None' if the service wasn't found.
the returned URL must be converted to a URL using llUnescapeURL()

http://yourappname.appspot.com/?type=list
Lists the available services currently stored.
Returns a Comma seperated list of service names, ending with the word 'END'.
if no services are defined, 'Empty' is returned.

------------------------------------------------------------

Once a service has been entered, you can use the serivce name in the URL to do automatic redirection.

Example:

You establish a service named 'intro1'

The URL for the service is:
[url=]http://sim3015.aditi.lindenlab.com:12046/cap/3ff4f3f2-ea08-76c1-cef6-a22b4a573a7c[/url]

By Navigating to [url=]http://yourappname.appspot.com/intro1[/url]

The page will be automatically redirected to the service URL:

[url=]http://sim3015.aditi.lindenlab.com:12046/cap/3ff4f3f2-ea08-76c1-cef6-a22b4a573a7c[/url]

If the service name used in the URL is invalid, 'None' is returned.

------------------------------------------------------------

Using this DNS, you can have servers register their URL 'service' and allow other in-world (and out of world) applications discover the URL for this service in a consistent manner. The Google App Engine is free and offers resource limits which should be able to support small to mid range applications. Of course, for a small fee you can purchase more resources from Google as well.
(Resouce limits: [url=]http://code.google.com/appengine/docs/quotas.html[/url])


This is the basics, and has no verification or encryption, but should be fairly secure as long as you don't go around giving your service URL to people. Feel free to add on to it as you like. The code is released to the Public Domain. Enjoy.
_____________________
Kelly Linden
Linden Developer
Join date: 29 Mar 2004
Posts: 896
06-03-2009 15:34
The LSL code for an object that wants to maintain a service url will look something like (completely unverified, sorry):

string url = "";
string service_name = "my_service";

setup()
{
llSetObjectName("HTTP Server";);
url = "";
llRequestURL();
}

default
{
state_entry() { setup(); }
on_rez(integer n) { setup(); }

changed(integer c)
{
if (c & (CHANGED_REGION | CHANGED_REGION_START | CHANGED_TELEPORT) )
{
setup();
}
}

http_request(key id, string method, string body)
{
if (method == URL_REQUEST_GRANTED)
{
url = body;
llHTTPRequest("http://yourappname.appspot.com/?type=add&name=" + service_name + "&url=" + llEscapeURL(url),[],"";);
}
}
_____________________
- Kelly Linden
Kelly Linden
Linden Developer
Join date: 29 Mar 2004
Posts: 896
06-03-2009 15:36
Unfortunately if I understand this right any object that wants to contact this service url will need to follow this pattern:

* Request the url for this service from the dns
* Cache that url and use it forever
* If we get a 404 when using it, throw it away and go back to stage 1 (request the url again).
_____________________
- Kelly Linden
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-03-2009 16:26
From: Kelly Linden
Unfortunately if I understand this right any object that wants to contact this service url will need to follow this pattern:

* Request the url for this service from the dns
* Cache that url and use it forever
* If we get a 404 when using it, throw it away and go back to stage 1 (request the url again).


Yes, that's right, although you could also request the URL at regular interval to ensure it's always up to date. This would of course increase the load on the DNS server, but would limit the likelihood that the URL was bad. You could in fact request theURL just before you use it, every time. Then you would ensure it's always the latest.

P.S. Thanks for adding the LSL example Kelly, I was thinking of adding an example but hadn't gotten to it yet. :)
_____________________
Papalopulus Kobolowski
working mind
Join date: 11 Aug 2006
Posts: 326
06-03-2009 17:01
Question...

What happend if the sim as not available url´s ?

the object "http server" stop working until some url´s are released?
_____________________


RAW terrain files - terraform your SIM!!
http://www.wishland.info/
PD:the wiki its your friend ;)
http://wiki.secondlife.com/wiki/LSL_Portal
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
06-04-2009 07:22
If you're going to have to request the URL repeatedly instead of letting the DNS cache timeout work, wouldn't you do just as well using an HTTP redirect?
_____________________
Argent Stonecutter - http://globalcausalityviolation.blogspot.com/

"And now I'm going to show you something really cool."

Skyhook Station - http://xrl.us/skyhook23
Coonspiracy Store - http://xrl.us/coonstore
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-04-2009 09:09
From: Papalopulus Kobolowski
Question...

What happend if the sim as not available url´s ?

the object "http server" stop working until some url´s are released?


The way I understand it, if the sim the 'http server' is rezzed in goes down, anything or anyone trying to access that URL would receive a 503 error. When the sim comes back up, the 'http server' would receive a new URL, which is the reason some sort of DNS becomes necessary. The 'http server' would have to have some way to communicate this new URL to objects that use it. But since the address of these objects can change too, trying to tell them directly isn't assured to work. Thus the DNS service presented. It's a go-between that both the 'http server' and the user objects can rely on to never change it's address.

Hope that answers your question. :)
_____________________
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-04-2009 09:12
From: Argent Stonecutter
If you're going to have to request the URL repeatedly instead of letting the DNS cache timeout work, wouldn't you do just as well using an HTTP redirect?


I'm not quite clear on what you're saying. This isn't true DNS per the Internet protocol standard, it is a DNS in concept only. The post above gives a general outline of operation.
_____________________
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
06-04-2009 09:19
From: Darien Caldwell
I'm not quite clear on what you're saying. This isn't true DNS per the Internet protocol standard, it is a DNS in concept only. The post above gives a general outline of operation.
Oh, I misunderstood the way your application works. I thought you were setting up an actual DNS CNAME for the sim somewhere.

So a redirect would be an enhancement. If you could make "http://something/something/servicename" generate an HTTP redirect to the actual object's URL, you would only need to perform the lookup once per session and any HTTP client that supported redirect (which is pretty much all of them, including static command line clients like curl, fetch, or wget) would just work.
_____________________
Argent Stonecutter - http://globalcausalityviolation.blogspot.com/

"And now I'm going to show you something really cool."

Skyhook Station - http://xrl.us/skyhook23
Coonspiracy Store - http://xrl.us/coonstore
Kenn Nilsson
AeonVox
Join date: 24 May 2005
Posts: 897
06-04-2009 09:40
Very nice code app Darien ...

Potentially you could additionally have each newly added object register a unique 'key' on the app-engine at the same time that they record their new URL. (This key would NOT be updated when a URL was simply updated - just when a new object was registered)

By adding that unique 'key', you can direct objects to your app-engine URL using that key - have your app-engine fetch the current URL for that key ... and relay the information down.

Sure ... it would take some change to the coding ... but then you'd end up with a "static URL" (once a key is registered for the object) that can always communicate down to the dynamic URL of the object.
_____________________
--AeonVox--

Computer games don't affect kids; I mean if Pac-Man affected us as kids, we'd all be running around in darkened rooms chasing ghosts, eating magic pills, and listening to repetitive, addictive, electronic music.
Cenji Neutra
www.apez.biz
Join date: 30 Oct 2004
Posts: 36
Note that this is not actually DNS
06-04-2009 10:40
You might want to edit the subject to call this 'pseudo DNS' or some-such, not to mislead people into thinking this is a real DNS solution (which is what many users will want).
Papalopulus Kobolowski
working mind
Join date: 11 Aug 2006
Posts: 326
06-04-2009 11:44
From: Darien Caldwell
The way I understand it, if the sim the 'http server' is rezzed in goes down, anything or anyone trying to access that URL would receive a 503 error. When the sim comes back up, the 'http server' would receive a new URL, which is the reason some sort of DNS becomes necessary. The 'http server' would have to have some way to communicate this new URL to objects that use it. But since the address of these objects can change too, trying to tell them directly isn't assured to work. Thus the DNS service presented. It's a go-between that both the 'http server' and the user objects can rely on to never change it's address.

Hope that answers your question. :)


In my case this is not a problem because i just update the new url to my DB to get comunication up un running ( as im doing right now with xml-rpc).
The cuestion is , if the sim is running out of free available addresses the comunication can't be possible.
So, how many free url slots can have the SIM?
_____________________


RAW terrain files - terraform your SIM!!
http://www.wishland.info/
PD:the wiki its your friend ;)
http://wiki.secondlife.com/wiki/LSL_Portal
Escort DeFarge
Together
Join date: 18 Nov 2004
Posts: 681
06-04-2009 11:57
From: Papalopulus Kobolowski
In my case this is not a problem because i just update the new url to my DB to get comunication up un running ( as im doing right now with xml-rpc).
The cuestion is , if the sim is running out of free available addresses the comunication can't be possible.
So, how many free url slots can have the SIM?

It's my guess from past experience that there is a limit, and that the issue you raise would come under the unapologetic catch-all explanation of "oops, we didn't think about that, can you jira it?".
_____________________
http://slurl.com/secondlife/Together
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-04-2009 12:22
From: Kenn Nilsson
Very nice code app Darien ...

Potentially you could additionally have each newly added object register a unique 'key' on the app-engine at the same time that they record their new URL. (This key would NOT be updated when a URL was simply updated - just when a new object was registered)

By adding that unique 'key', you can direct objects to your app-engine URL using that key - have your app-engine fetch the current URL for that key ... and relay the information down.

Sure ... it would take some change to the coding ... but then you'd end up with a "static URL" (once a key is registered for the object) that can always communicate down to the dynamic URL of the object.


Yes, that's a good idea. This is my first jab at App Engine, so I'm still getting up to speed on all it's features. If I end up improving it I'll certainly update this post. But there's nothing to stop anyone else from doing so as well. :)
_____________________
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-04-2009 12:27
From: Papalopulus Kobolowski
In my case this is not a problem because i just update the new url to my DB to get comunication up un running ( as im doing right now with xml-rpc).
The cuestion is , if the sim is running out of free available addresses the comunication can't be possible.
So, how many free url slots can have the SIM?


The URL limit is based on how many primsthe land you are using supports. From the Information Page (http://wiki.secondlife.com/wiki/LSL_http_server):

From: someone
# The number of allowed URLs is the same as the number of allowed prims on the parcel the object is over.

Object owner does not matter, all objects over a parcel will use the resource pool for that parcel.
Like prims, all the parcels owned by the same owner and in the same region share the same pool of resources.
If you have two parcels in a region that each support 100 URLs, then you could use all 200 in object(s) on a single parcel.

# The region's object bonus factor does not apply to available URLs.

If a parcel has a max of 300 prims in a region with a 2x bonus factor there will only be 150 urls allowed.



For attachments, it's 1 per attachment point:

From: someone
# Each resident has their own unique pool of available URLs with a max of 38 URLs per resident.

* This is 1 per attachment point, but all 38 could be used by a single attachment for example.
_____________________
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-04-2009 12:28
From: Cenji Neutra
You might want to edit the subject to call this 'pseudo DNS' or some-such, not to mislead people into thinking this is a real DNS solution (which is what many users will want).


It won't let me modify the title sorry. But anyone reading will quickly see what it can and can't do.
_____________________
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-04-2009 12:30
From: Argent Stonecutter
Oh, I misunderstood the way your application works. I thought you were setting up an actual DNS CNAME for the sim somewhere.

So a redirect would be an enhancement. If you could make "http://something/something/servicename" generate an HTTP redirect to the actual object's URL, you would only need to perform the lookup once per session and any HTTP client that supported redirect (which is pretty much all of them, including static command line clients like curl, fetch, or wget) would just work.


Sounds like you and Kenn are thinking along the same lines. It may be possible to do that, but I've not delved into App Engine functions enough yet to know. It certainly would be a better solution. :)
_____________________
Very Keynes
LSL is a Virus
Join date: 6 May 2006
Posts: 484
06-04-2009 12:44
Excellent Work Darien, I started something like this myself, but having only just mastered LSL I was a bit daunted by now trying to learn Python too, especially as I am also busy learning Assembler for the PIC Microcontrolers for Professional reasons.

I look forward to reading your source code and learning from it.
Void Singer
Int vSelf = Sing(void);
Join date: 24 Sep 2005
Posts: 6,973
06-04-2009 16:34
does anyone know of a free service that allows automated updates of both IP and hostname? because something like that with a FreeDNS type service would solve a LOT of problems and not require any user installation.
_____________________
|
| . "Cat-Like Typing Detected"
| . This post may contain errors in logic, spelling, and
| . grammar known to the SL populace to cause confusion
|
| - Please Use PHP tags when posting scripts/code, Thanks.
| - Can't See PHP or URL Tags Correctly? Check Out This Link...
| -
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
06-04-2009 16:49
Another possibility would be if there's one of those URL shortening services that allows you to update your short URL with an API.
_____________________
Argent Stonecutter - http://globalcausalityviolation.blogspot.com/

"And now I'm going to show you something really cool."

Skyhook Station - http://xrl.us/skyhook23
Coonspiracy Store - http://xrl.us/coonstore
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-04-2009 17:36
Ok, based on the feedback of Argent and Kenn, I've added a most powerful ability.

After adding a service, you can have your item use the service name as part of the URL, and it will be automatically redirected to the HTTP-IN URL.

for instance:

if you establish that your service name is 'intro1', and it's URL is

http://sim3015.aditi.lindenlab.com:12046/cap/3ff4f3f2-ea08-76c1-cef6-a22b4a573a7c

Then anytime you navigate to http://yourapp.appspot.com/intro1

you will actually be sent to

http://sim3015.aditi.lindenlab.com:12046/cap/3ff4f3f2-ea08-76c1-cef6-a22b4a573a7c

This solves the problem of having to retrieve the newest URL each time. Thanks for the feedback on this. :) I"ll update the first post.
_____________________
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
06-04-2009 17:49
3 cheers and a tiger
_____________________
Argent Stonecutter - http://globalcausalityviolation.blogspot.com/

"And now I'm going to show you something really cool."

Skyhook Station - http://xrl.us/skyhook23
Coonspiracy Store - http://xrl.us/coonstore
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
06-04-2009 19:36
From: Argent Stonecutter
3 cheers and a tiger


What, no ferret? :p
_____________________
Argent Stonecutter
Emergency Mustelid
Join date: 20 Sep 2005
Posts: 20,263
06-04-2009 19:46
_____________________
Argent Stonecutter - http://globalcausalityviolation.blogspot.com/

"And now I'm going to show you something really cool."

Skyhook Station - http://xrl.us/skyhook23
Coonspiracy Store - http://xrl.us/coonstore
Kenn Nilsson
AeonVox
Join date: 24 May 2005
Posts: 897
06-04-2009 20:11
Awesome work! Looks like something that a lot of people will be able to put to great use. And ... I learned a little bit of Python by looking at your code :-) ( I've never used Python before ).
_____________________
--AeonVox--

Computer games don't affect kids; I mean if Pac-Man affected us as kids, we'd all be running around in darkened rooms chasing ghosts, eating magic pills, and listening to repetitive, addictive, electronic music.
1 2 3 4 5