Question for the HTML masters of SL?

I am attempting to make a program that logs into SL with a username and password. Problem is, the post information does not seem to work:

https://secondlife.com/account/login.php
?type=second-life-member
&nextpage=/account/index.php
&persistent=Y
&username=foolish
&lastname=frost
&password=rightlikeimgoingtogivethatout

Now, the above is normally on one line, but for some reason, SL refuses to acknowledge it and login.

Oh mighty gurus of web wisdom, what is keeping the post data from committing a manual login?

(If I can get by this hurdle, I MIGHT have a nice toy for everyone to play with later. )

omg h4x

b&

<snort> Like if I was hacking SL or anything else, I would post about it here.

Nope. Wanting to make legal use of the SL site with a program. <shrugs> Actually, now it's just become a war between me and the POST command.

I HATE html.

 curl -c cookies.txt -i -d "form%5Btype%5D=second-life-member&form%5Bnextpage%5D=%2Fcurrency%2Fsell.php&form%5Bpersistent%5D=Y&form%5Busername%5D=foolish&form%5Blastname%5D=frost&form%5Bpassword%5D=iluvcurl&submit.x=13&submit.y=20" "https://secondlife.com/account/login.php"
 curl -b cookies.txt "https://secondlife.com/currency/sell.php"

You can find curl here:

http://curl.haxx.se/

(release your toy or face the wrath of the karma chameleon!)

/smacks forehead

Thanks for the curl tip!

I was having the same problem trying to get my transaction history. I totally forgot about the cookies. I have only been writing scraping scripts since 1996, so I am really feeling like an idiot!

Thanks. Now just have to translate this into C# and figure out what I'm doing wrong in general. <sigh> Ain't software development grand?

There is a .NET implementation with samples at sourceforge.

http://sourceforge.net/projects/libcurl-net/

I am going to fire up c# tonight and give it a go.

Do I understand correctly that curl is a clever method of working around a problem with the straightforward approach shown in Foolish's initial post?

I have wanted to be able to automate the process of getting all posts by specific Lindens stored on my computer where they can be searched in an effective way rather than by the limited and crippled search system this forum uses, to use both for reference and to put in a Markov chain text generator.

Well, Foolish has the parameters wrong .. there are a bunch more.

Also, if you want to access the page, you need to be sending cookies in your request headers which tells the system you have logged in.

My suggestion is to get the developer extensions for Mozilla. They will reveal all.

As for your project, you'd probably have to write some scripts to get all the data. Or, I guess, you could code it up .. but I generally find scripting faster for this sort of thing.

For example:

curl -b cookies.txt "https://secondlife.com/currency/sell.php" 2> /dev/null
| grep "Best buying rate" | grep -o "L$..." | grep -o "[0-9][0-9][0-9]

Well get me the best buying rate on SL. As far as I know, unless you did it in perl or some other similar scripting language, that would take quite a few lines of code.

Actually, I'm trying to use the C# web functions and figure out how they work.

So far, I can get a copy of the web page, but am having trouble with the cookies and posting.

While the extra features are nice, I'm trying to figure it out from the basics. Will just take some time.

Woot! Found a C# code snippet I'm taking apart to figure out how it works, but it does!

He was right, there was a LOT of background post data that was invisible. Sadly, I'm not sure what all of it does, but it does not seem to matter as of the moment.


C# sourcecode for FormLoginGet object.

        private static string FormLoginGet(string requestUri, string loginUri, string username, string lastname, string password)
        {
            string loginData = "form%5Btype%5D=second-life-member&form%5Bnextpage%5D=%2Fcurrency%2Fsell.php&form%5Bpersistent%5D=Y&form%5Busername%5D=" + username + "&form%5Blastname%5D=" + lastname + "&form%5Bpassword%5D=" + password + "&submit.x=13&submit.y=20";

            // cookieContainer is used to store the cookies used by the login
            CookieContainer cookieContainer = new CookieContainer();

            // First hit the login page
            HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(loginUri);
            req.CookieContainer = cookieContainer;
            req.Method = "POST";
            req.ContentType = "application/x-www-form-urlencoded";
            ASCIIEncoding encoding = new ASCIIEncoding();
            byte[] loginDataBytes = encoding.GetBytes(loginData);
            req.ContentLength = loginDataBytes.Length;
            Stream stream = req.GetRequestStream();
            stream.Write(loginDataBytes, 0, loginDataBytes.Length);
            stream.Close();
            HttpWebResponse res = (HttpWebResponse)req.GetResponse();

            // Then grab the content of the desired page
            req = (HttpWebRequest)HttpWebRequest.Create(requestUri);
            req.CookieContainer = cookieContainer;
            req.Method = "GET";
            res = (HttpWebResponse)req.GetResponse();
            StreamReader sr = new StreamReader(res.GetResponseStream());
            return sr.ReadToEnd();
        }

Now, with this bit of code, you just run the subroutine:

string webpage = FormLoginGet(string requestUri, string loginUri, string username, string lastname, string password);

The login url should always be: "https://secondlife.com/account/login.php"

And this will pull any page you like! Told you I would share...

Back to my project now... <mutters to self about flux capacitors and ion brushes>


You know, I think the next this I will do is split this into two parts, the one that get's the cookie, and the one that rips the data USING the cookie later. Hmnnnn. Save having two page calls for every one.