Passing a key via on_rez (concept)

Must be my mistake. I thought Argent said that there was no reliable way to compress a key down into an integer, and I thought you were trying to find a way to do it. Looks like I was wrong, because now it seems like you're agreeing with what Argent said. Or maybe I'm still confused.

Edit: Just saw your last post, and now I'm completely lost, and I have no idea what we're discussing any more. So I will step out of this conversation since I don't think I can contribute anything meaningful any more

I do agree with Argent... I attempted to find a way to reduce key size to four bytes, but nothing I can do is reversable. There isn't yet a way to reduce 32 bytes into 4 bytes without losing something, unless you pass a decompression key as well, which would have to be passed in some way other than on_rez.

i have however stated that you can hash the key, pass the hash, and have the child prim find the corresponding key by hashing all the keys it detects. (using sensors)

If we could use quantum particles to store the data, we could easily transpose a key into a quantum variable... seeing as they have (if i'm not mistaken) 32 possible states of existance.
Rather than binary (0 or 1) it's .... thirtytwoary (0 to 31). Two quantum bytes would have tons of possible states. 32 to the power of 32....

Yep.

We should totally pass quantum particles on_rez().

on_rez(doublequantum start_param){
llSay(0,"Holy crap that's a big variable!";
}

The whole point, I always thought, was that if you have X bytes of random data, you simply cannot compress it below X bytes, because that's the amount of information present there. Otherwise you would be able to repeatedly apply the same algorithm indefinitely and end up with any amount of information being represented by one byte. You *might* be able to, but you can't guarantee it.

What I don't know is the size of the space of possible keys in SL. I doubt it's the same in practice as the space of possible keys based on [a-z0-9].

Okay here's the deal:

Keys are 36-byte strings... 32 bytes if you take out the dashes.

However, they are hexadecimal (characters 0 1 2 3 4 5 6 7 8 9 A B C D E F and nothing else).

Two hexadecimal values can be combined to make one byte (as each hex is 1/2 a byte).

For example:

0 and F can be combined:

0 = (binary) 0000
F = (binary) 1111

0F = (binary) 00001111

Now, if you take the binary value 00001111 and convert it to an integer, it becomes:

15.

So now you've taken the string "0F" (two bytes large) and converted it to the number 15 (one byte large).

Now, since a key is 32 bytes but represents 16 hex pairs, you can effectively reduce it to 16 bytes in size, 1/2 of the original size, without losing any of the information.

Then, if you use huffman compression, you can reduce that 16 bytes to around 10 bytes (less if you're really lucky) without losing any information.

Of course, with huffman compression you must have a decompression key to go with it. So for 16 bytes you'd have a 16 byte compression key. Chances are, you'll never be able to compress the whole amount to less than 16 bytes. (with the data and key together).

Yes, you can compress something infinitely (until it reaches one byte) if you want, but you won't be able to decompress it again unless you've got keys for each step. And those keys make up more space than the original uncompressed data...


I'm rambling.... this is all theoretical anyways.


The best solution I can think of would be the hash. For practicality.

It's still not really sending an actual key through a single integer, but it allows for the rezzed prim to be able to quickly find the referred key once it scans.

Ignoring the option of having the rezed object just ASK for the texture after it rezes...

If the llRezAtRoot/llRezObject command doesn't care about the position or rotation of the object (because it is going to position/rotate iteself) then you can also use that information to pass a key. There should be more than enough room across all the parameters.

Random data is hard to compress. Text, images, data, and video all get compressed differently since the techniques vary based on the contents and common patterns in the information. But random data... Any given technique MAY save space, maybe even a lot, but it could also INCREASE the size for some given patterns of random.

And at some point, the impact of time needed for compression and decompression outweighs other options like llWhisper-ing keys back and forth.

Ziggy -- your intuition and reasoning IS the mathematics, and it is sound.

Stated mathematically, it's pretty straightforward:

32 bits cannot store 128 bits of data, because 32 < 128.



Stop apologizing for your math!

If that 128 bits of data represents smaller amounts of data, then yes, you can compress 128 bits into 32.

For example...

the string "00110011 00110011 00110011 00110011" (minus the spaces) is 32 bytes of data. That's 256 bits.

However, since the data is all ones and zeroes, you can effectively reduce this to only 32 bits, by changing each 1-byte "1" or "0" into a 1-bit 1 or 0.

Thus, 256 becomes 32.


All that is required to get that 256 bits from the 32 is an algorithm to undo the changes.

And it's not hard... just have it detect each bit and transform all 0-bits into "0" characters and all 1-bits into "1" characters.

The 1-bits and 0-bits are not the original characters, but they are representative of such, and as long as you know what they represent then there is no data loss in the process.

A key is 32 (ignoring dashes) base-16 digits, which is 512 bits.

Expressed in base-64, it would use 8 digits.

An SL-friendly base-64 character set could be made with:
A-Z (26 characters)
a-z (26 characters)
0-9 (10 characters)
(that's 62, so two more...)
. and #

.0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#

You could then compress a key into an 8 character string.

But that's a string, not an integer. Integers are only base-10, so it would take 52 digits to express the same key, and integers only have a little over 9 digits to work with.

No, you have teh same 16 bytes of data that you originally had.
Where are you putting the compression tables you're using? How does the receiver know that 102 is 0, and so on? You have to include the tables in the message, and on such a short message the tables will be MUCH larger than the original.
Only when there is redundancy that can be exploited. If there's no redundancy in the data, as there isn't in the original 128 bit key encoded in the 36-character string, there's no opportunity to compress it. That's basic information theory.

I know.

I never said anything to the contrary...

Even if there is no redundancy, huffman compression can lessen the size a byte or two.

You still have to have the table, but you can pass that via llSay() (which is what I stated previously).

I agree with you on this.... no need to keep attacking my posts.



Here's where it goes differently:

There wasn't originally 16 bytes. There were originally 36, one byte for each character in the key (which is formatted as a string). You've got to remove the dashes, leaving you with 32 bytes. It is still 32 bytes because it is not literally in hexadecimal, it is still a string, one byte for each character.

You can convert it to 16 bytes though. Thus it goes from 32 to 16.

Keys are made up of hexadecimal characters, but they are not really hex. They are still strings made up of one-byte characters.

That scan will use more resources and be slower than using listens, unless you're doing the scan anyway. For a smart bullet, a hash of the keys would be a useful technique. It's not compression (compression, as I said, isn't possible here) but for your application it's perfectly reasonable.

The whole idea of huffman compression in this post was to reduce the key size to fit into a 4-byte integer.

I never stated that the table would be transmitted via on_rez... I specifically stated that it would have to be transmitted in another way. However, the key would still go through the on_rez().


Nothing I've said goes against the fact that we can't transmit a key through on_rez without transmitting other data in some other way.

Yes and no... qubits don't behave like bits, and qubits don't have a discrete number of states. And, of course, if we were using quantum computing we wouldn't be using mere 128 bit UUIDs, instead you'd pass a qubit that was entangled with the object itself.

I agree; passing a hash would be the most practical method.

But that really is only applicable for scans.

If you're trying to pass a texture key, however...... might as well just llSay().

Mmmmm.......

We're not talking about 128 bits "representing" anything. The 36-byte string is a representation of 128 bits of data. That 128 bits, is the data... there's no simple correlation between the bits that can be used to stop transmitting some of them.

Your example of "1" and "0" is different, because the bytes in "10101100" all have the same bits in positions 1 through 7, only bit 0 is independent.

The 32-byte string might be a representation of a 128-bit data key, but the actual key itself (in a format readable by LSL scripts) is a 32-byte string. That was my thoughts on the matter.

You cannot pass a 32-byte string through a 4-byte integer.

However, by converting the 32-byte string into a 16-byte string, you reduce its size by 1/2.

You haven't lost any of the original 32 bytes of data, because with the proper algorithm you can pop it back into 32 bytes.

This is an example of taking one piece of information (a 32-byte string) and compressing it (into a 16 byte string) without losing any of the original 32 bytes of information. It doesn't matter whether you as a human can read it as hex representative of something else. The computer reads it as a 32-byte string.

Sigh...

That table is the keyin a different encoding. Every key is going to have a radically different huffman encoding, because they're effectively random, so you'll have to send a separate table with each key to get any useful compression... so the total number of bits you'll have to transfer for each key becomes MUCH larger than the original.

I'm not attacking, I'm trying to teach. Obviously ineffectively.

The key is formatted as a string, but that's an implementation decision by Linden Labs, prsumably to avoid having to either modify the database code to treat UUIDs as native objects or to avoid converting the hex string to the binary representation and back again when it's just being copied. The key is not the 36 bytes of the string, it's the 16 bytes those 36 bytes encode. The rest of the bits in that string are completely constrained and do not contain any actual information.

Information can be treated as an incompressible substance. Anything you can compress has duplicate information in it somewhere. Once you have re-encoded all the duplicates, that incompressible core is the actual data.

Within LSL they are implemented as byte strings, but UUIDs are not a concept invented by Linden Labs, and not all implementations maintain the hexadecimal dash-separated strings. The 128 bits encoded by the strings are the real key.

I understood what you were saying, but what I said was not invalid either. We were both right, to an extent.

I understand how huffman encoding works. I understand that in the end I'd be sending a lot more data via llSay(). I was not trying to say it was practical, I was saying that it's possible to send just the key (compressed) through on_rez, but I also stated that you'd have to send the table via say. I never said it would have a smaller data size.

Part of the problem was that I kept jumping back and forth between what applied to SL and what applied to the real world. I should have been more clear.

Anywho...






We do however agree on the fact that there's no way, without a key, to compress the data enough to be able to force it through a little integer hole.










I'm sorry..... I'm too stubborn to conceed. This might be due in part to the fact that I've been programming since I was 9 years old, and I've developed something of an ego about it. Sorry if I offended you.

No, it'd use 8 digits if each digit conveyed 64 bits. That requires a bit more than 64 characters.

A base-64 character set conveys 6 bits per digit, which is what my compressor script does.

I understand. The problem is you're showing a fairly basic misunderstanding about information, it's a common one, but it's one that leads people to waste a LOT of time chasing rabbits.

It's great that you've been programming since you were 9. But I've been programming since I was 12, and that was back before there was such a thing as a home computer.

I think at this stage we have to move to asking, how ARE keys generated?

Are they truly random, or based on something else? If they're seeded from the wallclock time, like a lot of random number generators seem to be, then since it'll only be a few seconds between an object getting rezzed and its on_rez method running, the script can read the clock, work out the list of potential keys that could have been generated at that time, and then the rez parameter only needs to be the tiebreaker.

(remembers that the guy who writes the compression FAQ has a "challenge" at the end - to compress an arbitary amount of nonrepeating random data - which he sets anyone who denies the conservation of information)

(also remembers that somebody did actually beat the challenge and caused some furore )

^.^ is this online? I'd love to read.




By the way, I wrote the scripts for the transmission of a hash from the parent to the child prim so that the child detects the user requested. I'll post it here later, once I comment it (for the new users who may not understand anything yet).