Trigonometry/Rotation help

Okay, I've gotten myself into a math tangle that hopefully someone can help me out of. I had hoped this would be a simple action, but I'm not seeing a simple function to do this.

The situation is that I have a simple pointing object, one that orients itself to point directly at an avatar, which most likely will be slightly "down" from its vantage point. Well, using the code at the WIKI (http://rpgstats.com/wiki/index.php?title=LlLookAt), I can get the object to point to the avatar, but I'm currently wanting to use the local X-axis of the object rather than the Z, so am using the code alternative for the llRotLookAt function rather than the llLookAt function.

The problem arises in the fact that often, as the object rotates to point its local X to the avatar target, it barrel rolls around the local X to end up with the local Z no longer pointing up, but off in a random direction. In trying to re-orient this object, I realize that I can't get the local Z axis to point straight up (global Z), if the object itself is tipped in the Y to have it point down. In truth, I need to find out how to orient the object such that the local Z-axis is coplanar to the plane formed by the local X-axis and the global Z-axis.

So, I'm guessing my code will look something like:

float myAngle = 42;
rotation myAngleRot = llEuler2Rot(<myAngle, 0, 0>); // Rotate around X to un-barrel-roll
rotation newRot = myAngleRot * llGetRot(); // rotate locally, rather than globally
llRotLookAt(newRot, 0.5, 0.5); // Apply the rotation

The question would be, in this scenario, how to find out what "myAngle" should be (I used 42 here as an arbitrary number), since it needs to be the amount that will bring Z back "up" as much as possible.

Or, is it possible to prevent this barrel roll in the first place? I tried setting llSetStatus(STATUS_ROTATE_X,FALSE); before doing the llRotLookAt to begin with, in the hopes of preventing that barrel roll in the first place, but that didn't work. Is there another way?

You might be able to get away with restricting the rotations, but here's how you can calculate a full rotation using a local coordinate frame. It assumes you want the only deviation from horizontal to be a pitch (local y-axis remains level). The code also doesn't check a couple of corner cases, like the target being AT the prim's position, or being straight up or down from it. Those are left as an exercise for the reader (either to code or search in the forums for where I've already provided such a complete set of logic):

(Note: This has not yet been compiled; a few small syntax fixes might be required.)

rotation rotationToLookAt(vector targetPos)
{
   vector localX = llVecNorm(targetPos-llGetPos());
   vector localY = <-localX.y, localX.x, 0.0>;  // Same as <0,0,1>%localX
   vector localZ = localX%localY;

   return llAxes2Rot(localX, localY, localZ);
}

I spent the last couple of weeks pouring my brain down a similar rat hole

Aha, the llAxes2Rot function: that was the link I was missing; I think I've got my function worked out just fine now. Thanks Hewee!

Sure thing. By the way, I realized that since localX isn't necessarily horizontal, localY technically needs to be normalized too:

rotation rotationToLookAt(vector targetPos)
{
   vector localX = llVecNorm(targetPos-llGetPos());
   vector localY = llVecNorm(<-localX.y, localX.x, 0.0>);  // Normalized <0,0,1>%localX
   vector localZ = localX%localY;  // No need for normalization since localX and localY are orthonormal

   return llAxes2Rot(localX, localY, localZ);
}