Pixel shader attachments

From this proposal:


Pixel shaders are awesome once you discover the sheer power of them, and I'd very much like to see this implemented personally for cell shading purposes: anime should have a place in SL too, no?

Hi Emma,

Not being very computer smart personally, it seems that what this would actually be, is like having a skin for objects. Would this be what you are saying with this proposal? If not...can you put this in terms people who have no idea what you are talking about can understand simply? Thanks!

Lovepeace

Okay, I'll try to attempt to explain pixel shaders, they're advanced material and hard to sum up because they can do so many things, though.

First off, where a pixel shader comes in for us. A pixel shader is generally attached to a 3d model so that would mean a prim in SL, though with some coding (if LL wants to go through the trouble) they could conceivably be attached to a simple texture, which is preprocessed before shown on an object.

What a pixel shader does is when an object is rendered it goes by the pixels it renders one by one and performs a mathematical operation on them which you can script with something called High Level Shader Language. A typical shader script like the one used in the first processed image below could be:

sampler2D g_samSrcColor;

float4 MyShader( float2 Tex : TEXCOORD0 ) : COLOR0
{
    float4 Color;
    
    Color = tex2D( g_samSrcColor, Tex.xy)*3;
    return Color;
}

technique PostProcess
{
    pass p1
    {
        VertexShader = null;
        PixelShader = compile ps_2_0 MyShader();
    }
}

For an explanation on what all that means I again direct you to the top bit of Facewound's Shader Tutorial. What the code here does is take every pixel and multiple it's components by 3. This makes the colors much lighter, obviously. This produces the following result when run on my pretty SL face:

Sample 1 Color components multiplied by 3.

Going from that same source image here are other samples of what a pixel shader can do (a simple pixel shader, that is):

Sample 2 Color's red value multiplied by 3, this causes a reddish tone

Sample 3 Red, blue and green color values multiplied by the sinus of the current vertical source position in the image, which causes something weird but nifty

Sample 4 The source coordinate in the vertical direction of the image is halved, the output coordinate is not, this causes a stretched image at 200% in the Y direction.

Sample 5 The source coordinate in the vertical Y direction is modulated by the sinus of the source coordinate in the X direction, which creates a wave effect on the image.

These effects are nifty, but not really very impressive in and of themselves, since obviously they might not have a practical application, but this system can blur, sharpen, emboss, greyscale or B&W the image entirely as well. These are small little scripted routines that are sent to clients and which are uploaded into their video hardware for lightning speed execution when textures or objects are drawn.

Now, on to cell-shading. Mind you, the sample I have provided is partly FAKED, since I can't attach pixel shaders in SL, but I wanted to show you a real live, SL example of cell shading. Also note that this is a very scrappy simple cell shader, so the result isn't mindblowing, for better results see the list of cell shaded games linked in my original post.

Now, this is a picture of yours truly against a pure blue background. This is what I've used to illustrate what a cell shaded object might look like in SL. I've made the image greyscale so color values don't get twisted, ramped colors over larger distance creating distinct flat shaded portions of color, like in cell-shaded anime and manga. I then had to go back in and color these areas seperately. Also, there are a lot of floating pixels that make it look less clean than it should, in a proper cell-shaded object textures would be made with cleaner lines to keep in mind cell shading, and every differently colored "part" of an object like this would be a seperate object and shaded seperately.

What you might get is something like this. Now I hear texture artists crying "Why not simply texture things with flat areas of color!?", well the problem with that is that the lighting done be SL would create gradients over this flat area of color, meaning it is no longer flat at all. It's also static, so no matter which way you turn, the same pieces of color will still be in the same place. Using cell-shaded the regions of color turn with the shading.

I hope this explains a little better, it only took me a good hour to come up with this quite extensive post.

Actually you will be able to accomplish this to some extent in 1.7 with the FULL_BRIGHT prim option (or edit window checkmark.)

This has the same effect as setting something to be a lightsource but without the lag, glow and bad lighting effects that light-material objects cause now.

No shadow/shading or sun position effects at all.

I hadn't tried building a cell-shaded object in 1.7 with this new option (lots of home-brew cell-shaders have used the light material setting previously) but it sounds like a good thing to go try in preview.

L8r,
UU

One of the bluntest strengths of pixelshader implementation, as Emma describes extensively, would basically be great for a LOT of impressive eyecandy.

For example, the scene in the Matrix where Neo can see through the Agents at the end, and reality around him turns into these falling waterfalls of code. Imagine that in SL.

Or, we could all have old gibbly bits wiggle across the screen and go insta-monochrome like an old film reel, stuttery and stuff.

Or seeing us all as cellshaded cartoons (I know this point has been brought up before), or one of my faves: BLOOM LIGHTING!!!

Lighting in SL, as beautiful as it can glow with Local Lighting on, still looks kinda flat.

HOW ABOUT DRUNK-O-VISION WITH HUDS? Keep drinking and the screen keeps getting wavier and blurrier! Caution against avatar liver damage! This could even make Virtual Hallucinations more educational.

THE DYNAMICS OF SIGHT!

Okeydoke get the ball rolling...

I hadn't thought of that. Would you truly lose all definition? or just most? I haven't seen the effect in action yet. However, it's still pseudo-cellshading, the patches of color would be static, and it looks kinda odd in my eyes (just like when I've seen skins that are heavily shaded in SL now, it looks fine while standing still but as soon as the person wearing them starts turning and moving, it gets weird)

Although full bright options would make it easier to make a pixel shader for cellshading too, 100% ambient lighting is indeed often used for it. ^^;

Indeed all the things Torley is saying are true soo. You could have a wavey/blurry underwater effect like WoW, a blurry drunk effect that gets increasingly worse as you drink more, hell someone might even make an acid object and make your screen do all sorts of funky things on your virtual trip.

The possibilities are really endless, IMHO. (if properly implemented)

SL may not have it for awhile. But something great to hold up on a pretty postcard pointing the way on the road to the future.

There are 108 entries in this ShaderTech archive of vertex and pixel shaders, covering all manner of effects and non-effect processing. It's a useful resource.

More pertinent to this thread though is this question which I posted in April to the Hotline to Linden forum on factoring out graphics to GPU shader hardware, but which sadly received just a PR answer which said nothing useful.

I followed it up with this thread in Technical Issues where I tried to explain why the previous answer missed the point entirely on technical grounds, but it received no response.

The very powerful shader hardware on our modern GPUs is being almost completely ignored at the present time, while the very intense graphic environment of SL is grinding our high-end machines to a standstill with dismaying regularity. Yet, apparently it's not an important issue in the client design.

Pretty sad. Oh well.

I was waiting for you to post here, Morgaine. I'm so happy you did.

Maybe in Havoc 2 they'll be more prominent and available to designers? I'm not holding out too much hope they'll be implemented in SL as is anytime soon.

Just pointing out to the devteam that they're there and maybe its good for everyone if they started using them and started letting us use them.

Sadly, this could be said of pretty much everything. LL's rate of (visible) development has dropped to a crawl.

They say that they're resource-constrained. Well doh, who isn't. Open-source is there to be harnessed, if there were a will for it, but there isn't, despite occasional positive noises from the management.

So, SL will just limp into its future at this snail's pace ... and then be overtaken by the first open-source effort with a viable architecture that captures widespread interest. And that'll be the end of the field leadership dream.

Q: Does OpenGL have intrinsic support for this?

If the answer is no, don't even bother fantasizing

umm PS is a microsoft inovation. SL doesn't use MS stuff for it's back end. Try selling Cg instead.

uhhh i just read something interesting...
http://en.wikipedia.org/wiki/OpenGL
microsofts plans for (killing) OpenGL.

OpenGL has total support for it, in many different ways. What's more, there is high compatibility between shader languages from nVidia and Microsoft (they cooperated on their development), and separately there's also OpenGL's own shader language as yet another player on the shader scene.

So yes, shaders are very well supported in OpenGL. Just not by LL.

I wish!

In my stoned state, I just realized something else: falling underwater and looking up, there's a type of graphical filtering—it even affects pixels. Not sure what to describe it as in more technical detail. What is this? More varieties of whatever's being done would be fascinating overlays. There've been some neato alpha transparency effects, is this the same sort of thing?