Transparency Problem (Maya)

I've run into a rather annoying transparency problem in Maya. I made this flat rectangular sculpt that's supposed to be completely transparent in the middle, so I can fit a half-open door there. I made a tga in Photoshop, containing an alpha with the appropriate parts blacked out.

I was hoping there'd be nothing more to it and the area would become completely transparent. However, the transparent area is in fact still collecting light from the scene (at least that's what I think is happening), which is making it look slightly opaque. I'm working with a blinn, and it's the 'shine' that's still showing even in the completely see-through parts of the object.

Is there a way around this? Or am I doing something wrong? Should I not be using tga files for transparency? Or are there things I need to be mindful of while making tga files with an alpha?


Suggestions greatly appreciated

If you think about what's happening, it makes perfect sense. You're using a Blinn material, so your surface is shiny. The fact that part or all of it might be transparent doesn't automatically make the shine go away. Unlike SL's ridiculously primitive texturing engine, Maya employs a fully functional material shader system, with individual channels controlling literally dozens of properties, besides just color and transparency.

In real life, a material can be both transparent and shiny at the same time. The same is true in Maya. If you want some part of the surface to be less shiny than some other part, you have to tell Maya that that's what you want. And the way you do that is -- you guessed it -- with alpha mapping.

(Once again, this is one of the reasons I constantly harp about how foolish and shortsighted it is for anyone not to learn to use alpha channels. They're the very basis of graphics, and are used for all sorts of things besides just transparency. Those who shy away from them will NEVER be able to do very much. Anyone who wants to do anything of much importance with digital art MUST learn to embrace alphas.)

The primary channels which control a Blinn's apparent shininess are reflectivity and specularity. If you leave those channels unmapped, then every part of the material, including the transparent parts, will be just as reflective and just as specular as every other part. Hence, the whole surface appears to be shiny.

Right now, your alpha channel from your TGA is controlling the transparency of your material. Maya does this by default with 32-bit TGA's. The image's alpha channel won't control any of the material's other channels unless you tell it to. So, if you want it to control shininess, you need to apply it the material's reflectivity and specularity channels.

There are a number of ways to do that. (Maya offers tons of ways to do every one thing.) I prefer the Hypershade (Window -> Rendering Editors -> Hypershade).

If you open up your Hypershade, and map your Blinn material, chances are you'll see just a few components. Attached to the primary Blinn shader node will be a file node, and attached to that will be a placement node. The file node is the active link between Maya and your actual texture file on your hard drive. The placement node controls how the texture image is mapped to the surface.

To view a node's parameters and properties, simply double-click on it. It will open up in the Attribute Editor.

For example, if you double-click on the placement node, it should look somewhat familiar to you. SL's texture tab in the editor window has many of the same parameters. You can't actually see nodes in SL, since it has no hypergraph or hypershade style interface, but all the in-world editor does is control the parameters of various nodes and node links, just like your Maya Attribute Editor is doing right now.

If you hover over the links between the file node and the shader node, you'll see they connects the file's color output to the material's color input, and the file's transparency output to the material's transparency inupt. This is why it's not affecting the shininess. All you've told it to do so far is control color and transparency, nothing else. To change that, we need to make some more connections.

To create connections between nodes in the Hypershade, simply middle-drag one node onto the other. Do this now. Middle-drag the file node onto the shader node. In the menu that pops up, click on Other, to open the Connection Editor.

On the left side of the Connection Editor, you'll see all of the file node's outputs. Notice outColor and outTransparency are in italics. That's because those two outputs are currently connected to something. On the right side are all the shader node's inputs. Notice color and transparency are also in italics, because they're also connected to something. Hover your mouse over the connections in the Hypershade window to see what's connected to what.

We want the image's alpha channel to control reflectivity and specularity. So, click on outAlpha on the left side, and then click on Reflectivity on the right side. outAlpha is now controlling reflectivity. Areas of the image that are black in the alpha channel can no longer reflect anything. Now click the little plus sign next to Specular Color, to expand the tree. With outAlpha still highlighted on the left, shift-click the three specular color channels, to form connections to them as well.

Do a test render, and you should now no longer see the transparent parts of the image appearing to be shiny.

Note, this was a fairly crude way of doing things. The instructions were just the simplest way I could think of to get you on the right path. There are far more powerful ways to proceed. Explore the Hypershade and Connection Editor in more depth. Also look into how to create PSD shader networks, which will allow you to use layers in a Photoshop file to control the channels of a material.

Also, be aware that pre-existing imagery from Photoshop or wherever is just one way to create maps. Another way to go is to use Maya's built-in procedural system. For example, connect a simple black/white ramp to a material's transparency channel, and you'll cause the surface to grade form transparent to opaque. You can do the same thing with black/white gradient alpha image, of course. But by doing it parametrically, you can change it on the fly, directly within Maya. So, for example, if you want the gradient to change over time, you can keyframe the color stops on the timeline, and watch the transparency change before your eyes when you play the animation. This particular example isn't directly applicable to SL, of course, but hopefully you're starting to see the potential.

Connect a noise node to a transparency input, and you can make a snow texture in about 2 seconds. Connect it to a bump channel, and you've got instant stucco or concrete. Tweak it a little, and you've got leather or plastic. Etc., etc., etc.

But that's just the very skin of it. Take a look at the amount of 2D and 3D texture nodes in the Hypershade's Create Nodes column, take into account the shear number of parameters they all have, think about how you can connect them not only to materials, but also to each other, and it's plain to see that the possibilities are endless. There are plenty of people who almost never use Photoshop texturing, instead preferring to create nearly everything parametrically right in Maya. And if you compare the results, you'd never know they weren't in fact using imagery. Parametric shaders are INCREDIBLY powerful. I encourage you to dive into them, experiment, read, learn as much as you can about them, and practice, practice, pratice.

Happy rendering.

Thanks so much for your in-depth answer, Chosen

I just tried your solution and it works perfectly. You can see the change on the node preview as well. I didn't pay attention to it yesterday, but before, you could see even on there that the transparent parts were also shiny. It does make sense now that you mention it, that transparent areas would still reflect. I just wasn't getting the behavior I'd envisioned, which is usually the case when I run into 'problems'.

I've been aware that a lot can be done in Maya itself by using materials creatively, but I haven't delved into that myself. It seems quite overwhelming from an outsider's point of view. I've mostly stuck to basic (tweaked) lambert, blinn and phong nodes, most of the time with bumps applied. I do most of my texturing work and bump mapping preparation in Photoshop. I really should learn more about hypershade, just so it wouldn't look that foreign to me as well.

Are there perhaps Maya communities you would recommend where I could learn more easily?

Thanks again for your help
All the suggestions you've given me so far on various subjects have been incredibly helpful. I really appreciate it.

I'm glad you were able to follow that, Anya. Reading through it again, I noticed there were some mistakes in it, which I've now corrected. For example, I had said to connect outAlpha to Transparency instead of to Reflectivity, and there were one or two incomplete sentences. Sounds like you were able to read past my late-night typos without incident, though. (I almost never drink, but I did hit the Vodka a little bit last night, so you're lucky the post was even in English, hehehe.)

As for Maya communities, you might want to check out the Autodesk forums, and the highend3d.com forums. I've never actually used the highend3d forums, myself, but I really should. The tutorials on that site are generally very good. I'd imagine the forums are even more useful.

(Hmm, it looks like they've changed their name from HighEnd3D to CreativeCrash. I hadn't been to the site in a while. Not sure when they made that change, but it looks pretty recent, since it says the site is in beta.)

There is an in-world Autodesk group, but from what I've heard, the leadership can be a little tyranical. That's extremely third hand, though, so take it for what it's worth. I've got no direct experience with them. What I heard could very well have been exaggerated or altogether wrong.

Xenius Revere's classes (see xeniversity.com) are also an excellent resource. I recommend them to every Maya user, new and old. They're the only courses I know of that teach Maya in a manner directly aimed at the SL user's sensibilities.

Xeniversity.com sounds good. That's always been my problem a bit, that a lot of information out there is (understandably of course) not aimed towards the SL community.

I would love to expand my knowledge outside of SL eventually and do animation in Maya for example, but it would be great if I could actually work from the experience I already have. Everything I've learned so far has been with SL in mind. When I've googled specific tutorials in the past, I've on occasion gotten stuck because they weren't aimed at newbies like me and the terminology wasn't always understandable. I have some basic knowledge of 3D and editing software in general, but it's not always easy.

And yep, I did in fact get an error last night when I tried to connect outAlpha to Transparency, but when I followed your second step I got the desired result anyway so I thought I just hadn't followed the procedure completely right. (Transparency looked right on the node and holding the sculpt against a black background the hole looked completely black as well.) I'll connect outAlpha to Reflectivity in the future as well just to be sure

The reflectivity thing will make a difference if there are other objects around to reflect. Note, you'll only be able to see reflections when you do a rendering. Maya's viewer pane doesn't support reflections (because that would require realtime raytracing, which would likely bring your FPS down to single digits or less). It does support specularity, especially if you've got it in high quality mode, which is why you could see the difference with that immediately.

Ah yes of course. I'd dragged the object away from the scene a bit to check the transparency, which is why I wouldn't see reflectivity issues. It's a bit of a complicated scene, with other items blocking the 'door' opening, so I wanted to have a good look at the sculpt by itself. Before making the necessary hypershade links it was still opaque because of the shininess of the object itself, even when looking at it on its own.

*looks around for Chosen*

I have a small follow-up question to this issue. The transparency is solved, but as a result I have another small issue with the blinn I'm using. I'm applying the same door texture (without the transparency then of course) to other parts of my project. For those blinns I changed the specular color from standard grey to something grey-blueish to make the shine more interesting.

However, now that I have my texture feeding the specular color slot for my 'door' blinn, the specular color is showing as white and I'm not sure how to go about making it like my other blinn nodes? If I can't get it right, I'll just change the specular color on those nodes, but I rather liked the shine I'd created for them.

Suggestions welcome

Remember I said there were about a million ways to proceed, and that the one I picked was just the simplest to explain at the time? Well, looks like it's time to pick another one.

Once again, there are many ways to solve the problem, from changing things in Maya to fixing the image in post in Photoshop.

Here's one easy way to go. Instead of using that alpha channel as your spec map, use a new image, with the appropriate shading already on it. In Photoshop, copy that alpha channel from the first texture and paste it into a brand new image, as a layer this time, not a channel. Now just paint the white parts blue, and save.

In Maya, delete the existing connections between the old texture's outAlpha and the shader's specular color channels. Then create a new file node, load up that new spec map image you just created, and connect its outColor to the shader's specularColor. You won't have to go through the connection editor this time, since this is a much simpler connection. Just middle-drag the file node onto the shader node, and pick specularColor straight from the popup menu.

You'll find that using separate images for each element, rather than just one channel from one image, will afford you a lot more options. Alternatively, you can set up a PSD network, and use layers from a single PSD, to do basically the same thing while making organization a bit easier.

The only reason I went with alpha-channel-for-everything method the first time around was it seemed like the simplest way to go at the time. If nothing else, it showed you how to dive into the connection editor. But it certainly wasn't the most flexible strategy, as you discovered.


If you'd rather go the fix-in-post route, which is what many artists do prefer, then bake out each of the major properties of the image (color, spec, reflections, etc.) as separate maps, and then layer them together in a PSD. Don't worry too much about the specific ordering of the layers, since you can always change that around for different kinds of effects. Just make sure you get everything in there somewhere. It's usually best to start with the color map on the bottom, but you don't HAVE to.

You'll now have an enormous amount of (fast) control over the coloring and intensity of each element. If you want the spec to be purple instead of blue, for example, you can just apply a hue/saturation adjustment to that layer, and you're all set. If you want the reflections to be less pronounced, just turn down the opacity of their layer. If you want the shadows to be more bold, maybe change their blending mode to Hard Light instead of Soft Light, or apply a levels adjustment. Etc., etc., etc. You can make all kinds of changes, without having to wait for the whole texture to rebake again.

Blending modes will be really important. You'll probably want to set the spec layer to something like Hard Light. Multiply can be a good choice for reflections. But don't take those as hard rules. Play around with each layer's mode. As you get more practice, you'll develop an instinct for what works best with what.

Thanks again, Chosen

That new method works great as well. It's hard to see if the reflection is correct, because you can't really tell on the node and where in the preview (before I render the frame) I can clearly see that the other parts have a blue shine I can't see it on the one with transparency. But it should be ok.

I'll have to push myself to possibly render out different properties separately in the future. For now I think it'll make things too complicated for me. Though I have considered rendering out ambient occlusion separately for my project. I found a light setup that works for me, but to then enable raytrace or depth map shadows and make it look good is tricky. (I know about light and shadow linking, but I still find it hard.) Rendering out an occlusion pass might be nice there, so I get some shadow between objects that way.