Ambient Occlusion (Maya)

Second question from me. Some wonderful forum people helped me solve my last problem, so I thought I'd give this a shot as well

I recently learned how to do ambient occlusion in Maya. I read up on the subject a bit beforehand. I know ambient occlusion is a kind of global illumination that's supposed to get you more realistic lighting. I did some tests and love the results.

However, I am still really confused as to how I should use it. I could combine the rendered (white) occlusion textures in Photoshop with my plain texture to get baked lighting, but is that all? I mean: I'd still like to apply bump, shine etc. to my textures. How do I go about combining this with ambient occlusion? I can't just combine the occlusion pass with another baked texture can I? Wouldn't I lose all those beautifully subtle shadows created by the occlusion?

I guess my question comes down to: how do I combine occlusion with materials and properties like shine and bump mapping?

Any advice would be greatly appreciated.

Two methods:

1. You could do a separate pass for each property (occlusion, bump, spec, etc.), and then composite them all in Photoshop afterward. The advantage with this method is you can control the exact amount of each property, simply by adjusting things like layer opacity in PS. The disadvantage is it's very time consuming to do all the separate passes.

-OR-

2. Bake all properties at once, and output to a single texture. The advantage is it's faster than doing a bunch of individual passes (although considerably slower than just a single-property pass, of course). The disadvantage is if you want to adjust the strength/weakness of any single property, you need to do the whole thing all over again.


Specific instructions for both methods will vary, depending on what renderer you're using.

If possible, I'd like to bake everything at once, for the reason you stated and also because I wonder if the result with a PS overlay would really be the same. From tests I've done now this doesn't seem to be the case, but I'm not sure I'm actually doing it right.

I found a tutorial that recommends linking the ambient occlusion texture to the 'ambient color' attribute of whatever material I'm using. I tested this with a lambert and it does work, in that sense that I can have my occlusion and bump mapping (from the lambert) all at the same time. I've noticed however that the textures come out very, very light. The shadow areas are about as light as the original texture, the lighter areas come out very heavily lit. I'd want the lighter areas to be the same color as the original texture, and the shadows darker, not what I'm seeing now.

I can get around this a tiny bit by making the 'Light' color in the occlusion attributes darker, but this way I'm washing out my shadows.

I'm using mental ray to bake this. Are there perhaps particular settings I should pay attention to? I set the quality to Production, but left the rest as is.


(And howdy again Chosen, was hoping you would answer )

It's been a while since I've used Mental Ray in earnest. Turtle's been my renderer of choice for years now. Turtle makes it really easy because it puts everything in one place. You just check the options you want included in the bake, and it takes care of the rest. Mental Ray can yield results that look almost as good, but the setup is significantly more complicated, as you're discovering.

As for why the lit areas got brighter, here's the logic. The ambient color of a material is the color the surface will be when in shadow. Before you applied your AO map to the ambient color channel, the ambient color for the whole surface was probably default black. Your AO map changed all that, though. The most brightly lit areas would now have their ambient color mapped to white. So of course they got a lot brighter. Take a surface that is white when not lit (impossible in the real world), shine a light on it, and you get "super-white". It basically glows.


Here are a few options you could try, in order to lessen the effect. Note, I'm speaking purely theoratically, since I haven't used MR in so long. You'll have to experiment to see which, if any of these options work best:

1. If you set the background color to white when you created your AO map (common practice), set it back to black before you do your final bake. This might or might not make any difference. I'm really not sure.

-AND/OR-

2. After you create your AO map, and assign it to the material's ambient color, relight your scene to compensate. Crank down the intensity on all lights by maybe 50% or so.

-AND/OR-

3. If you created your AO map with default Bright and Dark settings (white and black, respectively), try redoing it with Bright set to 50% gray instead of white.

-AND/OR-

4. Try redoing your AO map with an increased Spread setting. The default is .8, I think. Higher values will spread the occlusion shadow further over the surface.

-AND/OR-

5. Try adjusting the levels on your AO map in Photoshop, to lower the intensity of the whites.


That's about all I can think of for now. Again, I'm years out of practice with Mental Ray. Turtle effectively eliminates most of these concerns. I guess I'm a little spoiled.

Note that ambient occlusion tends to produce darker shadows in corners than you would get in reality... including some places that don't get a shadow at all, or that would actually look lighter in RL due to Mach banding.

It looks dramatic, not realistic.

Apply it with a light hand, you'll get better results.

That was another thing I was wondering actually, if AO can be used on its own. Personally, for now, I really like how it looks. But I'd have to get the project I'm working on properly textured to really have a clear idea. Up until now I've always used kinda harsh shadowing in my products. It looks less realistic in a way, but I think in-world harsh sometimes looks so much better. In the beginning I used ambient light for a while, but then I switched to directional for nearly everything. I guess part of lighting is personal taste? I haven't found THE perfect setup for me yet, I bet I'll evolve a lot still.

Chosen, I'd actually tried all those things except altering the texture in PS. I'd indeed turned the background color to white, changing it back unfortunately doesn't help, neither does decreased lighting on the scene (even turning the lights off makes no difference). The third suggestion I mentioned in my second post. It does help, but also brings the shadow and light areas closer together, which I thought wouldn't be ideal. Changing spread helps just a tiny bit. The texture is still very 'glowy' in light areas.


Ah, I was under the impression that the whites of the occlusion would overlay, like they would if you recombined separate passes in PS. What I did do now however, something that I should've thought of before (but it's perhaps kinda stupid): I placed something around my shoe in Maya. This does dramatically help, ofc, since the occlusion takes this new object into account and the lighter areas are no longer considered 'white'. The lit areas are now my base color, the shadows are darker. This solves it I suppose, just seems like such a noob to do it lol.

That 'solution' is not reliable really, did some more testing and rendered out a few textures. The shadowing isn't what I'd expect when I look at the occlusion separately.

I guess I should render it out separately. But I'm really confused as well as to what type of lighting I should be using then on my blinns, phongs, lamberts. (I'm assuming I should render these out with shadows as well and then add the occlusion.) I don't know why this bothers me so much now. I used to always do kind of what I thought looked good, but I've been trying to get it 'right' and I don't really know how.

Is there a preferred approach to the kind of lighting used in these cases? Especially when keeping in mind there's going to be occlusion?

Ambient Occlusion isn't accurate, so don't worry about being accurate, just use it where it makes the texture look better, and don't use it if it doesn't seem to be helping.