This is a halfway solution between hearing everything a particular avatar has to say and 'ignoring' them and having them disapear from your client.
In the Chat History Window each avatar's name will have a small box on the left side. If you want to follow that avatar's conversation the box has a '+' symbol in it and you will see what they have said. If you do not want to follow an avatar's conversation you click on the box, which switches to showing a '-' symbol and you then only see their name in the Chat History Window, not what they have said. You can toggle it back on again by clicking the box once more.
This way you can see if that avatar is talking, but not be distracted by their conversation if you're not actually interested in it.. and this makes it easier to focus on the conversations of those avatars that you do want to talk with.
http://secondlife.com/vote/vote.php?get_id=2704