Instant Message button for Group Spam is a problem...

I for one reject all group invites and keep my character group free because of one reason. The only time I ever want to the the "Instant Message" button on my screen is when I receive at actual personal message. Group spam keeps that button turned on all the time which is bad because it causes me to miss personal communications. I've looked through the options and can't find a way to make group spam not make the button stay on. Frankly, this really sucks.