I'm not sure why all of the Solaris mail programs are actually set-gid
If you strip set-gid mail from /usr/bin/mail,, /usr/bin/mailx,
/usr/SUNWale/bin/mailx, /usr/dt/bin/dtmail, /usr/dt/bin/dtmailpr,
/usr/openwin/bin/mailtool nothing should break.
(At least not if you /var/mail directory has the standard 1777 permissions)
By forcing a file permission of 600 on mailboxes, group mail should not
gain you anything.