Role Manager Privileges That Exclude ability to add to administrator group

It would be ideal if someone could have role manager access but have that access limited to specific user groups, especially the administrator group. Currently if they want someone to be able to add new users to user groups they have to be an administrator or have maintain role features, but giving the maintain role features allows the user to add themselves as an administrator.