How do you make your Black Ops session not joinable?

I have Xbox Live, is there a way to not make your server or session not joinable for one friend(He's annoying) and still play on public servers. But let other friends in. Is it possible? If so, please tell me. Don't tell me to remove him, I dont want to hurt his feelings.|||You can try to change your party to (invite-only) But I'm not sure if it will stay if you move to a public match. Try Though.|||Set your party to invite only in the lobby and only invite the friends you want to join.