I don't believe this is currently possible. My recommendation would be to create a new conversation with the people you want to include. You can then leave the old conversation.
And just to add, if you feel someone is breaking forum rules in a conversation, you can always report their message.