Would it be wrong of me to ask my colleague not to participate?

I'm so confused. Are there 5 people in the group, or is the "good friend" the same person as the "exgirlfriend"/"colleague". Can you qualifty how many people there are? Who is doing work besides you? Are you getting class credit for this project? My advice would depend on whether or not there are 5 people. What do the others think? If there are only two of you and you did all the work, I think you should have a serious discussion about the possibility of you doing this by yourself. Talk it over and see if you can come to an agreement.