Mmm yes it's difficult. Many grey areas. In your scenario obviously if you end up in front of a judge with the pussy hat (you think I'm joking, it's real) you're screwed.

But I would say you should get off, because you grabbing your stuff back is not unlawful. You did not "escalate" nor caused the altercation.

More gray area would be if I punch you in the face, with a single punch and I stop. Then you pull out a knife, I pull out a gun and shoot you. Imagine we are both men of same age / size. While I did attack you first, it's you who escalated to deadly force - in fact my punch does not give you the right to stab me.

Interesting, what do you think?