From the tone of your post I'd say the guy might've gotten pissed about your attitude towards him.
Here's an example:
I'm someone who had a bad day at work today. I am usually aggressive, demand respect and am also particularly sensitive about racial disparities.
I'm driving like shit because I'm pissed thinking about the bad day I had and, all of a sudden, I find myself with two white boys looking at me and talking as if they were the kings of the road.
While I wait for the green light I hear one of the two say "nevermind ... [muffle muffle] that black guy [muffle muffle]"
I get pissed thinking these little pricks are acting superior and I ask them menacingly if they actually said what I think they said: "did you say black?"
The little prick actually answers back (openly confronting me)... *that son of a bitch*
=========== got my point? =============
That was very stupid of you... you might get yourself in some serious trouble acting that way. When you notice that the other guy is getting aggressive you need to make them understand that it was a misunderstanding and then calmly decide whether it's appropriate to continue argueing or not with the man... sometimes it's best to take shit from someone if you know that the odds aren't in your favor.
You never know who you might find on the road. The guy could've been carrying a gun and could've just blasted you and your friend's ass for that. I've seen worse for much less.