To each his own.
But if you pick fights just to prove yourself, then you're the idiot. I'm a big guy (6'6", 260 pounds), but I hate fighting. I'll only fight if my life is threatened. Saying that big guys are idiots is a harsh generalization. I'd much rather talk my way out of a situation then face potential jail time and/or getting my ass kicked by my opponent's "posse". Fighting for sport, such as boxing or MMA is fantastic, because it's regulated, and there's no possibility of someone pulling a knife or gun, and the situation generally won't get out of control. Where I grew up, fights in public places usually escalated into a brawl, with usually a stabbing or two. I've actually seen four people get stabbed on four separate occasions, so I guess that accounts for my disdain of needless violence.
You referred to big guys as "idiots", and I know the type you are talking about. I see them when I go to the gym, strutting around and trying to act hard. If you feel the need to beat these people up, again, that's your choice, but why even bother? They're terribly insecure with themselves already, and why risk giving them the pleasure of winning a fight, regardless of the size of their opponent? Let them wallow in their own misery.
It's not wrong that you like to fight, but I think you are channeling your aggression towards being self destructive. Perhaps you should join a boxing gym. I have several friends that participate in amateur MMA fights, and they seem to really enjoy it. What's there to lose? You get training, plus, you get a sizable audience when you do beat someone up. You'll learn ground fighting techniques as well as become a better striker, so if a situation arises in the real world where you;re required to resort to violence, then you'll have even a better chance of coming away the victor.