Different mediums of entertainment appeal to different people, and within said medium, there are plenty of shows to choose from. The problem is that the anime on TV mostly range from decent to utter shit, and that's seems like what most anime fans out there think of when it comes to anime. Don't get me wrong, shows like Code Geass, FMA: Brotherhood, G Gundam and even Monster were shown on TV, and those were amazing anime that I would recommend to anyone in a heartbeat, but for all those shows, there were a lot more crappy anime out there, some of which are still airing right now.
Another thing is, IMO, the wrong shows seem to get the most pub and attention, and a lot of the good ones seem to be barely mentioned or treated as an afterthought. Take FLCL as an example, personally, I simply couldn't stand to watch that show, much less enjoy it, partly due to the fact that it tried to jam many different tropes and cheesy jokes into 6 episodes, and lack of a deep coherent story, which the fans claim it has, but I certainly haven't found it.
What makes it worse is that for a long time, Adult Swim kept airing this damn show for over 10 years on and off, and much better shows were given the shaft, usually by messing around with the air time or even the airing of the episodes themselves, {Code Geass is the prime example} almost to the point where I got pissed off at AS for screwing around with the schedule because they seem to have a beef with anime not named Cowboy Bebop or FLCL. Of course, you could simply get all the episodes online or wait for the DVD box set to come out, {I did} but that still doesn't make it right to put anime through that type of limbo, especially if the anime is amazing.
I'll stop here before I ramble any more, but just remember that there are far more anime available than what's aired on American TV, and more importantly, there are many different genres and styles of anime designed for a specific audience and taste, you just need to know where to look.