The fact that I frequently see posts that say things such as "vote X, this movie sucks" proves that people are the problem and not the system. Everyone should know that "X" is reserved only for racist, stolen, or ridiculously offensive movies and that the only difference between it and "0" is that it notifies the NG staff. Because, there are too damn many idiots that don't know how to rate a movie, doesn't mean you should change the system.
I understand why you are thinking about changing it. I also understand that you are trying to make up for the people who seem too imbecilic to understand the X-5 system. I don't think it will work, although.
Also, (I know this was proposed before and it is a good idea) when someone votes "X," they should have to fill out a form explaining why the movie is offensive.
Perhaps, the 0-5 system should stay for early movies and be replaced by the thumbs up/down system once a movie has attained a few hundred votes. That is when 1-4 becomes a bit useless.