I think quite often it can be black and white, other times it's not that simple.
Sometimes if people don't like a track they just move onto the next one, which I think is fair enough because if you don't like something, your vote is already going to be biased and unfair, so having the choice to leave it alone I guess would be the right one.
I think it's better for people to have the choice to walk away without voting, rather than them having to vote and their vote being bias in a bad way. You can't force people to listen to your music after all, and it's up to the listener if they want to vote or not.
I also believe you have to have some degree of interest in a track to be able to vote fairly enough, which again is biased because if you like something you arn't going to vote low on it, unless it's done out of jealousy and spite.
And of course you will always get the people that see a final score and want to alter it, for example if a song has had 50 votes and has a total score of 4.5 and they think it's only worth a 4, the're going to vote zero to make that more of a reality.
If anyone can make sense out of what I just wrote, I salute you!