Quality can be recognized on its own.
A good flash, a really good flash, does not need a caveat saying "Please don't vote low if you don't like this". I mean, I know tastes vary, but funny is funny, and if you put together a bunch of inside jokes that only a handful of people will enjoy, then don't whine when the general public gets disappointed.. and votes accordingly.
Newgrounds is a public venue, not a private club. If you want a Flash to be successful, it has to have humor that the majority of people will enjoy. This flash.... not so much. I don't mean to spit on your hard work, but it just wasn't that funny. I'm sure your friends enjoyed it, and that's great. Be glad that you entertained them, and accept that after they're finished voting, the rest of the world (who could care less about your inside references) will bring the score back down to Earth, which is what happened. Doesn't mean you did a bad job, it just means that your flash didn't have universal appeal. And hey, if it did, maybe that'd just mean you're like everyone else, and you probably don't want that anyway.