Newgrounds.com — Everything, By Everyone.

At 11/9/06 11:51 PM, attemptedperfection wrote: but I used exactly that situation for my example to prove my point

Alright - I'll PM you later - perhaps we should do a test run using your system - we could use one of the past MAC's as an example - rejudge it and see how we feel about the results.

At 11/9/06 11:55 PM, WinTang wrote: Btw I believe that Maestro's comments on the Christmassy/Not Christmassy thing were pointing out a discrepancy

I'll freely admit the possibility that I misinterpreted those comments - all in all I agree with everything that Rage said - my previous post wasn't meant to challenge or debate him in any way -

Rage - if your reading then I apologize if I got your comments mixed up!

I'm sorry guys is there a page that gives the comments for every piece?

At 11/10/06 12:08 AM, XBrav wrote: I'm sorry guys is there a page that gives the comments for every piece?

Woot top ten lol..... less critical with the numbers though its like u don't have any faith in the portal's musician great contest guys.

At 11/10/06 12:12 AM, Syntrus wrote:

At 11/10/06 12:08 AM, XBrav wrote: I'm sorry guys is there a page that gives the comments for every piece?

Woot top ten lol..... less critical with the numbers though its like u don't have any faith in the portal's musician great contest guys.

Yeah it was an awsome competition. I missed the show, nobody's posted the mp3 rip of it, and I wanted the reviews for pure critique.

At 11/9/06 11:43 PM, LJCoffee wrote: "Can I propose a workable solution that will solve this problem?"

First, a "theme" slider. I agree with others that the rankings are pretty fucked up. Like how the hell did RGH's jungle-safari song place 4th in a XMAS contest? I couldn't disagree anymore with that outcome, the song may be good but because of it's distinct jungle qualities and sound clips, which can be considered to be the exact opposite of xmasy, I wouldn't put that song anywhere near the top 10 for this contest. Second, when you look at some of the scores you can clearly see which people got the harsh judges and which didn't, and some of those harsh judges were so harsh that they had a big impact on the accumlative overall score of the song. Then there were some people with crappy songs who got lucky enough not to have any harsh judges assigned to them and ended up with a higher ranking than some of the good songs which deserved the spot more. Apparently, having an "originality" slider or a "clarity" or "diversity" slider just isn't good enough because individual judges have individuals perceptions and definitions of what originality is and what clarity is and what diversity is. The only way a system of judging like this, where you assign different judges to different songs and don't have every judge rate every song, ever works and produces fairly accurate results is when you throughly screen all the judges and have them study a specific rubric or criteria to judge the songs on, otherwise, their own personal misconcepts or biases get in the way of judging and totally screw some songs over.

My solution basically summarized:

1. Replace the "appeal" slider with a "on theme" slider.
2. Possibly write up a specific rubric or criteria for every category that the judges can follow so they don't have varying definitions or perceptions of the categories.
3. Find yourself some judges that would be willing to judge EVERY SINGLE SONG for the sake of accurate results. If you can do this step then performing step 2 would be rather unnecessary, since then a harsh judge would just judge every song harshly and only bring down the general range of the scores, yet it'll still be fair since everyone got judged by this same person.

There.

Pehaps I'm misreading your post, Rage, or perhaps I'm simply too tired, but I feel personally attacked by your comments. Forgive me if I'm way off.

I rated RyeGuy's song all tens except for the overall score, which I docked considerably for not being blatantly christmasy. But I don't see anywhere in the rules where it says exactly what is required for a song to be considered appropriate, and I hear several aspects of that song that could be related to christmas or the season.

I see that my scores are very high compared to Win's, and accept that fact. That having been said, i stand by every one of my ratings. For the most part I was extremely impressed with the quality of the submissions I reviewed, and congratulate every single artist on their hard work. In my mind, they each earned the EXACT point totals I gave them. I'm sorry you don't approve.

yo dudes lets chill.. this mac stuff is an intresting deal every time and the audio every time gets better and better. ths is good for the audio portal and trisekele truly he speaks from the heart.. i would encourage more cooporation with them damn flash artists so they don't constantly use px9 and actually look around for there music lets not have a big name come in but instead lots and lots of small name flash artists get in on the action perhaps an audio flash collab where we compete music vids. the project and this point seems in comprehensable but if we give it some time and we play our cards right maybe just maybe we could do somthing like that... i don't hole any h8 towards the judging its hard to judge and artform acurately because art is different for everyone. Krizzle is right in the sense that judges should judge every song so they are all held to a standerd. look at american idol all those guys are held to the standerd of the three judges. no drastic changes just the three of em. but still just a great contest i'm like a philosiphizer i write a hell of a lot but it never really goes anywhere in closing lets go ride bikes on tuesday.

Heres some reviews for some of the tracks i judged. I didn't review them all, just a few.

cornandbeans - {Christmas Grooves} (original) Score = 75.4

Had a nice beat to it, very, very happy indeed. The drums where brilliant and fitted perfectly with the song, and the other instruments gave the song a very lively spirit. Definately a winner in my books.

MaestroRage - _-={The Epic Christmas!}=-_ (original) Score = 84.6

Very evil sounding. I loved the bells in this song, they were great! The orchestra played very well and made it sound very Professional. By the way i meant to ask, what is it that santa says at the start of the song? Coz i really wanna know!

MusicalSerenity - MAC6 October X-mas Submission (original) Score = 81.7

Very, Very different from your other pieces of music. The sound was amazing and the whole song turned out spectacular. Too bad it was so short, i would have loved to hear more out of this style of music. Maybe another one like this next year for the MAC7? Anyway the song was great and a lot better than your trance tracks, you should be proud of yourself!

MusicalRocky - Familiar Blanket of Snow (original) Score = 78.1

The rhythm was very stylish and sounded very christmassy. The length was good and the instruments used were great too. Hope to hear more songs like this from you for the other MAC's.

SBB - Mirrors in the Snow (original) Score = 85.1

Really good song, i loved the entire build of this song. Great piece, you have alot of talent and if you can keep this good, there will be lots of great opertunities to come.

fr3kn - The Assassians Christmas (original) Score = 93.4

This song was my absolute fvourite song, i'm surprised it didn't get into the top 5, but at least you still got 7th place. The shotgun sound at the start is just to die for, and t gives the impression that Santa is gonna go kill some Punk-Ass Bitch for playa hatin him. Anyway great song and keep at it!

Spikrodd - Mamberiences (original) Score = 30.1

Not really very Christmassy. It wasn't equally catchy either. A rather dull and boring peice.

I would like to hear rucklo's/lj's take on this (the use of christmas theme as a scoring element) and here's why.

Judges' instructions in keeping with the christmas theme was the following:

Remember, this contest is for the best Christmas music so please take that into account when you judge.

Now, that's obviously vague. But I'm not pointing fingers - in fact, I think leaving the constraints very loose was an excellent idea, allowing excellent non-traditional christmas music to gain noteriety, and to encourage originality. In the Woods was one such tune, and as I said before, I completely stand by my review.

At 11/10/06 12:47 AM, Triskele wrote: Pehaps I'm misreading your post, Rage, or perhaps I'm simply too tired, but I feel personally attacked by your comments. Forgive me if I'm way off.

Triskele my comments were in no way an attack to you, I was merely using you as a judge on the opposite spectrum of WinTang, there were others I could have used, but your reviews were the first to hit my eye, please don't think I was out to get you.

I rated RyeGuy's song all tens except for the overall score, which I docked considerably for not being blatantly christmasy. But I don't see anywhere in the rules where it says exactly what is required for a song to be considered appropriate, and I hear several aspects of that song that could be related to christmas or the season.

But he still got a much much higher rating then he should have gotten, you see this is what I meant. You rated the song mostly for the song, and then with the christmas theme on the side. Should WinTang have been assigned to that song, he very well may have just gone "This isn't a christmas song - REMOVED!" or severely lowered the mark. Clarity, Diversity, effort and all those other fields really shouldn't have been the heavy weight here, I am not attacking you, I am pointing out how differently you and WinTang were looking at things.

I see that my scores are very high compared to Win's, and accept that fact. That having been said, i stand by every one of my ratings. For the most part I was extremely impressed with the quality of the submissions I reviewed, and congratulate every single artist on their hard work. In my mind, they each earned the EXACT point totals I gave them. I'm sorry you don't approve.

And this is what I meant as well, you see, you speak very leniently too, in truth I respect and encourage that, for there WERE quite a few artists who have not yet mastered their own styles or craft, and so words of encouragement are preferred, but your words of encouragement were also a huge factor in overall scores for many artists who deserved better.

LJCoffee, please don't think I wrote that all just to complain, I wanted to first provide what I saw to see if these were indeed problems or I was just imagining conspiracies "Da AP code", now that I am aware I am not the only one to think this, I WILL send you all of my ideas that i've been brewing up, AND I will post some of them here to get feedback.

Attemptedperfection, giving judges a set amount of points to disperse won't work, what if a judge runs out on the last song he has, does he have to go "well... I didn't have much points left so I give you 1.3 !!!!!"

I realize no judging system is perfect, but even in the Idol reality show, there are three judges for EVERY person.

I would be willing to become one of those to go through EVERY piece if that is going to help in anyway, and forfeit my right to compete effectively.

I think it may help if we assign judges to categories.
Let's say a soon-to-be-judge lists the types (corresponding to the various portals, e.g. Classical, Drum 'n Bass, General rock) that he/she is proficient with - if not in creating it, then at least in listening to it often. Rucklo and LJC can then assign every judge to 2 or 3 categories, and fill up the gaps.

I also think that you should only judge if you KNOW you have good stuff to say about music, not just because it "might be fun". This is not to criticize any of the judges of this MAC but just a general thing to keep in mind.

At 11/10/06 08:22 AM, MaestroRage wrote: Attemptedperfection, giving judges a set amount of points to disperse won't work, what if a judge runs out on the last song he has, does he have to go "well... I didn't have much points left so I give you 1.3 !!!!!"

no, I meant let the judges listen to all of their assigned songs, and afterwards deliberate over which songs are most deserving of their points and which songs should receive less

At 11/10/06 10:15 AM, WinTang wrote: I think it may help if we assign judges to categories.
Let's say a soon-to-be-judge lists the types (corresponding to the various portals, e.g. Classical, Drum 'n Bass, General rock) that he/she is proficient with - if not in creating it, then at least in listening to it often. Rucklo and LJC can then assign every judge to 2 or 3 categories, and fill up the gaps.

I also think that you should only judge if you KNOW you have good stuff to say about music, not just because it "might be fun". This is not to criticize any of the judges of this MAC but just a general thing to keep in mind.

I'm not sure how effective this would at solving the problem. Although it's a good idea to have people judging genres they are familiar with, it still wouldn't help against scoring fluxuations.

For the most part, it looks like the judges felt the same way about the pieces, it's just some judges were lower scorers than others. I think the idea brought up a page or two back by Attemptedperfection is the best so far. By giving each judge a limited # of points to distribute, then you don't have to worry about scaling issues. For example, you could add 50 points per submission to the total points, and then tell them a score rage (0-100). Having a rubric could help, but I think this is one of the better ways of keeping things fair.

At 11/10/06 05:01 AM, DarKsidE555 wrote:

Greensleves...Water/Wave sounds? Do I have to say more!? Also, the bloody original is a ballad not a christmas tune...

The tune of "Greensleeves" has become associated with Christmas over the years through the lyrics of "What Child is This?" to the point where it's hard to find an instrumental Christmas CD without it. So what if I used its original name? That doesn't change the music, which many people now associate with Christmas. As for the "water/wave sounds" I was trying to be creative as well. At the time I was working on this, I didn't suspect that greensleeve's Christmasness would be brought into question, and since I was doing a remix, I didn't feel it necessary to make the music itself particularly joyful or saturated with bells and chimes. I thought the wave-static effect contrasted nicely with the flute and ambient sounds I was using, and so I used it, as it also adds to the effect of breaking from the song structure for a moment. On the whole, I was hoping for this to be used in a flash with a harsher Christmas theme, as there are sure to be many violent renditions of the jolly holiday. At any rate, I don't think it's quite fair for you to talk about creativity like that after downplaying mine. Yes, the scores are unfair, but what's done is done, and hopefully those who were judged unfairly this time around will get their deserved scores in the next mac. Please understand I mean you no disrespect; I'm just defending my song.

Okay based on the vast differences between each judge’s scores, I agree with a lot of the points that have been made by MaestroRage and others regarding such huge gaps in scoring

I would like to put forward a few suggestions of ideas I’ve been trying to formulate, some of which may be agreeable by some of you and some of these ideas may not!

Well the idea that attemptedperfection came up with, I feel could perhaps work, not exactly how he put it, but in a kind of way

I feel that each track should each start with a particular score, rather than the accumulated score that the judges have used thus far. Possibly set it at 100 or maybe even 50 then work backward from that

With this in place, I would like to put forward the idea of the Emotional Impact each submission had on the listener/judge

For the Christmas themed MAC (as an example) the idea of Emotional Impact could work and perhaps for other seasonal pieces too (should there be any other seasonal based MACS)

Now based on the current rating system in place for music reviews (minus Overall) and add to that the “Emotional Impact” of the piece, that now gives 5 criteria for reviews, 5 being divisible by 100 = 20 as is 50

Emotional Impact
Diversity
Originality
Clarity
Effort

So using those Criteria, it should be possible to review the piece, bearing those criteria in mind

So say with this idea a particular piece gets the following scores:

Emotional Impact: 6 remainder (out of 10) 4
Diversity: 4 remainder (out of 10) 6
Originality: 8 remainder (out of 10) 2
Clarity: 7 remainder (out of 10) 3
Effort: 6 remainder (out of 10) 4

Total 31 or perhaps 62 whatever if you wanna make it out of 50/100

remainder Total: 19 or perhaps 38 whatever if you wanna make it out of 50/100

then deduct the remainder from the 50/100 Starting score

Emotional Impact could mean that the judge/listener has ideas in his mind of what exactly Christmas music should be about, what would you expect to hear? would it be lively? etc… (needs a little more thought perhaps) I’m just throwing an idea out there, perhaps some of you could correct me, add to this or whatever even ridicule the idea if you must

what do you think?

I like the idea of emotional impact, but what makes the deduction system any different from just giving them, in your case, a "4, 6, 2, 3, 4" to start with?

or "6, 4, 8, 7, 6" rather...

At 11/10/06 11:09 AM, attemptedperfection wrote: I like the idea of emotional impact, but what makes the deduction system any different from just giving them, in your case, a "4, 6, 2, 3, 4" to start with?

Ok yeah well It does need a bit more thought definitely! just an idea, the deduction system may not be a good way! as I think I see your point with that, I suppose that is really the same as the accumulative scoring, I dunno really just throwing ideas out there, the main one being the emotional Impact of the piece

Each judge/listener may have to decide and agree between them, just exactly what a seasonal piece should be about

At 11/9/06 09:01 PM, WinTang wrote: TJA
CHRISTMAS MAGIK – 54.7

im surprised i got a review that good. Trance isnt my strong point AT ALL, since ive been doing drum and bass and trip-hop for so long. It was suppose represent christmas in dream sort of way the good feel you get from christmas but when it quite not there yet, thus why the track never built up. the ending was rubbish because i never finished in time/ finalised it unfortunately, due to collage work geting in the way. by the way did you notice that you cant hear a bass... i forgot to put it back in after editing it in audacity :P
thanks for spending the time to listen/review it.
i think ill go back to the music im good at :)

Congrats DavidOrr btw, I didn't mention that yet :)
I think my idea will remove some of the inconsistency, because it will make it easier for judges to have a general idea of what the overall quality of the submissions is. Plus, it will be an overall improvement in the accuracy of the ratings - in case of attemptedperfection for example, which seems to be a very controversial submission, it would be very beneficial to have him judged by people who appreciate an unconventional style to begin with, to eliminate both "sorry no X-mas dude, all zeroes!" and "whoa you must have an IQ of 600, all tens!"

To stealth-annihilator: in the MAC judging there is an extra category apart from the ones we're used to in the reviews: "Catchiness/appeal". I think that pretty much covers 'Emotional impact' or whatever impact the judge is looking for. It weighs 25% of the score total.

To anyone: let's try not to get anal about this, bottom line is that we have a well-deserved win for DavidOrr, I think most will agree on that. It's good to lay bare the weaknesses of the judging system, but as LJCoffee said, whining doesn't help, making suggestions might.

Yeah, I don't think anybody has any issues with the top spot, so it's not like the system is completely invalid; the best is typically going to be recognized as the best. Sure, the top 5 may be a bit iffy, myself included, but it's not like we're the ones getting the $250, or whatever it is... and it's not like we're necessarily getting more exposure either; I have seen neither an additional review nor a download since the results were announced. So really, it isn't that big of a deal; we all know the results are flawed, and now we're working on unflawing them a bit for future contests. no worries. turn those frowns upsidown! and remember: whether you're the pitcher or the catcher, always wear a glove.

wow, apparently I'm practically guaranteed to be used in a flash!
:D
awesome.

but you attempted perfection, and just didn't get there ;).