Scoring and judging discussion

[Pokemon Trainer Sarah]

Alright so this is what I'm thinking, so that we can go ahead and recruit judges:


JUDGING

• Judges should provide feedback to all participants after each challenge and must have a clear grading rubric.
• Judges will be assigned 100 points to distribute among the entries, with the best entry receiving the biggest share of points, etc. Each entry must receive at least 1 point, but it is otherwise up to the judge to distribute.
• No single entry may receive more than 35 points. (Therefore not all points must be awarded if there are too few entries.)
• Judges cannot participate in their section, but will receive points for submitting their judging each week in a timely manner.
• Judges should submit their 4 week plan for their section (number of challenges, what each challenge is) before PXFIRE begins.

We could drop the bold rule if there is no issues with participation... What do you guys think?

Participation recognition could be handled perhaps by having separate ranks that aren't worth more points but can get you a token or a cool postbit thing? I know I did this for Team Trainer. By participating in at least one thing each week you would reach the maximum rank. We could either award medals for participating in any event at least once per week, or alternatively, medals for participating at least once in each event. But that's something we can still sort out once we have judges. :)

Any comments or ideas?

[Soups]

I see no issue with the bold rule. It's fair and it's a great safety net. I still think it would be easier to let the teams themselves reward their members for participation.

[Morzone]

when you say that judges will receive points for submitting their judging in a timely manner, how do you define a "timely manner" and how many points would they get? Other than clearing that up I think it all looks good.

[Pokemon Trainer Sarah]

when you say that judges will receive points for submitting their judging in a timely manner, how do you define a "timely manner" and how many points would they get? Other than clearing that up I think it all looks good.

Usually there will be a deadline every week as to when the judges need to submit scores by. Generally a few days after submissions close. So if the events run from Sunday to Saturday, then the judges could have until Tuesday or something to judge and submit scores. If they don't get the scores in on time then it's not such a big deal except they won't get the points. I'm not really sure how many points are fair... any ideas?

[Morzone]

Maybe...

If the judge can get the results out within 24 hours of the closing they get 20 points, if they do it within 48 hours they get 10 points, outside that they don't get any?

[Soups]

Maybe...

If the judge can get the results out within 24 hours of the closing they get 20 points, if they do it within 48 hours they get 10 points, outside that they don't get any?

Personally, I don't think we should give points for judging... While it's important to submit the results in a timely manner, I wouldn't want to see any judge rush through judging simply because they'll receive more points, you know? I'm not saying that WILL happen, but if there are points on the table... You catch my drift?

It's cool if we do though.

[Neo Emolga]

Personally, I don't think we should give points for judging... While it's important to submit the results in a timely manner, I wouldn't want to see any judge rush through judging simply because they'll receive more points, you know? I'm not saying that WILL happen, but if there are points on the table... You catch my drift?

It's cool if we do though.

Ehhhh... give judges points. Otherwise their motivation will wane quickly and they may drop out after they get tired of working for free. It also discourages people from judging when they could not judge any sections at all and simply take part in the section themselves. At least if you give them points, it's a motivation to do a good job, submit results on time, stick with it for the entire length of PXFIRE, and thank them for their time and dedication, especially after giving up the chance of participating in that section themselves.

[Soups]

Ehhhh... give judges points. Otherwise their motivation will wane quickly and they may drop out after they get tired of working for free. It also discourages people from judging when they could not judge any sections at all and simply take part in the section themselves. At least if you give them points, it's a motivation to do a good job, submit results on time, stick with it for the entire length of PXFIRE, and thank them for their time and dedication, especially after giving up the chance of participating in that section themselves.

Hm... I didn't think of it that way. Both sides could be argued, but I see your point. The pros outweigh the cons if we give judges points.

[Neo Emolga]

Hm... I didn't think of it that way. Both sides could be argued, but I see your point. The pros outweigh the cons if we give judges points.

I'd say go with Morzone's plan. 20 points if you do everything on time. 10 points if you're late, and nothing if you're super-late or don't post results at all.

I would say the deadline is Midnight on Monday night (we just need to decide on the timezone we're going with but usually it's EST). Results have to be in before then to get the full amount of judging points. I like Morzone's plan where if you're late but not by 24 hours, you can still get half the amount for 10. After that, you get zip (and risk having your judging hijacked by a PXFIRE leader who can declare winners and set the next week's theme AND/OR possibly get replaced).

In the past, I set a judging plan where it goes like this:

MONDAY-FRIDAY: Standard time for submissions. There will definitely be no judging here.

SATURDAY: Last minute submissions. If you submit something before the section is judged, it counts. However, judging CAN happen on this day as well, and if the judge posts results before you submit, boom, you're too late.

SUNDAY: Submissions for the week are over. Judging now happens, but once judging for the week is done and the judge has posted the next theme, it's open season for submissions again.

This gives plenty of time for submissions and gives judges the weekend to get results up. We've used it before and I feel it's a pretty solid and fair system.

For things like RP where posting never really stops, the judge can decide where the cutoff is between what counts for last week and what counts for the next week.

[Morzone]

When looking for a roleplay judge we'll have to make sure they tell us when their cutoff will be each week ahead of time so we can make sure they stick to it instead of trying to worm in extra points when they were slow at posting results.

Alternatively we could say the cutoff is a certain time every week and make it standard across the board. (like 5 PM on saturday or something)