Post by redangel121 on Feb 15, 2015 10:28:19 GMT -5
This depends on how many maps get submitted. I think we should close out the mappers signup on schedule as to not increase/overload the judges time in getting them rated. If everyone gives 1 map that's at least 22... what if some give us more than that? When the maps are all finalized and submitted for judging we should deliberate then on how long the judges have to judge them. 1 week seems a little short on time to judge ~25-40 maps unless we limit them all to only one map to submit. I really don't want to limit the mappers because then they have to choose what they think is better and submit that, instead of the judges choosing the best.
I guess our advertising campaign worked like a charm, and maybe was a bit too effective I'm also in favor of not limiting anyone who has already signed up.
If nobody has any objections, I'll be closing the mapper sign up at the end of today. (in less than 9 hours) So if you know anyone else who'd still like to join the contest, try to let them know as soon as possible, because time is running out.
The signup deadline is there in consideration for the judges most of all.
According to Wixen, even a contest of a lesser size was a hefty logistical challenge and a real headache to judge. Mind you, the previous contests were tiny in comparison and we've broken all records for Q2 cafe at least. We're now in uncharted territory in terms of size and some judges might not feel comfortable taking any more mappers due to the added workload. One potential solution might be to randomly split the judges into two or more groups, and randomly assign maps to each group for fairness sake. Either way, we need to take necessary steps to ensure new procedures are put into place, to deal with a contest of this scale, before we continue on this path, and so we don't have any headaches this time around. I'm willing to listen to suggestions, but we need a resolution soon if we're to keep the signup door open.
Wixen has first hand experience with the last two contests so he'll be able to tell us more about his concerns and past experiences. He directly presided over the results from contests 8 and 9, so his input needs to be instrumental in forming a new structure if he feels we've outgrown (might strain) the old one. It'll then be up to us to revise and streamline the judging procedures in a way that ensures fair and unbiased ratings to be obtained from all judges. For example we'll need to instruct the judges before hand step by step and have them go through a sample map selection to ensure that they clearly understand the regulations and rate accordingly. Certainly they'll need to understand how to properly take r_speed measurements which will be tricky without prior training. We might need someone to come up with an instructional tutorial for that. It's important to do this all before the end of the contest, so that when we get there, our rating machine is nicely oiled up and ready to go. Ideally we want this to be like second nature to all the judges, so they produce fair ratings at the other end, ready for me to collect and put up as final results for contest 10.
I think, as we now have so many maps coming in, that judges need at least 2 weeks to do their job. I have been judging before and it is very time consuming if you want it done right. I don't think it's a good idea to split up the judges, as I feel the overall judgement will somehow turn up incorrectly. All judges must vote on every map in my oppinion. In earlier contests we made sure that the judges who signed up had the necessary qualification and experience before approval, I am not sure if this has been done here?. If we want it to be a proper judgement, this is a vital issue. And not to forget, the head admin has to recalculate all the scorecards before making them officiel, this was for me a mayor task, as many was with filled with errors. We can not have a single flaw in the final presentation. And yes, maybe we need a deeper explanation about how to judge a map according to the rules. Maybe the headjudge already has the scorecards ready for this contest?
Maybe the headjudge already has the scorecards ready for this contest?
Actually I was thinking of reusing the old ones, and maybe just tweak them a little. Since everyone should be able to fill in a PDF table without buying any software, maybe we could use that instead: www.foxitsoftware.com/Secure_PDF_Reader/
9 - All mappers will keep their maps secret until the deadline. At that point, the download links or zip files will be supplied to Head Judge in a private message. This way, the maps will all be released to the judges on the same day. At that point, Head Judge will forward all of the download links or zip files to the judges by private message or email.
RULE 9 (CHANGES)
Since we already have a screenshot thread I find this rule to be obsolete. I'd like to propose a change. I've examined the past contests to see what we could do better this time around, and I think there's room for improvement. It will require providing server access to help all the contestants create DM suitable maps. Currently rule 9 stands in the way because everyone is required to work in secret. In the past there's been a few contests where this worked against us, and death match performance wasn't taken seriously enough. As a result even a few winners didn't stay long in the death match area. DM performance therefore has to be at the fore front of this contest because it's the main thing with staying power that will ensure your maps actually get played or not. And in this contest we'd like to see more winners than losers, as far as game time is concerned. I understand it's hard to create death match maps and know exactly how they will perform with players unless you can actually test them. Most novice mappers or SP mappers might have limited exposure to DM testing and will not know what the players expectations might be, or even how their map will perform under the death match settings. Testing with bots has its limitations, and will not really tell you that either, unless you hear it first hand from the players. I doubt most mappers here will want to wait till after the contest to find out that their map isn't suitable.
For this very reason I'd like to propose a new dynamic format, where each mapper is able to test their map live with players before it is submitted to the judges for final ratings. To do this at the end of each week they'll be able to send their current map version to the head judge for early upload to the server with a temp man name, which they'll change later of course. This will ensure that there are no naming conflicts and that each mapper has equal access to player feedback and can iterate / perfect their map with death match in mind. Certainly we want a close race, but since this is a death match contest we also want to help all our mappers create maps that will succeed with players and get played a lot, not just once. I'm sure nobody wants to spend a lot of time creating maps that players will reject, so through player feedback they can all be forged in the fire of creative criticism and come out as smooth and solid as steel on the other end. Some maps will still be more perfect than others and will be awarded top spots, but at least all will be usable. I believe this way there will be more winners than losers overall, and it's what we all want. But if anyone feels that the element of surprise still works better in their favor they can ignore these server tests (& test independently), and instead use the dates below to check in to let us know they're on track and capable of finishing, and if not we could at least try to offer some help.
Dates (for weekly DM server testing / mapper check in / status report)
Sunday Feb 22 - (mapname_v1) Sunday Mar 1 - (mapname_v2) Sunday Mar 8 - (mapname_v3) Sunday Mar 15 - (mapname_v4 for last tweaks and later just mapname as final version)
I have nothing against testing the map, gameplay is the most important part of a DM map in my opinion. It think it is a great idea.
But: I do have little time on the weekends though and if you make it mandatory to check in / play / submit a new map version every sunday, I may not be able to do this. I will try to make it at least once though, I think it's a great offer.
Spirit, Since we have players on the server week round, you could do death match testing any day / time you feel works best for you. I'm sure you'll be able to find a more appropriate time So feel free to upload to mediafire anytime you got something and just send me a pm when you do, so I know when to upload it to the server. I only suggested Sunday as an example day for check-ins and catching up, considering we still haven't heard from some of the mappers since they joined :-)
Yepp. I just felt I had to write that. I got confused reading the scoring sheet. There are so many areas to score.. that I feel there are things overlapping. A part of me wants to simplify it, group things together; less areas to score. Example: Visuals, Design, Sounds, Performance, (Subjective)
I am not 100% convinced about Subjective. Isn't that something that will shine through anyway? As a judge you have to be professional and score as objectively as you can, but you will still have preferences that will make for higher scores. And finally, just to explain the above simplified scoring sheet:
I am not 100% convinced about Subjective. Isn't that something that will shine through anyway? As a judge you have to be professional and score as objectively as you can, but you will still have preferences that will make for higher scores.
This has been debated long ago and we decided, that as the subjective is only 1/10 of the total score, it will not be able to shine through.