CAP Update: Survey Results and Going Forward

Sadly, my opinion varies depending on the poll. When it comes to things like movepool, I'd rather have Finite, but Art Submissions should be infinite due to the size of the slate compared to movepool. That said, in each circumstance, unweighted is definitely the better option, since I know when I submit ballots for Art Submissions I just put down the ones I like going from the top.

I prefer Finite for more of the process, but I think the number should never be one, since then that restricts things to the point where you are unsure if someone is cheating or not. With at least two votes, you give open ground for anyone to win. Having say, two or three ballot options allows others to see what the opportunity cost of that one user's vote is. Let's say X and I were in a poll and someone wrote down mine and X's for his options. It shows that ours were the two best options, although which one he likes more is unknown. Either way, it gives us both a chance to win, since if he's voting for the pair of us, it shows he's happy with either movepool.

Weighted, finite bold vote

I can't make a judgment on the fairness of any of these three methods.

To be honest, I'm not sure if it's even necessary to find a way to 'level the playing field'. A quick look at our list of CAPs reveals the same names cropping up again and again responsible for the competitive elements. There may be an element of bias there, but who is anyone to say that they didn't submit the best possible selection of moves or stats given the structure of the process at that time? These names show up consistently because these people generally choose the most favored direction in the first place, and the winners are after selected based on a few speed points here or whether or not the CAP gets taunt.

The art polls actually already seem to demonstrate a reasonable amount of diversity. We have a number of CAPs whose designs were submitted by relative unknowns (who often don't show up again because they then become inactive around the forums anyway).

Concept polls have clearly the greatest diversity in winning submitters. It shouldn't come as a surprise that they are well respected community members who would already have the best idea of what a good concept looks like.

What I think the weighted poll really addresses are feedback and the various forms of manipulation and cheating.

In forming a vote, the member has to consciously order based on preference. While they don't necessarily voice their reasoning, it does provide at least some information to the submitter as to where they stand.

An argument I can see for Unweighted/Infinite is that it is at least inclusive and less discouraging to someone who might not see their name appear much at all in either of the finite methods. However, I don't think the voting booths is the place one should be getting their critique or a self-esteem boost. Ideally, a submitter would be getting all of their feedback in the submission thread, and it would be great if, as a community, we tried to give feedback to everyone. Even a passing comment can do much to stave off discouragement and is worth a lot more than a throwaway vote amongst many that won't really mean much in the following polls anyway.

I don't know enough about how early polls were affected by manipulation or cheating, but the very fact that it requires a specific ordering and number seems like it could be worthwhile in deterring such actions. A person may be discouraged from having to make a similarly ordered vote as someone else, and it's much easier to spot trends in such a case anyway. At least recruited voters may have to make an informed decision on the rest of their vote if they were recruited for one specific submission.

The method may be more vulnerable to gaming, but I don't see this as a huge problem. Is it dishonest to vote for your second or third favorite option if your first option really doesn't stand a chance? People are going to want to get the best possible result, and it's unreasonable to suggest people shouldn't make an attempt at that if their ideal choice is no longer an option.

Cheating may be easier to spot by comparing the order of lists. But, I think it's a greater issue in the later polls. It may do something to help weed it out early, though, which I don't see either of the other two poll options doing.

Firstly, thank you for taking the effort to type this up. I'm sure it must have taken a long time. :)

Secondly, to answer your request:

Unweighted, infinite bold vote

In my opinion, this is perhaps the most manipulatable form of voting because it can mean 10 votes for a single entry. From my limited knowledge of how much cheating was done during CAP 11's art polls, this could allow the small minority of cheaters/manipulators to have a large voice in the polls. For this reason, this is not my choice.

Unweighted, finite bold vote

This is seemingly the most manipulation-resistant option presented here. By forcing the voter to include a certain amount of other entries in addition to their favorite, it prevents some aspects of cheating. However, this could be circumvented by voting for the least popular entries in addition to your entry of choice. In addition, this form seems to be susceptible to swarming and band-wagoning because it fails to measure the intensity of the voter towards its favorite. The vote of an apathetic is equal to the vote of a fanatic. For these reasons, this is not my choice.

Weighted, finite bold vote

This is my choice. It allows for voters to show the strength of their opinions while forcing them to consider other entries as secondary candidates, making it less influenced by manipulation. Mmm compromise. Please don't make the polls heavily weighted though.

Unweighted, infinite bold vote: this poll allows each voter to submit an unlimited amount of votes on their ballot that each have the same weight. If someone votes for only three submissions on their ballot and another person puts seven on theirs, each of those 10 votes will count equally.

The current method being used, but not the best. It's effective in keeping check of cheaters, but the individual voter could simply pick anything he wants or even go as far as to give one vote to every single participant.

Unweighted, finite bold vote: This poll allows each voter to submit a fixed amount of votes on their ballot that each have the same weight. All voters must adhere to that fixed number on their ballot or their ballot will be invalid.

Yes - I think this one is the best out of the three. Not only does it allow the artists to see more accurate feedback through the individual voters ranking their options, it does not give cheaters an easier time like the third option does. It makes the voter pick his options more carefully and not blindly pick one that he simply maybe likes or something along the lines of that.

Weighted, finite bold vote: This poll allows each voter to submit a fixed amount of votes on their ballot, but in order of their preference to advance. Votes listed higher on the ballot have a higher weight than those listed on the lower end of the ballot. All voters must adhere to that fixed number on their ballot or that ballot will be invalid.

No - Sorry, but in my opinion the fact that the poll would be weighted means the problem of manipulation and cheating would be even more apparent. Although on the good side the individual would have to consider his votes much more carefully as his polls are going to be weighted, this also means the people cheating on the polls or manipulating them would have an easier time allowing the ones they want to win.

I would prefer the unweighted, infinite bold vote over the other two options listed. Mainly, over the finite bold vote, this option allows more freedom for a voter to "tell the truth" about his/her preferences, while over the weighted vote, there's less of an ability to game the polls.

I believe I raised a concern that a voter listing more options amounts to inflating the impact of his/her vote, but now I'm not so sure about that. It may only really have been a problem if a voter were allowed to vote more than once on the same option. Now I see this as a like/dislike vote for each poll choice.

The infinite bold vote is the best option in terms of fairness. With the finite bold vote, there are problems with determining how many votes should be enforced. Additionally, the like/dislike mentality in the infinite vote is preferrable for the "underdogs" over the "vote the best options" mentality in the finite vote that may inevitably push underdogs out in favor of the veterans.

The infinite bold vote is also the best option in terms of feedback. Since not voting for a choice amounts straight up to an expression of dislike. Feedback doesn't matter during the actual polling, only before and after. If a submission is eliminated, then too many people disliked it, plain and simple. I'm just not sure that exact feedback is that important in early rounds. (I'm assuming that the last poll(s) will be single-click, but I suppose that's not necessarily set in stone...)

In terms of manipulation and cheating, in the end I don't see how the other two options would be a satisfactory improvement over the single-click poll. A sufficiently weighted vote verges on being the same as the single-click (albeit with better feedback, I suppose), while a weakly weighted or unweighted finite vote only partially solves the problem of not voting for one's most preferred options.

The unweighted, infinite bold vote is the best option out of the three for maximizing fairness and the inclination of the voter to express his/her desires both accurately and precisely.

I am a fan of unweighted, finite bold vote. Unlimited votes is a clusterfuck, but weighted votes can shut out some cool shit. I think it is a good medium between the two.

Recruiting is allowed, idk why it wouldn't be.

Cheating is a BIG problem, as has discussed in the past. idk how to fix or mitigate it, but imo it is the worst issue with cap atm.

I think that Unweighted, Limited / Unlimited Voting are pretty legitimate polling methods, although they're outdone entirely by IRV (Which CAP should absolutely be using to expedite the polling process). Weighted Voting is absurd and should never be used. There's literally no advantage in this method other than catering to dishonest individuals.

I'd also like to propose that we use IRV over all of these methods. We can leave voting open for a day and easily vote for the final winner all in one fell swoop. I'd like to hear Doug's opinion on this voting, however, as it means that we skip the visible poll stage, which admittedly is a bit exciting for participants. It's possible that he favors that more than expediting the process and eliminating skewing or cheating.

I don't feel I'm qualified enough to provide good commentary on the possible choices because I almost never participate in the competitive portion of the project. But I will say that the way feedback is received differs between the Art and Competitive Polls. In the competitive polls the submissions are all constrained by specific parameters, and if someone doesn't do well in the polls they can figure out why by looking at the choices that scored higher than theirs. But in the art polls the submissions will differ so much that comparing your submission to the ones that received more votes won't help much. While the competitive polls one submission may win because it's a stat spread with 101 speed and is very bulky and another submission loses because it has 120 speed and is frail, in the art poll one submission may win because it is done in the same art style as the official Sugimori art and one loses because it looks like a western comic. The competitive polls can give feedback through comparison, but the art polls require more individual feedback.

I definitely think out of all three of these options, the Unweighted, Finite bold vote is the best choice, followed by Weighted, Finite and then lastly Unweighted, Infinite.

Unweighted, finite voting requires users to select a certain one or two people (whatever is allowed) to represent themselves and their own personal opinion. By limiting the amount of choices one can actually choose from, I believe that it helps to better specify what individuals want in terms of the project itself. Furthermore, the fact that each vote counts as one vote and/or one point, not more or less, creates a much more level playing field for all candidates. One person means one vote, plain and simple. It places a fair limit on our own opinions, which is exactly what this project should strive for.

Weighted, finite voting accomplishes the same thing as unweighted, finite voting in terms of limiting the candidates available to vote on in one sitting. However, by placing more power in the position of voting, we have made the playing field less even. For example, assuming 1ist place gets three points, and third place gets one, it's possible for someone who gets 15 first place votes to beat someone who gets 44 third place votes, even though it's possible that these people actually like the third option very much. While some may argue that the result is still determined by popularity, it definitely puts some at a disadvantage, since name recognition and personal inflection may play a part in the ordering and position of voting ballots. I wouldn't be terrified if this goes through, but it's not the best option.

I am a huge disbeliever in letting individuals vote for as many choices as possible, as while there is a chance that some responsible users will tactfully choose (and not choose) certain people, it gives an outlet for people to just choose people "for the heck of it" and lessen the concreteness between a voter and his or her opinion. The unweighted, infinite voting style does exactly this. How can we accurately judge the opinion of the public when we have individuals choosing applicants for small reasons as a result of "just being able to"? It becomes less of a "This person is the best and should win." contest and more of "Oh, well, I like this too. And this. And this." kind of a vote. It's by far the worst of the three, and should not be considered.

Re: Fairness: In my opinion, I think that the votes are slanted towards the more well known artists a tiny bit. However, it really isn't a huge difference, and no matter what approach is taken for voting, this is bound to occur. For example, if we were to make all entries and votes anonymous, a chain reaction is almost bound to occur when one artist tells one user which one they made. That user tells another, etc, and they all figure out who made what, and then the point is lost. So, I do think that the table is slightly tilted in favor of better known submitters, I don't think it's a terribly pressing problem, and nothing really practical can be done about it.

Re: Feedback: Between the visibility of the number of votes one user gets on click polls, and the Topic Leader announcing which submissions are allowed to advance to the next round, I think it's pretty easy to see where you sit rank-wise (top, middle, bottom). If someone got cut out fairly quickly, it lets them know well enough to try a little harder the next time. As for more advanced placement details, I'd rather we not go very much further than what is already in place. I know that this is much more important issue when it comes to bold vote polls, but it's not terribly hard to keep track of how many votes there are total, and how many are for the submitter. I guess we could have someone (probably the Topic Leader or Assistant Topic Leader) periodically post the numbers of votes and percentages overall, but that just seems like dumping more work on them when the people who are really interested could just do it themselves.

Re: Manipulation, Gaming, and Cheating: Cheating is a very straightforward one, and it is fairly obvious that anyone caught cheating will be punished. (From what I can tell, the moderators have been pretty effective at catching cheaters in the past, so I trust them.) Gaming is much more difficult to regulate, and there are few options for it, the best of which is simply encouraging people to vote for their favorite option, and putting good effort in keeping up a positive attitude throughout the entire project to stop any grudges that could cause it to occur. Finally, recruiting is definitely the hardest one to stop. Users with little to no experience in the CAP project are very easily persuaded to click a button (clicky polls) or post some bold votes (postcount+) when asked. On the other hand, things like disallowing votes from users with less than x number of votes will slow interest in the project, likewise, having a select group of experienced CAP members vote doesn't really reach CAP's goal of being a community project. With those things in mind, I can't think of a compromise, but I'm sure others will have, and I hope to see what they think.

Lastly, of the three choices of votes listed, I would like the Unweighted, finite bold vote to be used. I think both of the unweighted votes are superior to the weighted version, because the weighted version encourages slightly more gaming and recruiting than actual just picking the choice the voter thinks is the best. With unweighted votes, the user can vote for their desired choice(s) more freely without gaming being as large of a factor. I also chose the finite bold over the infinite one, because it can be a lot of work for the TL to count up every vote made when all the choices are simultaneously selectable.