If we believe that there's an issue with "cool" concepts being the ones that go ahead when they're not necessarily the best for the metagame, we could put new limitations on the concept submissions. For example: "The description must detail exactly why we think that our concept will improve the metagame for reasons other than simply being fun to use."
I think this'll be a good idea to use weed out the big majority of concepts.
I would then add a stage within the polls, like cyberzero said, when 3 concepts are left to properly discuss those 3.
I would then still add the proposed concept assesment stage,
Soooo.
Stage 1 concepts are being made but a rule is added where users must try to describe what that CAP would add to the metagame.
Voting starts.
Voting stops when 3 concept remains.
Discussion stage on the 3 start to see what effect it will have on the metagame in general if we chose to work with that specific concept.
Voting starts again.
Now the concept assesment start to properly work out more details on what is wanted. What the goals are for that CAP. How those goals should be met. How it should be able to function in the metagame what role it would exactly fill on a team. What should at least be shown in the stats and the movepool (what should have emphasis and be a must, i.e. a tanking CAP should have at least def stat boosting moves, and good Def/SpD stats.)
I'm still wondering how detailed everyone wants this concept assesment to be though. I'm all for quite detailed personally. Programmers also come up with a plan and a (detailed) concept before they write the first line of code. (Yeah, I work in the IT branch) The benefit of working of a more detailed concept is that you can assume that the concept is properly thought out on all fronts. It also a discussion on how all individual stages work together (movepool, stats, abilities, etc.). Because of this, it's probably hard to still go in the wrong direction in any of the following steps because you could then viably point to the assesment and mention how the CAP is really missing it's (well thought out) goals because of the proposed movepool, stats, etc.