New Scoring

How about p^(1+100/max(#challs,100)) or p^(1+100/(#challs+100)) or something? I think a power of 2 is a nice minimum. Going beyond it diminishes the points for low percentages so much that it almost isn't "worth" doing a site unless you know you can finish pretty much all challenges. This also avoids having to adjust for sites with few challenges (and then again if they add a bunch).

You might still want to be able to adjust the effect, so you can still add a factor somewhere. I think it would be useful because the reason to do all this is to compensate for the fact that some sites don't reflect the difficulty of challenges done in their scoring.

Adding a minimum of 1 point per challenge doesn't seem very interesting to me, but I do agree that getting something instead of nothing might be quite a bit more inviting to starters. In any case, I don't think it's very hard to do. Something like #solved+(site_score-#challs)*p^whatever should be ok, shouldn't it?

While I'm at it, why not do away with the base_score? What's the point of it? A score for a site for just existing? I'd say that the only thing that matters are the challenges. If the average %solved isn't sufficient, you can use the factor f in site_score = f*#challs*(2-avg_solved) to more appropriately compensate for the difference in difficulty between sites.

Quote from dloser

Feb 06, 2011 - 10:18:25

How about p^(1+100/max(#challs,100)) or p^(1+100/(#challs+100)) or something? I think a power of 2 is a nice minimum. Going beyond it diminishes the points for low percentages so much that it almost isn't "worth" doing a site unless you know you can finish pretty much all challenges. This also avoids having to adjust for sites with few challenges (and then again if they add a bunch).

This is very much what I was trying to argue, though without the formula. It is frustrating to start a site knowing that that you are going to solve 30 challenges for a handful of points.

I'm not committed to the 1 point minimum. I was just thinking that anything would be better than 0 for half a dozen challenges. It would also be nice if the system worked without ad hoc tinkering, which is a plus for dloser's idea.

If points had avg%solved involved at all, it would have to take into account a lot. what if a site all of the sudden gets like %50 more users? then everyone's score for that site will change when they didn't do anything. avg%solved would only work if %avg challenges solved had a normal distribution or if we took into account the amount of time that a user has been signed up. I'm not for taking away the base score, we need a static or mostly unchanging way of relatively comparing the difficulty between sites.

What I am for changing however, is the way that the base score is determined. This is difficult to do because "difficultly" is a relative term in itself. We would have to do some statistical analysis of the distribution of challenges. It would be best to base it on what percentage of challenges (or range of challenges) have the highest population density. This is different from the average. This way, a site where most people only solve 1-20 percent of challenges would have a higher score than a site where most people are at 50-70 percent.

Alternatively, we could have a running poll regarding the difficultly of some sites vs others.
or both!

This is always going to be incredibly difficult and subjective.
I found it the hardest part of setting up WeChall.

There are so many things to take into account:
What is worth more:
- solving 10 challenges each on 100 sites or solving 100 challenges each on 10 sites?
- solving 10 challenges on Electrica or solving 10 challenges on ThisIsLegal?
- solving 10 challenges on a site you just started at or solving 1 challenge on a site you almost finished?
and how do you calculate these things?

How do you distinguish between good but hard challenges (like on Electrica or +Ma's) and bad challenges that are either broken or make no sense or cannot be solved (like some on TBS or DareYourMind)?

What about the language differences, or even theme/area of interest differences?

In my humble opinion the WeChall score should reward the people with the highest IQ and the most knowledge.
Not the ones with the most free time or those with the most friends.

I think the original scoring scheme got pretty close. Later changes have shifted the balance a bit too much to quantity over quality for my taste, but it is still reasonably accurate.
Discussion and suggestions on this topic are always appreciated!

@adkatrit: As far as I know (from reading /scoring_faq) the average %solved is (or was?) already used. I only mentioned it as something that is already in place, but can be extended with an additional factor (vs. tweaking the base_score). Not as something that should replace the base_score.

I agree with Kender that it's virtually impossible to get the scoring accurate or even decide what that means. Also, it's not like any of it really matters... ;) In light of that it probably best to keep it as simple as reasonable.

This sounds like it may be an ongoing battle for some time.

For me rank and score does not matter, but I am aware that wechall's primary function is to provide a global ranking and therefore for many it will matter a lot. I think the changes work well but as certain site score systems are currently edited manually, I feel that in time this again may cause issues.
I hope you find a suitable solution. The issue is in defining what makes an 'easy' or 'hard' challenge. I mean something hard for a certain user will always be easy for another.
Challenges like TBS' 'contact' however in infamous for its obscurity and therefore associated difficulty. If you are going to get into talks about electrica vs thisislegal, then taking tbs as an example it would also therefore beneficial to differentiate points between 'contact!' and 'an easy starter'. This however is an impossibility unless the site itself scores challenges based on difficulty and therefore gives more points for a challenge which is less-solved than one which has been solved by every man and his dog.
There is no easy solution for this but I hope the admin team can reach a suitable agreement and algorithm to best differentiate between the difficulty : points balance.

Respect and regards to all,
sabretooth