Stupid ipad cut and paste.

@Guaroz: We do not need to compare the players on the site with each other.
In school you have some classes with many very good kids. That makes a A in this class more common, but over all not worth less.
Sure, many players here are A, it's not the norm to go CD all the time. But this does not make it average.

I would say we should gather more information what people think in therms of reliability using this wording:
A = Excellent, B = Very Good, C = Good, D = Acceptable, F = Fail

I know this is a very subjective matter, and for a A-player a RR of 90% is unacceptable (that's why he can set a filter on his games), but if you CDed in 2 of your 10 games your rating will be a D, and at least you should be able to play with people that have a similar rating, and keep the F-rated people out of your games.

@Amby: Was just uploading the code. Should already work again.

"I would say we should gather more information what people think in therms of reliability using this wording:
A = Excellent, B = Very Good, C = Good, D = Acceptable, F = Fail"

Please Oli, for everything that is good on this site, do NOT use that wording. A "C" rating, especially if we use the "A=90+, B=80+, C=70+, D=60+, F=59" suggestion that you made on page 7, is anything but "Good". Anything under 90% reliability is not good. And a 65% reliability is most definitely not "acceptable" to any except those who don't care at all about the games they're in.

I'm definitely more in line with Guaroz on this one. "A=Normal, B=Acceptable, C=Bad, D=Very Bad, F=Fail" makes way more sense to me, because while a "D" is passing in school, a 60% reliability means you're a major f*ckup in reliability if you're missing 40% of your phases/CDing in 40% of your games. I mean, really, missing 1 out of almost every 2 turns is definitely not "acceptable" right?

@drano, or anybody else: I need more ideas about how to spread the % between the given explanations. If we use a grading system people do expect D=aceptable so I need more ideas how to spread the %. Please @all make some suggestions.

Oli -

If a "D" is going to = Acceptable, then a "D" needs to be like 90-93% at worst. Claiming that a 75% is acceptable makes a mockery of this site. You'll end up in exactly the same situation, where all your top players ignore the RR because it means nothing. You'd have people thinking they can CD in 1 out of 4 games and this site "accepts" that behavior. That's an insult to those of us who respect each others time and a mockery to Diplomacy in general.

I still vote we abolish the idea of a "grading system" and simply use percentages while educating people on what's considered "good". If we HAVE to use a "grading system" though, my vote is for the following:

Excellent = A = non-CD 100% and non-NMR 98%+
Very Good = B = non-CD 98-99 and non-NMR 96-98
Good = C = non-CD 95-98 and non-NMR 94-96
Acceptable = D = non-CD 92-95 and non-NMR 90-94
Fail = F = non-CD 0-92 and non-NMR 0-90

It may be a bit harsh, but it will actually teach people to be better players instead of "rewarding" them for being mediocre players.

Thanks does not look that much different than mine version from the last page. There we had D=79,99 or lower...
Please some more distributions from the other players...

Oli -

The difference is I'm using 2 factors instead of one joint factor for reliability. Since I think most people would agree that CDs are more severe than NMRs, the (CD + NMR)/2 ratio is probably not a wise decision. And I"m sure it'll be hard to come to a consensus as to what a good ratio is. My ranking therefore, uses both factors. In the event that you're in the "A" range for non-CD, and the "B" range for non-NMR, you'd be a "B" as we would use the worse of the two stats. After all, we don't want to make people seem more reliable than they really are do we?

Other than that, my rankings are just a bit more severe than yours. The "A" range is the same, but after that, we diverge pretty quickly. You'd give a 90 a "C" where in mine, they'd be a F basically. Big difference there.

Another idea comes to my mind. Maybe instead of a school grade (that's even different depending on your country and school system) it might be an interesting idea to use medals or symbols. This way we would be much more free in distributing the symbols to the %.

@drano: I thought about the difference of importance on NMR and CD for some time, but if you compare NNR to total phases played and CD with games played they are both equal important. Someone who misses a turn each year for a complete game (about 15 years each game) is not better than someone that CDs 1 out of 5 games. Regularly NMRs do ruin games the same way a CD does.

@Oli - thanks mate working fine now.

I do believe the point is being missed here. What matters isn't necessarily the % attributed to various "Medals" or "Grades". What matters is that we need to educate people that a 80% non-CD rate is absolutely horrendous! And that a 90% non-NMR rate isn't anything to be proud of either! The best way to do this is either with simple percentages, or by using words such as "excellent", "good", "bad", "very bad", "horrendous" etc. It might hurt some feelings if someone is labeled "horrendous", but hey, it's the truth! We can't sit and coddle everyone's feelings.

I just feel like we're getting too wrapped up in what things to use like "medal" and "grades", when the important thing is having people realize that anything under a 90 (if not higher), is simply bad. No ifs, ands, or buts, about it.

@oli -

"Someone who misses a turn each year for a complete game (about 15 years each game) is not better than someone that CDs 1 out of 5 games."

You're absolutely right, those people are both absolutely horrendous in reliability. A 50% non-NMR rate and a 80% non-CD rate are both terrible. And letting people see both those stats and basing their decision simply on percentages is still the best option IMO.

Sorry, I was unclear. I meant 1 missed phase per year = 80% = one missed game of 5.

Oli -

1 missed phase/year isn't necessarily 80%. If i don't have retreats or builds, it's 50%. If I have only builds, it's 66.7%, 1 retreat and builds is 75%. Only with 2 sets of retreats and a build is it 80%.

That said, it doesn't change the point I made in my post. 80% is still terrible.

- Oli -

1. Re: Stats.
You wrote"We do not need to compare the players on the site with each other."
Oli, I don't want to know "WHO is where". The purpose of a RR System is not to win it, it's to make clear to members whether and how much their behaviour meets what's expected from them.
What I want to know is "how many" active players do match a given percentage of rating. A stat like:
From 99.00 to 100.00 - x players (= y% of total)
From 98.00 to 98.99 - x players (= y% of total)
From 97.00 to 97.99 - x players (= y% of total)
...
From 80.00 to 80.99 - x players (= y% of total)
Less than 80 - x players (= y% of total)

This would not only be extremely useful while building a grading scale, it would also be indispensible for expert players creating games with the "Custom" option. They need to know how many "potential joiners" would have the game they're going to create if they set 98+ or 97+ or 96+ etc.
Either way, we need to know *how many* - not *who*

2. Re: Grading scale.
I'd abandon the idea of a school-like grading scale and just make our own scale. School wording means nothing in here: it's not a skill rating. Again, there's no excellence in being reliable like you're supposed to be, while we need to inform players who aren't reliable that their behaviour is frowned upon and barely tolerated.
It's not a competition, there's nothing to win in RR. There's only to make understand unreliable players why good guys stopped inviting them.
So my advice is still to abandon the backfiring school-wording and take instead a more clear wording like the one I said before, that simply calls normal what is normal and it calls bad what is bad.
If it'll be possible to have some stats, I'll propose some.

@drano: a year has 2 turns with 2-3 phases each...

@Guaroz:
1. I will try to code an overview for this. You are right, this is an usefull information, but given that most of the players do have next to none missed phases or CDs I just wanted to say the "normal" grade will be A anyways. There is no need to confirm this by an overview and set the "normal" grading to C. Becasue than we would have A=100, B=99, C=98,D=95. Not an ideal solution for some predifined filters.

2.Yes. I think this is right. We should create out own grading system. Or somehow ad a factor that moves the current RR-% to a already known system.

If you take for example (RRold = (noCD+noNMR)/2):
RRnew=RRold-(100-RRold)*1,5
You change the RR from now 98% to 95%, 88% => 70% 80%=>50% and you get a good match from my old proposal of 98%=A, 79%=D, just with more meaningfull %.
This would result in:
100% => 100% = A+
98% => 95% = A
94% => 85% = B
88% => 70% = C
80% => 50% = D
79% or lower = F

Thanks Oli, I will wait for stats result before making my proposal.

"given that most of the players do have next to none missed phases or CDs"
Yeah, I was under the same impression. Stat is useful also to check whether a given percentile has an abnormal numerosity. We both suspect that numerosity will be concentrated on the top percentiles, so perhaps there may be need to split some percentiles in 2 or 3 in order to have a meaningful stat. Opposite ratio for bottom percentiles, we might need to merge them to have a readable stat.
Anyway, it's too soon to worry about it, let's first see what's the outcome for each integer percentile.
I guess the biggest difficulty now it's having only *active* players into that stat.

I've changed the system again, and I think I will leave this as it's now for some time to get some real usability-results.
Basicall it works like this:
It takes the CD and NMR-ratio and substract this 2.5 times from 100.
This results in:
(CD+NMR/2) = 100%-99,6 => R100-R99 (Excellent)
(CD+NMR/2) = 99,5%-98% => R98-R95 (Good)
(CD+NMR/2) = 97%-94% => R94-R85 (Acceptable)
(CD+NMR/2) = 93%-88% => R84-R70 (Bad)
(CD+NMR/2) = 87%-80% => R69-R50 (Very Bad)
(CD+NMR/2) = 79% => R49 (Fail)
I will not group these values, but keep the actual numbers. Please note, there is no "%", as these values are no %. In the profile I will add the explanation-text for these ratings.

In the gamecreation-setting you can specify and value you like for RR and I will keep the noCD and noNMR setting too.

Oli, thanks for the great job.

I have two comments
1) you might give a theoretical justification to the new "R" formula
RRold = (noNMR + noCD)/2
R = 100 - 2.5*(100-RRold)
by first noting that (for RRold not too far from 100) it's quite close to
100*[(RRold/100)^2.5],
and then remembering that p^N is the probability that an event with probability p (e.g., regular submission of orders by the player) happens N times in a row (e.g. in a game with N phases). So, if you neglect the (rather large) difference between noCD and noNMR, the "R" formula somewhat gives you the probability that the player will not cause trouble in a game of ~ 2.5 turns (btw, I'd use an even larger factor.. at least 5; but that's a detail).

2) on a quite different topic, Joseph the most Excellent was suggesting to have separate rankings for takeovers, since TOs positions are usually "doomed".
I agree with that request; however, I would be very careful in saying that a victory from a TO position would "say something about my skill as a player". In fact, there are cases where you take over decent or even excellent positions (just an example: http://vdiplomacy.net/board.php?gameID=15663 )

@diatarn_iv:
That's a good oberservation and makes the calculation much more streamlined.
I did some tests in Excel and got similar results using ((NoCD + NoNMR)/2)^3.
That's a really easy to use calculation. I will change this to this calculation, because it's much easier and more intuitive this way.

@diatarn, no doubt there are outliers. In fact, I saw someone recently offering the best position in a game because he was tired of entering orders for so many units. But the general point remains.

Oli -

Each year has 5 potential phases: spring orders, spring retreats, fall orders, fall retreats, and winter builds. However, there aren't always retreats and builds thus, every year fluctuates between 2 phases and 5 phases depending on what happens. Unless the system counts retreat phases even if there's no retreats, and the same with builds?

That aside, I don't like how we're arbitrarily assigning multipliers here. Stats shouldn't have arbitrary multipliers assigned, they should be hard numbers. Randomly deciding 2.5 is the multiplier to me only serves to confuse people. If someone has a noncd of 100 and nonmr of 99, their "R" becomes 98.5 right? That's going to confuse people, and rightfully so. Its just a random multiplier decided upon for no real reason. Same of we changed it to x5 or any other suggestion. It doesn't make sense to me, even though I do like that it makes the scale a bit harsher.

Just noticed olis latest post. So now its (nocd+nonmr/2)^3?

What is the justification? Can someone explain? It seems like an arbitrary decision to me and doesn't make much sense.

This was being discussed in the WW4 game going on now. It would seem some have no idea about this discussion going on. A person there was upset because he took great care to repair his reliability rating by taking over CD positions. Now it would seem that care was for naught. It does seem kind of harsh to offer a repair and have people take advantage of that and then pull it out from under them, but I also see the greater reason to fix what has been broken.

One thing that puzzled some people there was how could I have a perfect rating when I just started playing on this site and I have to agree with them. Yes I missed no phases but I am in my first game here and my "reliability" is still unknown, not perfect! This rewards people who simply sign up with new accounts, you encourage a lousy player who drops from any game that goes bad to simply rejoin as anew player and presto, he is now thought of as "reliable" when you really know nothing about him, isn't reliability EARNED?

I echo the sentiments a few have made to drop any sort of "rating" and simply post the stats. Let people know that player X missed 10 phases of 400, Player Y missed 2 phases of 88, Player Z missed 0 phases of 10 and let them figure out what they feel about each of these players themselves. Simple statistics that don't lie and don't give some sort of "rank" that is or is not really deserving.

Just my 2 cents anyways....

Spot on Tom. I agree 100% that new players shouldn't be considered "reliable" simply because they've entered orders on 5 of 5 phases for example. There should be a "on probation" period for new players until they reach a certain number of phases (again, 1v1 games don't count for this), whereupon their reliability becomes a normal stat like the rest.

Is it possible to add a last CD/last NMR stat? I want to show I improved my reliability without having to play 100s of games, but I definitely don't want a clean record.

Just a way of showing I improved without removing the very bad record I deserve.

I had suggested something earlier based on time and that was not possible. But it is a real good idea if maybe eventually possible down the road?

@ Drano:
I think Oli liked what I called an (a posteriori) theoretical justification of his (previous?) formula: if you assume that an "average" game lasts for N phases, noNMR^N (with noNMR between 0 and 1) gives you the probability that a player will have a "perfect" game (in terms of reliability). So, elevating to the N-th power just moves the question which is "answered" by the rating from "will this player give trouble in the next phase" to "will this player give trouble in the whole game?" [btw, I think the first to give the suggestion to raise to some power was Retillion, though I'm not sure he was completely serious].
Even if you accept this premise, you might still criticize both the fact that (noCD+noNMR)/2 is different from noNMR, and the fact that N=3 is low (the average duration of a "Classic" game is about 17 turns); but in my opinion they are tolerable faults, if the statistics about noNMRs and noCDs are displayed separately.

@ Tomaha:
about the example with players X, Y, and Z. The point you are making can be put into quantitative terms if you assume that there is a "gaussian" uncertainty in the number of NMRs by a player: if you do that, the noNMR rates of the three players are
X: (10+-sqrt(10))/400 = 2.50%+-0.79% --> between 1.71% and 3.29%
Y: (2+-sqrt(2))/88 = 2.27%+-1.61% --> between 0.66% and 3.88%
Z: (0+1)/10 = 0%+10% --> between 0% and 10% [btw, here I'm using a sort of "poissonian" error]
so, what you are saying is essentially that the upper limit for X is (slightly) better than that for Y, and (much) better than that for Z, even though the central values (2.50%, 2.27%, 0%) are in reverse order.
But what can you do? Friday I was suggesting that people with 0 NMRs should be "sorted" by the number of phases they played (so that 0 NMRs with 1000 phases played is better than 0 NMRs with 10 phases played).. essentially because of this "upper limit" argument; but I think Retillion was mostly right (the exception being players with too few phases played.. say, < 100) in answering that if you have 0 NMRs, you should be considered "perfect" regardless of the number of phases played.