Ranking and Grading

Prepared by:

Joseph Malkevitch
Mathematics and Computing Department
York College (CUNY)
Jamaica, New York 11451-0001

email:

malkevitch@york.cuny.edu

web page:

http://york.cuny.edu/~malk/

There are many situations where an individual must rank or grade some collection of things. Voters must evaluate candidates so they can decide how to vote in an election such as when they vote for President of the United States or what movie was their favorite over the last year. Teachers must grade students on their performance in a course. Members of a jury must decide if a person on trial is guilty or not, and many sports events require "judges" to evaluate how well a figure skater or a diver performed.

To help you think through the issues here suppose we have two "competitors" who might be divers in a sports event or two candidates for mayor.

The ballot used to evaluate these contestants involves assigning "grades" or "scores" to the contestants. Sometimes scores are given as numbers, 6 points, 5 points, ...., or 1 point where more points means that the "judge" thought the contestant made a stronger performance. Sometimes the scores are "letters," such as A, B, C, D, E, and F. Note that unlike the usual scale used by teachers, this scale has an E grade between D and F, and A is the highest grade.

Example

There is one prize to be awarded based on four "groups" of ballots with "grades" from A to F.

Group 1: 40 votes of A for candidate X and 40 votes of F for candidate Y

Group 2: 38 votes of A for candidate Y and 38 votes of F for candidate X

Group 3: 11 votes of A for candidate Y and 11 votes of B for candidate X

Group 4: 11 votes of C for candidate Y and 11 votes of F for candidate X.

Activity 1

Decide which candidate you think should win based on the 100 ballots cast above.

Note: If you work as part of a group to discuss what to do, after you discuss how you might choose a winner, decide for yourself who should win, and be prepared to describe how the "method" you used to choose a winner would work if there was a different pattern of ballots but using the same 6 grades.

Activity 2

In the example above the "graders" only used 4 of the 6 available grades. Grades D and E were not used though perhaps their presence had some effect on the grades that were assigned. Decide which candidate you think should win based on the 100 ballots cast above but this time where there are only 4 grades available whose names are A, B, C and F.

Note: If you work as part of a group to discuss what to do, after you discuss how you might choose a winner, decide for yourself who should win, and be prepared to describe how the "method" you used to choose a winner would work if there was a different pattern of ballots but using the same 4 grades.

Things to think about:

1. Does it matter if society is choosing a winner from a collection of people running for office or society is choosing a party where the winning party will be asked to try to form a government in parliament?

2. When deciding to give a "grade" were you looking at the choices from the point of view of an "absolute" scale - say, the best dive that any human can perform versus a "relative" evaluation where you were only "grading" the particular choices (divers) in this particular case?

3. When people vote or grade do you think that the way they vote or grade (assuming the ballot type is the same) is independent of their knowledge about what system is being used to produce a "winner?"

4. Some voting/grading situations produce a single winner while in some such situations what is desired is a "ranking." Do you feel that the best system for choosing a single winner among many choices is different from the best system for ranking the original choices from best to worst?

5. What are the pros and cons of using a grading scale with letters (A, B, C, D, E, F), numbers (99, 98,...., 0) or words (very high, high, neutral, low, very low) when grading candidates, films, or ice dancers?

6. When people vote in elections do you think they are concerned with electing the "best candidate" or with voting for the candidate they most want to be in office?

7. What difference in approach, if any, is required when the number of "choices" (above 2) is more than 2?

8. What are the nice properties and/or fairness conditions that the method you think is best would obey?