Non Objective Test Essay Grading

Constructing tests

Designing tests is an important part of assessing students understanding of course content and their level of competency in applying what they are learning.  Whether you use low-stakes and frequent evaluations–quizzes–or high-stakes and infrequent evaluations–midterm and final–careful design will help provide  more calibrated results.


Here are a few general guidelines to help you get started:

  • Consider your reasons for testing.
    • Will this quiz monitor the students’ progress so that you can adjust the pace of the course?
    • Will ongoing quizzes serve to motivate students?
    • Will this final provide data for a grade at the end of the quarter?
    • Will this mid-term challenge students to apply concepts learned so far?

The reason(s) for giving a test will help you determine features such as length, format, level of detail required in answers, and the time frame for returning results to the students.

  • Maintain consistency between goals for the course, methods of teaching, and the tests used to measure achievement of goals. If, for example, class time emphasizes review and recall of information, then so can the test; if class time emphasizes analysis and synthesis, then the test can also be designed to demonstrate how well students have learned these things.
  • Use testing methods that are appropriate to learning goals. For example, a multiple choice test might be useful for demonstrating memory and recall, for example, but it may require an essay or open-ended problem-solving for students to demonstrate more independent analysis or synthesis.
  • Help Students prepare. Most students will assume that the test is designed to measure what is most important for them to learn in the course. You can help students prepare for the test by clarifying course goals as well as reviewing material. This will allow the test to reinforce what you most want students to learn and retain.
  • Use consistent language (in stating goals, in talking in class, and in writing test questions) to describe expected outcomes. If you want to use words like explain or discuss, be sure that you use them consistently and that students know what you mean when you use them.
  • Design test items that allow students to show a range of learning. That is, students who have not fully mastered everything in the course should still be able to demonstrate how much they have learned.

Multiple choice exams

Multiple choice questions can be difficult to write, especially if you want students to go beyond recall of information, but the exams are easier to grade than essay or short-answer exams. On the other hand, multiple choice exams provide less opportunity than essay or short-answer exams for you to determine how well the students can think about the course content or use the language of the discipline in responding to questions.

If you decide you want to test mostly recall of information or facts and you need to do so in the most efficient way, then you should consider using multiple choice tests.

The following ideas may be helpful as you begin to plan for a multiple choice exam:

  • Since questions can result in misleading wording and misinterpretation, try to have a colleague answer your test questions before the students do.
  • Be sure that the question is clear within the stem so that students do not have to read the various options to know what the question is asking.
  • Avoid writing items that lead students to choose the right answer for the wrong reasons. For instance, avoid making the correct alternative the longest or most qualified one, or the only one that is grammatically appropriate to the stem.
  • Try to design items that tap students’ overall understanding of the subject. Although you may want to include some items that only require recognition, avoid the temptation to write items that are difficult because they are taken from obscure passages (footnotes, for instance).
  • Consider a formal assessment of your multiple-choice questions with what is known as an “item analysis” of the test.
    For example:
    • Which questions proved to be the most difficult?
    • Were there questions which most of the students with high grades missed?

This information can help you identify areas in which students need further work, and can also help you assess the test itself: Were the questions worded clearly? Was the level of difficulty appropriate? If scores are uniformly high, for example, you may be doing everything right, or have an unusually good class. On the other hand, your test may not have measured what you intended it to.


Essay questions

 

“Essay tests let students display their overall understanding of a topic and demonstrate their ability to think critically, organize their thoughts, and be creative and original. While essay and short-answer questions are easier to design than multiple-choice tests, they are more difficult and time-consuming to score. Moreover, essay tests can suffer from unreliable grading; that is, grades on the same response may vary from reader to reader or from time to time by the same reader. For this reason, some faculty prefer short-answer items to essay tests. On the other hand, essay tests are the best measure of students’ skills in higher-order thinking and written expression.”
(Barbara Gross Davis, Tools for Teaching, 1993, 272)

When are essay exams appropriate?

  • When you are measuring students’ ability to analyze, synthesize, or evaluate
  • When you have been teaching at these levels (i.e. writing intensive courses, upper-division undergraduate seminars, graduate courses) or the content lends it self to more critical analysis as opposed to recalling information

How do you design essay exams?

  • Be specific
  • Use words and phrases that alert students to the kind of thinking you expect; for example, identify, compare, or critique
  • Indicate with points (or time limits) the approximate amount of time students should spend on each question and the level of detail expected in their responses
  • Be aware of time; practice taking the exam yourself or ask a colleague to look at the questions

How do you grade essay exams?

  • Develop criteria for appropriate responses to each essay question
  • Develop a scoring guide that tell what you are looking for in each response and how much credit you intend to give for each part of the response
  • Read all of the responses to question 1, then all of the responses to question 2, and on through the exam. This will provide a more holistic view of how the class answered the individual questions

How do you help students succeed on essay exams?

  • Use study questions that ask for the same kind of thinking you expect on exams
  • During lecture or discussion emphasize examples of thinking that would be appropriate on essay exams
  • Provide practice exams or sample test questions
  • Show examples of successful exam answers

Assessing your test

Regardless of the kind of exams you use, you can assess their effectiveness by asking yourself some basic questions:

  • Did I test for what I thought I was testing for?
    If you wanted to know whether students could apply a concept to a new situation, but mostly asked questions determining whether they could label parts or define terms, then you tested for recall rather than application.
  • Did I test what I taught?
    For example, your questions may have tested the students’ understanding of surface features or procedures, while you had been lecturing on causation or relation–not so much what the names of the bones of the foot are, but how they work together when we walk.
  • Did I test for what I emphasized in class?
    Make sure that you have asked most of the questions about the material you feel is the most important, especially if you have emphasized it in class. Avoid questions on obscure material that are weighted the same as questions on crucial material.
  • Is the material I tested for really what I wanted students to learn?
    For example, if you wanted students to use analytical skills such as the ability to recognize patterns or draw inferences, but only used true-false questions requiring non-inferential recall, you might try writing more complex true-false or multiple-choice questions.

Constructing Essay Exams

What happens: Learner

  • Hears and reads instructions
  • Interprets the question
  • Recalls relevant information
  • Prepares a response according to the verbal directive,
    either mentally or written, either outlined or "mapped",
  • Writes response
  • Reviews and edits if time permits

Essay tests can evaluate more complex cognitive or thinking skills
assuming that rote memory and recall tasks are assessed more appropriately through objectives tests as true-false and multiple choice questions. These cognitive challenges are reflected in the verbs of the questions themselves, from simple to complex (c.f. lists of verbs in objects...)

  1. Knowledge: recall, define, arrange, list, label, identify, match, reproduce
  2. Comprehension: describe, explain, recognize, restate, review, translate, classify; give examples; (re)state in own words
  3. Application: apply, illustrate, interpret, operate, solve, predict, utilize
  4. Analysis: analyze, compare, contrast, distinguish, examine, experiment, diagram; outline
  5. Synthesis: design, develop, formulate, propose, construct, create, reorganize, integrate, model, incorporate, plan
  6. Evaluation: evaluate, argue, assess, compare, contrast, conclude, defend, judge, support, interpret, justify

(for a complete listing of verbs in these categories, see Essay terms and directives)

Advantages:

  • Require students to demonstrate critical thinking
    in organizing and producing an answer beyond rote recall and memory
  • Empower students to demonstrate their knowledge
    within broad limits beyond the restraint of objective tests (true false, multiple choice)
  • Allows learners to demonstrate originality and creativity
  • Reduces preparation time in developing,
    as well as distributing, a test, especially for small number of students
  • Presents more possibilities for diagnosis

Disadvantages:

  • Grading is often subjective and not consistent, colored by
    preconceptions of student, prior performance, time of day, neatness and handwriting, spelling and grammar, and where the actual test falls in
  • Can be a limited sampling of content
  • Good writing requires time to think,
    organize, write and revise
  • Time consuming to correct
  • Advantageous for students with good writing and verbal skills
    as opposed to those who have alternative learning styles (visual and kinesthetic)
  • Essay questions are not always properly developed
    to assess higher thinking skills (often only test for recall and style)
  • Advantageous for students who are quick,
    as opposed to those who take time to develop an argument or may suffer from writers block

Mechanics:

  • Clearly state questions
    not only to make essay tests easier for students to answer,
    but also to make the responses easier to evaluate
  • Include a relatively larger number of questions
    requiring shorter answers in order to cover more content
  • Guard against having too many test items
    for the time allowed
  • Indicate an appropriate response length
    for each question
  • Set time limits if necessary
  • Note graded weights to questions

Ideal test items:

  • Integrate course objectives into the essay items
  • Specify and define what mental process you want the students to perform
    (e.g., analyze, synthesize, compare, contrast, etc.).
    Does not assume learner is practiced with the process
  • Start questions with an active verb
    such as "compare", "contrast", "explain why";
    Offer definitions of the active verb, and even practice beforehand.
  • Avoid writing essay questions that require factual knowledge,
    as those beginning questions with interrogative pronouns
    (who, when, why, where)
  • Avoid vague, ambiguous, or non-specific verbs
    (consider, examine, discuss, explain)
    unless you include specific instructions in developing responses
  • Have each student answer all the questions
    Do not offer options for questions
  • Structure the question to minimize subjective interpretations

Directions:

  • Present the assignment both verbally and in writing.
    The initial oral plus written presentation to promote and inspire thought;
    written for reference within the test
  • Provide evaluation criteria
  • Focus on the mental activity to avoid rote answers,
    and/or repeating examples from the text
  • Teach students how to write an essay (test)
    explaining definitions of cognitive verbs
  • Teach the difference
    between presenting a position as opposed to presenting an opinion
  • Define requirements clearly
    State the number of points each question is worth
  • Warn students of possible pitfalls
    especially if you have strong ideas of what you do and do not want
  • Inform the students about how you evaluate
    misspelled words, neatness, handwriting, grammar, irrelevant material (bluffing)

Correcting:

  • Develop a model answer
    that contains all necessary points
  • Note additional content for extra points
  • Conceal or ignore students' names in the correcting process
  • Read through the answers to one test item at a time
    without interruption
  • Sequence best through worst responses
    for verification if time permits
  • Write comments on the students’ answers,
    both affirming and correcting
  • Do not give credit for irrelevant material
  • Mix or shuffle papers to vary subject's location
    before assessing the next test item
Curricular guides and resources:

Using feedback in the classroom | Teaching critical thinking | Bloom's taxonomy |
Teaching with questioning | Preparing guided notes |
A curricular idea! | Curricular resources and guides |
Learning Exercises & Games | Exploring learning styles |
Constructing true/false tests | Constructing multiple choice tests |
Constructing essay exams | Cross language resources including digital translators |
Online Learning/eLearning books and resources for teachers

0 comments

Leave a Reply

Your email address will not be published. Required fields are marked *