Ebel Scoring - How to set Ebel scores in Practique

Ebel is a standard setting method that is used for multiple choice assessment format. The Ebel method involves a group of experts providing a judgement based on the difficulty and relevance of each item in an exam. 

Practique has full support for Ebel standard setting, allowing each exam paper to be standard set by a group of invited expert judges. The Ebel matrix can be customised (you need to ask Fry to configure this for you), and difficulty and relevance can be input by each judge, and the Ebel score automatically calculated for each item in the exam. Ebel score histories are kept for each item in the item bank, allowing Ebel scoring trends over different exams and exam papers to be tracked separately in the item bank. 

Inviting judges

  1. To start the Ebel scoring process you will need to create an Item Set.
  2. Leave the Item Set in Draft 
  3. On the Item Set page click "Ebel Scoring"
  4. On the Ebel score overview page click "Invite Judges to Ebel scoring"
  5. Type in the names of the judges you want to invite. These must be existing examiners in the Practique system. If you do not have any judges or the judges you want to invite are not in Practique you will need to add them in the "Practique users" section in "settings".They will need a minimum set of permissions to be an Ebel judge "Item sets - Score item set
  6. Once you have entered the names of all the judges click "Invite"

Setting Ebel scores as a judge

  1. The Judges will each receive an email with an invitation and a clickable link that will take them to the Item Set you want them to judge.
  2. The Judges will click on the link and asked to log into Practique. If they have forgotten their password they can reset by clicking on "forgotten your password"

  1. Once the judge logs in they will be taken to the Ebel scoring page which will show a list of all the items in the Item Set. 
  2. The Judges can click on the item to view the content. The Item will open in a new tab in the internet browser.
  3. The Judge can then select the difficulty from a drop down menu (Easy/Medium/Hard)
  4. The Judge can then select the relevance from a drop down menu (Essential/Important/Acceptable)
  5. The Judge can say if they wrote the question by ticking the box
  6. The Judge can add any comments (Optional)
  7. When they have completed their judgements the judge must click "Save" at the bottom of the page 

  1. The Ebel score will automatically calculate based on the judges scores and will be visible on the Item Set page.
  2. To finalise the Ebel score the Item set needs to be approved. 

Setting the passing score with Ebel

  1. To set the passing score using Ebel, select "Ebel method" from the "standard setting method" drop down menu on the standard setting page.
  2. The Ebel score will be automatically calculated and displayed when you click "apply" on the standard setting page. 
  3. Practique will pass or fail candidates based on the Ebel score and display the results.