Practique Development Roadmap

Below you can see the features we are focussing on and when. We will aim to release improvements under the themes stated here and you will receive an update on what is released in our Practique Server release notes and Practique for iPad release notes. Remember that we regularly review our roadmap and priorities based on user feedback. This should, however,  give you a good overview of our plans. You can also see all of the features that we have captured from your feedback. If your feature is not here it may not have gone through our review process yet or we may have included it as part of another feature.


How we plan and focus our work


Next Release  - These are large features we feel are really important for our customers and the progression of Practique.  These are larger features or blocks of work that we feel are important. We will also allot a proportion of our development time budget to features decided on during the Practique User Group meetings. You should expect to see these features begin to appear in the next release. Generally we think the release should be ready for the next Practique User Group if they are every six months.

Next 12 months - Larger features that we have prioritised from user feedback that we know will make a great difference to Practique and are part of our longer term strategy. We expect these to be the first candidates when planning the work for the next release

Continuous - These are smaller features based on user feedback. We often can include these as part of our ongoing development as they take less time than the larger features. You should expect to see some of these features in the next six months.

Postponed - These are features that we have captured from users' feedback, but that are not seen as a priority for the next 12 months. 


Next release

Feature Name
Summary
Examiner Availability ToolBrand new tool that provides users with a calendar they can use to communicate exam dates to potential examiners. Flexible and configurable to allow users to capture and manage examiner information.
Candidate feedback report improvementsContinued improvements to the customisation of the candidate feedback report.
Batch operationsRe-allocate examiners after the exam has finished
Select multiple Items to archive in bulk
Senior MarshalAddition of a Senior Marshal role for OSCE exams who has an overview of all circuits and stations
P4B calculatorIn application calculator that is accessible during written exams 
Exam board report Single report with all the information you need for the exam board post exam. 
We are taking feedback from our users on what information to include. If you have specific needs please send us an example of your own exam board report.


Feature List

Note: This table is sortable, by clicking the up and down arrows next to the column headings.





ThemeSummaryFeaturesApplicable Exam TypesSchedule
1

Examiner Availability Calendar

This would allow users to recruit and book OSCE Examiners for future exams, at any point.

  • When scheduling an exam, you would be able to pull in examiner allocations from the calendar
OSCE
2Standard SettingWe would like to improve the Standard Setting workflow and add some new features.
  • Enhance live standard setting information while and exam is in progress
  • Allow the import of Ebel judging from spreadsheets
  • Allow Examiner-performance rescalling (hawk/dove
  • Have exam blueprint based Ebel grids
  • Allow mark re-scaling to university standards
  • Ability to sort standard setting screen by all fields PSD-256
  • Show passing and failing students (column in the download) PSD-256
  • Improvements to the UI of setting exam standard
  • Add Rasch modelling
  • Show negative -1, -2 SEM in Standard setting
Written, OSCE
3Exports/ImportWe are going to improve the way that data is imported to and exported from Practique.
  • Consolidation of the structure of exam data exports
  • Allow filtered candidates and examiners to be exported
  • Add the export of candidate and examiner scheduleing
  • Separate the import of items permissions
  • Allow the export of Item Sets with Item IDs and blueprint tagging
  • Simplify CSV field mapping
  • Auto detect CSV encoding in upload
  • Allow filtered Item bank export
  • Add Max score per station, and max score per exam in the CSV export of exams
  • Improve naming convention of results download
  • Allow export of Standard Setting question scores
  • Include Examiner ID in all results exports
Written, OSCE
4iPad exam settingsWe are going to improve the way that users can install and update the iPad application on their devices.
  • Control the settings for the iPad app on a per-exam basis (examiners editing marks, rotation management)
OSCE
5OSCE Marksheet ImprovementsWe are going to improve the way that OSCE marksheets are created and displayed on the iPads
  • Improve the way that marksheets are created, by
    • Allowing schemas to be searched for, and deleted
    • Improve marksheet 'Count As' tagging review process
    • Warn users if global criterias are incorrectly set up
    • Marking Schema instructions hidden in marking buttons
  • Improve the content of the marksheet by
    • Allowing photos to be captured on the iPad
    • Improve the way that structured feedback can be recorded
    • Adding a sliding scale observation criteria
    • Allowing custom text to be inserted on the marksheet
    • Improve the way that generic feedback statements can be recorded
    • Linking from the mark sheet to resources - Ability to embed links in observation criteria that will open specific resources attached to item.
    • Allow styling (bold, italic etc) in observation criteria
OSCE
6

Global Marksheet Improvements

We will be changing the Global Marksheet wokflow to improve versioning and control
  • Investigate versioning improvements to the Global Marksheet
OSCE
7Custom Marksheet ValidationWe will allow custom rules to be set for the marksheet.
  • Configure maximum/minimum number of words recorded
  • Set rules for mandatory information. For example, if 'Fail' is selected, feedback must be completed.
OSCE
8Permissions & OwnershipImproving the way that permissions and roles are managed, and adding more granular permissions.
  • Ability to separate exam-control permissions on a exam basis
  • Ability to manage only certain modules (ie. written or OSCE)
  • Permission based use of the item bank
Written, OSCE
9Security ImprovementsEnsuring that Practique conforms to latest security standards.
  • Improvements to user management
  • Streamline account creation process
  • Investigate two factor authentication
Written, OSCE
10Batch OperationsThe abiliy to perform existing actions on one or multiple candidates, examiners, items and exams.
  • Approve/reject/submit/retire items en masse
  • Master Blueprint management improvements
  • Change examiners on a session/block/exam basis
  • Changing examiner information en masse
  • Start all sessions at the same time
  • Advance Exam Days
Written, OSCE
11Instant MessageThe ability for OSCE examiners to send text messages to floor marshals.
  • Text field in the 'request help' field
  • Sent/recieved/read confirmation
OSCE
12Notification SystemImprove the way that system notifications are delivered and handled.
  • Notify users where items are obsolete due to blueprinting changes
  • Ability to set 'reply to' email notification address and set a custom rejection/bounce back message
Written, OSCE
13Blueprinting ImprovementsUpdating Blueprinting to improve its ease of use and to add some new features.
  • Blueprint tagging validations before an item can be submitted
  • Graphical representation of the exam blueprint
  • Define order of exam blueprints in the list
  • Functional dependencies between blueprint dimensions
  • Cloning Blueprint Dimensions and having same BP dimension in multiple maps.
  • Enhanced Blueprint reporting
  • Remove/archive exam blueprints
Written, OSCE
14Double MarkingAllowing more than one examiner/marker to assign a score to a candidate's response.
  • Enhance the way that double marking is handled in Practique for OSCE stations
  • Allow custom aggregation of multiple marks
  • Allow large scale peer marking 
  • Enhanced marking allocation and processing for written exams
Written, OSCE
15Exam PreparationImproving the workflow for scheduling exams.
  • Impove the way that error messages are allocated
  • Simplification of Exam Preparation Workflow
  • Add more checks and balances (eg Candidates/Examiners in impossible allocations)
  • Improvements to Double Length Station Rotation
  • Implement filtering for candidate select view 
  • Drag and Drop examiner and candidate
  • Sequential testing
Written, OSCE
16Paper OutputBeing able to do more with the paper output function of Practique
  • Enhanced OSCE exam day materials
    • Paper notes for rest station
    • Track and bundle all exam day documents
    • Production of candidate and examiner lables
  • Enhanced written papers
    • Multiple exam paper templates
    • Prepare and export a generic final exam paper
Written, OSCE
17Short Answer Question paper delivery.Being able to scan SAQ papers, and view and mark electronically.
  • Scanning of SAQ answer papers
  • Mark responses on-screen
Written, OSCE
18Exam in progress changesBeing able to make changes to the the candidates/examiners of an exam while the exam is running
  • Being able to make ad-hoc changes to OSCE participants
Written, OSCE
19Question authoringSpeeding up question authoring for written questions.
  • Ability to dynamically add and remove SBA and EMQ options
  • Option to automatically set SBA/VSAQ questions to 'Title Case'
  • Text formatting (including indenting)
Written
20Senior MarshalA new role for OSCE exams that gives someone an overview of all circuits and locations
  • Permissions to see specific marksheet submissions
  • Show Candidate not attending on Marshal iPad
  • Communicate from Examiner iPad to Marshal that there are marksheet amendments pending
  • Floor marshal should be able to see all rounds in a session
  • Floor marshal should be able to view station materials
  • To be able to Marshall all circuit and to have multiple users logged in as an observer
  • Start and stop exam and session controls on marshal screen

  • 'request help' to be flagged prominently on marshal screen 
OSCE
21Extra TimeAbility to set custom time per-candidate as exam goes on.
  • Extra Time could be amended when exam is in session 
Written
22Station iPadsStation specific iPads than can show resources. A substitute to door notes.
  • Notes accessible to various candidates
OSCE
23Candidate ImagesAbility to have a profile image for candidates
  • Candidate photos on iPad
  • Be able to run exam in anonymous mode so Marshall & Examiner do not see candidate name
OSCE
24Exam Control and CommentsAbility to add comments about an exam, and review exam log in details
  • Have a way of toggling users that have not logged in at all (written iPad)
  • Attach notes to exams
  • Attach Audio to sessions
Written, OSCE
25Post-Exam SBA changesAbility to re-define the correct answers after running an SBA paper
  • In the unusual case that an SBA question is set up incorrectly, this would allow users to re key the question. 
Written, OSCE
26iPad Audio ImprovementsImprovements to the way that sound clips are recoreded on the iPads
  • Improving the UI and adding some additional features
OSCE
27Help & DocumentationAdding to the help text and documentation in the system.
  • Increasing the amount of help and documentation within the system
Written, OSCE
28Custom Written QuestionsAbility to create your own question styles, using Practique's existing assessment tools.
  • Scenario based custom written question
Written
29MCQ Question TypeAdding the MCQ question type in addition to the SBA- Being able to have multiple correct answersWritten
30Resources Source FileAbility to upload a source file for resources PDFs
  • To allow source files to be uploaded to items
Written, OSCE
31Item Authoring notifications and permissions

More control over who sees and approves Items. To-do list style notifcation for Item Authors and approvers.


This will allow users to streamline question preparation

Written, OSCE
32Candidate Feedback Report ImprovementsMore control over content and appearance of Candidate Feedback Report.
  • Improvements to candidate feedback report generation
  • Allow pass mark in the candidate feedback report to be toggled
  • Hide 'pass/fail' in candidate feedback report
  • Customisable content from candidate feedback report
  • Text display based on score on CFR
  • Feedback in CFR that depends on the domain scores ie another column in this table that says 'Feedback'.
  • Ability to remove elements : Ability to remove the overall score percentage and pass/fail from candidate reports, so that candidates can just recieve examiner feedback
  • Feedback only CFR, without results (ie formative feedback)
  • Scheduled release of reports 
  • Question breakdown - Ideally these should all be toggable, i.e. Your Answer / Correct Answers/
    Examiner notes
  • Hide Result analysis (table & graph with the per station breakdown). At the moment the exam has only one station.
  • Keep Station Examiner feedback section.
  • Option to generate candidate feedback reports without the Overall Score and Overall Result rows (Added by Robert re PSD-1580)
  • Show pass mark on Candidate feedback report (PSD-1597)
  • Ability to choose either the overall score OR min stations passed on the candidate report or both
  • Toggle Examiner notes (show or not show) Examiner notes should be changed to "Feedback" 
Written, OSCE
33ReportingIntroducing a suite of new reporting functionality
  • Improvements to the way that available reports are presented
  • Add analysis of variance (annova) summary by session
  • Exam analysis report: Comparison of students mean scores across sessions
  • Point biserial report (linking to Ebel)
  • Item response model
  • Store timestamp of PDF generation & e-mail out operations
  • Report distribution should re-use generated PDFs
  • Report distribution depends on PDF generation
  • Distribution list & status
  • Results transcript
  • Figure out statistics for different versions
  • Correlation statistics across exams
  • Reports UX review
  • Station Cut score report - graphs for other Standard setting methods
  • In Written Exams, items are referred to as stations in exam reports
  • Include date and time of generation
Written, OSCE
34Mesh NetworkingAutomatic synchronisation of content through local wireless network, for improved resiliance.- This will allow synchronisation to marshal when there is no internet accessOSCE
35MultilingualInfrastructure for multilingual backend and multilingual delivery of exams
  • Ability top store translated items linked in the bank
  • Able to use the system in another language
Written, OSCE
36Exam AttendanceAbility to control and record attendance of exams
  • The ability to scan candidate id card (barcodes) to take attendance.
  • Show candidate photos on the iPad
Written, OSCE
37Spell CheckerAllowing on-device spell check to be enabled for some exam types
  • Ability to toggle spell check for exams
Written, OSCE
38Server Licence Infrastructure

Ability to validate Practique licence automatically

  • Ability to validate Practique licence automatically
Written, OSCE
39Multiple Cases per StationAbility to mark different items depending on the Candidate in a Station Room
  • Assign multiple cases to a particular station
  • Candidate can do all cases or select from a range
OSCE
40User Terms and ConditionsAbility to upload an organisations Terms and Conditions to cover users of Practique
  • Show T&Cs to Examiners before marking
  • Show T&Cs to admin team on first log in
  • Show T&Cs to written candidates

OSCE,

Written


41Candidate and Examiner ID changesMake it easier to change the ID of Candidates and Examiners in Practique
  • Allow the IDs to be updated

OSCE,

Written


42Filtering in Item SetsAllow filtering of Items within the Item Set
  • Similar to main Item Bank filtering
OSCE, Written
43Candidate Item OrderShow Item number when viewing Candidate Marks
  • Including when delivered randomly
Written
44Angoff Standard Setting WorkflowImprovements to the Angoff standard setting process to allow judges to score within Practique
  • Support Angoff judging methods
  • Allow custom Angoff arbitration
Written, OSCE
45Improve allocation of Item Sets to Exams Add standardised filtering to the Item Set allocation page in exam preparation
Written, OSCE
46Item Change DifferencesAbility to see the differences between items that have been changed easily
  • Allow approver to review the differences
Written, OSCE
47Ebel Judging ImprovementsImprovements to the Ebel Scoring process
  • CSV export of Ebel Score 
  • Show question paper number in addition to Item number
  • Include question stem with Ebel judging form
  • Improve scrolling of large Ebel Sets
Written
48Exam Paper Cover PageAbility to attach a PDF cover page to written papers
  • This would suplement the Edit Blueprint Cover template
  • Ability to offset automatic page numbering
Written
49Filtering ImprovementsAbility to be able to multi select review state and type when filtering item bank 
  • For example, this could be used to filter for draft or approved items
Written, OSCE
50Time taken ReportAbility to track and report on the time taken for candidates to complete written questions
  • Shown for individual questions
Written
51Formatting of Text on serverUpdate the styling of Practique to make text easier to read
  • Particularly, left align
Written, OSCE
52Names in approval processAbility to see te full name of a reviewer and author when reviewing items, as opposed to username
Written, OSCE
53Support Item Set Sections in P4BAbility to create sections within a written exam
  • Instruction text between the sections
  • Configure in Item Set page
  • Randomisation per section
Written
54Easier ordering of Rest StationsAbility to set order of rest stations as part of Item Set
  • This would work similarly to Item Set Sections
OSCE
55Item Set ExclusionAbility to exclude certain Item Sets from Item Set generation
  • Easily limit the Items available to be included in an exam
  • To be set when generating an Item Set
OSCE, Written

Completed Features


ThemeSummaryFeaturesApplicable Exam TypesRelease
1Editable Item DocumentsThis would allow item authors to create OSCE station documents within Practique.
  • Ability to create and edit basic text documents within the Resources tab
  • These resources will be versioned inline with current resources management
OSCE5.4
2APIWe are going to enhance the way that Practique communicates with other systems
  • This will allow users to have Practique integrated with other systems they use. Practique will be able to load information from, and push information to these systems
  • This will include candidate results and feedback
  • This will allow timetable and scheduling information for OSCE exams to be shared


Written, OSCE5.3
3Simultaneous OperationsTasks that are non-instanteous(such as reporting) will run in the background. 
  • Better communication of complex actions progress
  • Increase speed of paper generator
  • Improve speed of exports
  • Improvements to automatic item set generation
Written, OSCE5.3
4Blueprint Constraint Satisfaction ReportBe able to see how well your Item Set meets the Exam Blueprint
  • Warnings on item set constraint satisfaction
Written, OSCE5.4
5EMQ DeliveryAbility to deliver EMQs on iPad and Practique for Browser
  • Support EMQ on iPad
  • Support EMQ on Practique for Browser
Written5.3
6Marksheet & Item PreviewAbility to preview the marksheet and written questions, as you are editing them.
  • Able to generate PDF/word of single item (written)
  • Marksheet preview to check how it would look on iPad for examiner 
  • Export Marksheet with count As tagging so that it can be reviewed pre exam by external examiners etc
  • Generate generic paper marksheet as a backup that can be completed in an exam
Written, OSCE5.4
7Exam TimerAdding an indicative timer for candidates when taking a written exam
  •  This would not lock candidates out
Written5.3
8Multiple FilteringAbility to have multiple filters at the same time when searching
  • Add multiple 'AND' filter for same blueprint dimension
Written, OSCE5.3
9Question Authoring (Tables)Speeding up question authoring for written questions.
Written5.3
10Reporting Refactoring/InvestigationIntroducing a suite of new reporting functionality
Written, OSCE5.3