Examiner guidelines on question development, marking and confidentiality
Question development
All current Examiners will have been to at least one training workshop and/or a training session associated with an examination sitting. All Examiners will have prepared at least two or more full questions. As such, all active Examiners should have some experience of how questions and the marking guidance are developed and used.
A flowchart outlines the standard process of question development, including the re-editing, peer review and proof-reading safeguards, is attached. All Examiners undertake peer reviews and additional reviews following re-editing of questions if this is substantial.
All questions are designed on standard question proposal templates.
There is a detailed set of guidance for question peer reviewers. The role is vital to the production of high quality scenarios and examiners must be familiar, and ideally have some experience, or performing such peer reviews.
Question Selection for each sitting
There is also a standard matrix of scenario type and question content which assures each examination contains a similar range of scenarios and topics covering the required areas for the MFPH. The Chair of the Question Development Group (QDG) is responsible for ensuring the supply of scenarios conforms to the agreed mix in a timely way - allowing peer, proof and final review by Academic Registrar processes to be completed in a timely way prior to the examination. In future, it is anticipated that the QDG may be able to forward plan sittings for the coming year, but to date the intensity and complexity of preparing high quality scenarios has meant that in general the QDG are working to the next one to two sittings only.
Marking
The level of average competency is that expected of a trainee at the completion of Phase 2 of the syllabus (two to three years into training) and knowledge of the syllabus for DFPH is assumed. The guidance for marking candidate performance in MFPH continues to be refined. It is basically expected that an average competent performance is graded a 'C' pass. The marking guide in the questions give specific items that are required for a competent pass, and increasingly, there is refining of additional guidance to help specify what grades a candidate as a weak (fail) or excellent performance:
- General marking criteria for each of the five core competencies
- Specific general question guidance from each question
Each question has an Examiner Marking Guidance commentary where the overall thrust of the question is described and some of the main aspects the question structure is designed to elicit, within the framework of the five core competencies - Specific marking criteria for each question
Each Examiner marking guidance box contains a commentary on the nature of what is required for an average answer to that specific question. This may also include one or more explicit facts or applied skills that are required to be demonstrated to indicate a pass or to demonstrate excellence. These are written into the question at production and checked by editorial process and peer review
The candidates are guided in their approach to the answers by the use of general candidate guidance contained in a specific introductory section of the candidate briefing and in the use of scripting provided to the actor or non-marking examiner role-player to help elicit answers on areas being examined. The extent of the guidance is purposefully varied but is designed to elicit the required responses. To this end, the alignment between the factual briefing, the specific examiner marking criteria and the scripting of each question is carefully checked by editor, peer reviewer and proof reader.
For an illustration of the overall structure of a question, its component parts and the aspects of the marking criteria guidance described above, see the OSPHE sample questions.
Confidentiality
Confidentiality is of vital importance and examiners and question authors must keep all material completely confidential. All draft and complete scenarios are password protected but it is vital that authors and peer reviewers keep all material securely.
As with such activity, Examiners should not discuss scenarios once worked up ready for the examination.
Examiners may be asked to give advice to trainees on the preparation for the examination. They are warned to take care and are recommended to respond to trainees’ requested areas of revision rather than to initiate topics.
Copyright
Questions often utilise web or other sources of data used in the candidate briefing packs. FPH has a series of copyright permissions that enable many sites to be used with no specific infringements. Authors should be aware of the permissions, and utilise data from those sources wherever possible, as new permissions can be time consuming to obtain. Experience suggests that data constructed by the author as an original document is often simpler, clearer and easier to use, and does not require third party copyright permissions. FPH is in the process of requesting authors sign copyright declarations for scenarios used in the MFPH. If in doubt, please discuss with the Education Office.
