Using a sophisticated assessment authoring solution to create a central repository of question content is considered to be best practice when it comes to the creation of on-screen and paper-based examination content.
Question authors often come from a variety of professional backgrounds with different experiences in item authoring. Although they are subject matter experts, the whole process would undoubtedly benefit from a degree of structure and consistency, particularly when working with distributed teams with mixed levels of experience. By using on-screen authoring tools, the quality and creativity of the items can be improved, enhancing the value of your item bank. As getting this message across to stakeholders can sometimes be a challenge, we have put together a list of 10 benefits of moving to on-screen authoring, based on the experiences of BTL, our customers, and some additional guidance from educational consultants AlphaPlus.
1. Work to an agreed process and avoid missing out important information
There are a wide range of components which are essential and must be captured before an item is ready for live use. These can include: question stem text, answer options, answer keys, marking scheme, learning outcomes, units, mark weighting, authoring status, subject level, associated media, metadata, item statistics, source materials, and correct and incorrect feedback to name a few. Giving an author a blank canvas to work with can result in missing data which is hard to track and change. The best practice is to guide them through this process, ensuring that information is captured at the right stage in the workflow and all authors are working to the same creation process.
2. Increase productivity and feedback as part of a collaborative team
Feedback from reviewers is an essential part of the authoring process, therefore waiting for a batch of items to be completed before submitting can result in lessons being learnt too late in the authoring cycle. By providing direct on-screen feedback and collaboration, the time taken for feedback cycles to be completed is reduced.
[one-half-first] 3. Let nothing be lost in translation, reduce design and layout iterations
When a question is not being directly authored on-screen then detailed visual presentation descriptions will need to be provided to the multimedia designer. Even with detailed descriptions, the question item will often require some additional design iterations before the desired outcome is achieved. By allowing an author to directly set up the question using easy and intuitive creation tools, the author can be more creative and does not need to spend time providing additional descriptive information.
[/one-half-first] [one-half]
[/one-half] [clearfix]“A key factor in our selection of Surpass was how clean and simple the authoring environment is. Our Clinicians, who produce our question items, do not have time to learn a complex IT process, but we knew we could just give them access to the tool and they would have no problems creating questions.”
Romana Moss, Computer Based Testing Project Manager, Royal College of Paediatrics and Child Health (RCPCH)
4. Save time with automated item publishing
Professional assessment solutions like Surpass undertake a number of time saving tasks automatically. For instance, in the authoring workflow, once an item is set to publish by an authorised user, it will be instantly published to the item bank. This is a huge time-saving activity which also means that edits can be made instantly to assessments without the need for re-scheduling. In addition, template items can be copied with greater ease, meaning that production of complex items with similar interactions or presentation formats can be achieved rapidly.
5. Be proactive with production management, see at a glance how close you are to meeting your authoring quotas.
Items are often produced by a large, distributed authoring team with set production quotas. By using an assessment authoring tool such as Surpass, production targets can be monitored, for both the subject as a whole and for the individual author. Managers can therefore be proactive rather than reactive to failing targets; this in turn can be less stressful for the team as it leads to a more supportive team environment.
6. Find items and data quickly
If information relating to the creation of items is locked up in non-searchable formats, then it can be difficult to audit information based on the contents. By using standard data fields and custom metadata you can search on any individual characteristics of your authoring output.
7. Improve data quality with consistent and accurate metadata
Much of the same supporting metadata, such as Units or Learning Outcomes, will need to be captured repeatedly as multiple items are authored. If these are treated as free text within a flat file it can result in many variations for the same content. Not only is it time consuming to keep typing the same information, but it can also lead to consistency errors across your data which can mislead audits and lead to time wasted with data cleaning. By providing authors with pre-defined data to select from, consistency errors become a thing of the past.
8. Give your questions more impact with rich media
Often authors will create items on the assumption that media will be available, which can sometimes result in costly mistakes. By following an authoring workflow, checks can be made to ensure that the right media is available during early in the process, preventing the need for rewriting items to match new supporting media files or source material.
“One of the key benefits of on-screen authoring is that it provides the opportunity to be far more creative and imaginative than one could normally be in a more conventional authoring environment. It opens the door to a wealth of rich and interactive media that can easily be embedded into a range of different item types. Immediately the items become far more dynamic and can often capture real world contexts or issues in a way that brings the focus of the assessment to life. Conventional approaches to item design are challenged and often need to be re-thought as the new pallet available to the author is vast and vibrant.”
Gillian Whitehouse, Head of Project Development, AlphaPlus Consultancy
9. Get a greater sense of pride in your work and be more creative
Seeing an item you have created as soon as you have inserted basic information can give participants and teams a greater sense of pride and better visibility of their hard work, it can also enable authors to be more creative in their item writing.
10. Improve the candidate experience, see the question as a student would with instant preview
By authoring directly into Surpass, authors can instantly preview questions – checking interactions, marking and identifying problems such as missing or misleading information well in advance of publishing, often reducing the need for multiple checking cycles.
As more and more teams make the move to on-screen assessment authoring, Surpass will continue to evolve with more time-saving and quality features being created. We welcome feedback on our authoring solutions therefore if you are interested in experiencing many of the benefits discussed in this article then please contact BTL for a demonstration of Surpass, The Assessment Platform.
Looking for more information on how leading organisations create top quality assessment content? Visit the ‘Who We Work With’ and ‘Case Study’ Pages for more great advice and ideas.
[clearfix]