To continue my series on VTA Test Feedback features in version 8.0, I created a video that looks at another way to collect data from students about the Test Question… Feedback Measure Sets. This post will go over the details described in the video.
Feedback Measure Sets in VTA 8.0
Introduction
In a previous life, I helped manage the VTA LMS with a 12K+ item question bank. We had guidelines for what constituted a valid question, a valid test and we regularly pulled Item Analysis Reports to gauge student success rates by question. Anytime that success rate was less that 70%, we would examine the training materials, evaluations and discuss how can we help the student succeed. We could only guess at what might help, but if we had the ability to collect students’ thoughts about the question, we could have improved some success rates much faster.
To continue my series on VTA Test Feedback features in version 8.0 (see my post on the VTA Confidence Measure), I created a video that looks at another way to collect data from students about the Test Question… Feedback Measure Sets. This post will go over the details described in the video.
Feedback Measure Data Relationship
A Feedback Measure Set is a “Question” (Display Text) that asks students for feedback about the test question/answers with “Check All That Apply” selectable options (Feedback Measure Items).
A Feedback Measure Set is then associated on the VTA Test Templates for which you want to use it.
Display in VTA Learner
The Measure Set is presented on each Test Question in VTA Learner, below the VTA Test Feedback Confidence Measure (if the Confidence Measure is also turned on).
If the Confidence Measure is not on, the Measure Set is presented below the Question’s Potential Answers.
By clicking the word “Feedback”, the section will expand to display the Measure Set.
VTA Data Setup
To begin using VTA Feedback Measure Sets, you need to:
- Add Feedback Measure Items in the Master Site.
- Create a Feedback Measure Set in any Site.
- Associate that Feedback Measure Set to any Test Template in that same Site.
Add Feedback Measure Items in the Master Site
As mentioned, Feedback Measure Items are standard potential answers, controlled in the Master Site (00), that can be used on any Feedback Measure Set.
When entering Feedback Measure Item values, remember:
- Feedback Measure Sets can be used during the Test and/or during Review. Be sure the response would make sense where it appears (during the Test, during the Review or both).
- Keep responses simple. You don’t want to have to explain when/why an Item should be selected. It should not be too vague or use complex language (you don’t want to hear, “What does that mean?”).
Delete/Inactivate Feedback Measure Items
If you attempt to delete a Feedback Measure Item that is currently associated to a Feedback Measure Set, or if it was previously – and part of an online test, you will see the highlighted message below:
Feedback Measure Item Examples
Below are example of Item values you may consider using. Any reason you may need to update a question can be used as an Item.
- Answers are not clear
- Correct answer is not really the correct answer
- Image is distorted
- Media link is broken
- Not covered in training
- Question is not clear
- Reference Link is broken
- Translation is incorrect
- Typo in Answer Text
- Typo in Question Text
- Video does not play
You can also choose to be very specific for whatever information you are looking to collect.
Item Record
In Evaluator, at the Master Site (00), open the Feedback Measure Items grid and click Add.
In addition to the Item Name (which can be up 128 characters long) and being able to set its Active status, the following are displayed:
- Added Date
- Last Inactivation Date
- Last Reactivation Date
These dates are disabled from edit. They are for your informational purposes only.
Create a Feedback Measure Set in any Site
A Feedback Measure Set is the combination of a Question and a collection of the Feedback Measure Items (or potential answers) you want to use for a Test Template.
In Evaluator, at any Site (00), open the Feedback Measure Sets grid and click Add.
General Tab
When creating a Feedback Measure Set, although you will give it a Measure Name, the student only sees the text you enter in the Displayed Text field. Keep in mind:
- Be sure the Displayed Text that students will see will logically direct them to make Item selections.
- You can choose to inactivate a Feedback Measure Set to no longer have it available for selection on Test Templates.
Items Tab
Navigate to the Items tab to manage the Items on the Feedback Measure Set.
When you add a record, you control the Order that Items will appear for the Feedback Measure Set.
Associate the Feedback Measure Set
Once your Feedback Measure Set is turned on for a Test Template record, the next Student that starts that Test will have the opportunity to submit feedback.
To associate Test Feedback options to a Test Template:
- Open the Test Template record.
- Expand the Test Feedback section of the record (below the Description field).
- Select the options to include:
- Question Feedback – Students will be able to submit feedback during the Test.
- Review Feedback – Students will be able to submit feedback during the Test Review.
Summary
The Feedback Measure Set is a great tool to collect quantitative data you can use to improve test assessment measures.
If your test question’s format, language and/or structure are adding to the challenge of student success, fix them! People should be quickly answering your questions, not trying to figure out what is actually being asked.
Feedback Measure Sets can help you do that. Learn from your students… they may find typos, poorly written questions or confusing formats days, weeks or even months before you have a chance to analyze results.
By using Feedback Measure Sets, after your tests have been taken, you can use the VTA Item Analysis Report to compare success rates to the issues people find with your questions.
Watch the RISC Blog for a post about other Test Feedback reporting and one more Test Feedback feature…. Comments!
eLearning professional with over 20 years experience positively impacting corporate productivity through skillful management of staff development programs and implementation of RISC LMS solutions. Marc attained an M.S. in Industrial/Organizational Psychology from Springfield College in 2005 and is reputable for working with groups to identify issues and create tools, processes or training solutions to achieve and exceed measurable objectives.