Testing the Survey Once Coded
Once a survey has been programmed, it is time to test the operation of the survey before going ‘live’, in order to verify that the data you intend to pass to Mercury’s systems is being correctly formatted and passed.
- First, when testing make sure that Mercury’s moment-to-moment ratings player or text highlighting tool appears properly within the survey, as expected, when expected, and that it includes the correct ratings scales, and legends or, in the case of text highlighting, the right message, properly formatted.
- Second, add a few test ‘complete’ records and note down the respondent ID that you were assigned and the answers you gave so that you can make sure that Mercury’s systems were passed the correct values (often demographic data is added into Mercury’s systems afterwards so your survey may not yet show any of the demographic data). In the case of a dial-test, also note down how you rated the video (i.e. you rated the video at a steady 50 or 10 or 90 for example), while if a text highlighting exercise, make sure you know which words you highlighted – to verify that everything was recorded correctly. NOTE: It is recommended that when a survey is being tested (in test mode), that it print out the respondent ID number so as to make data verification easier.
- Third, you must make sure that the data passed to Mercury’s systems is correct. If a moment-to-moment dial-test, for each complete you’ll want to make sure the rating line matches the way you rated the video. If a text highlighting exercise you’ll want to make sure that the proper words have been highlighted. Lastly, in both cases, if the survey has been programmed to also pass some demographic variables, you’ll want to check each of those as well – again to ensure that the answer choices you selected in the survey match what was passed to Mercury’s systems.
Reviewing test data is easy because Mercury provides a test capability that allows you to test the data on your own. After adding several records, open the following Mercury test webpage:
Enter the identifier of the Mercury project you wish to test and enter one of the respondent IDs you used for testing. If you enter a valid identifier and respondent ID, the page will present you with a dump of the data that was passed and captured in the Mercury system for that respondent ID number. If you get an error attempting to access the respondent test data, make sure the identifier and respondent ID are correct. If you continue to have an error, this indicates that the respondent ID you should be passing to Mercury’s systems is not correct. It is essential that Mercury’s systems are passed the correct unique ID for each respondent, so immediately have the survey programmer check to ensure that they are correctly passing the ID. Always feel free to contact Mercury.
If demographic data was passed (NOTE: often demographic data is added into the respondent record at a later point so you may not see any demographic data), the display will show all the captured data including the question names and answer choices selected by the respondent. If a dial-test, the data display will also include a representation of the rating from that respondent. If a text highlighting exercise, the display will show:
For text highlighting exercises, the display will show the positional number of the words that were highlighted. By way of example, if a message being tested had 500 words, and you had highlighted the first word (word “0”) and the last word (word “499”), you would see the “Data: 0;499”. If the first through third word had been highlighted, as well as the last word, you would see “Data: 0,2;499” – meaning words 0 through 2 were highlighted along with word 499. For testing, we recommend that you simply highlight the first couple of words, and the last couple of words.
If the captured data is as expected and if the dial-test rating line or text highlighting data is correct, then everything is working properly. Remember — if your survey is passing demographic data as well, do tests that select every answer choice so that you can be sure that all parameters are being passed correctly.
For dial-tests, to accelerate your testing, you might prefer not to have to watch and rate one or more videos during each test cycle through the survey – so as to not waste significant time with the rating process (unless one is specifically testing to ensure that the correct M2M rating is being recorded for the survey).
Mercury can place each video that is being rated by the M2M system into what is called “staging” mode while testing. When in staging mode, a hidden option will be enabled that permits the rating of the video to be “skipped” if desired to allow testing to go quicker, without the need to watch videos or rate each video.
When in Staging Mode, simply right click anywhere on the rating recorder slider and you will see a “Skip Video” option. Choose this option and the M2M system will set the remaining moments of the video (or all of the ratings for the video) to “50” and will advance in the survey. This capability works with the M2M demonstration / help capability as well as on the display of video in M2M without rating.