Mobile assessments: The pros and cons

by |
Globally, mobile phones are more popular than basic sanitation – according to the United Nations, 6.6 billion people have mobiles, while only 4.5 billion have a toilet.

That means more jobseekers will be looking to use mobile technology for the job application process Australian recruiters are among the most likely in the world to embrace it, according to recent research.

But allowing job candidates to do online assessments on their mobile devices could lead to potential problems.
Advisory company CEB’s Global Assessment Trends Report showed that 59% of recruiters in Australia and New Zealand would support the use of mobile assessments, compared to the international average of 42%.

However, more than 50% of those potential early adopters want research to support mobile assessments.

Since it’s an emerging area, there have only been a handful of studies done and so far, research shows that using different devices can lead to different results on certain tests, according to Ken Lahti, CEB’s product development and innovation vice president.

“When we have these mixed modalities, if those candidates are applying for the same jobs, we need to be sure that the tests work just as well on a mobile device as they do on a traditional online internet test," he said.

“The research shows that that’s not always the case. In particular, with assessments that are tapping into cognitive ability or problem-solving type skills, people perform a lot worse on those on a mobile device than they do on a regular online test on something other than a mobile device.

“If you had those candidates coming in and you were to deploy a cognitive ability test and allow people to take it on mobile, what you’d see is that these people would have a disadvantage in the application process and would score lower and would likely be screened out at a higher rate.”
The wide variety of places that people use their mobile devices is one of the potential explanations for why there are differences in test scores, said Lahti.

The way in which people use mobile devices – in small bursts, rather than for long periods – could also affect the levels of concentration needed to complete a difficult cognitive ability assessment or an overall job application process.

Other basic multiple choice tests that are not about problem solving tend to work the same on mobile and non-mobile devices, said Lahti.

There’s also the possibility that doing online tests on a mobile platform could give candidates a greater chance to cheat, with about 20% of those surveyed citing it as a potential issue.

“It is possible to cheat or try to cheat on any kind of unsupervised test that is delivered over the internet. Mobile might make that easier or more common because of where people tend to use their mobile devices. They tend to have them with them much more frequently and they tend to have them with them in a broader variety of situations than they would have typically had a laptop or a desktop,” said Lahti.

“So rather than just getting help in a deliberate fashion from a spouse or friend or someone who happens to be at their home, the fact that the application process is mobile means that they could be riding a bus or out at a café or anywhere and potentially get help from a wider variety of people who they might have access to.”

Would you let candidates do mobile assessments? 
  • Joseph P. Murphy on 6/26/2014 12:30:29 PM

    The migration to mobile is inevitable. Managing expectations is the biggest challenge at this point.

    Very few ATSs are fully mobile ready. As such, you need a PC to get to the assessment stage anyway.

    In a conversation with the head of talent acquisition for a global consumer brand stated time on site from mobile users goes up. And its not because it is a good experience. Navigation issues, touch screen insensitivity, pages that slide and images that don’t scale are all at play in a negative candidate experience.

    Assessment design must undergo transformation to adopt to mobile delivery. In the early days of internet testing, the common practice was to migrate a bubble sheet to the web. Internet testing has now become multi-media, engaging and educational experience delivered via methods such as the Virtual Job Tryout.

    Now, in its early days, the practice for mobile is to migrate a PC design to 3 inch screen. Just like the bubble sheet was a poor translation to the new format, migrating PC designs to mobile does not transfer well either.

    Stay tuned though, I believe we will see a fascinating evolution in the mobile assessment space.
    Working with smaller real estate will force innovation. The way we think about measurement will have to evolve as well.

HRD America forum is the place for positive industry interaction and welcomes your professional and informed opinion.

Name (required)
Comment (required)
By submitting, I agree to the Terms & Conditions