User Testing is a vitally important part of the creation of any usable digital creation. Throughout the development of my application, I have been conducting informal user testing with the younger members of my family and their friends and I have recently conducted formal user testing at Holy Name R.C Primary School.
The user testing at the school was an extremely productive undertaking but before I start evaluating and explaining my findings, I want to take a little look at the value and processes of user testing.
User testing helps test the usability of a designed artefact, the Nielsen Norman Group define usability as:
“Usability is a quality attribute that assesses how easy user interfaces are to use”. The word “usability” also refers to methods for improving ease-of-use during the design process.
Usability is defined by 5 quality components:
- Learnability: How easy is it for users to accomplish basic tasks the first time they encounter the design?
- Efficiency: Once users have learned the design, how quickly can they perform tasks?
- Memorability: When users return to the design after a period of not using it, how easily can they re-establish proficiency?
- Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?”
- Satisfaction: How pleasant is it to use the design?
It is often hard to know when and how many usability tests are needed within the design process, usability expert Steve Krug says “Timing is everything”
“Most companies can only afford to pay for one round of usability testing, which they’ll do near the end of the development cycle, when the thing’s almost finished,” Krug continues. “Unfortunately, that’s the worst possible time to do a test. Yes, you’re going to find problems but you’re not going to be able to do anything about them any more. Some of them are going to be deep-seated problems. If you start testing at the very beginning of your design, though, you can pretty quickly uncover those problems.”
According to the Nielsen Norman Group “The best results come from testing no more than 5 users and running as many small tests as you can afford.” Nielson (2000) http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
This is good news for me, as it means that my smaller informal tests that I had previously conducted were a step in the right direction of this little and often approach and it means that my more formal school based testing, can be conducted with a smaller impact to the everyday business of the school.
User testing with children has its own challenges; Tom Stewart offers advice for usability testing with children in his article ‘Tips for Usability Testing with Children’. Stewart starts by explaining that clinical user testing labs can be counterproductive to the task when dealing with children, a friendly, familiar environment can often yield better results. A familiar face can also be helpful, as described by Stewart:
“When testing with younger children it is important to have a parent or familiar adult around to provide reassurance. The adult may or may not participate directly in the session depending on what we are trying to achieve and what sort of feedback we need.” Stewart (2010) http://econsultancy.com/uk/blog/5543-tips-for-usability-testing-with-children
Jamie Sands is a Usability Consultant at User Vision, he also agrees that the comfort of the user is very important:
“Usability testing is most effective when the respondents being tested are comfortable and therefore happy to think and discuss what they are interacting with. If children are not at ease, they are unlikely to respond in a natural manner and less likely to give their attention to the tasks at hand.” Sands (2010) http://www.uservision.co.uk/resources/articles/2010/usability-testing-young-audiences/
The pressure of a “test” can be scary for all people especially children, so you should also try and make the research less daunting:
“The experiment should be welcoming and not intimidating. All usability testing can potentially feel like a test that the user can pass or fail. To avoid this, the moderator needs to be comforting and reassuring to the respondent to ensure they do not feel as if they themselves are being tested.” Sands (2010) http://www.uservision.co.uk/resources/articles/2010/usability-testing-young-audiences/
When assigning tasks during testing, it is also import to consider their appropriateness in relation to the child’s ability. Stewart (2010) says “Make sure the length of the session and the difficulty of task is appropriate to the children’s age and development.” Children are also more prone to boredom and distraction which can also be a problem during user testing with children, Stewart explains:
“The facilitator has to keep a close watch for boredom, fatigue or children becoming too engrossed in one task and have a range of options ready to move the session on to more productive areas.
We always try to aim for sessions which are fun and tasks which are engaging with lots of different activities, games and breaks for snacks and drinks.” Stewart (2010) http://econsultancy.com/uk/blog/5543-tips-for-usability-testing-with-children
In order to get an accurate result from user testing, it is important to understand that children communicate differently to adults, and understanding this can make the interpretation of the experiment easier, as explained by Stewart:
“It is important to recognise that children communicate in different ways, many of them non-verbal. Facilitators need to be flexible and use appropriate communication techniques. Very young children (under 6) are often not able to express themselves verbally, so behavioural observations can be as important as verbal feedback (e.g. smiling, fidgeting, sighing, groaning).” Stewart (2010) http://econsultancy.com/uk/blog/5543-tips-for-usability-testing-with-children
Sands also discusses interpreting responses during user testing:
“Children are often less vocal, or less able to verbalise their opinions about issues they experience. An uncomfortable child may be too shy or uneasy in giving responses if they feel they are being tested and are concerned that they may say the wrong thing. Whereas adults will express their feelings vocally, children are more likely to offer clues non-verbally, by fidgeting, smiling and their body language etc. The experimenter should be aware of these cues and note these in conjunction with potential problems experienced with the site. Look for the child’s lack of engagement with the site. Yawning and fidgeting can be clear signs that the site is no longer capturing the child’s attention.” Sands (2010) http://www.uservision.co.uk/resources/articles/2010/usability-testing-young-audiences/
I structured my tests in a way that I hope will allow me to get the greatest amount of information from the testing experience.
I arranged for a school governor, who regularly visits the school to help with reading to accompany me, as I believed this would help the children to feel more at ease. I firstly introduce myself and explain that this is not a test for them but it’s a test for me, so there is no need for them to be nervous. The next step is to let the children have free roam of the application, while I make notes based my observations of their choices, usage and general behaviour. I then ask the children to complete a small amount of tests, followed by 4 questions about the experience.
Recording user testing is important, in order to review and document the events as they happen, for evaluation at a later date. This recording can be done in a number of ways, including video recordings, audio dictation and written accounting based upon observation. Due to the age of the testers and the setting in which the testing is taking place, the most appropriate method of documentation and the one I have employed is written accounting based upon observation. Video and audio recording of young children raises too many problems, that could jeopardise my opportunity to conduct testing.
I was originally planning to produce some form of visual reference to demonstrate how the testing went, i.e. a chart but in some ways the testing went too well.
The users also completed the tasks I set them successfully, showing a good understanding of how to navigate within the application and how to operate all of the interactive features that they encountered. This could be interpreted as evidence that the design is effective and that no changes are needed but I feel another round of user testing, time permitting may be needed in order to reinforce these initial findings.
The most interesting results in my opinion, came from my observation during the free roaming time at the start of the test. During this stage I noticed that the children were drawn towards the highlighted interactive elements within the app and they sometimes missed elements that had been left un-highlighted. This was exactly the reason why I left some interactive content without highlights, so that I could establish whether the highlighting technique was effective; see the difference between the two below:
I did noticed that the hit recognition area on a couple of buttons needed increasing and the text on one of the buttons needed changing. I also realised that I needed to add arrows to the Viking house interior.
As a result of my testing, I have changed the elements indicated above which will hopefully improve the usability of my application.
All users tested said they liked the application and expressed a desire to use applications like this within their normal classroom lessons and indications to back up these statements were given during the testing by the children, in comments such as “cool” and “that’s class”
All of the children indicated that their favourite elements of the application were the interactives and in particular the students enjoyed seeing themselves wearing a Viking helmet. This was what I was hoping for but I do hope that the interactives do not divert attention from the other elements, as my application needs to work on all levels. I did however notice that all students read the text and some gave answers based on illustration and animations, which is a good indication that all elements are communicating information.
Prior to the user testing my main concern was that the user’s would struggle with the amount and level of the text within the application but to my delight all of the users tested managed very well with the text, although the head teacher did point out that the younger age users may struggle with the amount of text.
The school’s Headteacher raised the question of audio within the application; previously I have been encouraged by some people within my education establishment to include background noises to some of the scenes within my application. This was something I have resisted, as I believe it would not be appropriate if the application was been used within an education setting such as a classroom; can you imagine 30 children using the application at the same time but all at different parts of the app, each with differing sounds blurring out, mixing to make an unorganised hum of pure noise?
This would be counterproductive to the learning environment and process, and although the volume could be simply turned down, the time it would take to ensure each student had turned the sound off could be better used by actually teaching or exploring with the application. The head teacher agreed that sounds of that nature would not be appropriate for an application of this type but she did suggest that maybe some audible Viking words within the Viking Language section would be a good addition to the application, this was a good re-enforcement to previously stated intentions to add such a feature within the Viking language section of my application.
Overall my testing has been a great success; my application appears to be easy to use, whilst communicating information to and entertaining the user. I would like to leave this post with a quote from one of the children tested: “that’s class”.
DuVerneay, J. (2013) When to Test: Incorporating User Testing into Product Design,
[online] Available at: http://www.usertesting.com/blog/2013/03/04/when-to-test-incorporating-usability-testing-into-product-design/, [accessed 29th May 2013]
Krug, S. (2011) DIY Usability Testing, [online] Available at:
http://m.netmagazine.com/interviews/steve-krug-diy-usability-testing, [accessed 30th May 2013]
Nielsen Norman Group, no publication date, Jakob Nielsen’s Alertbox Articles, [online] Available at: http://www.nngroup.com/topic/user-testing/, [accessed 31st May 2013]
Nielson, J. (2000) Why You Only Need to Test with 5 Users, [online] Available at: http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/, [accessed 31st May 2013]
Nielson, J. (2012) Usability 101: Introduction to Usability, [online] Available at: http://www.nngroup.com/articles/usability-101-introduction-to-usability/, [accessed 31st May 2013]
Sands, J. (2010) Usability Testing with Young Audiences, [online] Available at: http://www.uservision.co.uk/resources/articles/2010/usability-testing-young-audiences/, [accessed 29th May 2013]
Stewart, T. (2010) Tips for Usability Testing with Children, [online] Available at:
http://econsultancy.com/uk/blog/5543-tips-for-usability-testing-with-children, [accessed 30th May 2013]