Week 5 Assignments Blog

Do you think some tasks are better suited to different types of usability testing? Why or why not? Give at least one example.

7 thoughts on “Week 5 Assignments Blog

  1. Do you think some tasks are better suited to different types of usability testing? Why or why not? Give at least one example

    There are definitely tasks that are better suited for expert analysis vs. user-based observation. One project that I was almost involved in was converting a web application from single authentication to two-step authentication. This would probably be best suited for testing by expert analysis since it was very technical in nature. Another project I worked on was taking the file output from a Scantron system and importing into a web application. It involved the user taking a number of tasks so testing this system would be best tested through user observation since it had so many user steps.

  2. Do you think some tasks are better suited to different types of usability testing? Why or why not?
    Yes. From my personal research experience, while I was testing shopping behavior of consumers based on the photographs they see on the website, an online survey/questionnaire was the best type of usability test that I thought could bring out better results because we are engaging the user in that real-life scenario of them shopping online. If we are employing a focus group for this research, the results might be biased based on hearing the other participants’ input.

  3. Security-related tasks are much better suited to expert evaluation. The underlying cryptography and workflow is often designed and reviewed by experts, but even the interface level often has more benefit from expert analysis. Take a login page for example. Many large sites (Google) or sites that require tight security (banks) will have the user enter their username, then hit a button, and on a new page, enter their password. Why do they not have both on one page (like most login forms). The reason is not obvious to an end-user, but makes a lot of sense if you know about security. One simple explanation is that it now takes more time to enter both pieces of information. The amount of time it takes to enter for the user is negligible, but if a remote computer was attempting to try millions of combinations to guess a password, this would add a large amount of time to their process. Another reason is that it allows different workflows. Perhaps it makes more sense to get the user ID, then send a text to the user’s phone, then ask for the password. These sorts of differences are not something that most users could identify, but security experts could.

    That is not to say end-users should not be involved, they should review the solutions experts come up with to ensure they are user-friendly and not unnecessarily difficult to use. Security/convenience balance is important, and often only the end-user can determine where the acceptable line is.

  4. Each of the different types of usability tests have advantages and disadvantages which leads to some being better suited for a task. Within my organization we send out packets of specific task for each user to perform and ask them give it a pass or fail and add comments if it fails. This method works for small alterations to the system such as adding a radial button that needs to calculate something in the background, or to update letter templates that are pre-populated within the system. We send these packets out to at least two individuals in each class that will be affected by the change leading to up to 15 test cases. I also go within the system and preform system checks, basically checking on things that we had issues with in the past.

    As we work on developing the brand new system that will be integrating all these systems we are working with expert analyst that have our thousand page guidance document to help them test and work on implementing the program. When it comes to this testing, I would not want the testers that pass/fail the system to be the ones assisting us. As we move to testing a final product, and not testing business rules and workflow, I would be glad to help write the testing packets for them myself. (Thankfully this is not my job, I get to be an expert tester!)

  5. Do you think some tasks are better suited to different types of usability testing? Why or why not? Give at least one example.
    I think there are definitely tasks that are better suited for different types of usability testing. There are certain tools or interactions that we may be designing that need to be field-tested. For example, if we are designing a new GUI for an ATM machine, if we place a machine in a lab we may or may not get the necessary lighting situations to determine if the GUI is easily readable, does the screen material still reflect/absorb the correct amount of ambient light to allow ease of use. These can also be done with a simple observational study as you can tell by a persons actions at the device if they can read the screen and if it is intuitive enough for them. Conversely, if we are simply designing a training program to run on Captivate, that can be done in a lab environment as we can simulate a cubicle environment with lighting and ambient noise and interruptions it is not necessary to be tested in the field. This would also necessitate a deeper interaction from the tester either as a participant or covert observation to get a real feel for how the widget is working for the users. I know there are cost considerations and budgetary concessions that would have to be made, but I feel that if possible the type of design should drive the type of experimentation and usability studies that should be conducted.
    While there is no exact science for how or when a lab environment is chosen over a field situation I think that common sense would tend to dictate how the testing should be done, with all other aspects being equal.

  6. Do you think some tasks are better suited to different types of usability testing? Why or why not? Give at least one example

    Definitely some tasks are much better suited to certain types of usability testing. Many interactions can be effected by the usability testing if care is not taken. For example, users may interact with a software in a different way if they feel they are being watched or a certain outcome is expected. In these cases using a usability study that involves the user feeling more secure such as using 1 way mirrors may give more reliable results than having a researcher right beside them.

    I actually participated in a usability study for Youtube at one point, and found it was extremely hard to be part of the study. It was over a video call where they had a live testing web page that they were having me interact with and give my opinions and feelings. I generally struggled to give them information I would have given them if I was not in the study, as I felt that they were looking for certain things and in trying to not be biased towards those results, I may have been biased in other ways. I think a better system would have been to give me a time to try to accomplish a certain thing with the testing site, and then afterwards ask me questions so I didn’t feel “under the gun”.

  7. I do think some tests make more sense for certain kinds of tasks. Depending on the technical nature of the back-end (so to speak), you may need to be testing things that aren’t transparent to a user or the task workflow. Security testing is an example for this. Additionally, certain kinds of user testing may work for certain interactions, and not for others. A complex task with complex outcomes which are difficult to verbalize while performing might be better suited to observational testing or physiological testing. Examples of this would maybe be physical or full-body, immersive tasks like driving, navigating one’s body through a space, or performing complex operations under pressure, when one doesn’t have the cognitive space to reflect on their own performance.

    Lastly, it seems clear that there will be big differences in the kind of results desired, regarding whether you want to make gains in subjective metrics like likeability or fun, versus objective data-driven statistics like efficiency and error frequency.

Leave a Reply

Your email address will not be published. Required fields are marked *