in these studies. The post run questions do ask how they felt they did in completing the task.
These questions are subjective forms of SA. However, just because a user thinks they
performed well, it does not mean they actually did well; we observed many instances of subjects
reporting they had not hit anything with the robot, when they had actually caused damage to the
arena [Yanco et al., 2004].
3.1 Test Arena
For all the tests that were performed, the test arenas were all relatively the same. The studies
discussed by Yanco, Drury and Scholtz [2004] all took place in the National Institute of
Standards and Technology (NIST) USAR arena [Jacoff et al., 2000, Jacoff et al., 2001, Jacoff et al.,
2002]. For the studies performed at the University of Massachusetts Lowell (UML) we
constructed our own USAR test arena. Our arenas were constructed in large open lobbies. We
joined 4-foot square sheets of wall paneling together with door hinges to create sections that
could quickly and easily be configured into arbitrary patterns representing damaged rooms and
corridors. We covered some of the areas with tarps to create dark areas. These arenas
compared well with the concrete building that houses the standard NIST arenas.
All of the studies performed at UML had multiple arena orientations as well as starting positions.
These orientations were permuted so that a single test subject only saw each arena once and also
so all the test subjects did not run each course in the same order. This permutation helps
remove learning effects from the resulting data. The configuration of the wall sections,
obstacles and victim locations was the same for all test subjects. We marked the locations of
everything on the floor with tape and drew a map that was used to record the robot's path
during the subject runs, which made it possible to recreate each arena exactly the way it was set
up for the previous users.
3.2 Timed Run
In all the studies, except for the last one that was performed on version 3 of the interface (which
is talked about in section 5.3), the user had a set time limit to complete the task that was asked of
them. In most cases we explained that a disaster had occurred and that the subject had X
minutes to search for and locate as many victims in a damaged building as possible. The time
limit was anywhere from 15-25 minutes depending on the study. This portion of the testing
15
Comentários a estes Manuais