International Usability Study
Building Global Law Enforcement Collaboration
Project Brief
I led an international research effort, aimed at understanding the online needs and communication preferences of international law enforcement. As President and Lead Consultant at Ideas2Image, I was engaged in all aspects of this project to include business development, proposal writing, study design, results analysis, system architecture, and structural design. I worked with DoS stakeholders and in-country representatives to identify participants and conduct interviews. This successful effort helped the Department of State (DoS) to create on online Alumni Portal for law enforcement personnel who had attended a course as one of the International Law Enforcement Academies (ILEAs) located around the world.
Designing the Study
Working with stakeholders at the Department of State (DoS), I identified study goals and a general approach for gathering the required information. Through a series of stakeholder interviews, I defined baseline audience segments that could be used to identify initial participants. With these parameters as a guide, I crafted an interview protocol that was designed to categorize responses into two facets of of response.
Defining the Concept
Each ILEA maintained a web site of some sort. Some of these sites consisted of fully developed platforms, while others were little more than a homepage with basic informational content. The DoS stakeholders wanted to create a consistent representation of the ILEA program and integrate the various websites with the agency’s online platform, the ILEA Global Network (IGN). However, it had not yet been determined how this objective would be met.
A key project goal was to create a unified network of websites that employed the IGN as the hub.
Setting the Baseline: Existing Website Heuristic Audit
Historically, each ILEA had maintained a web presence of some kind. These websites ranged from rudimentary one-pagers to sophisticated platforms that incorporated a Content Management System. A key initial task was understanding what material was being produced and how the current sites were organized. I led a structured review of the existing websites to evaluate the strengths and weaknesses of each platform. Using a heuristic evaluation model, I evaluated and scored various aspects of each website to provide a common basis for reporting.
Identifying Participants
My team and I created an online survey, using SurveyMonkey, to identify potential participants for the study. A distinct instance of the survey, with a unique URL, was used to gather responses from alumni at each ILEA location selected for the study. The screener questions were constructed to segment respondents to determine who might most appropriately be considered for participation in the user interviews. The basis for segmentation were: 1) Recency of training; 2) Frequency of online use in a professional capacity; and 3) Access to the technological equipment to successfully participate in a virtual interview.
Filtering Responses
To evaluate questionnaire responses, I created a weighted scoring system that objectively determined which respondents were the most relevant to the study. In creating this score, each closed, polar (i.e., yes/no) question in the screener was assigned a numerical value between 1 and 4, which represented the question’s importance in segmentation definition. Those who’s responses were between .6 (60%) and .74 (74%) were retained as alternate/backup participants.
Conducting the Interviews
In a one-on-one interview with potential users, we examined feelings, opinions, and barriers to participation around the ILEA websites. We conducted a total of 22 interviews (88% of the project goal) across Roswell, Budapest, and San Salvador. The interviews sought to develop an understanding of how alumni users engage or might want to engage with the ILEA website and its communication components. Specifically, we explored users’ awareness of website features, opinions of the system, barriers to participation, and ways the participants are currently communicating with their colleagues.
Understanding the Results
In a one-on-one interview with potential users, we examined feelings, opinions, and barriers to participation around the ILEA websites. We conducted a total of 22 interviews (88% of the project goal) across Roswell, Budapest, and San Salvador. The interviews sought to develop an understanding of how alumni users engage or might want to engage with the ILEA website and its communication components. Specifically, we explored users’ awareness of website features, opinions of the system, barriers to participation, and ways the participants are currently communicating with their colleagues.
Structural Design
My team and I created an online survey, using SurveyMonkey, to identify potential participants for the study. A distinct instance of the survey, with a unique URL, was used to gather responses from alumni at each ILEA location selected for the study. The screener questions were constructed to segment respondents to determine who might most appropriately be considered for participation in the user interviews. The basis for segmentation were: 1) Recency of training; 2) Frequency of online use in a professional capacity; and 3) Access to the technological equipment to successfully participate in a virtual interview.
High-Fidelity Annotated Wireframes
To demonstrate the the proposed functionality of each page within the portal, I created a series of wireframe diagrams. The client requested that these be developed as high-fidelity renderings, with the intent that the wireframes would be used as the basis for further usability performance testing.
