SIMS 213
Spring 2002

In developing the second interactive prototype, we addressed most of the violations identified in the the heuristic evaluation, replaced most of the "Wizard of Oz" techniques from the first interactive prototype, and polished the interface by increasing consistency among the web pages.

UI Redesign

The following table lists the major (severity 3 or 4) identified heuristic violations and our responses:

Violation	Description	Response
H5 [Severity 4]	Submit page cannot be displayed during submit process	New gateway page avoids problem
H4 [Severity 4]	Clear all button doesn't work	Could not duplicate bug
H4 [Severity 4]	Underlining should not be used to denote hypertext because briefs contain underlining	Use local stylesheets for header or hardcode header to avoid underlining in brief itself
H1 [Severity 3]	Search doesn't clearly indicate which facets must be used for searching	Provide help/documentation
H1 [Severity 3]	Search doesn't indicate that "All Courts" is the default for the Court facet	Implement "All Courts" option
H2 [Severity 3]	Cannot search using multiple courts	Research with legal researchers indicates they already know what juristiction, and limit to one
H9 [Severity 3]	Search doesn't alert users when they search without selecting facets	Search results reports no results
H6 [Severity 3]	Inconsistent colors for links legal briefs in search results	Could not duplicate error; probably a browser settings issue
H1 [Severity 3]	User must accept terms & conditions again whenever revising the submit info	Implemented gateway page to avoid problem
H5 [Severity 3]	User can enter nonsensical info into the submit page	submit is not automated, goes to human reviewer
H4 [Severity 3]	The submit page does not allow users to submit a URL instead of attaching a file	A file attachment is required (to be specified in BB Terms & Conditions)
H1 [Severity 3]	Use of "submit a brief" on the nav bar is confusing when completing the submit page	Reworded nav bar "submit a brief" to "contribute a brief", also provided overview of submission process on gateway page
H5 [Severity 3]	Unclear which fields in the submit page are required	Indicated required fields on submit page
H9 [Severity 3]	When user doesn't enter all required info into the submit page, error message only reports first missing element	Dynamic error alerting would require extensive javascripting implementation, won't do it for now.
H10 [Severity 3]	Insufficient help and documentation	Provide more help/documentation
H2 [Severity 3]	Need pdf versions of all briefs	We currently cannot enforce standardized formats on contributors

Following is a link to an Excel file listing all of the violations and our responses:

Heuristic Violations [Excel, 24kb]

In addressing the heuristic violations, we significantly redesigned the navigation bar to clearly separate links from function buttons. In particular, we changed the changed the name of a navigation link from "Submit a Brief" to "Contribute a brief". This change is intended to prevent users from mistakenly clicking the navigation link instead of the the function button for submitting a brief.

In addition to addressing violations, we replaced most of the "Wizard of Oz" techniques from the first interactive prototype. Significantly, we implemented the search function. In addition, we implemented the inclusion of additional metadata when listing briefs as search results or listing briefs in conjunction with the browse function.

Lastly, we polished the interface by increasing consistency among web pages.

Preview of Usability Test

We look forward to testing legal professionals on a version of BriefBank that offers much more interactivity than previous iterations. Since our system is designed to distribute and collect briefs, we will focus our testing on those two activities.

Search is nearly completely implemented and our submit process, in a user's eyes, will appear to function correctly. Therefore, we feel confident in proceeding with fairly complex user tasks. The following tasks are preliminary versions of the tasks we plan to use for our pilot usability test.

Task: Search our collection (Part I)

"Your boss at your law school's legal clinic wants you to write a brief on a DVD encryption case similar to US v. Elcomsoft. She remembers that she found a particularly good brief at BriefBank several weeks ago, but she doesn't remember its name. Your professor wants you to find that brief, an amicus brief arguing (X issue), and study the arguments."

Task: Search our collection (Part II)

"You've been assigned to research Universal v. Reimerdes for your supervisor. Find all the amici briefs for the plaintiff that relate to the Universal v. Reimerdes case."

Task: Submit a brief

"You've written a brief that recently has received cert from the Supreme Court. Your colleagues have been driving you crazy asking for copies. You decide to send your brief to BriefBank. That way, you can just refer other lawyers to that site when they want a look at your research."

Measures to be used in Evaluation

We will evaluate the system with the following broad measures: the user's observed behavior/comments, task performance, and survey results.

Observation

Subject headings: Do users comment on/seem frustrated by the lack of subject headings?
Page flow: Can users navigate the system efficiently? Do they use the back button excessively? Can they make reasonable guesses about the consequences of their actions (i.e., pushing a button)?
Labels: Do the system's labels and vocabulary match the user's conceptual model?
Relevance of retrieval: Does the metadata retrieved from search/browse match the user's expectation of such? Does the metadata help the user make relevance judgments?

Survey

Match of system user's research context: Does the system compliment Lexis and/or other legal research tools?

Satisfaction

Relevance of retrieval: Does the metadata retrieved from search/browse match the user's expectation of such? Does the metadata help the user make relevance judgments?
Trust/ownership/privacy/terms and conditions: After experiencing the system, does the user understand the conditions of use and trust BriefBank to keep them?
Performance: Do connection time and display time meet the user's expectations?

Task Performance

Relevance of retrieval: Does the metadata retrieved from search/browse match the user's expectation of such? Does the metadata help the user make relevance judgments?
Task completion: Can our user complete the tasks in a reasonable amount of time?

SIMS 213 Spring 2002

SIMS 213 Assignment 7

Second Interactive Prototype

Introduction