SIMS 213 Assignment 6

Heuristic Evaluation for MDTP Project

Introduction

We conducted a heuristic evaluation of the interface for the Medical Dictation / Transcription Program (MDTP). The program facilitates the transcription of audio dictation files recorded by physicians. For our evaluation we used Nielsen's heuristics.

Group Evaluation

Violation Statistics

Descriptions of Violations

    Severity 3

[H1 Simple and Natural Dialog] (Severity 3) (Identified by 1 tester)
The tabs "All Jobs" and "Macros" do not look like hyperlinks/clickable tabs. The darker color appears to be "grayed out" and inactive. Only one out of three testers found the functionality of these tabs, and this one instance was by accident, but the true nature of the problem is that two out of three testers did not discover the use of these tabs.

[H3 Minimize Users' Memory Load/H6 Clearly Marked Exits] (Severity 3) (Identified by 3 testers)
When you click the Save button, you have the option of submitting the transcript, saving it, or cancelling it. We discovered the cancel button only after saving. The buttons for this functionality should be included in the main window for the transcript record.

    Severity 2

[H1 Simple & Natural Dialog] (Severity 2) (Identified by 2 testers)
Scenario One requires the user to identify the highest priority task from the Job List. The Priority markings are very clear by themselves (high is in red and boldface). But when given several items all of the same priority level, I was not able to tell which item was highest priority. I started work on the first item on the list, assuming that the system may sort the highest priority item for me, or else a real transcriptionist would be able to identify the priority as a combination of metadata" "high" priority ranking + date or length or status. One possible solution would be to group jobs requiring action by the transcriptionist separately from jobs requiring action from someone else.

[H2 Speak the Users' Language] (Severity 2) (Identified by 2 testers)
Unclear difference between "QuickSave" and "Save."

[H4 Consistency and Standards] (Severity 2) (Identified by 1 tester)
Sorting the job list by priority is not consistent. The default setting by high priority seems to re-sort unpredictably when re-sorted in ascending order. That is, the top of the priority list in the descending list does not move to the bottom of the ascending list.

[H4 Consistency and Standards] (Severity 2) (Identified by 1 tester)
Within the job list window, order by length is displayed in inconsistent time formats, which leads to an incorrect sorting of length.

[H4 Consistency and Standards] (Severity 2) (Identified by 1 tester)
Cannot open the jobs from the "View All Fields" window.

[H5 Feedback] (Severity 2) (Identified by 2 testers)
How can the transcriptionist tell what edits the doctor added to the document? Doctor cannot change status of the document to show that he has approved it.

[H5 Feedback] (Severity 2) (Identified by 2 testers)
The Job List frame does not update to show which job is currently open in the typing window. Even though the current job number does exist as a data field in the main window of an open job, it is not easily discerned at first glance, a graphical view, for example, highlighting the job in the job list would be more visible.

[H5 Feedback] (Severity 2) (Identified by 2 testers)
When you change the status of a transcript record, there is not a corresponding change in the job list. For example, if you choose Submit the Transcript for a record, the status seems to refresh automatically, but not immediately unless the user refreshes the page. When the Job List updates, it is distracting.

[H5 Feedback] (Severity 2) (Identified by 1 tester)
The audio control is usefully built into this interface. While this feature is not deployed at this time, we consider the feedback that matches blank spot in the typed transcript to a precise section of the audio track. The blank spot in the transcript is easy to find. The precise place in the audio transcript is likely harder to find. It's hard to imagine a busy physician playing back the whole transcript to fix a certain phrase. Movies use a "time code" to synchronize audio and visuals, and there may be some similar way to match typing and audio. Is there something similar in use in medical transcription? Even better, there might be a way to speed up the feedback between blank and the sound of the physician's voice, such as click on the blank to play the exact spot in the transcript, or "Play Blank Spots" button on the audio controls.

[H6 Clearly Marked Exits] (Severity 2) (Identified by 1 tester)
There should be a means of deleting or editing a shortcut.

[H7 Shortcuts] (Severity 2) (Identified by 1 tester)
Adding keycodes is fast and effective. However, the alphabetical sorting of these may not be the most personalized way of sorting. For example, I was able to quickly add a lot of keycodes. The small problem that follows is managing the volumes of keycodes that a transcriptionist would build in regular use. A sorting feature or an index to keycodes could be improve this feature.

[H7 Shortcuts] (Severity 2) (Identified by 1 tester)
Using the keycode shortcuts requires clicking the link for the designated keycode. Expected to use this functionality by typing the keycode when I wanted the actual word.

[H9 Prevent Errors] (Severity 2) (Identified by 1 tester)
The metadata for a given record, for example, Patient Name, Date of Visit, sometimes is not complete. When the transcriptionist has to enter this information, the text fields should be arranged to enforce consistent entry of the information. For example, the date input could be enforced as mm/dd/yyyy.

[H9 Prevent Errors] (Severity 2) (Identified by 2 testers)
We noticed that Cathy was able to open other transcriptionists jobs under "All Jobs." This could be very disruptive for other transcriptionists, but we imagine this is almost certainly to be implemented soon.

    Severity 1

[H3 Minimize Users' Memory Load/User Control & Freedom] (Severity 1) (Identified by 2 testers)
I like the large font button, especially that for some technical reason the transcription window seems immune to the browser font sizing built into MSIE 5. Clearly, it's better to have a font size button built into the interface for a typing window, rather than depending on the setting hidden in the browser. However, there are two small things that may need improvement here: 1) there are only two font sizes, and more may be necessary for those of us with poor eyesight. 2) The larger size font requires making the window larger, which may not be an option for a transcriptionist with a small display.

[H4 Consistency] (Severity 1) (Identified by 2 testers)
Saving from the doctor's screen has the same options as the transcriptionist, but should this be different?

Recommendations for MDTP

First of all, we are very pleased with the intended purpose of these system. We are definitely trying not to need medical care until your system is implemented at our health care providers! Second, we are impressed by the degree of detail in this prototype. The hard work will definitely pay off soon when you show this project to the SIMS community and your client. But "the devil is in the details," thus our comments are meant to indicate detail-level improvements that would help your system succeed with your client and users.

1. The largest problem we identified is the interaction flow for saving, printing, closing, submitting documents. Reworking this flow will help simplify this very frequent task. Pay special attention to simplifying the number of steps and using the user's language.

2. While some amount of complexity is necessary in this interface, there are large areas of the interface we did not encounter during our testing scenarios. For example, the All Jobs view of the job list, and the Macros view of the Shortcuts. Is this complexity really necessary? Two out of three reviewers did not discover these portions of the interface.

3. The All Jobs and Macros parts of the interface were difficult to discover due to the gray navigation tabs. We suggest using a different color for the tabs. Navigation would help invite users toward the other parts of the interface: for example, navigation tabs could be more graphical and resemble tabs, or simple hyperlinks could be consistent with others in this interface.

4. Enforce consistency in the sorting orders of the Job List. The main problem here was the inconsistent sort by Priority, but closer attention to sorting in general will fix a lot of little bugaboos. We are very curious as to what goes into a Priority ranking. Also related are the time formats in sorting by Length, and the sort by Status.

Tom's Evaluation

1. [H1 Simple & Natural Dialog] (Severity 2)
Scenario One requires the user to identify the highest priority task from the Job List. The Priority markings are very clear by themselves (high is in red and boldface). But when given several items all of the same priority level, I was not able to tell which item was highest priority. I started work on the first item on the list, assuming that the system may sort the highest priority item for me, or else a real transcriptionist would be able to identify the priority as a combination of metadata" "high" priority ranking + date or length or status.

2. [H2 Speak the Users' Language] (Severity 2)
Unclear difference between "QuickSave" and "Save."

3. [H3 User Control & Freedom] (Severity 1)
I like the large font button, especially that for some technical reason the transcription window seems immune to the browser font sizing built into MSIE 5. Clearly, it's better to have a font size button built into the interface for a typing window, rather than depending on the setting hidden in the browser. However, there are two small things that may need improvement here: 1) there are only two font sizes, and more may be necessary for those of us with poor eyesight. 2) The larger size font requires making the window larger, which may not be an option for a transcriptionist with a small display.

4. [H4 Consistency and Standards] (Severity 2)
Sorting the job list by priority is not consistent. The default setting by high priority seems to re-sort unpredictably when re-sorted in ascending order. That is, the top of the priority list in the descending list does not move to the bottom of the ascending list.

5. [H5 Feedback] (Severity 2)
The audio control is usefully built into the typing interface. While this feature is not deployed at this time, I'd like to direct comments toward the feedback that helps match a precise section of the audio track with its place in the typed transcript. Movies use a "time code" to synchronize audio and visuals, and there may be some similar way to match audio and typing. Is there something similar in use in medical transcription? It's hard to imagine a busy physician playing back the whole transcript to fix a certain phrase. Even better, there might be a way to speed up the feedback between blank and the sound of the physician's voice, such as click on the blank to play the exact spot in the transcript, or "Play Blank Spots" button on the audio controls.

6. [H6 Clearly Marked Exits] (Severity 3)
I discovered the cancel button only after saving. Was looking for this in the main window.

7. [H6 Clearly Marked Exits/H2 Speak the Users' Language] (Severity 3)
There is no way to "close" a transcript record once finished working with it. The metaphor for close should map to putting the record away once finished with it.

8. [H7 Shortcuts] (Severity 2)
The main shortcuts in MDTP seem to be personalized keycodes. Personalizing a list of keycodes is fast and effective, and provided the functionality works as intended, this should be very useful. One small quibble about the keycode list, however. Playing around with the system for five minutes, I was able to quickly add a lot of keycodes. This feature is very powerful and works great. The small problem that follows is managing the volumes of keycodes that a transcriptionist would build in regular use. A sorting feature or an index to keycodes could be improve this feature.

Kaichi's Evaluation

1. [H2 Speak the Users' Language] (Severity 2)
What is the difference between Save & Quick Save? I was confused by the two types of saves and did not find any explanation for the differences between the two.

2. [H2 Speak the Users' Languag] (Severity 1)
It might be nice for the user to control the font size and font type. This type of control might possibly fit in more with a user's previous mental model of word processing type software.

3. [H5 Feedback] (Severity 1)
Audio control did not load, see unloaded image icon. Are plugins needed?

4. [H4 Consistency] (Severity 2)
When saving jobs logged in as a doctor, the save popup window still states "save for doctor's review".

5. [H5 Feedback] (Severity 3)
How can a transcriptionist easily tell what edits were entered by the doctor? Currently I find it difficult to really understand how the different types of users can track what changes have been made to a job by others.

6. [H9 Prevent Errors] (Severity 1)
If being able to save quickly (through a QuickSave button) is important to transcriptionists, might it not be userful and maybe more effective to have an auto-save functionality?

7. [H4 Consistency] (Severity 1)
Ordering jobs in Job List Windowby Length: different time formats give incorrect ordering (5:25 vs. 05:25). Need to standardize user input/format.

8. [H9 Prevent Errors] (Severity 3)
When viewing All Jobs, a user can access, modify, and save changes to another user's job. Is this supposed to be allowed? It seems like some access control should be implemented/designed to prevent possible errors and confusion.

9. [H5 Feedback] (Severity 2)
When a job is open, the only way to identify which job is open is by looking at the job ID field in the main window. A graphical way to easily see which job is currently selected would provide better visibility to the user, maybe through gray shading/highlighting of the current job in the Job List window.

10. [H10 Help/Documentation] (Severity 2)
Need help for entering shortcuts/macros.

11. [H1 Simple and Natural Dialog] (Severity 3)
Was not aware at first of being able to switch between the different tabs of Cathy's Jobs/All Jobs and Shortcuts/Macros. I didn't identify the darker tabs as clickable links, in fact, I found them only by accidently clicking on one.

John's Evaluation

1. [H3 Minimize Users' Memory Load] (Severity 3)
When you click the Save button you have the option of submitting the transcript report, saving it, or cancelling it. The buttons for this functionality should be included in the window for the transcript report.

2. [H1 Simple and Natural Dialogue] (Severity 2)
Jobs with different status ratings ("assigned", "submitted", "signed") can be given a "high" priority. Not clear how users decide between jobs with the same priority but diferrent status ratings.

3. [H5 Feedback] (Severity 2)
Once a transcriptionist submits a report, how can she tell that a doctor has reviewed the report and made changes?

4. [H5 Feedback] (Severity 2)
Once a job is selected, the job list does not update to show that the job has been openned.

5. [H5 Feedback] (Severity 2)
When you change the status of a transcript report there isn't an immediate corresponding change in the job list.

6. [H6 Clearly Marked Exits] (Severity 2)
There should be a means of deleting and/or editing a shortcut.

7. [H7 Shortcuts] (Severity 2)
Using the keycode shortcuts requires clicking the link for the designated keycode. Expected to use this functionality by typing the keycode when I wanted the actual word.

8. [H9 Prevent Errors] (Severity 2)
The metadata for a given report, for example, Patient Name, Date of Visit, sometimes is not complete. When the transcriptionist has to enter this information, the text fields should be arranged to enforce consistent entry of the information. For example, the date input could be enforced as mm/dd/yyyy.

9. [H4 Consistency] (Severity 1)
When a doctor makes changes to a report, should there be a new status that he selects indicating he has reviewed the report and made changes? Would the doctor change the status to signed?