We decided to keep the field, but renamed from “Title” to “Description” Provides context to each item on list view Gives each item a label to help with organisation Users need to be able to scan this item list and know what each item contains, without

Themis – Usability Testing Output

Case Review and Case Closure

Migration and Borders Technology Portfolio

Sub-title

Research Background

We tested with 2 users from CWI and 1 from Border Force National Intelligence Hub
Each user took part in one session
Testing was focused on examining how users would request a case review and close a case on Themis
All testing took place in Themis SIT environment

Themis Dashboard

A screenshot of a computer

Description automatically generated

All users were easily able to find “New cases to assess”

Some users thought it may not be necessary to have “Case in review” displayed twice

Sub-title

Dashboard - Notes

Is there a particular need for cases in Review to be displayed separately?

If “cases in review” needs a separate display, should it still be under “Cases being developed”?

Picking up a new case

All users were successful in picking up a new case

Sub-title

Case Review

Users were able to locate and click “Request review” button

All users successfully navigated to the “Review” tab

Sub-title

Request Review

Some users struggled to find back button here

All users identified the “Comments” text box as where they would type why the case needed a review and whom they’re requesting a review from

Users were easily able to find “Save button” and finish requesting a review

Sub-title

Review Request Complete

All users noticed the status change to “Case in review”

Should timestamps be recorded in 24hr clock format for consistency?

Sub-title

Overview – Awaiting Review Outcome

Users correctly identified that at this stage a case is now waiting for the reviewer to complete

Users expected activity log to reflect the case now being in review

Sub-title

Awaiting Review - Notes

Should users have to reallocate cases once they’ve requested a review?

There was an expectation for cases in review to no longer appear in a user’s “Your cases” section on the dashboard

Potential options include exploring the possibility of automatic deallocation once users have requested a case review
Should the review function allow users to send the case directly to the reviewer? – IMS users currently email cases to the reviewer

Sub-title

Migration and Borders Technology Portfolio

Recording Review Outcome

All users were able to use “Comments” text box to record review outcomes

Users easily navigated back to “Review” tab to record review outcome

Users recognised that some teams send cases back to review requester and some will do the review and close case themselves

Sub-title

Migration and Borders Technology Portfolio

Review Outcome

Users appreciated that comments and timestamps are displayed here for both reviewee and reviewer

Sub-title

Review Completed

Users noticed “case in review” status changes to “development in progress” once review outcome has been recorded

Users immediately saw status change as reflecting that the review is done

Sub-title

Case Closure

Users easily found button to “mark case as”

Users successfully navigated to the “Conclusion” tab

Some users questioned the placement of “Conclusion” tab – thought it would make more sense if it came after “Review” tab

Sub-title

Conclusion Tab - Notes

Is it strictly necessary for “Conclusion” to have its own tab?

Overview page already allows users to mark case as “Closed”

Intuitively, users expected the “Conclusion” tab to be placed after “Review”

Mark Case As

Users recognised “awaiting information” as something they would use

“Live/not live” confused users – would prefer “Open/Closed

“On hold” confusing for users – they feel it doesn’t make sense if the case has been transferred

Users were able to select “Closed” and “Continue” buttons

Sub-title

Mark Case As - Notes

There was some confusion over the “on hold” option – users felt a transferred case shouldn’t be labelled as “on hold”

Does this terminology reflect the needs of Themis users?

Mark Case as Closed Notes

Users recognised the “Closing case note” box as where they would summarise how case reached closure

Users were easily able to find “Save” button to record closing notes and mark case as closed

Sub-title

Case Closed

Users saw greyed out “Closed” status

Lack of any confirmation text on this page confused users – some assumed there had been an error

Sub-title

Case Review and Case Closure Notes

CWI user expected case to no longer be allocated to them after a review has been requested
Cases described as “Live/ not live” confused users as it’s unfamiliar terminology – “Open/Closed” would make more sense to users
There should be confirmation text displayed after a case has been marked as “Closed” – currently users thought there had been an error

Sub-title

Business Discussions

Migration and Borders Technology Portfolio

Sub-title

Where should cases appear after a user has requested a case review?

How can we guide users on correct actions after requesting a review?

Is there a need for the dashboard to display a separate “Cases in Review”?
What is the purpose of “On hold”?

Should a case which has been transferred to another team be described as “on hold”?

Should Themis support the ability to record a review on closed cases?
Is the “Conclusion” tab necessary? Overview page allows users to select “Closed” when they click “Mark case as”

If conclusion tab is needed, should it be positioned below “Review” tab?

Themis – Usability Testing Output

Actions & Outcomes

Migration and Borders Technology Portfolio

Sub-title

Research Background

We tested with 1 user from OIU, 1 from CDT and 1 from HO International Operations (Netherlands)
Each user took part in one session
Testing was focused on examining how users would record Actions & Outcomes on Themis

Migration and Borders Technology Portfolio

Themis Dashboard

All users successfully navigated to “new cases”

“Cases being developed” and “Development in progress” confused a user as they believed they meant the same thing

Sub-title

Migration and Borders Technology Portfolio

New Cases

Users easily discovered how to pick cases up

Sub-title

Migration and Borders Technology Portfolio

Actions and Outcomes Tab

Sub-title

Users were able to navigate to the “Actions &Outcomes” tab

Migration and Borders Technology Portfolio

Add Action and Outcome

All users were confused by the “Title” field here – they didn’t see it as necessary and wouldn’t know how to use it

Users were easily able to locate “Actions taken” which relate to their teams

Users identified the sequencing as reflecting the way they work

Sub-title

Migration and Borders Technology Portfolio

Add Actions and Outcomes

Users easily found the check boxes for their relevant “Outcomes”

All users recognised the “Details” box could be used to further expand on which actions were taken and add context for anyone picking up the case

HOIO user did not recognise any of these “Outcomes” as applying to their work

OIU user felt “Tasked to another team” should be an option here

Sub-title

All users were able to easily record their actions taken and outcomes

Migration and Borders Technology Portfolio

Mandatory Fields

In addition to not knowing how to use the “Title” field, users were frustrated that it is a required field – You currently cannot progress past this stage without filling it in

Sub-title

Both Actions & Outcomes are required to be selected – users identified this as a problem as some teams will only record an “action”

Migration and Borders Technology Portfolio

Add Actions & Outcomes - Notes

Users commented that this stage reflects the back-office form on IMS
An OIU user felt “Tasking to another team” should be an Outcome
Users expected recorded Actions & Outcomes to appear on the “Overview” page

Actions & Outcomes Recorded

Users were able to recognise the value of “Actions and Outcomes” being summarised here

HOIO user felt the timestamps should be recorded using the 24hr clock

Sub-title

Actions & Outcomes Recorded - Notes

Users expected the actions and outcomes recorded on a case to be displayed on the case activity log

Can this information be displayed without overcrowding the timeline?

Sub-title

Business Discussions

Migration and Borders Technology Portfolio

Sub-title

Further Discussion

We need to have a clearer understanding of the intended purpose of the “Title” feature

How does the Business envisage its use?
Once we know how it is supposed to be used, we can find an appropriate name and guide users on expected behaviour

Actions and Outcomes should not both be mandatory fields

Should Themis allow users to progress a case without both Actions & Outcomes being selected?
Should Actions & Outcomes be displayed on the timeline?
Should events on Themis be timestamped using a 24hr clock?

Would a “Tasked to X team” meet the needs of HOIO users?

Design Updates

Migration and Borders Technology Portfolio

Sub-title

What we decided

We decided to keep the field, but renamed from “Title” to “Description”

Provides context to each item on list view
Gives each item a label to help with organisation
Users need to be able to scan this item list and know what each item contains, without opening individually

Adding hint text

Gives users a clearer understanding of what information they should add

Repositioning of field to bottom of page

Users were confused by the field being positioned at the top of page
We anticipate moving it to the bottom will help users understand it’s function
Once users have selected actions & outcomes, they are then asked to give a description
This is more intuitive and reflects the need for users to describe what they have just done

Updated Designs

Future design updates will include an additional outcome labeled “Allocated to another team” Users have told us this is a need

With the description box at the end of the flow, users will be more familiar with the actions & outcomes