Measure internal tools before you redesign them

Measure internal tools before you redesign them

Measure internal tools before you redesign them

Design process

The short answer

Internal and enterprise tools rarely come with revenue or retention numbers, so good design work on them goes unseen. Before the project starts, pick two to four measures that matter to people who must use the tool, such as errors, time on a key task or time to learn it. Record a baseline, then measure again after release.

Internal and enterprise tools rarely come with revenue or retention numbers, so good design work on them goes unseen. Before the project starts, pick two to four measures that matter to people who must use the tool, such as errors, time on a key task or time to learn it. Record a baseline, then measure again after release.

Original mobile concept in charcoal and bright yellow-green. An app called Measures shows a fictional internal tool project, Order entry redesign, marked: measure before design starts. Top task: create a purchase order. Users: buyers and new hires. Measures, 3 of 4 baselined: Task success (usability test, same tasks each round; baseline). Time on task (controlled test, not analytics; baseline). Validation errors (counted in the product from day one; baseline). Time to learn (new hires, repeat sessions; pick method). Note: after release, same tasks, same method; pair the numbers with research to learn why. Buttons: Add measure and Record baseline. Beside the phone, a dithered drawing of a slab with three task tiles and a long ruler, with one yellow-green tile being set on the ruler's first mark. Text: Illustrative concept. Measure before you redesign. Baseline first, then the same tasks after release. No real data is shown.
Original mobile concept in charcoal and bright yellow-green. An app called Measures shows a fictional internal tool project, Order entry redesign, marked: measure before design starts. Top task: create a purchase order. Users: buyers and new hires. Measures, 3 of 4 baselined: Task success (usability test, same tasks each round; baseline). Time on task (controlled test, not analytics; baseline). Validation errors (counted in the product from day one; baseline). Time to learn (new hires, repeat sessions; pick method). Note: after release, same tasks, same method; pair the numbers with research to learn why. Buttons: Add measure and Record baseline. Beside the phone, a dithered drawing of a slab with three task tiles and a long ruler, with one yellow-green tile being set on the ruler's first mark. Text: Illustrative concept. Measure before you redesign. Baseline first, then the same tasks after release. No real data is shown.

Illustrative concept, not client work or measured results.

Why the usual numbers don't fit

Much metric advice starts from Google's HEART framework, which maps product goals to signals and metrics. NN/g's Page Laubheimer points out that its engagement, adoption and retention dimensions are "meaningful only for consumer products, where the user can decide whether to use the product." When the tool is part of someone's job, retention just tracks staff turnover. His CASTLE framework measures cognitive load, advanced feature usage, satisfaction, task efficiency, learnability and errors instead.

Why the usual numbers don't fit

Much metric advice starts from Google's HEART framework, which maps product goals to signals and metrics. NN/g's Page Laubheimer points out that its engagement, adoption and retention dimensions are "meaningful only for consumer products, where the user can decide whether to use the product." When the tool is part of someone's job, retention just tracks staff turnover. His CASTLE framework measures cognitive load, advanced feature usage, satisfaction, task efficiency, learnability and errors instead.

Set the baseline first

NN/g's Alita Kendrick lays out benchmarking: choose about 5 to 10 top tasks, "aim for 2–4 metrics," measure before the redesign, then repeat with the same method. Pilot the study first. She warns that "bad numbers are worse than no numbers," so pick a plan your team can keep running. Laubheimer prefers time on task from controlled usability tests over analytics, since interruptions skew it.

Start with what frustrates people

Jared Spool's tip in a 2015 talk is simple: "Count the error messages." Errors are cheap to track and point at real pain. Numbers only say whether something moved, not why, so pair them with research. If past work was never measured, say so plainly and describe what you observed. Don't invent a number.

Set the baseline first

NN/g's Alita Kendrick lays out benchmarking: choose about 5 to 10 top tasks, "aim for 2–4 metrics," measure before the redesign, then repeat with the same method. Pilot the study first. She warns that "bad numbers are worse than no numbers," so pick a plan your team can keep running. Laubheimer prefers time on task from controlled usability tests over analytics, since interruptions skew it.

Start with what frustrates people

Jared Spool's tip in a 2015 talk is simple: "Count the error messages." Errors are cheap to track and point at real pain. Numbers only say whether something moved, not why, so pair them with research. If past work was never measured, say so plainly and describe what you observed. Don't invent a number.

Illustrative concept: From top tasks to a baseline. Step 1, name the top tasks: create a purchase order (buyers); find a past order (buyers and support). Step 2, pick 2 to 4 measures: task success; time on task (controlled test); validation errors. Step 3, measure twice: baseline before the redesign starts; after release with the same tasks and the same method. Below, pair the numbers with research. Numbers show whether a measure moved. Research shows why it moved, and what to fix. Never measured? Say so, and describe what you observed.
Illustrative concept: From top tasks to a baseline. Step 1, name the top tasks: create a purchase order (buyers); find a past order (buyers and support). Step 2, pick 2 to 4 measures: task success; time on task (controlled test); validation errors. Step 3, measure twice: baseline before the redesign starts; after release with the same tasks and the same method. Below, pair the numbers with research. Numbers show whether a measure moved. Research shows why it moved, and what to fix. Never measured? Say so, and describe what you observed.
Illustrative concept: From top tasks to a baseline. Step 1, name the top tasks: create a purchase order (buyers); find a past order (buyers and support). Step 2, pick 2 to 4 measures: task success; time on task (controlled test); validation errors. Step 3, measure twice: baseline before the redesign starts; after release with the same tasks and the same method. Below, pair the numbers with research. Numbers show whether a measure moved. Research shows why it moved, and what to fix. Never measured? Say so, and describe what you observed.
Illustrative concept: From top tasks to a baseline. Step 1, name the top tasks: create a purchase order (buyers); find a past order (buyers and support). Step 2, pick 2 to 4 measures: task success; time on task (controlled test); validation errors. Step 3, measure twice: baseline before the redesign starts; after release with the same tasks and the same method. Below, pair the numbers with research. Numbers show whether a measure moved. Research shows why it moved, and what to fix. Never measured? Say so, and describe what you observed.
Illustrative concept: From top tasks to a baseline. Step 1, name the top tasks: create a purchase order (buyers); find a past order (buyers and support). Step 2, pick 2 to 4 measures: task success; time on task (controlled test); validation errors. Step 3, measure twice: baseline before the redesign starts; after release with the same tasks and the same method. Below, pair the numbers with research. Numbers show whether a measure moved. Research shows why it moved, and what to fix. Never measured? Say so, and describe what you observed.

Before your next internal-tool project

  • Name the top tasks for each group of users.
  • Pick two to four measures that fit those tasks.
  • Record a baseline with a method you can repeat.
  • Count error messages from day one.
  • Measure again after release and write down what changed.

Before your next internal-tool project

  • Name the top tasks for each group of users.
  • Pick two to four measures that fit those tasks.
  • Record a baseline with a method you can repeat.
  • Count error messages from day one.
  • Measure again after release and write down what changed.