When we hear terms like “ experience design” and “interaction design”, we tend to think of products with nice and sleek interfaces, be they digital or physical. It seems like UX advocates (myself included) are always looking for the next big technology they can get their hands on to help bridge human and computer interaction. We tend to forget about what’s under the roof and what’s making all of this cool technology possible.

No one really thinks about our old friend, the (CLI).

Command line interface (CLI) is a text-based interface that is used to operate software and operating systems while allowing the user to respond to visual prompts by typing single commands into the interface and receiving a reply in the same way. — Techopedia

The CLI world is a mysterious and intimidating place. It’s like the Wild West in terms of its UX standards. This is because CLI follows its own set of standards that are unknown to designers and aren’t entirely consistent with modern UX standards. Here are some examples:

  1. Your syntax has to be on fleek. If you don’t write your command in 100% perfect syntax, you are guaranteed to get errors and will get pretty frustrated.
  2. When an action is taken in the CLI, sometimes you get feedback and sometimes you don’t. Some CLI systems like to give feedback when an action was taken and everything ran successfully. Other CLI systems don’t like to give feedback on successful operations.
  3. You can ask for help within the CLI but it may not be too helpful. It’s up to the CLI creators to make sure that the –help info they curate is relevant and informative to their audience.
  4. Error messages can be very cryptic. You could spend hours trying to figure out one error message. Error messages can be pretty cryptic to novice CLI users. And many users spend too much time googling or looking through help documentation to troubleshoot their error messages.

These are just a few issues with the CLI. The CLI is a hotbed for UX improvements. You don’t have to be an expert CLI designer or developer to help change the CLI user experience. Here’s a step-by-step for folks who want to user test the CLI like the of .

Before the Test

Research Goals

Write down what you want to get out of user the CLI.

  • What do you want to understand better?
  • What part of the experience with the CLI has room for improvement?
  • What do you want to validate?


Try to recruit between 5 to 7 users for the test. Find a mix of users that are novice and expert CLI users as well as a mix of users with technical expertise in your product. This diversity in range will provide a rich set of feedback. At Mesosphere, we’ve built a process for recruiting, scheduling and running usability test that automates a lot of the brunt work.

Task Script

Split whatever you are testing within the CLI into small tasks. This will help ensure you get through the entire test. Most importantly, this will give your test participants good direction and ensure they focus on the task at hand.

Simulate the Command Line Interface

You don’t have to code the CLI to test it. You can use online code editor tools like Codeshare, Codepen and Collabedit to simulate the CLI environment.

Create three pages within Codeshare that you’ll be using during the user test.

Help documentation. Put the –help documentation into a separate page so that test participants can reference it throughout the entire test. If you don’t already have help documentation within your CLI that you can use or that is relevant to your test, ask a developer on your team to help you create mock help documentation.

Documentation of the input and output for each task. Create a page where you, as the Wizard (tester), can reference what input you are expecting the test participant to enter and the resulting output associated to it. You’ll be using this page to copy/paste output into the user’s “terminal” (see below). Make sure to include some generic error messages for outputs in case the user’s output are false.

The user’s “terminal”. Create blank pages for each of the test participants to enter their command input and where you, as the Wizard, will enter the associated output.

During the Test

Set Expectations

Tell the user that they’ll be testing the CLI experience and that they’ll be using a simulated CLI. By telling the user that it’s a simulated CLI environment they have an understanding that all the capabilities of a CLI may not be available to them. So if they try a special command and it doesn’t work, they shouldn’t be discouraged. Also, it’s always good practice to tell the user that there is no such thing as a wrong answer and anything they do or say during the test is good feedback.

Think aloud

Tell the user to think out loud as much as possible as this will help you as the tester to better understand their pain points and frustrations. One way to initiate and encourage the user thinking out loud is to ask them to read each task out loud before beginning.

Play Wizard

Now to the fun part. The user will type in a command into their simulated “terminal” and press enter. Depending on their command, you will paste a corresponding output.

Watch out for the following:

  • Remember to have the user reference the help documentation if they’re fumbling on something.
  • When there are errors in syntax you should output a corresponding error message.
  • Decide which commands you’re going to provide outputs for. For example: some CLIs don’t provide feedback when an action on the system was successful.
  • If the user types in a command that you don’t have a response for, ask them: “What do you expect the response to be?”

Benefits to this Approach

  • You can test the CLI without investing too much time and resources in actually creating the CLI.
  • You can get quick feedback on the CLI in earlier phases of design and implementation.

Pitfalls to this Approach

  • The prep work involved before the test in creating the simulated CLI experience can take about 10 to 15 hours of your time. Delegate the work if you can to developers, product managers or other designers on your team.
  • Being a Wizard, a test lead and an effective observer/notetaker all at the same time during the test is hard. Record the session so you can come back to it and take notes. If you can, have another person on your team observe and take notes.

Things We Learned

  • In an ideal scenario, you’d want to have the Wizard in another room away from the test participant. This allows for little distraction with the simulated system from the test participant’s point of view. For example: the test participant may be wondering why you are franticly typing away on your computer all the time and get distracted from focusing on the test.
  • There can be a lot of moving pieces during the test. If you can, allot some time before the test to practice with a coworker as the test participant.
  • Consider having a CLI developer be the Wizard. This will help them gain insight into how users interact with their product. This could also make for a more efficient test as the developer is more familiar with CLI commands.

We’re Hiring

If you’re a Designer or a Developer looking for a new adventure, get in touch, we’re hiring in San Francisco and remote. You can also read more about our design stack, design workflow and how we conduct regular user testing.

Source link https://uxplanet.org/the-wizard-of-oz-guide-to-user-testing-the-command-line-interface-3847b4418d49?source=rss—-819cc2aaeee0—4


Please enter your comment!
Please enter your name here