User testing in 

In May and June, the Duke team made a trip to Nairobi to begin an initial round of user testing. Our objective was to test the content and SMS experience with a group of Kenyan women.

For this work we partnered with a fantastic maternity hospital in Nairobi called Jacaranda Health. Founder and Executive Director Nick Pearson is an affiliate of my home institution at Duke University, and we share the same vision for expanding access to care for perinatal depression. Jacaranda’s emphasis on innovation and research made them a natural partner.

The setup

We set up our user testing in a meeting space next to Jacaranda’s building on the outskirts of Nairobi. In setting up our space, we took inspiration from Heewon Choi, product design manager for Pivotal Labs, Tokyo. Testing materials included:

For each session, we seated the participant and a facilitator in front of the testing phone. A vertical camera stand held the second smartphone to record overhead video of the user’s interaction with the prototype. We positioned a MacBook Pro opposite the user and facilitator to record video of the testing environment using Quicktime player. As you can see in the photo, we dimmed the laptop screen brightness so that neither the participant nor facilitator would be distracted by the video recording.

We used the Reflector 3 software to mirror the screens of the prototype phone, the overhead video, and the laptop video via Apple AirPlay to a second MacBook Pro that recorded all three screens. A digital recorder captured audio from the testing session. This short video shows the 3-screen setup:

New mom Edith runs through a Tess session. (Video shared with permission)

One member of the Duke team sat next to the user and facilitated the session, while a second team member sat in the corner of the room, observed the session from the laptop running Reflector 3, and took notes. Our superb assistant Lilian from the Jacaranda Health team helped everything to run smoothly and contributed great insights.

In total, we conducted 23 testing sessions over 3 weeks with 10 women recruited from Jacaranda’s maternity hospital. We did not seek out women with a history of depression for this initial testing. Rather, we asked the staff at Jacaranda to recommend several pregnant women and new moms who might enjoy trying out a new service and giving detailed feedback. We ended up with a great group of women who ranged in age from 27 to 35 (M=30.3, SD=2.8). Nearly all of the women attended some post-secondary education, and all could speak and read English. Two women were pregnant, and the rest had kids under two years of age. This group does not reflect the broader demographics of this age group in Kenya, but that was not our goal at this early stage.

SMS sandbox

While X2AI worked on setting up our integration with Tess, we created a mock AI environment using a program called TextIt, a mobile messaging platform built in Rwanda. TextIt makes it easy to visually build SMS workflows like the one you see here. There is no setup cost or monthly fee, but testing in TextIt costs $0.02 USD for each incoming and outgoing message. You could build a production-ready app in TextIt, but you would pay twice for every message.

This is because TextIt requires a channel for sending and receiving messages to end users. We connected our TextIt account to a Kenya-based SMS provider called Africa’s Talking. For about $175 USD per month, we rent a dedicated 5-digit shortcode that lets us send and receive messages in Kenya.³ Every incoming and outgoing SMS costs us about $0.08 USD currently, but will drop to $0.04 at scale. Our shortcode is toll-free, so users can message us at no cost—we pick up the check.

What we learned

The biggest and most important takeaway from our user testing sessions is that women liked talking with our Tess mock up. Their enthusiasm for the idea really motivated us to learn about how to make it better.

To that end, we spent a few hundred hours revising the content and approach. Cut down on the number of consecutive outbound messages here, change the phrasing there, give up on that joke that fell flat.⁴

What comes next

We’re partnering with Dr. Edith Kwobah (psychiatrist at Moi Teaching and Referral Hospital in Eldoret), Dr. Christine Musyimi of the Africa Mental Health Foundation in Nairobi, and two medical doctors completing their training in psychiatry. This fantastic team will help to further revise the content and prepare for a pilot trial with 20 depressed women recruited from Jacaranda Health’s maternity hospital.

The team at X2 is finalizing our Tess SMS integration, and we’ll complete one more round of testing in the run up to the launch of the trial. Check back for more details about our trial design and analysis plan…and our new name! Tess is known as Karim in Lebanon, Sarah on Facebook, and soon to be ?? in Kenya.

Source link—-819cc2aaeee0—4


Please enter your comment!
Please enter your name here