Revisiting Old Ideas: A Case Study

by | Jan 28, 2021 | Last updated Feb 15, 2022

By Steven Keem, Product Manager

At Noom, we love the ideas that move our success metrics, but the truly successful ideas are the ones that teach us something.  

Our water tracking feature was one of our most recent wins. Of course, this was worth celebrating because we delivered an improvement many users had been clamoring for, for years. However, it wasn’t a new idea, so we naturally questioned, “Why now?

After an in-depth retrospective analysis, we gathered a few insights we believe will be valuable for continued success: refresh and reapply and build something easy to do something hard.

Refresh and Reapply

The idea of fast learning is all the rage these days. However, one thing that’s often overlooked is how long what we’ve learned remains relevant.

As an institution, science is filled with instances where prior knowledge and conclusions are overturned (hey there, Pluto). Changes can be uncomfortable (I’m still not used to saying there are eight planets in our solar system), but keeping knowledge fresh can lead to promising futures. Like the scientists, product managers should also find their Plutos.

The water tracker was an idea that sat comfortably in our backlog for several years. There were two reasons: we doubted many users would use it, and we questioned its value to achieve our vision. 

Like many other product teams, we have a long backlog of ideas, and each one has to be ruthlessly prioritized. We estimated that the feature’s usage would be low because our health coaches set goals with users to drink more water only 13% of the time (read more about our health coaches here). Compare that stat to the 50% of goals to add one healthy food item to each meal every day. 

Additionally, our product vision didn’t suggest a strong need for the feature. We believe that motivational and empathetic elements carry more weight in positive behavior change than purely functional capabilities. Not only that, daily recommendations of water intake aren’t strongly supported by research, so we were even more skeptical about the benefits of the feature.

Our perspective on the feature shifted at the beginning of the pandemic. Our user research team found that our users turned to Noom as a healthy distraction from COVID-19. People shared that we brought a sense of normalcy at a time when “normal life” was disrupted. One user revealed that the app’s daily activities helped her “feel like [she] accomplished something and brought some control” to her life. 

That feeling of accomplishment struck a chord with me. People craved control over their health, especially now, because of the pandemic’s upheaval of routine and balance.

We applied this revelation when the request to build a water tracker bubbled up again in our research. Though we were unsure how many people would use it, we were convinced it packed the empathetic elements needed to drive positive behavior change. Not only did program retention significantly increase, but we also were pleasantly surprised to find that 63% of users used the feature and 30% kept using it weeks later, a stat that demonstrated appeal almost on par with our essential features.

We love that these fresh findings led to an unexpected win. We can’t wait to find more Plutos.

Build Something Easy To Do Something Hard

I love training for races, but I have days where my motivation to run is low. I dread the thought of pushing myself for intense workouts, but I’ve learned a trick that gets me through this hurdle over the years. I simply tell myself, “just go for a light stroll.” Committing to a light run gets me out the door and, in about 10 minutes, I warm up to the idea of running my originally planned workout. At this point, my body feels energized, and ramping up to the intensity I need to be for the intense run doesn’t seem so daunting. It’s so much easier to motivate myself to do a tough run when I’m already outside doing an easy one. 

The results from the water tracker experiment brought users back to the app about 10% more often than the control group. It was possible that an additional tracker could have cannibalized the usage of essential and typically more strenuous app activities. Instead, we noticed higher engagement in them, such as meal logging. 

We believe that the easy and delightful water tracker helped users get over the hurdle of opening the app to use psychologically taxing features. Once a user finishes logging water, the next and more challenging task to start logging meals is a tap away. In other words, logging water is that light stroll that gets them to do the activities that are often not as fun.

Not everything can be that light stroll starter activity, though. They are ones that are grounded in three fundamental habit-forming strategies: 

  1. Make it easy
  2. Do it often
  3. Keep it fun

For example, another feature that potentially could engender the same effect is our pedometer. Tracking steps is automatic (it’s easy), and you open the app often to check progress (it’s done often) toward achieving your step goal (it’s fun). 

In contrast, our weight tracker is unlikely to be one. Logging weight itself is easy, but it’s not fun to step on the scale and risk confronting potential disappointment. For some people, how they feel for the rest of the day (or longer) hinges on this one activity. 

Over time, I expect that we’ll find more Plutos and develop more light strolls. This experience was a valuable reminder that continued learning is at the root of success, even from the ideas that don’t drive outcomes. Metric-moving ideas are hard to find, but insights will always be hidden somewhere.