Giving users a seat at the table: how we built our Android app

A few weeks ago, we launched our native Android app, which had been in the works for months.We knew after our iOS launch earlier this fall, that this release was highly anticipated by our Android users and we really wanted to get it right. But no one on the team was an Android power user (at first), so we didn’t trust our product instincts the way we may have designing for web or iOS. We had a vague idea of the features required to deliver a great app experience, but the details were fuzzy. So we decided to do things differently: we relied heavily on our users throughout the development process, optimizing it around learning — collecting and incorporating feedback from a special group of beta testers early and often. What resulted has shaped the way we approach product development at Asana — not just for Android. We’re just getting started.

Smaller MVP with room to grow

When we initially built the product roadmap, we knew we had a lot to learn. Sure, some information was available to us from the beginning — user research, competitive analysis, lessons we learned on other platforms, etc. But we immediately acknowledged that lots of new information would become available to us later. For example, our team was small, but growing fast, and we expected to have a couple of Android experts weighing in on key decisions soon. We also wanted to factor in any advancements in Android technology or design that Google was planning to announce later in the quarter. Most importantly though, we wanted to incorporate feedback from beta testers on the app as it developed.

So we negotiated down to a tiny set of features we knew we’d need in order to test the app ourselves. For us, this meant focusing on Inbox. By only committing to building Inbox, we avoided the costs inherent in gaining consensus on longer term roadmap decisions. We chose to make those calls closer to the time of actually executing on them, when we’d presumably have more information available to us. We left plenty of room in our schedule to expand scope beyond Inbox for that reason.

Design with the intention to iterate

Knowing that our plans were subject to change based on the arrival of new information, we approached design with a light heart. Without the pressure to “get it right” straight out of the gate, we were free to experiment with fresh ideas and take chances that otherwise may have felt too risky. We considered our initial designs to be learning opportunities more than anything else.

Our approach not only opened us up to design experimentation, but also allowed us to move faster than usual. We didn’t bother with redlines or other documentation ‘work about work’ because we expected most things to change. Instead, our designer put together rough mocks and tweaked them in code alongside an engineer to save time.

Since we were starting with just Inbox, we were able to narrow our focus on one feature rather than designing an entire app end to end, which would have required a longer lead time ahead of implementation. Of course, we knew we’d eventually add more features, so extensibility was always top of mind.

Extend the roadmap incrementally

Focusing on one feature first also allowed us to gather meaningful feedback from users early. We started with Inbox because we believed testing that feature internally would answer a lot of open questions we had about Android-specific use cases and allow us to make more informed roadmap decisions. Once Inbox was fully functional and decently polished, we distributed it internally.

Because Inbox only supported a single use case, the feedback we received was very focused and provided a clear signal for next steps. We heard things like, “Getting notifications about task updates is great, but I need to see the rest of the task for more context,” and, “Once I get a notification, I’d like to be able to create a task for myself to follow up.”

While it felt great to have more information than when we started, we knew we still had a lot to learn from real users. So we used the internal feedback to extend our roadmap a bit further out, but decided to hold off on committing to anything longer term. We would only build the features necessary to get us to our next goal, which was an app we could distribute to a small group of external beta testers in order to collect more feedback.

We added a couple of features on top of Inbox before distributing the app more broadly, but we didn’t build them all at once. Instead, we added each new feature one at a time and then collected feedback before moving on to the next feature. This prevented us from building more than what was absolutely necessary to reach our next goal of external distribution. It also allowed us to ship as soon as we recognized that our feature set was sufficient — we were never stuck with a large set of partially built features that all needed to be polished before shipping. Instead, we had a small set of fully functional and polished features ready to go as soon as we recognized our next MVP.

Fast feedback loops with real users

Once we had the right set of features required to collect meaningful feedback from people outside our company, we kicked off phase one of our beta program. First, we invited a handful of friendly customers into our office to meet the team, eat dinner, and get access to the app. This was not only a great opportunity to build the foundation for longer term relationships we hoped to have with our Android community, but it also gave our team (not just user research, but designers, engineers, and PMs) a chance to see first hand how our work was received. It was an incredibly humbling experience to watch users stumble through certain interactions. We realized that many of the assumptions we were making up front were not always true and learned a lot about ways we could improve.

We kept in touch with these users over the next few weeks and incorporated their feedback into our roadmap. Our user research team used Get Feedback to send out surveys gauging reactions to design iterations and new features every week. We also installed Instabug to collect one-off bug reports and feature requests. The app was transforming rapidly for these users and they were excited to be a part of the process. Working in sync with them was rewarding for us, too, because we got to see the impact we were making in real time, rather than building in the dark.

Once the app was in good enough shape to collect feedback from a broader group of external testers, we kicked off the second phase of our beta program. This was similar to the first phase, but included a larger group of about 50 testers. We hosted another dinner, observed their first experiences with the app, and then continued to collect and incorporate their feedback over the next several weeks. We eventually expanded the group of testers a third and final time to maximize learning before launch.

Knowing when to launch

To help us determine the best time to advance our app from internal testing to a small group of external testers to even larger groups and then eventually to launch, we used a sentiment score that tracked our progress.

Each week we asked our users to rate their experience on a scale from “tough going” to “totally awesome.” Initially, we saw huge gains in the sentiment score after each update to the app. This wasn’t surprising since we started with just Inbox and then quickly added features that significantly improved the overall experience for the majority of users.

Eventually, though, we began seeing diminishing returns. At that point, our updates either marginally improved the experience for everyone (design polish) or significantly improved it for only a subset of users (editing capabilities for power users). For us, this was a good signal that it was time to launch. We knew we’d learn much more about how to make great leaps again once we got feedback from users at large.

Never stop learning

One of the best parts about giving our users a seat at the table was knowing that we had cheerleaders waiting for launch with open arms. That made it both less scary, and more exciting.Hundreds of beta testers were already getting a lot of value from the app for months,and we couldn’t wait to let everyone else in on the new app. Our work is by no means finished; some of the most interesting insights are yet to come so if you’re an Android user, we’d love to hear from you. This experience has taught each one of us how important it is to involve users in app development early, and it made the process of working on the app more fun and more personal for us, too.

A version of this post originally appeared on Medium.

Want to help us build the next generation Asana for Android? Join our beta program.