How to Effectively Collect User Feedback in Mobile Application

According to lean development principles, developing a mobile application is a process that includes a sequence of phases --  design, development, release, feedback collection, modification to redesign, and so on -- with the aim of ensuring the successful development of an app at a minimal cost. User feedback is  an indispensable part of the product life cycle and the  basis to determine its evolution.

As Brian Harry, Microsoft Technical Fellow pointed out in his article on The Importance of Feedback in Software Development:

Feedback helps you sort out your understanding of yourself. It helps you see the world from a new Angle, correct the deviated direction and learn better. Besides, feedback makes you yourself more outstanding and your work better. Whether you follow a particular agile development practice or not, the early and frequent feedback is one of the important factors that make you more successful.

This is how Harry depicts the feedback cycle in his article.

As with traditional web applications, there are two main ways for obtaining user feedback in mobile applications: direct and indirect feedback. Direct feedback is gathered by encouraging users to directly state what they think about the app by means of questionnaires or pop-up boxes. Indirect feedback is based on the analysis of user behaviour, such as how they move through the UI, how much time they spend on a page, their click-frequency, user retention and active periods, etc. This article  will focus on the first kind of feedback.

The forms of direct feedback are diversified, and each of them has its own complexity. Often, email feedback options are concealed among app "Settings" and the resulting lack of context makes it harder for users to provide a focused opinion . Entering text information in a tiny mobile phone screen is also a big limiting factor. At the same time,  long questionnaires terrify users and too frequent pop-up boxes drive them mad. Alex Adamopoulos, Founder and CEO at Emergn Limited analyzes the cost of feedback in the What’s Wrong with Feedback. In his opinion, the introduction of any kind of feedback mechanism requires the team to pay a cost. In light of this, how to measure whether the embedded feedback loop has brought sufficient economic benefits and how to maximize the value of the feedback loop are key issues. Additionally, another factor to take into account is the availability of third party services to reduce the cost required  by the  design, development, and operation of a feedback mechanism, such as pgyer and other players in the market.

In summary, when choosing the most appropriate feedback form for their mobile products, product managers, designers and developers are often faced with complex questions, such as: How to choose among a variety of feedback forms? Would it be better  to develop the feedback mechanism in house or rely on a third party service? What shall the team do before and after receiving the feedback?  Does the team have the required capability for development and operation? Can we afford the design, development and operational costs of feedback?

This article, following the feedback life-cycle, aims to help product managers, designers, and developers to understand the problems that should be considered when adding a feedback module to a mobile app, and the typical design, development, and operational costs of different feedback mechanisms. Thus, it will help make the  most convenient choice, ranging from a minimal feedback module to a full questionnaire, and help shape the success of a product.

Before collecting the feedback

Specify the objectives

Product managers should understand very clearly what category their app belongs to, at what stage in its lifecycle it currently is, and what kind of feedback they need to collect from users.

Take two of the most common examples:

• apps for, e.g.,  the banking or insurance sector, which pay more attention to their core-business and usually want to know whether their users are satisfied with their services.
• productivity apps,  which pay more attention to the user experience with their function and design. A concrete example is Uber wanting to, first of all, know whether the functionality and the interaction design of their app meet the use preferences. Or, a real estate management app might want to know whether users found helpful a new "add to favorites" feature.

Consider the cost

Project managers should understand very well what different types of feedback mechanism entail -- such as who are the stakeholders, what skills are needed, how much effort is required, etc. -- in order to calculate the cost, to organize the team, and to arrange the delivery plan.

Adding a  feedback module to an app mainly involves three tasks: the first is to design and develop the app UI for the entry of the user’s feedback; the second is to design and develop the back-end system used to view and manage feedback data; and the third is to process and analyze what typically is a large amount of collected data, and to interact with users.

The costs involved in this process are mainly reflected at three levels:

• Design:  The UI to collect feedback in the app, being the part visible to the end user, has more compelling requirements in terms of user experience. For the back-end, data management platform, the design should pay attention to practicality, focusing on powerful data analysis features, and easy and logical operation.

• Development: due to the use of different development technologies in the front- and back-end，there are often two development teams involved. However, the team composition is strongly related to the specific development techniques used. The frameworks, languages, and tools  generally used to create cross-platform apps are usually similar to those used for back-end development, while frameworks, languages, and tools for native apps are more heterogeneous. If the front- and back-end systems are required to be developed independently, then two development teams are needed. Since the app only provides the means for feedback input, its development cost is relatively low. The back-end management system, on the other hand, has to provide summary, analysis, and management capabilities for large amounts of feedback data, so the development cost is relatively higher. Different forms of feedback have different requirements as to developers’ skills and development time, which needs specific analysis.

• Operations: After collecting feedback data, analysis, summarization, and classification need to be carried out to extract valuable information, thereby driving the subsequent design and development of the app. At the same time, it is necessary to respond to user feedback  timely so they can feel that their feedback is valuable.

Choose a form of feedback

Common forms of feedback in mobile applications mainly include the following:

E-mail

In this scenario, the user opens the mail client from the app and expresses their feedback by writing an email.

The biggest advantage of this way of collecting feedback is its low cost, being based on a single interaction with a standard design. In terms of development, just a few lines of code can handle this mechanism with no requirements for the back-end. At the same time, you  the user's email address, which makes it easy to reply, thus facilitating the maintenance of customer relationships. Considering that most of the times users who are willing to send an email to provide feedback tend to be more loyal, it is more likely they provide valuable data, and even unexpected ideas.

Faults of this approach are mainly related to the need to leave the app, which is likely to result in users not providing their feedback at all. Additionally, collected feedback is scattered through mail messages, making it inconvenient for later reviewing, tracking, and data analysis.

This approach is suitable for cases  of limited budget, short development time, and/or development team not enough skilled for back-end development.

Ask the user whether they like the app before asking for feedback

This can be done by presenting the user the choice between two options: “you like the app” or “you do not like the app”.

If the choice is the former, the user will be guided to give five points and write comments in the App Store. If the choice is the latter, then display a simple and general feedback form in the app, allowing users to send text, images and other information to the designated back-end.

Circa News APP

This technique, while obtaining the feedback, can at the same time greatly increase the rate of positive evaluation of an app and reduce the influence of negative evaluations. This can be explained with the popular proverb "bad news has wings”. In fact, there is a term in psychology called "negative bias" which is used to describe how, in comparison  with positive information, our brain has usually a more intense and lasting response to negative information. In theory, that is, we are more easily affected by negative news.

Compared with the approach of sending an email, the biggest advantage of this technique is that all feedback can be handled the same way in the back-end, which is more convenient for reviewing as well as tracking of feedback state, etc.

Of course, this will also bring extra cost, mainly reflected in the development of a back-end management platform which requires the team to have website development skills. A back-end integrating the basic functions of display, classification, statement analysis, and status tracking requires an effort of more than half a month by at least one senior web development programmer (except when using a third-party services)

This approach is suitable when there is  sufficient budget and the team has got the skills to develop websites.

Questionnaire

WeChat APP

Email and general feedback have strong commonality, that is, unless users are very clear about what they wants to express, it can be difficult to get targeted, valuable information. At the same time, both approaches require to enter a large amount of text, which is not a very convenient operation on mobile phones. Additionally, it is difficult to carry through a rapid classification on the collected feedback data on the back-end.  In comparison to this, a more effective way of collecting feedback is using a questionnaire, which lists a series of targeted questions and multiple-choice answers  for users to choose from. If the interaction is simple, users are easily willing to answer questionnaires. At the same time, in the back-end it will be  easier to carry through a statistical analysis of the results.

This approach requires to determine in advance the feedback points for which feedback is more urgent at the current stage of app development, and then to design the questions and the forms. If the questions are too long, it is easy to trigger user's resistance. Designing forms and questions requires a deep knowledge, and the principle is to get the most amount of information through the least amount of questions.

In terms of development, web page-like forms can be used, which does not only greatly reduce the development costs of the front- and back-end, but also provides a powerful back-end data analysis function.

Rich-content feedback integrating images, voice, text and multimedia

Pgyer SDK

Since textual feedback can be inconvenient to users when describing some complex scenarios, more and more apps adopt screenshots, annotation on screenshots, and allow feedback combining text and voice so that users can express their thoughts at any time and from any part of the app UI. This greatly increases the probability of users taking the initiative to provide feedback. In order to make the process of taking screenshots faster, in addition to the default screenshot feature of the system, a lot of apps support the "shake" gesture to trigger the screenshot. One of the biggest advantages of screenshot feedback is that there is no need to take up any screen space with dedicated controls.

The disadvantage of voice feedback is that the operational cost is high due to the analysis and classification of a large number of collected voice recordings. One way to reduce this operational cost is to carry through voice analysis in the app. That is, after the user record their voice message, the app first converts it to text information with the help of voice recognition technology, and then sends the resulting text to server. This shifts the work of verifying the correctness of recognition to users, thus affecting the user experience. We’re keeping our eyes open on the development of the voice recognition technology and other artificial intelligence technologies to see how they can benefit mobile feedback collection.

Feedback based on the context combined with Net Promoter Score

In order to strike a balance between user experience and feedback effectiveness, we suggest that a contextual feedback mechanism be adopted, according to the principle of Design in Context. This is: determine the feedback mechanism, its content, the means of input , the frequency of appearance, etc. based on specific scenarios and specific users.Net Promoter Score (NPS) is a common means to judge user loyalty. Through a brief question such as "will you recommend this product to your friends?" users can be divided into three categories: "Promoters", "Passives", and "Detractors". Once this classification is in place,  you can continue to ask the "Promoters" additional questions, thus realizing the ideal of "obtaining more valuable information from the users most likely to provide feedback, without annoying the average user". Therefore, this is the approach we prefer.

After users have had a given feature available for some time, the first step is knowing know what is the user’s attitude towards this feature using NPS; then, based on the user's answer (like or dislike), you can ask a few more targeted questions, all the same ensuring that the feedback module has a user-friendly design (avoid the pop-up box design which is kind of invasive).

This way, you can collect a large number of targeted and valuable feedback data points in a short time. This technique is particularly suitable for the situation where certain features have been on-line for a period of time.

Airbnb

While collecting the feedback

Respond to user feedback in time

Once an app has successfully integrated the feedback module, its product manager has gained one more responsibility: constantly checking emails, or the back-end  feedback management system to retrieve  first-hand user feedback. However, only "checking" is not enough. Whether user feedback is positive, provides constructive suggestions, or criticism, users will have spent their time and energy to provide it, and will want to know whether it has been received, valued, etc. Therefore, promptly replying to user feedback is a crucial step to improve customer satisfaction.

The first step is to immediately inform users that their feedback has been received , tell them that their feedback will be processed in a timely manner, and at the same time express sincere appreciation. The second is to timely contact with users based on the content of their feedback, which could probably mean answering customer’s questions, or clarifying the future product improvement plan, etc. The third step is notifying users when their suggestions have been adopted and are officially launched later on. This not only allow them to better use the app, but also let them take some pride for their suggestions have been adopted.

Email is one of the means that is most unlikely to disturb the user, thus it is important here that the design of any form of feedback take into account the provisioning of users’ email addresses for later tracking.

In its recently released iOS 10.3, Apple allows users to give scores and make comments on an app without leaving it. Users score and comments will be automatically synchronized to the App Store. Thanks to the fact that there is no need for the user to leave the app to provide a comment, it is expected that this will greatly reduce the ratio of user loss. At the same time, iOS 10.3 will allow developers to openly reply to reviews in the App Store, strengthening the relationship with the user. From this, we can see Apple’s emphasis on feedback loop between developers and users.

Note: to reduce development costs, an alternative is to adopt a comparatively mature third-party feedback service. Most third-party services not only provide a variety of app feedback interfaces, but also include powerful back-end  feedback data management features. This can greatly reduce development costs. The downside lies in the difficulty of customizing style and function, and of guaranteeing data security. Therefore, for teams that have their own unique design and high security requirements for feedback data, this option should be carefully weighed.

After the feedback

Analyze feedback data and adjust the app roadmap.

"In the face of feedback, we need the right attitude. Embrace change, and don't be afraid to rework", writes Adamopoulos in What’s Wrong with Feedback. The last step, therefore, is to summarize, classify, and analyze a possibly large amount of feedback data to identify the problems existing in the product, and to classify and prioritize them. If necessary, further user surveys and interviews could be required with the aim to formulate the next development and release plan.

Conclusion

The inclusion of a feedback module in a mobile app is not a simple matter. Before starting to collect feedback, you should be clear about the objective and comprehensively consider the functions and costs to build the feedback form; you should also accurately define the feedback data entry, timing, and frequency. After this step, the feedback module can be successfully integrated into the app. During the feedback collection phase, you should timely thank the users for their feedback and inform them about recent improvements. After collecting the feedback, you should conduct large data analysis on the feedback results, and effectively adjust the roadmap of the product. Only by following these steps can a feedback mechanism play its role and let users provide helpful information about your product.

About the Author

Jianing Zheng is a mobile developer in ThoughtWorks China. She is passionate about how to simplify people’s life by designing and developing mobile apps that users really like. During 5 years of helping a variety of clients to develop mobile apps for their customers, both as a developer and team lead, with deep understanding of the pros and cons for each form from a number of perspectives, she has accumulated much experience of how to use effective tools to get feedbacks from consumers to improve the products.