Mobile apps users' reviews analysis

Abstract

The aim of this report was to investigate factors that drive users to give bad and good reviews for smartphone apps. User reviews are perhaps the most important mean of advertising for apps. A content analysis study on a combine total of 190 user reviews for different ratings was conducted. The results indicate that the majority of both good and bad reviews have roots in whether or not users can complete the task the app is designed for. The report concludes that an app should only focus on what it is intended for and try to that gracefully. It is recommended that app developers consider their user group needs as well as other factors that can affect their app’s performance.

Introduction

The number of smartphone users has increased significantly over the past few users. Smartphones sale grew 61.3 per cent in 2011. Even though figures for 2011 are less than the 75.7 per cent growth for 2010, according to forecasts, double-digit growth for the foreseeable future is expected (International Data Corporation 2012). Besides communication purposes, mobile apps are playing an important role on how devices are used by their owners. As shown in the figure below, most smartphone owners have got more than 25 five apps on their devices even though not all of them might be used frequently. In such circumstances, companies that do not capture mobile markets will lose the competition to those that have (Curran 2012).

Figure 1: average number of apps on smartphones

Mobile app market is very tricky since apps are being judged directly by users. An app may not suffer as bad from anything as it does from bad user reviews. On the other hand, there is no better advertising than user recommendations either to their friends or in their reviews. This fact is the motivation of this report to analyse users’ reactions and determine important factors that bring apps good and bad reviews and ratings.

Method

This research was carried out by collecting user reviews from Apple app store and Google play. For each of 5 and 1 star ratings, 50 reviews, and for each of 2, 3 and 4 star ratings, 30 reviews were collected. The 5 and 1 star reviews were analysed by matching user comments with keywords taken from design patterns and paradigms. For each rating category, each keyword, and its number of mentions was recorded in a spread sheet for summarization and conclusion purposes. For reviews with 2, 3 and 4 stars, apart from keywords, the content is also considered in order to find supplementary information or support for findings from the analysis of 1 and 5 stars rating reviews. Extracted information was then recorded in text format to be checked against the quantitative data, stored in spreadsheets, to see whether there is any connection between them or not.

Results

A breakdown of factors for 5 stars reviews is shown in the following chart. As it can be clearly seen, effectiveness, that is users are able to complete their task and achieve their goal gracefully, is the most important factor for an app to get a good review. Ease of use, mentioned 18 times in sample user reviews, is the second important matter users would like to see about an app.

Figure 2: 5 star rated reviews’ factors

The chart below shows the factors that users have complained about when trying to use apps on their smartphones. According to the chart, the most significant problems that led to bad reviews were users not being able to achieve their goals, and the app does not provide expected functionalities or information.

Figure 3: 1 star rated reviews’ factors

Discussion / Interpretation of results

Effectiveness and ease of use

As shown by the results of this research, there are a variety of reasons for an app to become accepted or rejected by its users. Effectiveness, defined by Stone et al. (2005, p.6) as “the accuracy and completeness with which specified users can achieve specified goals in a specified environment”, is found to be the most significant factor in giving apps both good and bad ratings. According to users’ reviews, constant crashes is one of the reasons users get frustrated and finally run out of patience with an app and therefore, they stop using it. Simplicity is another cause that can bring satisfaction to users. It goes without saying that most of the reasons shown in charts above such as “ease of use”, “effectiveness” and “match with user expectations” are interrelated and success cannot be achieved without considering all of them at the same time. However, if users can get the functionalities they expect from their app and as a result, can achieve their goals, factors such as not being easy to use may be overlooked by them since practice can make a task seem easier after a little while when it normally requires a lot of attention for first time users (Dr Von Baggo 2011). Nevertheless, there have been cases that users counted “clunky” UI designs, which makes the app harder to use, as to why they were rating the app still good 4 stars but not 5. Not to mention providing adequate help and documentation, which has found to be the subject of both complaints and approvals, is helpful in such situation as well. Thus, it can be said that as long as the app let the users do what the app says it does without constantly needing to restart or frequent crashes, it might sound acceptable to users. Speaking of effectiveness, there are other issues that if considered, can reduce the number of bad reviews, stating user cannot complete their task, for an app, such as providing the minimum requirements to run an app. For instance, for a particular app that apparently works gracefully when used on iPhone 4S, there were lots of bad ratings from those who had tried the app on older devices.

Match with user expectations

Users’ expectations highly depend on their background which is formed by their experiences in the real world. AS demonstrated in the results of this research, it is essential to find out what features are expected from a specific type of app and include all those functionalities. Otherwise, it is not going to satisfy its users. For example, it was found from reviews that many users were complaining about apps such as spread sheets and budget planners that cover only a limited set of requests or about lack of accuracy when it plays a significant role in an app such as a ruler. Number of implemented features is not the only case though. Information and the way it is presented seems to be vital as well. In a particular app that provides astronomy data and has received many good reviews, it is stated by users that the content and how it is represented, is exactly what they want. It can be said that even minor mismatches with users’ expectations leads to dropped ratings even though the rating and the review may not necessarily be a negative one. In many cases, users giving 4 stars ratings to some apps have stated that the reason why they did not consider the app as 5 star is simply that they expect one or two more features from that otherwise desirable app. Editable items and duplicate entries in shopping lists and fortnightly calculations in budget planners are examples of this sort.

Targeting real users

Knowing exactly who is going to use your app and therefore, tailoring it to suit their needs also requires some good amount of attention. For instance, a specific app that was used by parents to entertain their children has added adult adverts and therefore, has lost a big user group. This issue might be quite tricky at times since your real user group may be dispersed across the globe. For example, users are driven to write good reviews because the app supports cross border functionalities. On the other hand, an app gets bad ratings because it only works in the US. Windows User Experience Interaction Guidelines (2010) says “your program is going to be more successful by delighting its target users than attempting to satisfy everyone”. Apple becomes more specific about it and recommends creating a list of characteristics users are likely to have. Then decide on 3 characteristics that best describe your target audience (iOS Human Interface Guidelines 2011).

Deceiving

A deceiving app is the one that says it is free; nevertheless, it does not support even a very limited functionality unless you get the paid version. People would not mind paying for an app, especially for the ones that do their job gracefully. Moreover, paying money does not seem to be a reason for giving good or bad reviews by users. However, people do not like to feel cheated. This has been found to be the reason for many bad ratings. As Shown in the 1 star ratings chart in results section, users have complained 10 times in the total number of 50 reviews analysed for 1 star ratings.

Adverts

Adverts have not always been the issue of bad reviews. Actually, there were few good comments specifying adverts are not intrusive, not annoying or are not hindering users’ tasks. However, it seems that sometimes adverts can get irritating. The list below shows a few of the situations when advertising have bothered users very much when using their apps.

• Frequent popups
• Bad choice of adverts (for example, 18+ adverts as mentioned before)
• Adverts close to functional elements

Proper feedback / Visibility

Sometimes, after pondering a bad review, it becomes obvious that the bad user experience is not derived directly from the app and there were other factors involved. Bad reviews received under such circumstances can be avoided by improving visibility of system status and providing proper feedback to users. With regard to this, Nielsen (2005) states that “the system should always keep users informed about what is going on, through appropriate feedback within reasonable time.” For example, a user has given a bad review for a video chat app simply because their voice was not being clearly transferred. This can easily be due to network status which user should be informed about; especially in such an app that heavily relies on the Internet to serve its users. There are other aspects of visibility to be considered. It is also found in this research that there were many occasions in which users could not find, or did not notice a feature or functionality at first, and they really liked it when they finally used it. In one particular review, this issue has led to a 4 star rating instead of 5, user stating a good feature is not immediately clear. Not to mention, many users may not be able to find these not-immediately-visible features at all, and therefore, they do not gain the experience the app was designed and intended for.

Conclusion

Effectiveness seems to be the most important factor form users’ perspective when using an app. An app does not have to be very complex or cover much functionality. All it needs to do is what it says it does in a good and reasonable way. Any effort while creating an app, that helps users complete their tasks efficiently, is highly likely to be appreciated by users. Making an app accessible free of charge will not necessarily lead to user acceptance or users overlooking apps shortcomings and mistakes. There is no point in building a free app and then trying to compensate for it by adding distractive and hindering advertisements which may bring a very bad experience for users.

Recommendation

There were quite a few recommendations in the discussion of this report; however, it is worth mentioning some of the key points again.

• Providing feedback about app status is crucial because as discussed, an app can get bad ratings for something that is not the app’s fault. This also includes the minimum requirements needed by the app to run.
• Features and processes should be made absolutely clear and accessible. Sometimes, it is hard to believe that an app is exactly as it is described in users’ reviews. It more seems to be that the user was not able to find their way out or work out how things are to be done.
• A competitive analysis or studying market trends to find out about users and their expectations will definitely pay off.
• Above all, it seems so hard for a constantly crashing app to get back on track. The more crashes can be avoided, the less severe bad reviews will be.

References

• International Data Group (IDC) 2012, ‘Smartphone Market Hits All-Time Quarterly High Due To Seasonal Strength and Wider Variety of Offerings. According to IDC’, IDC - Press Release, viewed 12 June 2012, .
• Curran, C 2012, 5 Smartphone Usage Trends for 2012 and Beyond, CIO Dashboard, viewed 12 June 2012, < http://www.ciodashboard.com/mobile/5-smartphone-usage-trends-for-2012-and-beyond/>.
• ‘[Figure 1: average number of apps on smartphones]’, [image] 2012, viewed 12 June 2012, .
• Stone et al. 2005, User interface design and evaluation, Morgan Kaufmann, San Francisco.
• Dr Von Baggo, K 2011, ‘Lecture 6: Module 10 Psychological Principles’, HIT2316/6316 Usability, Learning materials on Blackboard, Swinburne University of Technology, viewed 12 June 2012.
• Windows User Experience Interaction Guidelines 2010, Microsoft Corporation, Viewed 12 June 2012, .
• iOS Human Interface Guidelines 2011, Apple Inc., Viewed 12 June 2012, .
• Nielsen, J 2005, Ten Usability Heuristics, Jakob Nielsen's Website, viewed 11 June 2012, .