Mobile apps users' reviews analysis

Abstract

The aim of this report was to investigate factors that drive users to give bad and good reviews for smartphone apps. User reviews are perhaps the most important mean of advertising for apps. A content analysis study on a combine total of 190 user reviews for different ratings was conducted. The results indicate that the majority of both good and bad reviews have roots in whether or not users can complete the task the app is designed for. The report concludes that an app should only focus on what it is intended for and try to that gracefully. It is recommended that app developers consider their user group needs as well as other factors that can affect their app’s performance.

Introduction

The number of smartphone users has increased significantly over the past few users. Smartphones sale grew 61.3 per cent in 2011. Even though figures for 2011 are less than the 75.7 per cent growth for 2010, according to forecasts, double-digit growth for the foreseeable future is expected (International Data Corporation 2012). Besides communication purposes, mobile apps are playing an important role on how devices are used by their owners. As shown in the figure below, most smartphone owners have got more than 25 five apps on their devices even though not all of them might be used frequently. In such circumstances, companies that do not capture mobile markets will lose the competition to those that have (Curran 2012).

Figure 1: average number of apps on smartphones

Mobile app market is very tricky since apps are being judged directly by users. An app may not suffer as bad from anything as it does from bad user reviews. On the other hand, there is no better advertising than user recommendations either to their friends or in their reviews. This fact is the motivation of this report to analyse users’ reactions and determine important factors that bring apps good and bad reviews and ratings.

Method

This research was carried out by collecting user reviews from Apple app store and Google play. For each of 5 and 1 star ratings, 50 reviews, and for each of 2, 3 and 4 star ratings, 30 reviews were collected. The 5 and 1 star reviews were analysed by matching user comments with keywords taken from design patterns and paradigms. For each rating category, each keyword, and its number of mentions was recorded in a spread sheet for summarization and conclusion purposes. For reviews with 2, 3 and 4 stars, apart from keywords, the content is also considered in order to find supplementary information or support for findings from the analysis of 1 and 5 stars rating reviews. Extracted information was then recorded in text format to be checked against the quantitative data, stored in spreadsheets, to see whether there is any connection between them or not.

Results

A breakdown of factors for 5 stars reviews is shown in the following chart. As it can be clearly seen, effectiveness, that is users are able to complete their task and achieve their goal gracefully, is the most important factor for an app to get a good review. Ease of use, mentioned 18 times in sample user reviews, is the second important matter users would like to see about an app.

Figure 2: 5 star rated reviews’ factors

The chart below shows the factors that users have complained about when trying to use apps on their smartphones. According to the chart, the most significant problems that led to bad reviews were users not being able to achieve their goals, and the app does not provide expected functionalities or information.

Figure 3: 1 star rated reviews’ factors

Discussion / Interpretation of results

Effectiveness and ease of use

As shown by the results of this research, there are a variety of reasons for an app to become accepted or rejected by its users. Effectiveness, defined by Stone et al. (2005, p.6) as “the accuracy and completeness with which specified users can achieve specified goals in a specified environment”, is found to be the most significant factor in giving apps both good and bad ratings. According to users’ reviews, constant crashes is one of the reasons users get frustrated and finally run out of patience with an app and therefore, they stop using it. Simplicity is another cause that can bring satisfaction to users. It goes without saying that most of the reasons shown in charts above such as “ease of use”, “effectiveness” and “match with user expectations” are interrelated and success cannot be achieved without considering all of them at the same time. However, if users can get the functionalities they expect from their app and as a result, can achieve their goals, factors such as not being easy to use may be overlooked by them since practice can make a task seem easier after a little while when it normally requires a lot of attention for first time users (Dr Von Baggo 2011). Nevertheless, there have been cases that users counted “clunky” UI designs, which makes the app harder to use, as to why they were rating the app still good 4 stars but not 5. Not to mention providing adequate help and documentation, which has found to be the subject of both complaints and approvals, is helpful in such situation as well. Thus, it can be said that as long as the app let the users do what the app says it does without constantly needing to restart or frequent crashes, it might sound acceptable to users. Speaking of effectiveness, there are other issues that if considered, can reduce the number of bad reviews, stating user cannot complete their task, for an app, such as providing the minimum requirements to run an app. For instance, for a particular app that apparently works gracefully when used on iPhone 4S, there were lots of bad ratings from those who had tried the app on older devices.

Match with user expectations

Users’ expectations highly depend on their background which is formed by their experiences in the real world. AS demonstrated in the results of this research, it is essential to find out what features are expected from a specific type of app and include all those functionalities. Otherwise, it is not going to satisfy its users. For example, it was found from reviews that many users were complaining about apps such as spread sheets and budget planners that cover only a limited set of requests or about lack of accuracy when it plays a significant role in an app such as a ruler. Number of implemented features is not the only case though. Information and the way it is presented seems to be vital as well. In a particular app that provides astronomy data and has received many good reviews, it is stated by users that the content and how it is represented, is exactly what they want. It can be said that even minor mismatches with users’ expectations leads to dropped ratings even though the rating and the review may not necessarily be a negative one. In many cases, users giving 4 stars ratings to some apps have stated that the reason why they did not consider the app as 5 star is simply that they expect one or two more features from that otherwise desirable app. Editable items and duplicate entries in shopping lists and fortnightly calculations in budget planners are examples of this sort.

Targeting real users

Knowing exactly who is going to use your app and therefore, tailoring it to suit their needs also requires some good amount of attention. For instance, a specific app that was used by parents to entertain their children has added adult adverts and therefore, has lost a big user group. This issue might be quite tricky at times since your real user group may be dispersed across the globe. For example, users are driven to write good reviews because the app supports cross border functionalities. On the other hand, an app gets bad ratings because it only works in the US. Windows User Experience Interaction Guidelines (2010) says “your program is going to be more successful by delighting its target users than attempting to satisfy everyone”. Apple becomes more specific about it and recommends creating a list of characteristics users are likely to have. Then decide on 3 characteristics that best describe your target audience (iOS Human Interface Guidelines 2011).

Deceiving

A deceiving app is the one that says it is free; nevertheless, it does not support even a very limited functionality unless you get the paid version. People would not mind paying for an app, especially for the ones that do their job gracefully. Moreover, paying money does not seem to be a reason for giving good or bad reviews by users. However, people do not like to feel cheated. This has been found to be the reason for many bad ratings. As Shown in the 1 star ratings chart in results section, users have complained 10 times in the total number of 50 reviews analysed for 1 star ratings.

Adverts

Adverts have not always been the issue of bad reviews. Actually, there were few good comments specifying adverts are not intrusive, not annoying or are not hindering users’ tasks. However, it seems that sometimes adverts can get irritating. The list below shows a few of the situations when advertising have bothered users very much when using their apps.

• Frequent popups
• Bad choice of adverts (for example, 18+ adverts as mentioned before)
• Adverts close to functional elements

Proper feedback / Visibility

Sometimes, after pondering a bad review, it becomes obvious that the bad user experience is not derived directly from the app and there were other factors involved. Bad reviews received under such circumstances can be avoided by improving visibility of system status and providing proper feedback to users. With regard to this, Nielsen (2005) states that “the system should always keep users informed about what is going on, through appropriate feedback within reasonable time.” For example, a user has given a bad review for a video chat app simply because their voice was not being clearly transferred. This can easily be due to network status which user should be informed about; especially in such an app that heavily relies on the Internet to serve its users. There are other aspects of visibility to be considered. It is also found in this research that there were many occasions in which users could not find, or did not notice a feature or functionality at first, and they really liked it when they finally used it. In one particular review, this issue has led to a 4 star rating instead of 5, user stating a good feature is not immediately clear. Not to mention, many users may not be able to find these not-immediately-visible features at all, and therefore, they do not gain the experience the app was designed and intended for.

Conclusion

Effectiveness seems to be the most important factor form users’ perspective when using an app. An app does not have to be very complex or cover much functionality. All it needs to do is what it says it does in a good and reasonable way. Any effort while creating an app, that helps users complete their tasks efficiently, is highly likely to be appreciated by users. Making an app accessible free of charge will not necessarily lead to user acceptance or users overlooking apps shortcomings and mistakes. There is no point in building a free app and then trying to compensate for it by adding distractive and hindering advertisements which may bring a very bad experience for users.

Recommendation

There were quite a few recommendations in the discussion of this report; however, it is worth mentioning some of the key points again.

• Providing feedback about app status is crucial because as discussed, an app can get bad ratings for something that is not the app’s fault. This also includes the minimum requirements needed by the app to run.
• Features and processes should be made absolutely clear and accessible. Sometimes, it is hard to believe that an app is exactly as it is described in users’ reviews. It more seems to be that the user was not able to find their way out or work out how things are to be done.
• A competitive analysis or studying market trends to find out about users and their expectations will definitely pay off.
• Above all, it seems so hard for a constantly crashing app to get back on track. The more crashes can be avoided, the less severe bad reviews will be.

References

• International Data Group (IDC) 2012, ‘Smartphone Market Hits All-Time Quarterly High Due To Seasonal Strength and Wider Variety of Offerings. According to IDC’, IDC - Press Release, viewed 12 June 2012, .
• Curran, C 2012, 5 Smartphone Usage Trends for 2012 and Beyond, CIO Dashboard, viewed 12 June 2012, < http://www.ciodashboard.com/mobile/5-smartphone-usage-trends-for-2012-and-beyond/>.
• ‘[Figure 1: average number of apps on smartphones]’, [image] 2012, viewed 12 June 2012, .
• Stone et al. 2005, User interface design and evaluation, Morgan Kaufmann, San Francisco.
• Dr Von Baggo, K 2011, ‘Lecture 6: Module 10 Psychological Principles’, HIT2316/6316 Usability, Learning materials on Blackboard, Swinburne University of Technology, viewed 12 June 2012.
• Windows User Experience Interaction Guidelines 2010, Microsoft Corporation, Viewed 12 June 2012, .
• iOS Human Interface Guidelines 2011, Apple Inc., Viewed 12 June 2012, .
• Nielsen, J 2005, Ten Usability Heuristics, Jakob Nielsen's Website, viewed 11 June 2012, .

Using ROWID to determine the primary key of a record

This function gets a rowid as an input parameter and then if valid, returns true, and as output parameters, gives the SQL statement, primary key columns list and a where clause statement including primary key columns and the actual values of the corresponding record. If the rowid is not valid, the function returns false, and if the rowid is valid but the related table does not have a primary key, then ... .

create or replace function rowid2pkwc(gRowid varchar2,
selectStmt out varchar2,
colsList out varchar2,
whereClause out varchar2) return boolean is
cursor c1(tableName varchar2) is
select * from user_cons_columns
where constraint_name = (select constraint_name from user_constraints
where table_name=tableName
and constraint_type = 'P');
r1 c1%rowtype;
objectId number;
gTableName varchar2(30);
sqlStmt varchar2(32767);
descTab DBMS_SQL.DESC_TAB;
colCnt integer;
cursorId integer;
rowsProcessed integer;
colType varchar2(100);
colTypeDt date;
colVal varchar2(100);
colValDt date;
begin
--gRowid := 'AAAWsyAAHAAATVgAES';

-- Finding out which table the given rowid belongs to
-- selectStmt out variable is formed here
objectId := dbms_rowid.rowid_object(chartorowid(gRowid));
begin
select object_name into gTableName from user_objects
where object_id = objectId
and object_type = 'TABLE';
selectStmt := 'select * from '||gTableName;
exception
when no_data_found then
return false;
end;

-- Finding out the primary key columns of the table found in previous step
-- colList out variable (primary key columns in comma separated format)
-- is formed here
sqlStmt := 'select ';
colCnt := 0;
for r1 in c1(gTableName) loop
if c1%rowcount > 1 then
sqlStmt := sqlStmt || ',';
colsList := colsList || ',';
end if;
sqlStmt := sqlStmt || r1.column_name;
colsList := colsList || r1.column_name;
end loop;
sqlStmt := sqlStmt || ' from '||gTableName ||' where rowid = ''' || gRowid || '''';

-- The sql statement formed according to table name and pk columns in previous step
-- is parsed for the purpose of retrieving the values of pk columns
cursorId := dbms_sql.open_cursor;
DBMS_SQL.PARSE(cursorId, sqlStmt, DBMS_SQL.NATIVE);

-- Indicating what values to expect from the query parsed in previous step
DBMS_SQL.DESCRIBE_COLUMNS (cursorId, colCnt, descTab);
for i in 1 .. colCnt loop
if descTab(i).col_type in (dbms_types.TYPECODE_DATE) then
dbms_sql.define_column( cursorId, i, colTypeDt);
else
dbms_sql.define_column( cursorId, i, colType, 100);
end if;
end loop;

-- Execute the dynamic sql query
rowsProcessed := DBMS_SQL.EXECUTE(cursorId);

-- Reading values of the selected columns (pk columns) for the purpose of forming
-- the where clause
-- whereClouse out variable is formed here
if dbms_sql.fetch_rows(cursorId) > 0 then
whereClause := '';
for i in 1 .. colCnt loop
if (length(whereClause)>0) then
whereClause := whereClause || ' and ';
end if;
if descTab(i).col_type in (dbms_types.TYPECODE_CHAR,
dbms_types.TYPECODE_VARCHAR2,
dbms_types.TYPECODE_VARCHAR) then
dbms_sql.column_value(cursorId,i,colVal);
whereClause := whereClause || descTab(i).col_name || '=''' || colVal || '''';
elsif descTab(i).col_type in (dbms_types.TYPECODE_NUMBER) then
dbms_sql.column_value(cursorId,i,colVal);
whereClause := whereClause || descTab(i).col_name || '=' || colVal;
elsif descTab(i).col_type in (dbms_types.TYPECODE_DATE) then
dbms_sql.column_value(cursorId,i,colValDt);
whereClause := whereClause || descTab(i).col_name || '=' ||
'to_date('''||to_char(colValDt,'DD, MM, YYYY, HH24:MI:SS')||''' ,''DD, MM, YYYY, HH24:MI:SS'')';
else
dbms_sql.column_value(cursorId,i,colVal);
whereClause := whereClause || descTab(i).col_name || '=''' || colVal || '''';
end if;
end loop;
if (length(whereClause)>0) then
whereClause := 'where '||whereClause;
end if;
end if;

DBMS_SQL.CLOSE_CURSOR(cursorId);

return true;
end rowid2pkwc;

Delegation Pattern

Android AsyncTask Example

Definition

In software engineering, the delegation pattern is a design pattern in object-oriented programming where an object, instead of performing one of its stated tasks, delegates that task to an associated helper object.

Figure 1: simple delegation

Delegation and Inheritance

In languages such as Java, in which multiple inheritance is not supported, delegation can be used to fill in this gap. From this perspective, delegation is like inheritance done manually through object composition.

Figure 2: delegation and inheritance

More Than Just Inheritance

By using interfaces, delegation can be made more flexible and typesafe. Flexibility and type-safety here means that the delegator doesn't need to refer to any ClassA or ClassB. As shown by the image below, it can delegate its tasks to any object that implements/conforms to the delegation interface.

Figure 3: delegation and interfaces(or protocols)

The example below shows how flexibility and type-safety are achieved in delegation pattern by taking advantage of interfaces.

A Use Case - Asynchronous Tasks

Asynchronous tasks run in background. It is vital for a multi-thread program to handle the communication between the background threads and the main thread gracefully. One convenient way of doing so is using delegation pattern where an object, running in the main thread, plays the role of the background operation's delegate, and gets notified of changes in background operation's status such as partial/full completion and errors.

Figure 4: delegation and asynchronous tasks

In the following, a simple Android example of what is discussed above will be explored. This example is created in eclipse IDE on (Source code available for download). The image below shows screenshots of the app in different stages.

Figure 5: sample Android app screenshots

When the "Start Long Operation" button is pressed, a long operation is started in the background. The long operation notifies the main thread of different stages of the task accomplished so that the main thread can update a progress bar to provide feedback to the user. When the long operation ends, the main thread shows a message to the user saying the task is completed.
Class LongOperation that runs in the background:

The main activity that implements the LongOperationDelegate interface and invokes the background operation:

(Source code available for download)

iOS vs. Android - Delegation pattern

This is actually a comparison between Objective-C protocols and Java interfaces. In Obj-C protocols methods can be defined as optional which means objects conforming to the protocol (conforming to a protocol is Obj-C terminology for implementing an interface), may or may not implement those methods.

Figure 6: an Objective-C protocol

Whether an object has implemented a method or not can be determined by the caller object using respondsToSelector method.

Figure 7: caller object uses respondsToSelector method to determine if a method is implemented by other objects

Optional methods facility makes Obj-C protocols very flexible. This is why iOS Cocoa Touch Framework relies on delegation pattern so heavily.

Java Adapter Classes

Java interfaces can be made more flexible using adapter classes. An adapter class is an abstract class that implements an interface. All the methods of the interface are implemented with an empty body or a body that only throws an exception. Other classes that were supposed to implement the interface directly, will extend the adapter class instead overriding only those methods that are needed. In this way objects do not have to implement a lot of empty and useless methods as a result of implementing an interface.

Figure 8: Java adapter classes

Adapter classes are used in Java API as well. The interface MouseInputListener, which includes many methods, comes with MouseInputAdapter which is an abstract class implementing all methods declared in the interface.

Unit Testing

This report shows an example of unit testing in Objective-C using iPhone Unit Testing framework for a map view based application. Unit tests target your code at the lowest possible level - the input/output of a single method (a method is often the smallest testable unit of code). If every single method behaves as you expect, so will your program. What percentage of your code is covered by unit tests is called unit test coverage. 100% coverage is not only rare, but unrealistic.  You should determine which classes and methods are to get the most attention from your tests. This can be achieved by answering a few questions like how often the implementation changes or if it takes input from an unpredictable source, like a user or a network connection.

External data sources are less reliable than, for example, a static read-only file on the file system. When applications rely on external data sources such as the Internet and device hardware, availability and validity of data are undoubtedly major risks. This is due to the fact that the likelihood of incidents such as information has been moved, information not being in expected format or network disconnection is high. Therefore, checking if an application handles data related matters properly can be the centre of focus in tests.

Example map view application

The example map view application in question here consists of two view controllers. First, there is a table view controller which loads some coordinates from a CSV file. By selecting any of the coordinates from the table view, the second view, the map view, appears showing the chosen point. Screen shots of the application are presented below (source code of the application is provided at the end of the report).

Figure 1: example map view application

The goal here is to make sure a proper error is raised when coordinates, either latitude or longitude, are out of range. The code snippet below shows data validation in the map view controller. Note that all data handling and validation methods are supposed to be in a separate class in model layer. However, as the goal here is not to demonstrate MVC pattern and to make it quicker, data related matters are being handled in view controllers.

- (void)setMapDetailsWithLatitude:(double) latitude andWithLongitude:(double)longitude

{

    @try {

       

        if ( latitude > 90 || latitude < -90 ) {

            NSException* e = [NSException exceptionWithName:@"Bad Location" 

                                          reason:@"Invalid latitude" 

                                          userInfo:nil];

            @throw e;

        }

        if ( longitude > 180 || longitude < -180 ) {

            NSException* e = [NSException exceptionWithName:@"Bad Location" 

                                          reason:@"Invalid longtitude" 

                                          userInfo:nil];

            @throw e;

        }

       

        MKCoordinateSpan coordinationSpan;

        coordinationSpan.latitudeDelta = 0.1;

        coordinationSpan.longitudeDelta = 0.1;

       

        CLLocationCoordinate2D locationCoordinate;

        locationCoordinate.latitude = latitude;

        locationCoordinate.longitude = longitude;

       

        MKCoordinateRegion region;

        region.span=coordinationSpan;

        region.center=locationCoordinate;

       

        mapView.mapType = MKMapTypeStandard;

        [mapView setShowsUserLocation:YES];

        [mapView setRegion:region animated:TRUE];     

    }

    @catch (NSException *exception) {

        NSException* e = [NSException exceptionWithName:@"Bad Location" 

                                      reason:@"Invalid Location Parameters" 

                                      userInfo:[exception userInfo]];

        @throw e;

    }

}

Code snippet 1: data validation

Fortunately, XCODE makes it fairly easy to add unit tests to applications. What needs to be done in the beginning is to make sure Include Unit Tests checkbox is checked when creating a new application.

Figure 2: including unit tests when creating a new project

Then all needs to be done is clicking and holding Run button in the upper left of XCODE IDE, and choose test from the popup menu.

Figure 3: build for testing

If you forgot to include unit tests in an application, or you want to add unit tests to an existing application, you simply add a new target to the project, and add a “Cocoa Touch Unit Testing Bundle” to your app.

Figure 4: adding a new target to project

Figure 5: add a Cocoa Touch Unit Testing Bundle to project

Next is to write appropriate test cases to see if data validation code works properly and effectively. There are 4 test cases in the example map view application.

• testInvalidLatitude – Tests if an invalid latitude would cause an exception or not (latitude should be between -90 and +90).
• testInvalidLongitude - Tests if an invalid longitude would cause an exception or not (longitude should be between -180 and +180).
• testValidCoords – Tests if valid latitude and longitude would not cause an exception.
• testAllCoordsFromFile – Tests all the values provided in the gps_coords.csv file.

Note that test cases names should follow the above format, which is “test” plus some other words as in testInvalidLatitude, to be considered as test cases and get executed. Below, the code for test cases mentioned above is presented.

- (void)testInvalidLatitude

{

    MapViewController* mapVC = [[MapViewController alloc] initWithNibName:@"MapViewController" bundle:nil];

   

    STAssertThrowsSpecificNamed([mapVC setMapDetailsWithLatitude:91 andWithLongitude:0],NSException, @"Bad Location", @"Invalid latitude or longitude values should raise an exception");

   

    [mapVC release];

}

- (void)testInvalidLongitude

{

    MapViewController* mapVC = [[MapViewController alloc] initWithNibName:@"MapViewController" bundle:nil];

  

    STAssertThrowsSpecificNamed([mapVC setMapDetailsWithLatitude:0 andWithLongitude:181],NSException, @"Bad Location", @"Invalid latitude or longitude values should raise an exception");

   

    [mapVC release];

}

- (void)testValidCoords

{

    MapViewController* mapVC = [[MapViewController alloc] initWithNibName:@"MapViewController" bundle:nil];

   

    STAssertNoThrow([mapVC setMapDetailsWithLatitude:27 andWithLongitude:153],@"Valid latitude or longitude values should NOT raise an exception");

   

    [mapVC release];

}

- (void)testAllCoordsFromFile

{

    NSString* filePath = [[NSBundle mainBundle] pathForResource:@"gps_coords"
                                                ofType:@"csv"];

    NSString* gpsCoords = [NSString stringWithContentsOfFile:filePath
                                    usedEncoding:nil
                                    error:nil];

    NSMutableArray* mutableArray = [[NSMutableArray alloc] initWithArray:[gpsCoords componentsSeparatedByString:@"\n"]];

   

    MapViewController* mapVC = [[MapViewController alloc] initWithNibName:@"MapViewController" bundle:nil];

   

    double lat;  

    double lng;

   

    for (NSInteger i=0; i<[mutableArray count]; i++) {

        @try {

            lat = [[[[mutableArray objectAtIndex:i] componentsSeparatedByString:@";"] objectAtIndex:0] doubleValue];       

            lng = [[[[mutableArray objectAtIndex:i] componentsSeparatedByString:@";"] objectAtIndex:1] doubleValue];

            STAssertNoThrow([mapVC setMapDetailsWithLatitude:lat andWithLongitude:lng],@"Valid latitude or longitude values should NOT raise an exception");

        }

        @catch (NSException* e){

            NSLog([NSString stringWithFormat:@"line %d is not in expected format",i]);

        }      

    }   

   

    [mapVC release];

    [mutableArray release];

}

Code snippet 2: test cases in the test class
The results of tests are shown in XCODE output window (as shown in figure 6 bolew). Tests marked with passed keyword are successful tests. It means that, for instance, expected exception was thrown, or no exception was thrown when it was not meant to, or the return value of some method are as expected and so on.

Figure 6: test results

Conclusion

Variety of tests can be implemented such as STAssertNotNil, STAssertTrue, STAssertThrows, STAssertThrowsSpecific, STAssertThrowsSpecificNamed and STAssertEquals. In addition, tests are debug-able using XCODE which is very helpful. Taking advantage of unit testing, to some extent, can guarantee the functionality of any application. Having a variety of test cases that are designed and implemented accurately can help to minimize the risk of failure. However, unit testing is absolutely not a substitute for pressure tests or tests including real users whatsoever. Unit testing is specifically useful to test the desired functionality of the application in known dangerous circumstances, where you know your code might break.

Example map view application source code (including unit tests)