Evaluation feedback

Last week I had an evaluation. All functionalities were tested by asking the test persons to run over some tasks, meanwhile using the think-aloud protocol. Afterwards they were asked to fill in a questionnaire based on the USE questionnaire. It can be found here.
Below are the results:

General
9 persons of which 7 male and 2 female.
Ages from 18 to 24.
5 master degrees, 4 bachelor degrees.
7 persons with a computer sciences background.
6 persons have a smartphone, 2 have used one before, 1 has no experience with it.

Questionnaire
The USE questionnaire can be divided into 4 factors:

• Usefulness
• Questions 1 to 6
• Overall score (on a scale from 1 to 7)  5,0555
• Ease of Use
• Questions 7 to 11
• Overall score (on a scale from 1 to 7)  5,1777
• Ease of Learning
• Question 12
• Overall score (on a scale from 1 to 7)  6,1111
• Satisfaction
• Questions 13 to 15
• Overall score (on a scale from 1 to 7)  5,2963

Positive remarks

There were some positive remarks that came up with many of the test persons:

• 7 persons mentioned having your course schedule with you is a very useful utility. Moreover, the ability to show on a map where a class is held and calculating a route to that class. 2 of these persons found it useful especially for new students, but less for the older ones.
• 5 persons found it useful to see which of your friends are around, for group assignments, course lectures, etc...
• 4 persons found the functionalities handy, having most of what a student needs in one single application, in stead of being spread out on several websites. They also found the menu easy to use, since it is limited to the main big functionalities.

Negative remarks

There were also some negative remarks though. These remarks can be associated with one or some factors:

• Design
• Sometimes the way to accomplish something is not the shortest.
• The check-in functionality was somewhat hidden. 7 persons had difficulties trying to check in.
• Users were expecting some kind of feedback when there were no courses or events in the next week.
• Functionalities
• User location is not precies enough.
• Long loading times (NOTE: the evaluations were done mostly with EDGE)
• (Reverse) geocoding an adress from the events doesn't work all the time.
• Automatically select current location as starting location.
• Users expected some search function for events.
• General smartphone remarks
• Users are not eager to tap on map markers.
• Users tend to instantly tap OK when dialogs are shown, without reading the message/instructions first.

Having analyzed the evaluation data, I already have some possible solutions in mind. But whether I will do another iteration will depend on how well and how quickly I record the study material for the exams.
Also note, that since the market release, the application was downloaded and installed 16 times, and 3 of these users have filled in a questionnaire. I will do a deeper analysis of this 'online' data later...

Market release.

Yesterday I uploaded my application to the Android Market.
It can be downloaded here.

On the page is also described what functionalities are currently supported,
and what I am still working on at the moment.

This week I am planning a last evaluation of my application.
If you are interested and want to participate you can choose a time slot on my doodle.

Status update

Last week I spent on fixing some bugs in my implementation along with making the thesis poster.
This poster can be found here.

Currently I am working on the implementation of KUL timetables in my application.
Because this was not in my paper prototypes yet, I first created a mockup/storyboard which was then reviewed by my supervisors.

So at the moment my application can do the following:

• show campus map
• search campus buildings, get building info, and locate them on the map
• get routing information to the buildings and display the route on a map
• check in at the buildings (through facebook places)
• show nearby and recent buddy check-ins and display them on a map
• search buddies and show profile and recent check-ins

Easter presentation + new planning

During the easter holidays I and the other students of our group had our second thesis presentation. My presentation can be found here.

Although less than at the presentations in december we received some helpful feedback. Most important was the further planning since we are at a crucial time in our thesis work. I myself have decided, in accordance with my supervisors, to delay my planning. This is because I have a couple of other courses that involve projects, and I will probably not get my thesis text finished in time. After all, if I have to rush it,  not only the quality of both my implementation and thesis text will suffer from it, but also all the work for other courses. So here is my new planning:

• 26/04 - 02/05: thesis poster
• 03/05 - 09/05: adding KUL timetables to the application
• 10/05 - 16/05: adding KUL events to the application
• 17/05 - 23/05: pc room reservations, favorites
• 24:05 - 30/05: evaluation
• 31/05 - 30/06: exams
• 01/07 - 15/07: thesis text ch. 2 literature study
• 16/07 - 22/07: thesis text ch. 3 analysis
• 23/07 - 05/08: thesis text ch. 4 implementation
• 06/08 - 12/08: thesis text ch. 5 evaluation
• 13/08 - 19/08: thesis text ch. 1 intro + ch. 6 conclusion

The reason I take 2 weeks for chapter 2 is to anticipate on problems for getting started, don't knowing how to get started, what to write exactly. The reason for 2 weeks for chapter 4 is because that one will be a big chapter. Furthermore, the intro and conclusion are chapters that should grow throughout writing the other chapters. The deadline is 24/08 so I have foreseen a couple of days at the end for adjusting the final text. Off course, I will try to finish the text faster than according to this planning.

Progress update

I haven't written anything on my blog for the past 2 months.
This is mainly because I am focusing on the implementation of the application, and have lost sight of my blog.

What have I done in the past 2 months?
Well first of all, I have written a scientific paper about the application. This paper can be found here.
Second, I started implementing the application for Android. The implementation will be done in 3 iterations which are each followed by an evaluation.

I have completed the first iteration along with it's evaluation. This first iteration focused mainly on the design of the graphical user interface. Almost the entire application was designed, all with hardcoded data. This design was then evaluated by 4 persons aged 18 to 25. The evaluation was done on an actual device, from the computer science department. The test persons had some time to explore the application shortly after which I asked them about their general impression. These are the results of the evaluation:

• For some screens, it was unclear that some text fields, input boxes, etc. were clickables leading them to an other screen.
• At the screens where I used tab widgets, it was unclear which tab was the selected one.
• Some screens had too many buttons, forcing users to do unnecessary extra work.



Choosing bad colors

• There was some inconsistency regarding the buttons. Since Android smartphones have a back button on the device itself, users can simply go back to the previous screen by clicking this device button. On some screens I added an extra cancel/back button, but not for all screens. The test persons advised me to either add them everywhere, or either nowhere, forcing users to use the device button.

Too many buttons

The first two remarks were mainly caused by choosing bad colors for the screens.

The last two remarks were caused by bad, inconsistent design of the graphical user interface.

Furthermore, since I only got the device shortly before the evaluations and didn't have time to test it myself, it showed that the buttons, text, etc. were too small when shown on the screen of the device.

At the moment I am working on the second iteration: fixing first iteration issues, adding a Google map, building locations, route information, a check-in functionality along with buddy locations (by integrating with Facebook, Foursquare and others). I will post more on this when I finish the second iteration and it's evaluation.

Paper: Mobile Learning: Context and Prospect

This paper is a white paper of the ELI online focus session about mobile learning.
It contains a lot of links to other useful papers, and can be a good good guide for my further literature study.

The paper can be found here.

Evaluation of second paper prototype

Below are the results of the evaluation of my second paper prototype.

# persons: 5

Age: 19 to 24

# years studying in Leuven: 1 - 5 years (1 person already graduated)

All persons followed different studies

2 persons had experience with multitouch of which 1 also with Android

Problems/suggestions (by test persons) while running scenario's:

- Problem: not clear how to switch between different map views (events, buddies, none)
- Suggestion: being able to view buddies, events on other locations (campus maps)

=> Possible adjustment: adjust the filter options, and add icons on map for quick switching between map views

General results:

Purpose of application clear?                                           8,8 / 10

Interaction with application clear?                                     8,4 / 10

Would you use application yourself?                                 8,2 / 10

Does the application have an added value for students?      8 / 10