Summary post

Human-Computer Interaction, Introductory Course DH2620

This is a summarization about our project and the steps we went through in our design iteration process towards our final product.

We are a group containing 6 members from different fields of study:

Elisabet Lövkvist- Master of Science in Engineering and in Education, communication and information technology

Sofie Borck Janeheim- Master of Science in Engineering and in Education, communication and information technology

Axel Hill Eriksson- Master of Science in Engineering and in Education, communication and information technology

Alexander Koski - Master of Science in Engineering and in Education, communication and information technology

Hanna Bohlin - Degree Programme in Media Technology

Yrin Eldfjell - Master's Programme in Computer Science, Stockholm University

Together we have designed the Enhanced Real Time Information.

We will here attach links to blog posts that describe the design process and the exercises which has helped us in each step of the project process. In the end of the post we will also attach links to our reading seminar notes that has helped us integrate theory from the literature with our design process.

Exercise 1: Field Studies, Interviews & State-of-the-art analysis

During the first exercise we decided how we will take the first steps in our project, and how to collect the data we need to proceed. This ended up in interviews and state-of-the-art analysis described in these blog posts:

• Interviews: Sofie, Hanna and Alexander, Yrin, Axel, summary of interviews
• State of the art research: Hanna, Sofie, Yrin, Axel, Elisabet, Alexander, summary of state of the art research

Exercise 2: Personas and scenarios

When we had data gathered from our route, we had to find out if there were any problems, or what we wanted to contribute with to this route. To better understand the travellers we created Personas and Scenarios and added their Pain Points, described in these blog posts:

• Personas & scenarios: Kicki, Nils

• Pain points

Exercise 3: Design, Low-Fi Prototyping

The next step in the process was to use the information from exercise 2 to come up with ideas for our design, and we used different types of brainstorming described in the first blog post under exercise 3. This resulted in our first low-fi prototype, described in the next blog post.

• Brainstorming exercises
• Parallel design that resulted in first low-fi prototype
• Second low-fi prototype in Balsamiq

Exercise 4: Evaluation

Now when we have our first prototype it is time go get feedback, so that we can improve our design. The feedback we got from our exercise group is described in the next blog post. After that we did Think-alouds with the next iteration of our prototype, and the Think-Alouds and the summarization of them are described in the following blog posts.

• Feedback from another group
• Think-alouds: Elisabet, Sofie, Alexander, Hanna, summary of Think-Alouds

Exercise 5: Design Iteration and Critique

During exercise 5 we presented our prototype, and received feedback from the group, and here are the last iterations of the prototype described after the last rounds of feedback.

• Updated prototype after feedback from think-alouds (with feedback we got after the presentation)

Exercise 6: Final Presentation

This is the last blog post and it describes our final presentation and the prototype. Thanks for following our design process and hope you enjoyed it!

• Our final prototype and presentation

Seminar notes

Here we have attached the links to our seminar notes, both as individuals and as a group.

• Seminar 1 individual notes: Elisabet, Alexander, Hanna, Axel, Sofie, Yrin
• Seminar 1 group notes
• Seminar 2 individual notes: Alexander, Elisabet, Sofie, Axel, Hanna, Yrin
• Seminar 2 group notes

Reading seminar 2 - notes (Yrin)

Note: I just realized that chap. 15 was supposed to be included in this review. I accidentally chose chapter 14 instead. Sorry about that. /Yrin

"Sometimes the findings of a field study can be unexpected."

-- Interaction Design (3rd ed.), p. 501.

As far as design work is considered, the most important reason for doing a field study is to pick up the unexpected aspects of the user vs product interactions, in my opinion.

Now, in a late-stage evaluation study you'd better hope that the product design is mostly right, but a key focus should still be to test (read: challenge) your assumptions of the design.

The course book explains the trade-offs between very controlled lab studies on the one hand, and "natural setting" field studies on the other. It's very important to keep in mind that all HCI experiments puts constraints on the subjects in the study. These constraints have to be put into place deliberately---the focus the study on whatever aspect you want to evaluate---and not be forgotten about when doing the analysis. It is equally important to select a relevant group of subjects, and to understand who these subjects are---as well as their motivations---, in order for the later analysis to be meaningful.

It is of key importance to remember that as an experimenter, you can rarely control all the variables when doing research with human subjects. Great care is needed when analyzing the results.

Reading Seminar 1 - notes (Yrin)

This reading seminar encompasses data gathering, data analysis and establishing requirements. What follows is my summary of what I consider to be the essential concepts.

There is an inherent conflict in doing data gathering and analysis in HCI design. On the one hand, you have to make choices of tools and study design that enable the gathering of the right kind of information (from the appropriate group of subjects). On the other hand (since you're designing new ways for humans to interact with technology) you often don't know exactly what you are looking for. As a result, you have to be constantly prepared to think outside the box.

Even when you think you have pinned down the design and are just doing studies to "fit the parameters", you have to be prepared for the possibility of a major design change emerging from the analysis of the data.

The course book surveys some of the most common data gathering and analysis techniques. It reminds the reader that one can't just ask the first question that comes to mind and hope to get a reliable answer. Rather one has to consider how the questions are framed and what assumptions they rely on. A point to remember, I think, is that this matters for internal design discussions just as much as actual subject interviews.

The book goes on to stress the importance of establishing requirements. The requirements have an important organizational role, in terms of being something concrete that one can reference during the design process. The merits of design decisions---inherently subjective and "loose"---can be measured by their impact in the requirements.

Notes for Seminar 2 (Hanna)

Chapter 13, Evaluation Framework

To check that the development of a product is going in the right direction it needs to be evaluated. Evaluating a product is done by asking wide and narrow questions about how the user interact with a product, for example "Do they enjoy it?" or "Will they find this menu option?".
To normalize the evaluation process and make sure important details aren't missed there has been developed a framework called D.E.C.I.D.E.. This is what each letter stands for:

1. Determine the goals
Determine and decide the goals and scope of the evaluation.

2. Explore the questions
Make the questions that will lead to the goal of the evaluation clear.

3. Choose the evaluation methods
The method depends on the type of product or stage in development process. It can for example be a usability test in a laboratory or an In The Wild study.

4. Identify the practical issues
Issues can easily be identified by doing a pilot study before the real one. The issues can for example be how to get appropriate participants for a test, knowing how long the test will be and what the participants will do, if you try to create a natural setting how would the evaluator not affect the result of behavior, where will the study be held and what equipment is needed, what does the budget allow, what kind of evaluator expertise is needed and so on...

4. Decide how to deal with the ethical issues
There are rules about how participants and their privacy needs to be protected. Sometimes there needs to be a consent form for participants to sign. The participants must be fully in the know about what data will be used for and if it will be stored or not. Guidelines to protect online users are being developed (it takes time to make guidelines).

5. Evaluate, analyze, interpret and present the data
The evaluator needs to be clear about the studie's reliability, that the same method was used throughout, its validity, that the method measured the actual issues in question, its ecological validity, that the method was carried out in a laboratory and the results can be applied to the real world or not, biases that evaluators have making them focus on certain kinds of data and affect the result, and the studie's scope which is the extent to which the results can be generalized when all real life factors cannot be taken into account.


Chapter 15, Evaluation: Inspections, Analytics and Models

These methods of evaluation can be used without having any user participants in tests, which can save time and money.

Inspections: The Heuristic Evaluation method consists of heuristics developed to be generalized over many products. Each heuristic concerns one area of usability of the product. Experts are hired to go through all the heuristics and they manage to find most usability problems with a product. Another inspection method is the walkthrough, where experts create user scenarios and walk through the use of the product together acting as users. This gives very detailed answers about users thought process.

Analytics: Analytics can be used to evaluate how for example a website works by giving the evaluators data about where users click, how long time they spend on certain pages and so on. It's a very cheap method since the data is obtained without having to hire experts, you just use a program.

Predictive models: Without trying out the product, formulas can be used to estimate information about the product. The GOMS model is used to describe step by step every action and thought a user will go through when completing a task using the product. When these are out on a paper it is easier to developers to see where trouble might occur and change the design to make usability better. The Keystroke Level Method is even more specific, for every step taken when carrying out an action with a keyboard there is an estimated time that can be added together to see what the total time for a task for a user will be. The problem with these methods are their scopes, they don't take mistakes or a users obtrusive surroundings into account.

Presentation of our final prototype

Last week we presented our final prototype for the other groups. We wanted to show the steps we've taken in the iterative design process to improve on our prototype. You can look at the presentation here.

The final prototype is here. Our last improvements were updating the light strip to indicate the direction of the train, and showing the symbols that indicate delays (green check or red x) in the list of the trains as well, so the user can see if there's a need to click to see more information or not.

For the presentation, Sofie, Yrin and Alexander prepared a video to explain the use of the app and the connection with the light strip on the information signs on the platform.

We didn't win the contest (we only got 1 vote :() but we are still happy with the outcome of our project!

Summary of our State of the Art Analyses

The apps and sites analyzed are typically intended to replace the current SL app. They often provide a subset of the standard "Travel planner" features but add some of their own, such as crowdedness estimates, train location, alarm clock or push notice updates. Sometimes they provide traffic information on "off-line" platforms such as public information displays or signs.

The user is often faced with having to learn new systems or deal with expected features being absent. (Which as a side note happened just the other day when I asked to use another group members phone to check the bus departure times. In this case the app had no "real time" information.)

There are frequent concerns about reliability of service and accuracy of the information. The alarm clock feature of "Google Now" apparently requires both working GPS and Internet connection, for example.
In general, these traffic information systems don't handle major traffic disruptions well, which is unfortunate since the information is much needed during such events.

In conclusion, despite the many travel planning apps out there, we believe there is still a need for a well designed and well-featured app. It should have a powerful UI, clear visualizations, be able to accommodate both novice and advanced users, and be using a reliable statistics engine capable of integrating real time information with historic data.

State of the Art Analysis of Pendelkollen

The Pendelkollen app
Pendelkollen is a smart-phone app developed specifically for commuters. The app is centered around a set of saved travel routes that the user specifies. Time of day and week day(s) is also set. For each such route, the app provides detailed information about departures, delays and crowdedness. This helps the user make a more informed choice as to what specific train to catch.

The app is fed information from already existing GPS and wheel-axis pressure sensors on the trains. The existing traffic information systems are used as well.[3]

Who uses the service?
The app has been available since March 2014 and received rather wide coverage in IT-related media.
However, as of November 2015, only less than 5 per cent of all search results on Google for "pendelkollen" are from the last year - somewhat suggesting that public interest faded over time. The app seems to be currently unavailable from the Android and Apple app stores. We're still investigating the cause of this.

We have not found the actual usage statistics.

What works well?
That app gives straight forward and seemingly accurate information on saved routes. The app combined with the integrated push notice system seem to work well for e.g. daily commuters. The interface is easy to use (based on looking at a video demo).[1]

What doesn't work?
The app is limited to the Pendeltåg train system, and is available only the modern X60 cars.[2] The end user has of course a need for this service on all types of public transport - including buses.

The main UI issue is that routes have to be saved in advance, which makes it cumbersome to use for one-off trips as well as re-routing a trip in the event of a delay. (It has to be stressed that we've not been able to test the actual app, so this is conjecture.)

References
[1] "http://feber.se/mobil/art/295359/hll_koll_p_pendeltgen_i_stockh/" on 2015-11-08.
[2] "http://www.pendelkollen.se/faq/" on 2015-11-08.
[3] "http://computersweden.idg.se/2.2683/1.550449/trangt-pa-pendeln-nu-far-du-hjalp-att-valja-vagn" on 2015-11-08.

Acknowledgements
Thanks to Elisabeth and Sofie for helping out with researching the app (notes summarized in an internal document).