Research Methods for Interactive Systems

Course manual 2023/2024

Course content

The main goal of the course is to illustrate the multidisciplinary nature of Information Science (which combines and integrates social sciences and ICT) through research articles, real-world examples and a research project which will be conducted by the students themselves. In particular, the course will explore the application of psychological research methods (experimental design, statistical analysis) to human-computer interaction (HCI).

The course aligns with the Human Factors theme, the Data Science theme and the AV theme on Research Methods. The course builds on aspects of human cognition and perception covered in the introductory course IK in Vogelvlucht in the first year. In addition, it builds on the first-year Analytical Methods and Techniques course and prepares students for the third-year course on Research Methods and Techniques as well as the Thesis.

The central theme of the course is how users interact with complex systems, dynamic or otherwise. We address this overarching theme through research articles from a variety of different subdomains of HCI: including but not limited to user experience with technology, voice assistant interaction, (serious) games, and human-AI collaboration.

Students deepen their statistical knowledge and skills, and conduct their own HCI research (in groups of 3-5 students). The aim is to complete all phases of the research project, from formulating a good question, to choosing and using an appropriate research method, to analysing and interpreting the data collected and finally, to reporting on the research, both orally and in writing. In addition, they gain an understanding of when and how to choose a specific appropriate research design and accompanying statistical tests. This is accompanied by a comprehensive overview of relevant concepts in research design within HCI, including reliability and validity, common constructs and questionnaires in HCI, mixed methods, and a special focus on timely challenges to the field and its methods (e.g., replication crisis concerns).

Study materials

Literature

• Literature is linked to, and the statistics materials are available on Canvas.

Objectives

• The student applies the most important concepts from literature to an example (e.g. theoretical concepts, research methods). (De student past de belangrijkste begrippen uit de literatuur toe op een voorbeeld (b.v. theoretische concepten, onderzoeksmethoden.)
• The student recognizes the conceptual basis of a number of statistical tests (knowledge). (De student herkent de conceptuele basis van een aantal statistische toetsen (kennis).)
• The student selects the correct statistical test for a research design (insight). (De student selecteert de juist statistische toets bij een onderzoeksontwerp (inzicht).)
• The student makes a well-considered assessment of the scientific value of the literature studied in order to position their own research (evaluation). (De student oordeelt op afgewogen wijze over de wetenschappelijke waarde van de bestudeerde literatuur om het eigen onderzoek te positioneren (evaluatie).)
• The student conducts research in a methodologically responsible manner. (De student voert onderzoek uit op een methodologisch verantwoorde wijze.)
• The student implements statistical methods well (apply). (De student voert statistische methoden goed uit (toepassen).)
• The student writes a research report of their own research that meets the current requirements of scientific reporting (apply). (De student schrijft een onderzoeksverslag van het eigen onderzoek dat voldoet aan de gangbare eisen van wetenschappelijke verslaglegging (toepassen).)
• The student explains their own research briefly and concisely to a broad audience (apply). (De student legt het eigen onderzoek kort en bondig uit aan een breed publiek (toepassen).)
• The student can reflect on their own contribution to the research and suggest areas for improvement for further research. (De student kan reflecteren op de eigen bijdrage aan het onderzoek en verbeterpunten aandragen voor vervolgonderzoek.)

Teaching methods

• Practicum Oriëntatie en Reflectie (POR)
• Seminar
• Lecture
• Presentation/symposium
• Self-study
• Working independently on e.g. a project or thesis

The teaching and learning methods used are lectures, seminars, assignments, poster presentations, and a group research report.

The course format is as follows: each week will feature one lecture (hoorcollege, 2 hours), one statistics seminar (werkcollege, 2 hours), and one research seminar (werkcollege, 2 hours).

Lecture (hoorcollege):

During the lectures, the academic content of the course (the required reading and associated concepts) will be presented and discussed. In order to prepare for the lecture, students should read the assigned articles (or chapters) and prepare any questions they would like to ask. In case students prefer not to ask questions during the lecture, such questions can also be sent in advance via email to the course coordinator.

Statistics seminar (werkcollege):

In this seminar, a specific statistical technique will be introduced in the context of its theoretical background, how it is applied and reported. Students will then be given the opportunity to practice applying it through an assignment using R. The material for each week (including relevant data files) will be linked to or provided on Canvas. In the final session of the statistics seminar, groups of students will also give a presentation on a relevant statistical topic.

In preparation for the statistics seminar, it is expected that students have:

• a working version of R
• studied any provided relevant materials before attending the statistics seminar each week

Success in this seminar depends on preparation. The goal is to acquire the required prior knowledge independently and to deepen and apply that knowledge during the seminar, and ask any questions that might arise.

The seminar content is assessed through two assignments and in the form of a group poster presentation. The groups are the same as in the research seminar (except in rare exceptions and with explicit permission from the tutors and the course coordinator).

Research seminar (werkcollege):

In this seminar, students will form groups of 4 students to explore a research question of their own choosing (as long as it concerns a problem relating to human-computer interaction). In rare cases, groups of 3 or 5 can be formed with permission from the tutors and the course coordinator. Each week, the tutor will guide the students in conducting a specific part of the research: choosing a theme, doing a literature search, writing article summaries, formulating a research question, deciding on an appropriate method and statistical analysis technique, collecting and analysing data, reporting, and final presentation. Each week, the groups will be required to submit the relevant section of their research report, which will culminate in a group report (along with an individual reflection written by each group member). The project will conclude with the resulting (refined) research report and a (poster) presentation.

Learning activities

Attendance

Programme's requirements concerning attendance (OER-B):

• For practical trainings and tutorials, with assignments, attendance is obligatory, unless stated differently in the course catalogue. When students do not meet the requirements for attendance, he or she cannot finish the course with a pass mark. The requirements concerning attendance for lectures/seminars, if applicable, are stated in the course catalogue.
  

Additional requirements for this course:

In accordance with OER-B (Teaching and Examination Regulations, Aanwezigheidseisen opleiding), this course sets a minimal attendance requirement of 70% for the seminars (statistics and research). When students cannot attend, they should contact their tutor prior to the session to be marked as “absence notified”. When students do not meet the requirements for attendance, they cannot finish the course with a pass mark.

Absence in the group presentations (i.e., the final statistics seminar session for the statistics group presentation, and the group research poster presentation session) requires valid proof of absence (sent to the study councilor) and a written explanation of why the student's contribution can be handled equal in grading to the group members who were present). 

Attendance in the lectures (hoorcolleges) is optional but highly recommended.

Assessment

Passing the Course:

To pass the course, the following requirements must be met

• the average of both exam grades must be 5.5 or higher
• the average grade of the seminar components (group research report, individual reflection, group research poster presentation, and group statistics presentation, individual statistics assignments) must be 5.5 or higher
• all seminar components are required, i.e., students cannot simply skip seminar components

In the event that a student does not pass the overall exam grade requirement, they can take the resit exam, the result of which is then their grade for the overall exam grade component. There is no retake option for the overall seminar exam grade requirement (except in very rare circumstances which must be handled in consultation with the study councillor).

Bonus rule:

Those groups that for the final report draft deliverable on Dec 18th submit an *actual* final report draft in the way that is intended - i.e., one that is actually only in need of polishing; an "accept with minor revisions" in peer review terms - are eligible for a 0.5 point bonus on the group research report grade component.

Criteria for the bonus:

1. The final report draft must fulfill the research paper checklist provided in Canvas in terms of content. 
2. It must be within a 10% margin of the maximum word counts listed for each section. E.g., the method section has the final maximum word count of 1250 words > the draft of this section must be between 1125 and 1375.
3. It must have a title and complete appendix. 

Late Policy:

Any assignment submitted past the due date will be graded with a 10% penalty on a sliding scale based on 24 hour blocks. For example, if it was submitted within 24 hours late, what was initially graded an 8.5 would become a 7.5; at 24+ hours late the 8.5 becomes a 6.5; at 48+ hours late the 8.5 becomes 5.5, and so on.

The group statistics presentation is an exception to this: the slides need to be uploaded on Canvas as a requirement to receive the grade for this seminar component, but the late policy does not apply as the grade refers to the presentation itself – there is no option for late presentations.

Inspection of assessed work

The manner of inspection will be communicated via the digitial learning environment.

Assignments

Statistics assignments 1 & 2

• These assignments are due at the mid-point and the end of the course and cover the first half and the second half of the course's statistics seminar, respectively. The assignments are graded. The assignment contents are closely related to the content covered in the practical parts of statistics seminar, where extensive feedback is available.

Group research report with individual reflection and poster presentation

• The research seminar is devoted to the students' own research, which will be developed in groups. Each week, the students' research project will be discussed and progressed during the seminar: choosing a theme, doing a literature search, writing article summaries, formulating a research question, deciding on an appropriate method and statistical analysis technique, collecting and analysing data, reporting, and final presentation. Each week, the groups will be asked to submit the relevant section of their research report for feedback, which will culminate in a group report (along with an individual reflection written by each group member). The final group research report is graded. The research project will also conclude with a presentation and a pitch.

Group statistics presentation

• In the final statistics seminar session, the student groups will be asked to present the information covered in an article covering a statistics-based topic. The article must be chosen from a list of candidate papers provided by the course coordinator. 

The course contains the following assignments:

• Two statistics assignments (individual)
• Research report (group) with reflection (individual)
• Poster presentation of research project (group)
• Presentation on statistical topic/article (group)

In addition, students are given multiple deadlines for submitting early drafts of the various sections of the research report, and can then receive feedback on this from their peers and the tutors.

Fraud and plagiarism

The 'Regulations governing fraud and plagiarism for UvA students' applies to this course. This will be monitored carefully. Upon suspicion of fraud or plagiarism the Examinations Board of the programme will be informed. For the 'Regulations governing fraud and plagiarism for UvA students' see: www.student.uva.nl

Course structure

Additional information

Canvas:

This course has a Canvas site, where you will find all relevant materials - including required reading! - and information, and where you can upload your assignments. Please check it regularly!

Contact information

Coordinator

• dr. K.S. Rogers

Staff

• prof. dr. S. Ben Allouch (guest lecture)
• prof. dr H. Alavi (statistics materials)
• Nazli Aydin
• A.S.M. Beuger BSc
• Abel Bunt
• Jesse Kommandeur
• Alvaro Millan Ruiz
• Tjomme Schilstra MSc