6 EC
Semester 1, period 1, 2, 3
5294DASP6Y
| Owner | Master Information Studies |
| Coordinator | dr. Lise Stork |
| Part of | Master Information Studies, year 1 |
This project stretches over the whole first semester and will let students from both tracks work together so that you can apply gained knowledge to solve a complex problem in a real world project. The Project is founded on two pillars:
1. Experience and understand the creative process of developing an interaction environment as part of research into complex systems, with a particular focus on stakeholder research, user-research, data identification, context mapping, interaction design from agile development to a technologic prototype, and evaluation (validation).
2. Stimulating personal & professional leadership, via activities that improve team building and project management skills, and activities that contribute to one's intellectual development, autonomy and employability. These activities are either organised by the students themselves, or are offered in the form of workshops. The aim, of this course is also to introduce students to a rigours application of academic skills, such as research question formulation, experiment design, and evaluation.
Within the 20 weeks of this projects the students work in groups of not more than 5 students and design, implement, and evaluate the interactive prototype for a complex application. The overall structure of each project is ideation (September - October), prototyping (November - December) and implementation / evaluation (January).
The application has to fulfill the requirements provided by the client, where the focus lies on the finding of a multi-disciplinary solution by
This course offers various types of working methods
This is the main working mode in this project. Depending on the arrangement of the level of cooperation between groups within a project, groups have to organise their work independently but are helped by the supervisor. This includes planning meetings with stakeholders, design and implementation sessions, preparing the final material that will be graded, etc.
There will be 2 workshops organised by INFORMAAT.
The address is: Seinstraat 32 - 1223 DA Hilversum (top floor) .... roughly 15 minutes walk from the Train Station.
The first workshop will address methods to help you during the ideation phase. This will be held between September/October. The second workshop helps you to better cope with prototyping. This workshop will be held between November/December (dates to come).
5 lectures are planned. The aim is to provide enough pointers to further help you to work structurally in your project.
The first lecture provides you with an overview on why scientific research is useful, how it is embedded in human thinking, and how it can be integrated into project work. The second and third lecture addresses quantitative research methods. The fourth and fifth lecture covers qualitative research methods.
Each group has a supervisor assigned. During the ideation and prototyping phases, meetings should be arranged twice per block, where each group gets an adequate time for discussion. In January, groups will get a supervision session of every Monday and Friday.
Activity | Hours | |
Presentatie | 8 | |
Werkcollege | 130 | |
Self study | 30 | |
Total | 168 | (6 EC x 28 uur) |
In TER part B of this programme no requirements regarding attendance are mentioned.
| Item and weight | Details |
|
Final grade | |
|
10% Symposium day - Submit research poster, demo video, and teaser video. | Mandatory |
|
65% Final report | Mandatory |
|
15% Presentations | Mandatory |
|
10% Team management | Mandatory |
Grades will be announced to students via an email.
Students are assigned to projects in groups of 4-5 students.
The complete set of contributions will be evaluated as such:
Final report: 65%
Presentations: 15%
Team management: 10%
Symposium deliverables: poster, demo, teaser. 10% (all components contribute equally)
Each student has to complete the course with at least a 5.5.
DETAILS FINAL REPORT:
The final report (group assignment) covers:
The report should be max 12 pages (all inclusive, excluding references) in the 2-columns ACM format (https://www.overleaf.com/latex/templates/association-for-computing-machinery-acm-large-2-column-format-template/qwcgpbmkkvpqLinks to an external site.).
Below you can find a detailed description of what these components should contain. However, they are just guidelines. You're welcome to discuss with your supervisor different ways to structure the report.
The report starts with an abstract: a summary of the paper, in which you concisely present the problem and why it's relevant, the target solution, the method applied and of the results.
1) Introduction: Generally to catch the attention of the reader you start from the motivational aspect. Overall you must let the reader wonder: why do we care about the problem, and what would come from solving it? In general this part should hint both to a societal and/or business gaps (eg. in terms of needs, of an organization and of society) and to a research gap (eg. in terms of what exists in the literature and current practices). If the problem isn't obviously interesting from a scientific point of view, but yours has potential of impact, it might be better to put social/business gap first; but if your work is incremental progress on a problem that is widely recognized as important, then it is likely better to put the research problem statement first, to indicate which piece of the larger problem you are breaking off to work on. Pay attention to convey the importance of your work, the difficulty of the area, and the impact it might have if successful. You should refer literature relevant for the reader to understand why the findings of the paper are an advance on the knowledge in the field.
After you motivate the reader with preliminaries/background and put your work in context (citing relevant literature), you can state your objectives clearly. You need to give a concise but precise problem statement: What problem are you trying to solve? Start from the requirements elicited from the stakeholders, to a broader view on societal impact, and narrow down to what actually you've been working on. What is the scope of your work (is it a generalized approach, or aiming for a specific situation)? Once you identify the problem (and ideally relate it to research questions), you provide an overview of the chosen approach, of how it has been applied/developed; you can also provide a very short anticipation of the solution, possibly very briefly justifying why it was chosen to address the problem, and how you validated it.
In many papers, one or several major conclusions of the work (the contributions of the paper) are presented at the end of this section, so that the reader knows from the start the major answers to the questions just posed that the document will elaborate upon.
2) Related Work, your work is grounded in earlier work done by others and may be alternative to other works. That means you use techniques, and methods; ways of measuring and evaluating; datasets; terminology; that were used by others before you are working on the same or a related problem. You may want to compare your results with those of others, in particular with the state of the art (SOTA). Besides, informing the reader on the state of the problem and the solutions shows that you know what you are talking about, and that you use the best there is available (or at least, the best for the conditions you have to operate with). In this section, (a) you refer and describe technologies/methodologies that you will use for answering your RQs, (b) you describe how other people have dealt with similar problems as you are dealing with (and the pros and cons of those solutions). Provide as much as possible concise comparisons with respect to your solution/approach.
3) Material and Methods: As we have repeated in several occasions, a data systems project concerns several dimensions: requirement elicitation, data collection, data cleaning/pre-processing, inference (data processing), interface (visualization/controllability), outcome/impact, and, at a meta-level, team management. The report should provide relevant elements on all of these dimensions, of how they have been approached and with what results. Amongst these there is an experimental core component (generally consisting of some data collection, data exploration, internal validation of processing, and external validation of interface), on which you'll have to focus further.
In this section, after providing an overview on how the project has been approached on all dimensions, you provide all the detail required to repeat your experiments (link to dataset, to code, describe the system architecture, the inferential pipeline, testing, etc.). Be precise in describing measurements and include errors of measurement. So for instance, in the description of the data subsection, you paste your most insightful graphs from your EDA, next to the basic statistics on your dataset and descriptions (statistical and/or a population density diagram) of your variables. Moreover, you give all the settings used in your experiments. Describe eg. the data preprocessing steps used, hyperparameter settings (for all of them), how you created a train-validate-test split or otherwise did your training and testing. More importantly, exactly how the used metrics are calculated (think of the difference between micro and macro F1 for classification, and how often this is not explicitly stated). If you want or need to tell more, consider using the Appendix or a reference to a nicely structured notebook in which all experiments are done. Pay attention to highlight all changes that you add to existing methods.
4) Results: in this section you present your findings, display items (figures, and tables). Present the data, digested and condensed, with important trends extracted and described. Because the results comprise the new knowledge that you are contributing to the world, it is important that your findings be clearly and simply stated. The results should be short and clear. Do not say "It is clearly evident from Fig. 1”, however, don't be too concise. Readers cannot be expected to extract important trends from the data unaided. Few will bother. Combine the use of text, tables, and figures to condense data and highlight trends, focusing in particular to the findings which are relevant to your conclusions. In doing so be sure to refer to the guidelines for preparing tables and figures below. Recommendation: structure your section such that the reader should only read these two things, and can safely skip all else: a) the question and, b) the table/figure and the caption (elaborated so that it answers the question). No need to explain things in words that are already evident from your table (don't be pedantically descriptive, but meaningful). The reader should use 100% of her brain to understand the outcomes, not to try to figure out what was meant.
6) Discussion: In this section you focus on the reflection and justification on design decisions, alignment of the solution to the needs of the stakeholders, limitations. You should discuss what principles have been established or confirmed; what generalizations can be drawn; how your findings compare to the findings of others or to expectations based on previous work; and whether there are any theoretical/practical implications of your work. When you address these questions, it is crucial that your discussion rests firmly on the evidence presented in the results section, so refer briefly to your results to support your discussion statements. Do not extend your conclusions beyond those that are directly supported by your results. Some speculative text about what your results may mean (in particular w.r.t. hypotheses for why things did not work as expected) is appreciated but should not form the bulk of the discussion. On the other hand, be sure to address the objectives of the study and to discuss the significance of the results. Don't leave the reader thinking "So what?" It is also good to show reflection on what are the implications of your answers. Is it going to change the world (unlikely), be a significant "win", be a nice hack, or simply serve as a road sign indicating that this path is a waste of time (all of the previous results are useful). Dedicated insights should be given on whether your results are general, potentially generalizable, or specific to the particular case you have worked upon.
7) Conclusion/Future work. In your conclusions, address the following: a) reach conclusions about the initial objectives (for this reason we call the section "Conclusions"), b) show advantages of your method over previously published methods, c) state open problems, cases your method does not or insufficiently work, d) identify needed next steps in research on the problem. If the last two points make up more than one paragraph, put them in a separate section entitled "Future Work".
8) Appendix. Appendices contain information in greater detail than can be presented in the main body of the paper, but which may be of interest to a few people working specifically in your field. Only appendices referred to in the text should be included.
The report should be 12 pages max. (excluding appendix and refereces) in ACM Paper Format.
DETAILS SYMPOSIUM DELIVERABLES:
During the Symposium Day on the 31st of January, each group will present the final results of their project work in the form of a poster and video session. For this, each group is to prepare and submit:
a 2-3 minute teaser video. Please upload your teaser video on youtube (public, not indexed) and provide here the link (only one person per team). The pitch/teaser video is meant to provide a 2 minutes presentation of all your project (eg. motivation, requirements, method, solution, validation) (please put subtitles to the video too as there will be no audio!). You can see a few previous examples here: /courses/45978/pages/examples-of-outputs-from-previous-projects?module_item_id=2142963
All these together (30 points) count 10% toward the final grade.
DETAILS TEAM MANAGEMENT:
Team management counts for 10% towards your final grade, and is measured by:
How was the overall working attitude of the group?
Did the group take initiatives by themselves to carry out the project?
How did the group organise/plan work?
Did the students actively participate in work discussions?
How was the cooperation of group members during the research?
DETAILS PRESENTATIONS:
After each phase ( ideation, early-prototyping, and implementation) each group has a meeting with the supervisor and stakeholders to discuss their progress via a presentation (10min + 5min Q&A).
DETAILED GRADING RUBRICS:
|
FINAL REPORT |
Percentage |
| Exploration Successful exploration and utilization of a set of strategies in response to the problem or assignment. Is there an explanation of why the chosen approach is appropriate (based on discussions with experts or a literature review)? Is it shown that the chosen strategy for the solution is performed properly? Was the knowledge needed to carry out the project properly acquired? Is the methodology appropriate and informed by the state of the art? |
14 % |
| Reflection Reflection and justification on design decisions. Does the report show a critical reflection of the pros and cons of the decisions made? Are arguments for decisions made sufficiently grounded? If applicable, was the relevance for society well recognised (technological aspects, ethical aspects, historical context, or environmental aspects). Is the description of the context readable for a non-expert in the field? |
14 % |
| Validation Validation user-experience of the prototype. Are the validity problems of the study accurately addressed? Is the method described appropriately? |
24 % |
| Sustainability and technical quality Sustainability/feasibility and technical quality of the final product + discussion. Is the chosen approach easily extendable and generalisable? Would the approach chosen work in a daily setting? Are the pros and cons of the chosen method discussed and is it compared with other alternatives? Does (part of) the prototype seem reusable (e.g., Github page with README.md, some form of unit testing)? Is there an available running demo? |
24 % |
| Alignment Alignment of the solution to the needs of the stakeholders. Is the problem of the stakeholder described? Are requirements specified? Is the alignment of the solution with the stakeholder's need properly described in teh discussion? |
24 % |
|
TEAM MANAGEMENT |
Percentage |
| Overall working attitude of group - How was the overall working attitude of the group? | 20% |
| Initiative - Did the group take initiatives by themselves to carry out the project? | 20% |
| Organisation/planning - How did the group organise/plan work? | 20% |
| Participation - Did the students actively participate in work discussions? | 20% |
| Cooperation - How was the cooperation of group members during the research? | 20% |
| POSTER | Excellent | Above average | Below average | Percentage |
| Content | Important information is readily available and easy to grasp | Most relevant information is available but relations are difficult to establish | Relevance of information is unclear | 40% |
| Poster organisation and response to questions | Layout captures interest, information is logically placed, visuals are used effectively, organisational information (e.g. logos, group member names, etc.) provides a professional feeling. Fully informative and to the point |
Layout captures interest somewhat, Somewhat informative and to the point |
There is some layout, requires effort Not informative and not to the point |
40% |
| Administrative details | Relevant information is present (names, group ID, UvA logo, stakeholder logo, etc). | Relevant information is missing |
20% |
| TEASER VIDEO | Excellent | Very good | Sufficient | Insufficient | No marks | Percentage |
| Quality of the produced teaser video | Video gives clear and concise summary of the project results, challenges and solutions well identified OR video provides a clear demonstration of results AND video visually attractive | 100% |
| DEMO VIDEO | Excellent | Very good | Sufficient | Insufficient | No marks | Percentage |
| Quality of the produced demo video | Video gives clear and concise summary of the working prototype AND video visually attractive | 100% |
The 'Regulations governing fraud and plagiarism for UvA students' applies to this course. This will be monitored carefully. Upon suspicion of fraud or plagiarism the Examinations Board of the programme will be informed. For the 'Regulations governing fraud and plagiarism for UvA students' see: www.student.uva.nl
| Weeknummer | Onderwerpen | Studiestof |
| 1 | ||
| 2 | ||
| 3 | ||
| 4 | ||
| 5 | ||
| 6 | ||
| 7 | ||
| 8 | ||
| 9 | ||
| 10 | ||
| 11 | ||
| 12 | ||
| 13 | ||
| 14 | ||
| 15 | ||
| 16 | ||
| 17 | ||
| 18 | ||
| 19 | ||
| 20 |