6 EC
Semester 2, period 4
5244CMLM6Y
| Owner | Master Brain and Cognitive Sciences |
| Coordinator | R. Garrido Alhama |
| Part of | Master Logic, year 1Master Brain and Cognitive Sciences, track Cognitive Science, year 1 |
This course aims to provide the student with the necessary background and methodological tools for
carrying out research in language acquisition with the use of computational models.
The course will mostly focus on modeling empirical findings in Artificial Language Learning
experiments, specially those based on speech segmentation and on generalization of structural
regularities. With the aim of developing a critical sense on model comparison, we will review the
most recent models, and we will discuss their strengths and weaknesses, as well as issues regarding
model evaluation.
In order to see how models may help resolving theoretical issues, we revisit classic (but yet largely
unresolved) debates in the field, which revolve around questions such as one vs. multiple
mechanism theories, and the nature of cognitive representations (related to the symbolic vs.
connectionist debate in other areas of language and cognition).
Finally, we will have some invited lectures to provide a broader perspective of cognitive models in
other areas, including music, sentence comprehension and distributional semantics.
https://blackboard.uva.nl/bbcswebdav/pid-6522421-dt-content-rid-10667759_1/xid-10667759_1?target=blank
https://blackboard.uva.nl/webapps/blackboard/content/listContentEditable.jsp?content_id=_6408852_1&course_id=_210002_1&mode=reset
At the end of this course the student should be able to:
• describe the state-of-the-art cognitive models of language acquisition
• critically discuss and analyze cognitive models of language, based also on their
explanatory power and their validation against the empirical data
• understand some of the most pressing research questions and theoretical debates, and
realize how models can help answering those questions
We had lectures, computer labs, final project and oral presentation.
|
Activity |
Number of hours |
|
Werkcollege |
32 |
|
Zelfstudie |
136 |
Requirements of the programme concerning attendance (OER-B):
Additional requirements for this course:
Attendance to lectures was encouraged by having a percentage of the grade based on comments and questions.
| Item and weight | Details |
|
Final grade | |
|
0.6 (60%) Open Project | |
|
0.1 (10%) Forum Questions | |
|
0.2 (20%) Computer Labs | |
|
0.1 (10%) Questions for Lecturers |
This is part of the information that I provided to the students, regarding evaluation:
The course will consist on content lectures by the main lecturer (Raquel G. Alhama), guest lectures
by invited speakers, and computer labs.
Attendance to the lectures is expected, especially for the guest lectures (please contact me if you
have problems of overlap with other courses). During the guest lectures, you should try to come up
with relevant questions or comments for the invited lecturer. This will be part of your final grade
(10%).
Each week you will have to read scientific papers that have some relation with the content of the
lectures. A forum will be made available on Blackboard, where you will have to post your questions
and/or comments about the papers. The expected activity is one question/comment or one reply to
another question/comment each week, for each suggested paper. We will grade the relevance and
originality of your contributions (10%).
The computer labs take place is a room with electricity sockets to plug your laptops (so please bring
them to the sessions). The assignments require the use of Unix commands and of Python. Therefore,
you will need a shell to run Unix commands. We recommend to use Linux; if you do not want to
have an installed Linux in your computer, the easiest option is to create a Live USB drive. We will
provide instructions on how to do this before the first lab session.
The computer labs will be based on tutorials and exercises to perform during the session, and
individual assignments that you will hand in before the next session. The grade of these assignments
will amount to a 20% of the course evaluation. If any other assignments are suggested during the
lectures, the grade will also be included in this 20%.
Finally, there will be an open project, which you will do in small groups (2 to 3 people). The idea is
that you choose an existing modeling paper, and (1) you present it to the rest of the class, (2) you
replicate the modeling work (and possibly extend it), and (3) you write a technical report about your
work and your findings. The final project will amount to 60% of the grade, with 20% attributed to
each of the abovementioned steps.The course will consist on content lectures by the main lecturer (Raquel G. Alhama), guest lectures
by invited speakers, and computer labs.
Attendance to the lectures is expected, especially for the guest lectures (please contact me if you
have problems of overlap with other courses). During the guest lectures, you should try to come up
with relevant questions or comments for the invited lecturer. This will be part of your final grade
(10%).
Each week you will have to read scientific papers that have some relation with the content of the
lectures. A forum will be made available on Blackboard, where you will have to post your questions
and/or comments about the papers. The expected activity is one question/comment or one reply to
another question/comment each week, for each suggested paper. We will grade the relevance and
originality of your contributions (10%).
The computer labs take place is a room with electricity sockets to plug your laptops (so please bring
them to the sessions). The assignments require the use of Unix commands and of Python. Therefore,
you will need a shell to run Unix commands. We recommend to use Linux; if you do not want to
have an installed Linux in your computer, the easiest option is to create a Live USB drive. We will
provide instructions on how to do this before the first lab session.
The computer labs will be based on tutorials and exercises to perform during the session, and
individual assignments that you will hand in before the next session. The grade of these assignments
will amount to a 20% of the course evaluation. If any other assignments are suggested during the
lectures, the grade will also be included in this 20%.
Finally, there will be an open project, which you will do in small groups (2 to 3 people). The idea is
that you choose an existing modeling paper, and (1) you present it to the rest of the class, (2) you
replicate the modeling work (and possibly extend it), and (3) you write a technical report about your
work and your findings. The final project will amount to 60% of the grade, with 20% attributed to
each of the abovementioned steps.
10% grade based on questions and comments asked in lectures. Individual, with feedback.
10% grade based on comments and questions over the literature, through the discussion board in blackboard. Feedback was given during the lectures.
20% grade was based on assignments from the computational lab sessions. Individual. Feedback was given throughout the sessions, although unfortunately we were quite late providing feedback for the evaluation of the assignements (preparing the course materials consumed much time).
60% of the grade was based on a final project, which consisted on a replication of an existing computational model (with encouraged but not compulsary extension). The students worked in groups for coding and developing the project, and gave group presentations, but they handed it individual reports, which allowed for better assessment.
Onderstaande opdrachten komen aan bod in deze cursus:
The 'Regulations governing fraud and plagiarism for UvA students' applies to this course. This will be monitored carefully. Upon suspicion of fraud or plagiarism the Examinations Board of the programme will be informed. For the 'Regulations governing fraud and plagiarism for UvA students' see: www.uva.nl/plagiarism
| Weeknummer | Onderwerpen | Studiestof |
| 1 |
Introduction to Cognitive Modeling; Modeling Music Cognition 08/02 |
|
| 2 | Modeling Music Cognition (ctd.); Modeling Speech Segmentation Readings: Choose one of these two: ◦ Large, E. W., Herrera, J. A., & Velasco, M. J. Neural networks for beat perception in musical rhythm. Frontiers in systems neuroscience, 2015, 9, 159. ◦ Jacoby, N. and McDermott, J. H. Integer Ratio Priors on Musical Rhythm Revealed Cross-culturally by Iterated Reproduction, Current Biology, 2017 And also: ◦ Saffran, J. R.; Aslin, R. N. & Newport, E. L. Statistical learning by 8-month-old infants Science, American Association for the Advancement of Science, 1996, 274, 1926-1928 Lectures: 14/02 Guest Lecture by Bastiaan van der Weij (ILLC, University of Amsterdam): Cognitive Modeling of Rhythm Perception Guest Lecture by Carlos Vaquero (ILLC, University of Amsterdam): Introduction to Music Cognition and Computational Models (ctd.) Goals of Modeling; How to validate models15/02 Computer Lab 2: Introduction to Python. Computing Transitional Probabilities.Modeling Music Cognition (ctd.); Modeling Speech Segmentation Readings: Choose one of these two: ◦ Large, E. W., Herrera, J. A., & Velasco, M. J. Neural networks for beat perception in musical rhythm. Frontiers in systems neuroscience, 2015, 9, 159. ◦ Jacoby, N. and McDermott, J. H. Integer Ratio Priors on Musical Rhythm Revealed Cross-culturally by Iterated Reproduction, Current Biology, 2017 And also: ◦ Saffran, J. R.; Aslin, R. N. & Newport, E. L. Statistical learning by 8-month-old infants Science, American Association for the Advancement of Science, 1996, 274, 1926-1928 Lectures: 14/02 Guest Lecture by Carlos Vaquero (ILLC, University of Amsterdam): Introduction to Music Cognition and Computational Models (ctd.) Goals of Modeling; How to validate models15/02 Computer Lab 2: Introduction to Python. Computing Transitional Probabilities. |
|
| 3 |
Segmentation and Generalization
Marcus, G.; Vijayan, S.; Rao, S. & Vishton, P. Rule learning by seven-month-old infants Science, American Association for the Advancement of Science, 1999, 283, 77-80 Peña, M.; Bonatti, L.; Nespor, M. & Mehler, J. Signal-driven computations in speech processing. Science, American Association for the Advancement of Science, 2002, 298, 604-607
Types of Models Modeling Segmentation in Artificial Language Learning
Computer Lab 3: Training segmentation models. Finding the best fit to empirical data.
|
|
| 4 |
Generalization with Neural Networks
Marcus, G. F. Rethinking eliminative connectionism. Cognitive psychology, 1998, 37, 243-282 Pinker, S., & Ullman, M. The past and future of the past tense. Trends in Cognitive Science, 2002, 6 (11), 456-463
Generalization with Neural Networks. Overview of existing models and the representational debate.
Computer Lab 4: Introduction to Numpy. Recurrent Neural Networks for Language. |
|
| 5 |
Readings: Frank, S.L., Monaghan, P., & Tsoukala, C. Neural network models of language acquisition and processing. To appear in Human Language: from Genes and Brains to Behavior. The MIT Press. Demberg, V. & Keller, F. Cognitive models of syntax and sentence processing. To appear in Human Language: from Genes and Brains to Behavior. The MIT Press.
Guest Lecture by Dieuwke Hupkes (ILLC, University of Amsterdam): Hierarchical compositionality in recurrent neural networks
Modeling Sentence Comprehension with Recurrent Neural Networks (title T.B.C)
8/03 Computer Lab 5: Recurrent Neural Networks for Music. |
|
| 6 |
Distributional Semantics
Turney, P.; Pantel, P. From frequency to meaning: Vector space models of semantics Ákos Kádár, Grzegorz Chrupała and Afra Alishahi. 2015. Linguistic Analysis of Multi-modal Recurrent Neural Networks. EMNLP Vision and Language workshop.
Introduction to distributional semantics.
Representations in visually grounded neural models of text and speech.
Student presentations: present a paper that will be used for replication in final project.
|
|
| 7 |
Readings: Baroni, Marco; Dinu, Georgiana; Kruszewski, Germán. Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of the 52nd annual meeting of the association for computational linguistics, 23–25 June 2014, Baltimore, MD, Vol. 1 (pp. 238–247). Stroudsburg, PA: Association for Computational Linguistics.
Pereira, Francisco, et al. A comparative evaluation of off-the-shelf distributed semantic representations for modelling behavioural data. Cognitive Neuropsychology, 2016, vol. 33, no 3-4, p. 175-190.
A computational approach to the emergence of meaning in online communities.
Computer Lab 6: work on your own project. |
|
| 8 |
See
minimum of 10, maximum of 24.