Language, Speech and Dialogue Processing

6 EC

Semester 2, periode 4

5082LSDP6Y

Eigenaar Bachelor Kunstmatige Intelligentie
Coördinator dr. Tom Lentz
Onderdeel van Minor Kunstmatige Intelligentie, jaar 1Bachelor Kunstmatige Intelligentie, jaar 3Bachelor Bèta-gamma, major Kunstmatige Intelligentie, jaar 3Bachelor Future Planet Studies, major Kunstmatige Intelligentie, jaar 3

Studiewijzer 2017/2018

Globale inhoud

An overview of the structure of dialogue and communication and possible formalisations will be given at the start of the course. This overview will include prosodic and syntactic markers of information structure and formal modelling of the pragmatics of dialogue. In other words: how do humans in a dialogue express and mark information in order to reach succesful communication (querying the knowledge of interlocutors, correcting incorrect beliefs, responding to information requests) and how can computational agents mimick these skills? Due to the wide nature of possible applications, the course takes the form of a project, allowing students to explore one aspect of the course topic in depth. The following possible projects will first be introduced by the teacher to the whole class, after which students will work on them in (small) groups. Students can also propose variations.

Possible projects:

  1. Programming / extending a chatbot with dialogue building properties.
  2. Studying and improving automatic annotators of the information exchange in a corpus of spoken dialogue
    Students will work on the project in a group, but write an individual essay that starts from the results of their project.

Studiemateriaal

Literatuur

  • Articles, available through library / Canvas

Leerdoelen

Students can recall, review and summarize basic concepts/tenets of:

  • information structure,
  • prosody,
  • syntax,
  • pragmatics, and
  • interfaces between information structure / pragmatics on the one hand and prosody / syntax on the other hand and are able to expand their knowledge by finding and reading relevant literature.

Building on the concepts mentioned above, students are able to:

  • evaluate state-of-art algorithms on speech/language processing, both on performance and as scientific/ linguistic
    models;
  • use (one of) these algorithms and expand them to dialogue processing;
  • test and criticize an algorithm, describing their limitations and identify how they could be solved, or why they are
    tenacious;
  • write a formal processing tool (algorithm), applying linguistic understanding of speech and logic/epistemic
    understanding of dialogue; and
  • evaluate their own addition to an algorithm in both performance and on linguistic quality (did the algorithm capture
    or yield new understanding of the linguistic structure of dialogue?)

Onderwijsvormen

  • Werkcollege
  • Laptopcollege
  • Zelfstudie
  • Zelfstandig werken aan bijv. project/scriptie
  • Presentatie/symposium

Aanwezigheid

Aanwezigheidseisen opleiding (OER-B):

  • Voor practica en werkgroepbijeenkomsten met opdrachten geldt een aanwezigheidsplicht. De invulling van deze aanwezigheidsplicht kan per vak verschillen en staat aangegeven in de studiewijzer. Wanneer studenten niet voldoen aan deze aanwezigheidsplicht kan het onderdeel niet met een voldoende worden afgerond. .

Toetsing

Onderdeel en weging Details Opmerkingen

Eindcijfer

25%

Test basic knowledge

during class, third week

15%

Homework (lab sessions in groups)

weekly

40%

Project final report

individual

20%

Project execution & presentation

group

Your final grade is the weighted average of three grades. Homework grades reflect participation in the course.

Homework: per group
• Group composition, ideally: someone who knows Praat, someone who knows SVM, someone who speaks (a
bit) of Dutch (for corpus). Three people per group.
• Presentation: either propose a date your group would like to present / a session for presentations will be
planned.

Inzage toetsing

De manier van inzage wordt via de webpagina van het vak gecommuniceerd.

Fraude en plagiaat

Dit vak hanteert de algemene 'Fraude- en plagiaatregeling' van de UvA. Hier wordt nauwkeurig op gecontroleerd. Bij verdenking van fraude of plagiaat wordt de examencommissie van de opleiding ingeschakeld. Zie de Fraude- en plagiaatregeling van de UvA: www.uva.nl/plagiaat

Weekplanning

Schedule
Note: schedule can be adapted to fit presentations

Week 1
Lecture: what is parsing (refresh), what is speech recognition (refresh), where does prosody come in?
Lab: the parsing of ambiguous sentences. Parser certainty?
Homework: brief report with examples - lab session used to install Anaconda/Jupyter
Reading: Bod 1998, Spoken Dialogue Interpretation with the DOP Model; continue reading next week’s material!

Week 2
Lecture: Pitch contours. What is highest pitch? Downstep? Duration? How to identify region of interest?
Reading: Hirschberg 2004, Vallduví & Engdahl 1996.
Lab: Practice on dataset of elicited prominence markings. Think of prosodic features and extract them with a Praat
script.

Week 3
20-2: Test on basic knowledge: all material discussed in the first two lectures
23-2: Lab: SVM on data extracted in Lab session 2: does it work? Machine learning with given labels.
No extra reading

Week 4
27-2: Lecture: Dialogue structure and AIML/chatbots
2-3: Lab: continue 3 or start 4 (chatbot)
Literature: Polanyi et al. (2003) (Dialogue structure); Krifka (2007) (Information structure)

Week 5
6-3: Guest lecture Julian Schlöder on belief grounding
9-3: Lab: chatbots continuation OVIS or CGN; is annotation of boundaries and accentuation enough to identify
topic/focus-shifts and possibly improved boundary / attachment detection.
Literature: Krifka (2007) (again); Schlöder & Lascarides (to appear)

Week 6
13-3: Guest lecture Sharon Gieske on Alignment / style accommodation
16-3: Lab: start on your projects: identify possible improvements of algorithms that you would like to implement
(hand in for feedback on possible grade)
Literature: Pickering & Garrod (2004)

Week 7
20-3: Presentations (10 minutes per group)
23-3: Lab: help with own project

Deadline Hand in project: 4 April 2018

Rooster

Het rooster van dit vak is in te zien op DataNose.

Aanvullende informatie

References


Bod, R. (1998). Spoken dialogue interpretation with the DOP model. In Proceedings of the 17th International Conference on Computational Linguistics, volume 1, pages 138–144, Stroudsburg, PA. Association for Computational Linguistics.
Giles, H. and Ogay, T. (2007). Communication Accommodation Theory. In B. B. Whaley & W. Samter (eds.),
Explaining communication: Contemporary theories and exemplars, pages 293-310. Mawhah, NJ: Erlbaum. Available online: http://doc.rero.ch/record/306556/files/2007_gilesogay_cat.pdf
Hirschberg, J. (2006). Pragmatics and Intonation, chapter 23, pages 515–537. Wiley- Blackwell, Malden, MA. (preprint available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.135.7810&rep=rep1&type=pdf)
Krifka, M. (2007). Basic notions of information structure. Interdisciplinary Studies on Information Structure, 6:13–56.
Pickering, M. and Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27(2):169–190.
Polanyi, L., Berg, M. v. d., and Ahn, D. (2003). Discourse Structure and Sentential Information Structure. Journal of Logic, Language and Information, 12:337–350.
Schlöder, J. and Lascarides, A. (to appear). Understanding focus: Tune, placement, and coherence. Available online:  http://homepages.inf.ed.ac.uk/alex/papers/intonation.pdf 
Vallduví, E. and Engdahl, E. (1996). The linguistic realization of information packaging. Linguistics, 34(3):459–519. 3

Contactinformatie

Coördinator

  • dr. Tom Lentz