HIVK03323U System Evaluation and User Studier, Constituent elective module A

Volume 2018/2019
Education
Content

The module includes advanced studies of the theoretical and methodological aspects concerning evaluation of interactive information systems in context, the emphasis being on the correlation between information needs, situation and relevance as well as search behaviour seen in the light of dissemination processes and associated technologies.  Taking its starting point in the research literature, the module introduces models for information behaviour and information interaction in order to identify approaches to evaluation of system performance and usability, and explicitly to discuss methods for evaluation of interactive information retrieval systems.

The module consists of the following three sub-courses and a project seminar:

Information behaviour

  • Basic information behaviour models and interactive information retrieval research approaches
  • Types of information needs, including work task related needs, everyday needs, information mapping and fulfilment of needs including usability
  • The concept of relevance and evaluation in information seeking & behaviour,
  • The concept of information value, affective responses and user experiences
  • Digital behavioural observations and traces and their interpretation

 

Data collection and analysis

  • Research focus: behavioural studies vs. performance tests
  • Research design – validity & reliabilty of the test instrument, including constructs and test variables
  • Data collection methods: qualitative methods and statistical methods of analysis

 

Evaluation of interaction design

  • System centric evaluation; Cranfield-TREC-INEX; Performance measures
  • User centric evaluation; usability evaluation and analysis
  • Validation and reporting of test results
Learning Outcome

Competence objectives for the module

The objective of the module is to provide the student with:

Knowledge and understanding of:

  • Different types of evaluation and evaluation approaches of relevance to information searching and assessment of the performance of information retrieval systems,
  • Key concepts and models on information behaviour and search/retrieval and the associated processes and functions as well as methods for studying these.

 

Skills in:

  • Reflect on key models and methods for the study of information be-haviour and search/retrieval and the associated processes and func-tions,
  • Assess relevant methods, and combinations of these, for the study/evaluation of system performance and information behaviour – including evaluation of interactive information retrieval systems,
  • Identify and compare key evaluation concepts and methods for sys-tem evaluation and user studies, and to assess and validate evalua-tion results in relation to given circumstances.

 

Competences in:

  • Prepare/design research methods for studying/evaluating system per-formance and information behaviour – including evaluation of interac-tive information retrieval systems,
  • Conduct various types of system evaluations and user studies.

 

Academic objectives:

Students are able to

  • Give an account of various types of evaluation and evaluation ap-proaches as well as key concepts and models of information behav-iour and search/retrieval, and also the associated processes and func-tions as well as methods for studying these,
  • Assess relevant methods, and combinations of these, for the study/evaluation of system performance and information behaviour,
  • Independently conduct different types of system evaluations and user studies.
Literature

Sample literature:

  • Ingwersen, P. & Järvelin, K. (2005). The Turn: Integration of Information Seeking and Retrieval in Context. Berlin/Heidelberg : Springer. 448 s.
  • Kelly, D. (2009). Methods for Evaluating Interactive Information Retrieval Systems with Users. Foundations and Trends in Information Retrieval, 3(1-2), 1-224.
  • Lazar, Feng & Hochheiser (2010): Research Methods in Human-Computer Interaction. Chichester, Wiley, 426 s.
  • Saracevic, T. (2009). Information Science. In: Bates, M.J. & Maack, M.N. (eds.). Encyclopedia of Library and Information Science. New York : Taylor & Francis. 2570-2586.
Class teaching, lectures, student presentations, workshops and project seminar.
  • Category
  • Hours
  • Class Instruction
  • 56
  • Exam
  • 90,5
  • Exam Preparation
  • 264,3
  • Total
  • 410,8
Oral
Collective
Continuous feedback during the course of the semester
Credit
15 ECTS
Type of assessment
Oral examination, 30 min
Exam language: Danish or English
See Academic Regulations for the Masters program in Information Science and Cultural Communications http:/​/​hum.ku.dk/​uddannelser/​aktuelle_studieordninger/​informationsvidenskab-og-kulturformidling/​informationsvidenskab_og_kulturformidling_ma
Aid
All aids allowed
Marking scale
7-point grading scale
Censorship form
External censorship
Exam period

Summer Exam 2019

Re-exam

Same as ordinary exam, August 2019