A natural language processing framework for automated assessment

Allott, NM, 2000. A natural language processing framework for automated assessment. PhD, Nottingham Trent University.

Preview

Text 10290306.pdf - Published version Download (45MB) | Preview

Abstract

A novel knowledge representation schema for application in natural language processing is presented along with the algorithms for its automatic population from text corpora. The natural language understanding task under consideration is the task of the automated assessment of students' single sentence responses to technical questions. This knowledge representation schema shares much in common with a localist connectionist network and consequently possesses both representational and computational properties. The task of automated assessment is used as an empirical framework within which the performance of the knowledge representation schema can be evaluated. An initial experiment tested the ability of hand crafted knowledge structures to effectively encapsulate a correctness decision procedure. A correlation of 85% with the performance of an independent human marker was achieved. A second experiment tested the knowledge schema's ability to generalise to new unseen data; this correlated 65% with human performance. To address the issue of knowledge base creation two algorithms are presented which produced networks consisting of composite and clustering node-t3rpes. These activation-passing networks provide a perceptual function generating higher order descriptions of the input data. As a control, a system using Latent Semantic Analysis as an automated decision procedure was input with raw student sentences to mark. A correlation of 37% with human marking was achieved. By feeding the same LSA procedure with perceptually augmented input, derived from algorithmically produced activation passing networks, correlations of 55% were achieved. For this application the perceptual enhancement provided by these networks produces a 48% improvement in correlation scores, demonstrating the network's empirical utility in the automated assessment domain.

Item Type:	Thesis
Creators:	Allott, N.M.
Date:	2000
ISBN:	9781369325553
Identifiers:	Number Type PQ10290306 Other
Divisions:	Schools > School of Science and Technology
Record created by:	Laura Ward
Date Added:	06 Jul 2021 09:06
Last Modified:	20 Mar 2024 16:46
URI:	https://irep.ntu.ac.uk/id/eprint/43332

Actions (login required)

Edit View

Statistics

Views

Downloads