Quick links Home page Staff members Contact details Research overview Research projects Technical reports

Using REDEEM to Create Intelligent Tutoring Systems from Royal Navy Software

Shaaron Ainsworth

Ben Williams

David Wood

REDEEM allows authors to create their own ITS from existing computer-based teaching material. They enhance this material by overlaying it with their own instructional knowledge to produce richer environments that are designed for the needs of specific learners. REDEEM also incorporates default teaching knowledge supplied from research in the nature of learning and instruction.

Aims

The central questions addressed in this project are:

• Is the REDEEM environment appropriate for use with adult trainers and trainees?:-
• Can existing Navy software be readily reused within REDEEM?
• Can Navy trainers use REDEEM to create Intelligent Tutoring Systems (ITSs) that reflect their teaching objectives?
• Is the REDEEM approach time efficient and effective in this context?
• Do the ITSs that result lead to improved learning, as compared to traditional CBT?

Background

This three-year study aims to establish whether the REDEEM ITS authoring system can provide an efficient and effective means of producing Intelligent Tutoring Systems for use in Naval training. It builds upon a number of other evaluations of REDEEM, and is the first application of the system to adult training.

The project has been funded by the U.S. Office of Naval Research and has been carried out with the support of the Royal Naval School of Educational and Training Technology (RNSETT) at HMS Nelson, Portsmouth and the Training Support Squandon at RAF Waddington.

An example of the REDEEM authoring tools and RN courseware

On-Going Research

There are two main phases in this study - An authoring phase, in which Royal Navy (RN) Trainers produced simple Intelligent Tutoring Systems using REDEEM; and an experimental phase, which involves a comparative evaluation of naval trainees' learning with either the existing Computer-Based Training (CBT) or REDEEM versions of the CISP course.

Authoring phase

Two RN trainers (one training full-time RN personnel and the other training part-time reservists - both subject matter experts) used courseware from an existing seven-chapter Navy CBT course on Communication and Information Systems Principles (CISP) to produce a number of ITSs using REDEEM. The resulting systems had to provide tuition that would meet prescribed RN training requirements, but the authors were able to call upon their past experience of varying trainee needs to develop enhanced and differentiated instruction.

Experimental phase

In order to assess the effectiveness of the REDEEM approach to training, learning outcomes are being compared between RN trainees using either a non-adaptive CBT version of the CISP course or the REDEEM ITSs. The first study involved 23 trainees, at HMS Collingwood in July 2001. This was followed up by a second, smaller study at the HMS Sherwood reservists training centre. Comparisons will also be made with trainees who have been taught the same course in a classroom (i.e. non-ICT) setting.

Subsequently, two shorter studies under more controlled conditions have been carried out using the Navy Material. The first compared University students learning about personal computing and networking with either REDEEm or CBT and the second used the same courses with RAF personnel.

Summary /Conclusions:

The REDEEM application has proved reliable in both authoring and training, and trainer/authors and trainees have been able to use it without difficulty.

Initial findings indicate that, although there are issues regarding the sequencing of course material using REDEEM, the basic design of the system and the methods it employs are acceptable in this context. The authoring method used is both flexible and efficient, and the ease with which authors can review and modify their work provides the opportunity for experimentation with, and reflection upon different instructional approaches.

The results of the experiments showed that REDEEM led to significantly greater learning outcomes in both the Undergrad and RAF studies, though not in the Navy (see Table 1). Analysis of process measures is on-going but prelimary results suggests that REDEEM mainly improves learning through increasing interactivity rather than through macro-adaptation.

Selected Publications

Ainsworth, S.E & Fleming, P.F. (2004) Teachers as instructional designers: Does involving a classroom teacher in the design of computer-based learning environments improve their effectiveness? In Gerjets, P., Kirschner, P. A., Elen, J. & Joiner, R. (Eds.) (2004). Instructional design for effective and enjoyable computer- supported learning. Proceedings of the first joint meeting of the EARLI SIGs Instructional Design and Learning and Instruction with Computers, pp 283-291

Ainsworth, S.E., Williams, B.C & Wood, D.J. (2003) Comparing the learning effectives of REDEEM versus CBT. 11th International Conference of Artificial Intelligence in Education.

Ainsworth, S.E., Williams, B.C & Wood, D.J. (2001) Using the REDEEM ITS Authoring Environment in Naval Training. IEEE International Conference on Advanced Learning Technologies.

Major, N., Ainsworth, S.E., & Wood, D.J. (1997) REDEEM: Exploiting Symbiosis Between Psychology and Authoring Environments, International Journal of Artificial Intelligence in Education 8(3/4), 317-340.

Acknowledgements:

This project is funded by the U.S. Office of Naval Research Grant no N0014-99-0777

We would like to thank staff at RAF Waddington, RNSETT, HMS Nelson, HMS Collingwood, and HMS Sherwood for their cooperation and support in conducting this study