Deliverable 6 : Training sessions evaluation methodology

- Full text of the Deliverable (0.5 MB PDF file) -

Abstract

Presentation

The Work Package 6 consists in a validation phase and aims at demonstrating:

  • Real situation applicability for the developed methodology and tools
  • The impact of training on production of ODL environments and course material, as a contribution to Quality, analysing actual production in real situations (including real students).

This validation required careful methodology and was time-consuming as trainees were observed in their professional context. The result is compiled in this document, using online document sharing and communication tools.

Methodology

The evaluation procedure is based on a commonly accepted methodology which measures the gap between the initial situation (trainees with or without ODL experience, with no experience in quality issues) and the final situation (trainees with some experience of quality implementation in their professional context). Due to scarce data available and the wish to obtain comments from trainees, the selected approach is qualitative rather than quantitative.

Description

  • The validation questionnaire measures the trainee’s intentions on integrating a quality standpoint in his professional context. This questionnaire handed in to the trainee at the end of a training session is returned to the trainer.
  • The post implementation questionnaires collect information about the reality of the implementation. It comprises two sections :
    • Post implementation questionnaire for trainees : this questionnaire is trainee-centered
    • Post implementation questionnaire for beneficiaries: this questionnaire is beneficiary centered (students, colleagues, hierarchy members etc.)

These questionnaires were distributed about 6 months after the end of the training sessions to provide trainees with enough time for implementing concrete quality actions.

Study conclusion

Even if it is difficult to make a scientific evaluation based on few entries, some elements have nevertheless been analyzed and some interesting information deducted.

  • Trainees are more interested in ODL issues than in quality. First and foremost, quality is perceived as something to be achieved IF some conditions are available: enough time, money, hierarchy support etc. Hence quality seems to be more of a personal choice, the icing on the cake, rather than a compulsory element to be integrated in a successful ODL system, project or course.

Since many trainees are inexperienced in ODL (excepting Spanish trainees) their preoccupations are more oriented on ODL issues rather than quality, and these perceived as separate topics.

  • Because of the sophisticated ODL approach of the educational field, an ODL novice is challenged by different problems often without training and sometimes without support from more experienced persons. In such a context, quality becomes an additional constraint for the ODL action rather than an ODL definition modality.
  • ODL trainees (Spain) understood that quality approach is NOT dissociable from an ODL project implementation; they have fully integrated this in their project action implementation.
Many ODL projects are implemented through a bottom-up approach (except when an institution totally devotes its activities to ODL training, like in Spain). This kind of project upraises from personal effort and impulse, and after realization, testing etc. is presented to the board of directors. Hierarchy often validates the initiative and sometimes grants financial funds (hours, real money etc.) to enlarge the project. The quality vision of an ODL project being different, is, thus, often implemented as a top-down approach. The more quality (norms to be applied, ISO certification, quality convention, etc) is integrated to the global vision, project management and courses, the more micro ODL actions need to be quality-oriented, and quality issues are closely mixed with ODL action implementation.
European Universitary Pole of Montpellier and Languedoc-Roussillon (F) University of Montpellier 2 (F) Open University of Catalonia (ES) University of Tampere (FI) Szczecin University of Technology (PL) University of Applied Sciences Valais (CH) University of Lausanne (CH)