Assessment

Definition

Assessment is the process of documenting - usually in measurable terms - knowledge, skills, attitudes, and beliefs.

In the E-Quality project

Preparation of the validation phase

During the training sessions, the trainers gave the trainees personalized advices and helped them to define concrete actions, in order to implement quality. These actions are related to quality criteria and measurable quality indicators, which are to be observed during the validation phase.

• to stand measurable indicators and measure tools;
• to foresee iterations in the quality process;
• to give indicators related to different time scales.

Validation phase

The main objective of the validation phase is to assess the trainees' intentions to apply e-quality methodology in their own professional contexts. This validation requires a sufficient time for observation, evaluation, and analysis. The output of the work is compiled into a document, using online documents sharing and communication tools, and will be put into the best practice data base.

We aimed at measuring the impact of the training on the production of ODL environments and course material as an increase in quality, analyzing such real production in a real situation (including real students) and real tutoring in a professional context.

To do so, we decided to measure the gap between the appropriation of the methodology ( learning process : what I understood and learned) and the application of the concepts with supposed improvement for final students and other actors in the e-learning developing ( transferring acquired concepts on other concrete situations : what I intended to do with this instrument, what I effectively did and what has been improved in my practice using the methodology.

This phase has been divided into three sub-phases:

1. measurement of the appropriation of the methodology through the training sessions
2. measurement of the application of the methodology in concrete professional situations
3. measurement of the final user satisfaction rate (student, administrative staff, or e-learning course developer)

Validation concept

The purpose of validation in ODL is to:

• improve ODL quality by indicating the essential activities promoting quality in different processes;
• help recognize best practices and blocking factors;
• act as guidelines and support media for the development of education in a new situation using new technologies;
• act as a tool for quality assurance;
• act as a learning process for the whole team working with the course;
• increase the understanding of the success of different areas;
• act as a comparison when looking for best practices and examples for the improvement of ODL;
• act as a quality guarantee or recommendation in relation to students, producers, and other users of ODL.

The quality criteria and their indicators should not be seen as separate aspects of education. The criteria are an elementary part of the construction of learning events or learning objects and of the strengthening of the various processes needed for modern higher education. Even in traditional modes of education, such criteria are always in place as an integral, unexpressed aspect of the cultural tradition of the university teaching. The criteria and their indicators are thus not artificial tools or evaluation scales for ex post facto quality measurement, but guidelines and support media for the development of education in a new situation using new technologies.

Instrument implemented

•  Collecting data instruments

The validation concept has been built on three main developments:

• Implementing pre- and post-training session questionnaires to measure knowledge acquired during the training session.
• Implementing questionnaires to verify the intentions of the trainees, what concrete measures they have decided to apply to their professional context, and how these measures have been applied practically. These questionnaires are designed in relation to the roles trainees play in the ODL process.
• Implementing interviews to assess the end user's satisfaction rate.

The questionnaires take into account the country origin of the targeted public and allow the survey to build interesting comparison. These are aimed not only at increasing ODL courses quality, but also at presenting the reason why - should it be the case - the whole quality level has not been improved.

•  Grid and indicators

To build the set of questionnaires, indicators had to be produced. These were created to cover the two main sub–processes studied within the e-Quality project: student support, and learning material production.

The whole validation concept, centered on ODL actors, follows the whole ODL lifecycle and starts from the trainee's intentions to apply e-Quality methodology to reach reality of this implementation. Where the implementation had been successfully integrated to the ODL process, the selected criteria will allow the project to answer the following relevant issues:

Has the application of the e-quality methodology notably increased

• The global quality of the ODL course?
• The global student's satisfaction?
• The knowledge appropriation by students involved in an ODL course?