Deliverable 2.1 : Quality issues in partner's countries

- Full text of the Deliverable (0.5 MB PDF file) -

The aim of this deliverable is to elaborate a comparative analysis of quality and evaluation issues in different cultural and educational contexts (limited to Higher Education) in order to determine how these contexts have an impact upon the implementation of Quality process in ODL services in Switzerland, Poland, Finland, France and Spain. The report also includes a blocking factor list for the implementation of quality in e-learning. The main points of this document (and also of the deliverable) are:

The information was collected from five national reports (one per country). Those reports were made according to a template which included several items (questions) as indicated below:

• Contextual information about the partners’ country (country, population, policy on the use of ICT, labour market situation: unemployment rate, market needs, trends)
• General information about the access to computers and Internet (percentage of the population having a PC at home, percentage of the population having a low-speed access to Internet at home, percentage of the population having a high-speed access to Internet at home, percentage of the SMEs having a high-speed access to Internet, ratio student-computer in primary and secondary schools)
• General aspects of the Higher Education system in partners’ country (number of institutions being part of the Higher Education sector, number of registered students in these institutions and equivalent of full time students, typology of institutions being part of this sector, distribution of students in these different types of institutions, evaluation process for Higher Education institutions, teachers and their roles, use of ICT in traditional (face-to-face) teaching, ratio of equipment for students and high speed access to Internet, technical assistance for teachers using ICT in their traditional (face-to-face) teaching, training courses for teachers using or wanting to use ICT in their traditional (face-to-face) teaching)
• General aspects of ODL in the Higher Education system in partners’ country (in the institutions offering only ODL courses, how the teachers' work is rewarded, in the institutions offering ODL courses and traditional ones, how the teachers' work is rewarded, ODL policies: What are the main official objectives for ODL, what is (are) the main concept(s) of ODL accepted and used in the Higher Education system, pedagogical trends in ODL in the countries, present situation of access to e-learning for registered students in full or partly ODL courses, registration fee for ODL courses)
• Representation and implementation of quality in the countries (models of quality applied in the countries, national agency(ies), main challenges for implementation of quality in your country (problems to solve, areas to improve...), level of quality implementation in the countries, the organizational and cultural specificity of the countries that can influence the representation and the implementation of quality in the public organizations, the organizational and cultural specificity of the countries that can influence the representation and the implementation of quality in the Higher Education institutions, the cultural specificity that can influence the representation and the implementation of quality in ODL services or departments, implementation of quality in traditional universities (the ones only offering traditional courses): How many of them are using any quality system?, current situation of quality implementation in institutions currently offering only ODL courses, current situation of quality implementation in institutions currently offering both ODL courses and face-to-face courses)
• Implementation of ODL quality systems in the participating institutions (main characteristics of the institutions, number of current registered ODL students, equivalent of full time students, percentage of ODL students within the whole institution, number of ODL courses and characteristics of the courses, financial issues: in ODL, to which sectors is money allocated)
• Implementation of ODL quality systems in partners’ institution (are the institutions applying any quality model and since when?, is it a political decision and who took it?, who is responsible for the implementation and its follow up?, is there a specific budget allocation to support it?, what kind of difficulties has been or is still encountered?, what tools are used to manage quality?, in which areas is it implemented?, impact of the quality measures, how is the quality plan made public?, criteria about student's pedagogical support that are taken into account in each quality plan, some indicators)

The analysis of these indicators provides us with the essential data that will take us to set conclusions about the Blocking factors and Best practices in quality implementation.

Part 1 of the deliverable. The educational context in the countries

The collected information was listed in these main categories, in each category the dimensions asked are available too:

•  Policies in the use of ICT ; the items refer to the Involvement of citizens in the information and communication society, Introduction of ICT in the SME, ICT as element of integration, Introduction of ICT to enhance competitiveness and productivity, Regional equilibrium, Improving access to information, Safer use of the Internet and child protection, E-business and E-commerce development, Use and content development , Use of Internet for health issues, The digital gap, Promoting ICT in the administration.

•  ICT Policies in education ; with items such providing Equipment and facilities to schools, Internet and other training courses on ICT, Use of the Internet to develop cultural services, Production of online content, Interactive educational television, Innovation and development of ICT applications, Cooperation in ICT issues with another countries, Guarantee ICT skills for everybody, Development of virtual learning environments, Strengthening vocational and professional training, Promotion of higher education, Use of ICT to improve accessibility, Creation of a distance learning market in any of the education stages, Improvement of quality in the learning process and mobile sets for educational issues.

The analysis reveals some interesting statements in this category such as:

• The delivery of technology is a minimum basis in countries with less tradition in ICT policies such as Spain.
• Although some e-learning observatories such as L-Change have reported a decrease in the production of online content, the main policies in Spain, Finland, France and Poland aim to develop this sector.
• The widespread of ICT skills shows that ICT are not still commonly used in educational systems in most of the countries. In advanced parts of this report we will go in depth with this use in ODL courses.

The report also includes a table in which we find the access to computers and to the Internet in the different countries. Finland shows the higher indicators in all the cases, which highlights the valuable experience of this country in the use of ICT. In France and Spain, we can observe SMEs are mainly working with high-speed connections. In Poland the average of people per computer is considerably lower than the rest of the countries. This should justify one of the points in ICT educational policies concerning the delivery of technological infrastructure in the educational institutions. The table also shows the little use of high-speed connections at home, with percentages between 4 to 7%, but in Finland.

•  General aspects of the higher education system in the countries . Dealing with four aspects which are the higher education institutions from a quantitative perspective, the evaluation process of higher education, the human resources in the universities and the ICT issues in the universities.

•  The evaluation process in the universities ; which proposes the evaluation, objectives of the evaluation, roles of national agencies and universities, annual plans of evaluation, fields of the evaluation, description of the process, consequences of the evaluation.

•  Teachers. Comparison of several indicators about the existing types of teachers. The conclusions establish that it is essential to report about the current situation of teachers in each country, because most of the time they will lead the main tasks in ODL courses. Some of the teachers’ profiles show special interests in pedagogical issues as lecturers in Finland or PRAG in France.

•  The use of ICT in traditional teaching . Types of use. With informations related to the general information about the subject and degrees, learning materials and other resources, e-mail and other communication systems, forums and debate areas, cooperative work, web learning activities, administrative facilities for teachers, administrative facilities for learners, links to other resources of interest, videoconference services, complete online subjects, databases and internet searchers.

The conclusions of this point were that services are completely widespread in all the countries: learning materials, forum and debate areas, links, and cooperative work, learning activities delivered through the web...Some other more related to the use of medium/big CMS or LMS are not still working in all the countries: administrative facilities for teachers and learners, databases and Internet special searchers, complete online subjects...On the other hand, the use of the videoconference is not still widespread because of the low level of high-speed connection.

•  Technical assistance and training of trainers in the use of ICT. In some countries the technical assistance depends directly on universities and their teacher services. In France, for example, 70% of teachers using ICT declare that their institution is providing them with some logistical help. In the case of Poland there is a strong collaboration between students and teachers. This facilitates the transmission of technical skills especially from students to teachers. In the case of Spain most of the universities have created specific groups that dedicate part of their schedule to solve technical problems. But there are not many cases in which these groups also offer training  addressed to teachers interested in ICT. The Finnish case shows that technical assistance is more widespread in Finnish universities. It is usual to have PC advisors in the department and also units like Learning Technology Centre and Computer Centre offer technical assistance.

In general the state does not propose nor obligate universities an assistance service in ICT for teachers.

•  ODL in the national Higher Education systems. This chapter is dedicated to some quantitative data about the presence of ODL, the ODL policies addressed to Higher education institutions in these countries, the concept of ODL and the pedagogical trends, the access to e-learning, the financial costs, especially comparing the differences between joining an ODL course and a traditional one.

•  ODL policies about information society skills for everyone. Networks as a learning environment, digital information capital, information society structures in education, training and research, ensure that incorporation of ICT is included within the budget allocation for universities, training of the staff, development of infrastructure, development of administrative, creation of virtual educational communities, some type of financial help, development of courses to support ODL, use of ICT for vocational and continuous education, development of infrastructure, development of administrative services, creation of virtual educational communities, some type of financial help, development of courses to support ODL, use of ICT for vocational and continuous education, use of ODL for handicapped people, promoting quality and visibility of online campuses

•  The concept of ODL and general pedagogical trends. Collaboration learning, constructivism, focus moving to the learning process of the individual, personalization of learner, multidisciplinarity in e-learning research.

•  Access to e-learning. Chapter dedicated to the state of development of the required infrastructures in order to enhance the access to e-learning in the five studied countries.

•  Financial issues . There is a common trend in all the countries: ODL courses are more expensive than traditional courses.

Conclusions: In Finland ODL courses are part of a degree, but these e-learning courses can be more expensive than a traditional ones. In France students have to pay regular registration fees plus specific fees (for instance learning material and tutoring). In Switzerland online courses are in general as costly as face-to-face courses. In Spain and Poland ODL courses are in general more expensive.

Part 2. Representation and implementation of quality in the countries

In the present section of the report we go in depth with the current situation of quality in the five countries, particularly focusing in:

•  The quality models, systems or approaches that the governments are using at the moment.

In Spain , the LOGSE (education legislation for compulsory education) opted for the application of the European Model in Quality Management. The philosophy of the model is to promote the users' satisfaction and the impact in the society through a leadership which fosters planning and strategy. The EFQM model follows these principles, and it is also a systemic model that facilitates the evaluation and implementation of quality. But there is no systematic model of quality in the case of universities.
In Switzerland ISO 9000 is especially used in case of Universities of Applied Sciences. A specific norm called EduQua has been developed for continuing education institutions. OAQ is trying to define its own certification model.
In France there are specific and formal models applied such as ISO 9000 and TQM. But the most interesting instrument is the “book of reference” of the CNE, introducing officially the assurance of quality in the evaluation of universities (following Bergen decisions).
The rest of the countries have designed specific models that can be adapted to their particular situations in Education. In Poland the PCBC (Polish research and Certification Centre) leads the certification issues. The quality model in education has not been discussed beyond the education institutions (like ministry of education or SAC).
In Finland quality management systems are not very used in education, as it is considered that these does not suit well enough for the quality in education.

•  The description of existing national agencies that support quality assurance, their roles, functions, activities, efforts that have been done...

All the countries have a specific agency on behalf of quality issues.
In Switzerland the certificate EduQua qualifies education institutions.The effort depends on every institution. The students will preferably choose a certified institution, but it is only one of the criteria for them and not the most important one. EduQua main objectives are: Quality - Transparency - Coherence in the continuous formation. The Certificate EduQua qualifies a good formation within continued education institution, assures and develops quality in the continued education institution, offers more transparency for the consumers.
In Poland The State Accreditation Committee (SAC) is the only organisation authorized in-law operating to control quality in education process in higher education institutions. SAC's negative rating implies the withdrawal of the Ministry of Education’s authorization for particular institution which does not respond to applicable quality indicators. The objective of SAC is to control and evaluate quality in education (in higher education institutions). There are no evaluation criteria addressed to ODL courses.
In France AFNOR is responsible for all of the standardisation activities but not for Education. The only official body for the universities evaluation is the CNE.
In Finland FINNHEC is the agency which mainly advises universities.
In Spain autonomous communities have the possibility to create specific quality agencies. The Catalan and Andalusian regions have done it. In the rest of the country ANECA is on behalf quality issues.
The objective of ANECA is to help assuring quality in the Spanish university system, fostering transparency, comparability, cooperation and competitiveness, improving teaching, research and management and informing society in general about the quality of university programmes and services.

•  The challenges for implementation of quality in these countries.

The questions were related to the following topics; giving a more pedagogical point of view, the lack of suitable infrastructure, ODL not considered legally as valid as traditional learning, lack of quality certificates, low funds, conception of quality as “consuming work time”, difficulties for the creation of general models valid for several fields, independence of teaching, lack of transparency in administrative decisions, European higher education area, to be part of the management system, continuous and documented process, to enable the participation of all the members of the university community, to redefine the critical processes in educational institutions, clear indicators, to reflect about leadership and management in these institution, to enable the participation of all the members of the university community, to redefine the critical processes in educational institutions, to clarify  indicators, to reflect about leadership and management in these institutions, to go in depth with the subject “autonomy of educational institutions”, to get support from the administration.

Some conclusions: it is very interesting to observe that most of the partners share five of the challenges whose could be included in the category of ‘Institutional aspects': to involve the members of the university community in the design and implementation of quality systems, to reflect about the management of quality, to receive more support from the administrations, to increase the autonomy of the universities in order to improve quality and to set up suitable infrastructure.

•  The level of implementation of quality. Questions are related to the categories; society does not compel quality standards, lack of formal support on quality initiatives, highly centralized management of public services which decreases operationally, good infrastructures, high standards of education, high technologic skills, positive representation of quality: competitiveness, marketing, quality, contractual system, diverse concepts of success in education, role of the teacher, function of the school, diversity of educational reforms, importance of the value “autonomy”, language issues, low value of pedagogical issues, focusing in teaching instead of learning, traditional value of ODL, slight changes in ODL vision, unrecognizing ODL teaching.

•  The description of the cultural elements that are having or may have influence on the perception or implementation of quality.

This description is addressed to four types of institutions:

Part 3. Implementation of ODL Quality in several institutions

In the third part of the report several experiences of the implementation of ODL quality systems in each partner institution are presented, being very heterogeneous.

The Part 3 is divided in three subparts:

•  The description of the institutions

Université Ouverte UO-MLR, Universitat Oberta de Catalunya, Tampereen yliopisto (University of Tampere), Politechnika Szczecinska, Haute Ecole Valaisanne Spécialisee and Université de Lausanne

•  The implementation of quality in the whole institution

Some conclusions. Most of the universities started applying quality models between 1997 and 2000, and all of them took as starting point the decision of the respective university councils.
All the universities have created specific groups that mainly design the model and give support to the areas or departments in which such model is applied.
We identify several types of models: Institutions like UOC and Haute Ecole Valaisanne Spécialisee are currently applying standard models (ISO and EFQM) that have been adapted to their contexts. In the case of the UOC, EFQM has demonstrated not to be very suitable to focus in the knowledge or learning process aspects, so that UOC is currently defining a specific model to implement beyond the management aspects.
The rest of universities are using not-standardized models. The UO-MLR has produced a Quality Chart which defines set indicators (minimum requirements) that courses should follow in order to be attributed with a quality label. The quality approach is focused in student support and design of learning materials and other teaching resources. Some faculties or departments already apply this quality chart.
Similarly, the University of Tampere has developed a specific model which fits the learning process. As such, the core of quality work is student's learning process. Faculties and departments have to adapt the university's quality system according with their own context.
The University of Szczecin has developed a set of structured questionnaires aiming to get students' opinions, graduates and employers' opinion, to evaluate young tutors and lecturers and the rest of teachers.
Finally, most of the universities are applying quality systems globally, while UM is applying Quality chart only in some departments and the University of Lausanne on teacher demands.

•  The implementation of quality in the ODL sector of the institutions.

Focused in the following points (the quality plan: description, objectives, model on which is based the implementation, budget, the difficulties in the implementation of the quality plan, tools to manage the quality, mode of impact, criteria for students' pedagogical support, some indicators of quality

Blocking factors

Conclusions

The analysis of all the indicators leads to the conclusion that there are specific key aspects influencing the current situation and implementation on quality. These are:

• What is the current situation of quality implementation in the face-to-face courses in the traditional universities?
• What are the characteristics of teachers and especially those who potentially participate in ODL courses? What kind of training and technical assistance are they receiving?
• What is the current use of ICT in face-to-face subjects?
• Are the standards models of quality such as ISO or EFQM being used? How are they being applied?
• What are the functions of national agencies of quality in the implementation of quality models to both traditional learning and ODL?
• What are the current challenges in quality?
• How organizational and cultural specificities affect quality implementation?
• What is the specific situation of quality implementation to ODL courses and services?