News 2019
“Evaluation und Organisationsentwicklung”
Kurs im Rahmen des Weiterbildungsprogramms Evaluation
der Universität Bern
Maria Gutknecht-Gmeiner hielt am 18.-19. Oktober an der Universität Bern einen Kurs zum Thema “Evaluation und Organisationsentwicklung”.
Informationen zum Kurs und zum Programm allgemein am Zentrum für Weiterbildung ZUW der Universität Bern hier
(30.11.2019)
Qualität ist kein Zufall
Wie kommt die Qualität zum Lehren und Lernen?
Der im ÖGB-Verlag erschienene Band “Qualität ist kein Zufall” befasst sich mit der Evaluation und Qualitätssicherung von Lehr-/Lernprozessen durch die Lehrenden selbst.
Maria Gutknecht-Gmeiners Beitrag “Wie kommt die Qualität zum Lehren und Lernen?” befasst sich mit den Herausforderungen, denen sich Lehrende, aber auch die Bildungseinrichtungen als Ganze stellen müssen, um eine lehr-/lernorientierte Qualitätsentwicklung zu implementieren.
Gutknecht-Gmeiner, Maria (2019): Wie kommt die Qualität zum Lehren und Lernen? In: Qualität ist kein Zufall, hrsg. von Schratter, Daniela, Steinklammer, Elisabeth; Taucher, Philipp, Wien: ÖGB Verlag, 26-41.
Weitere Informationen finden Sie hier
(10.7.2019)
News 2018
Peer Review für Anbieter von Validierungsverfahren von non-formalem und informellem Lernen
Das Projekte Peer Review VNFIL Extended wurde erfolgreich zu Ende geführt. Es ermöglichte erstmals einen grenzüberschreitende Pilotierung von Peer Review als entwicklungsorientiertem Evaluationsverfahren. Teilgenommen haben Validierungseinrichtungen aus den Niederlanden, Portugal, Frankreich, Österreich, Slowakei und Litauen.
Die offiziellen Projekt-Webseiten sind
www.peer-review-vnfil.eu (österreichische Homepage in deutscher Sprache)
www.peerreview.work (Projekthomepage in englischer Sprache)
(31.07.2018)
Qualitätskriterienkatalog für die Validierung von informellem und non-formalem Lernen in Österreich
Eben wurde der Kriterienkatalog zur Förderung der Qualität von Validierungsverfahren im Bereich der Berufs- und Erwachsenenbildung in Österreich veröffentlicht.
Maria Gutknecht-Gmeiner erarbeitete die wissenschaftlichen Grundlagen für den Kriterienkatalog durch umfassende nationale und internationale Recherchen, moderierte die für die Entwicklung einberufene ExpertInnengruppe, entwarf die Struktur des Katalogs und verfasste die Texte.
Hier geht’s zum Kriterienkatalog
(28.04.2018)
Qualität in Bildungseinrichtungen: Wie kommt die Qualität zum Lehren und Lernen
Vortrag auf der Tagung “Qualität ist kein Zufall” am 22. März 2018
Maria Gutknecht-Gmeiner hielt auf der von AMS Österreich, AK/VÖGB ReferentInnen Akademie, GPA-djp und abif veranstalteten Tagung “Qualität ist kein Zufall. Aus der Praxis lernen” am 22. März in Wien die Keynote zum Thema “Qualität in Bildungseinrichtungen: Wie kommt die Qualität zum Lehren und Lernen?
Die Präsentationsunterlagen finden Sie hier
(15.04.2018)
News 2017
Qualitools Workshop for managers
Maria Gutknecht-Gmeiner gave a workshop on the integration od learner-centred quality methods into the overall quality management of educational institutions. The workshop was held within the scope of the ERASMUS+ project “QualiTools for IT Trainers” at the Integrationsfonds Österreich. Further information upon request.
Website of the project QualiTools: www.qualitools.at
(17.10.2017)
Working group „Revision of the Recommendations on Education and Training in Evaluation“
Kick-off Workshop, 20.9.2017
Maria Gutknecht-Gmeiner and Dörte Schott organised and moderated the Kick-off Workshop of the expert group for the revision of the DeGEVAL “Recommendations on Education and Training in Evaluation”. Minutes can be found here
(29.9.2017)
News 2014
Workshops “Developmental Evaluation, 23.1.2014, Bonn
1) Workshop for the evaluation units of the German political foundations
2) Workshop at the annual meeting of the evaluation units of the German organisations for development cooperation
On January 23, 2014, Maria Gutknecht-Gmeiner gave two workshops on “Developmental Evaluation” (Michael Q. Patton).
(31.1.2014)
Launching Conference Erasmus+, 22.1.2014, Hofburg Wien
Round-table discussion “Thematic Forum 1: Mobility under Erasmus+ following European and national political goals – Mobility for all”
On January 22, 2014, Maria Gutknecht-Gmeiner took part in a round-table discussion on mobility under Erasmus+. During the discussion, experiences of different educational sectors with the “Lifelong Learning Programme” were reflected and the opportunities and challenges of the new programme highlighted.
Information on the Launching Conference Erasmus+ in Austria more
(27.1.2014)
News 2013
Presentation on “Cross-border Peer Learning”
“Peer Learning in Local Youth Work and Policy”
2nd InterCity Conferenc in Helsinki
On December 9, 2013, Maria Gutknecht-Gmeiner gave a keynote on “Cross-border Peer Learning” at the second InterCity Conference in Helsinki.
The presentation can be found here
(15.12.2013)
Workshop “Evaluation” for Team Focus, Fonds Soziales Wien
On November 13, 2013, Maria Gutknecht-Gmeiner gave a workshop on evaluation for Team Focus, a department of the “Fonds Soziales Wien”, who conduct community and neighbourhood analyses commissioned by the city of Vienna.
The workshop tackled the question how evaluation differs from the usual studies and analyses of Team Focus, in what areas these studies already encompass evaluative elements and how their services can be developed further with a view to evaluation.
(20.11.2013)
Computer-Science-Management: Tentative “final” evaluation
During the third and fourth year of the implementation of Computer-Science-Management (CSM) at the Hertha Firnberg Schulen the evaluation and process facilitation by IMPULSE continued.
Special foci were: effects of CSM on school development, the planing the continuation within the framework of the new curriculum “Communication and Media Design” (Kommunikations- und Mediendesign) and the information and recruitment of the target group. In addition, a comprehensive analysis of all data since the beginning of the study programme in 2010/11 was conducted including time series.
The report is available upon request: contact
(15.11.2013)
Presentation on the “Europeanisation of Austrian education and educational policies”, 5.11.2013, Linz
Lecture series at the pedagocial university Upper Austria on the Austrian National Education Report 2012
Maria Gutknecht-Gmeiner gave a presentation on the “European chapter” of the National Education Report 2012 on November 5, 2013 at the Pedagogical University of Upper Austria
The presentation was recorded, the video can be found here
(10.11.2013)
Gender mainstreaming of evaluations
Maria Gutknecht-Gmeiner is member and co-speaker of the Topical Interest Group on Gender Mainstreaming in the DeGEval (Gesellschaft für Evaluation; German-Austrian Evaluation Society).
For the revisions of the “Standards of Evaluation”, the Topical Interest Group developed a paper delineating the consideration of Gender Mainstreaming in the standards. A short report on this topic has now been published in the “Zeitschrift für Evaluation” (Evaluation Journal):
Gutknecht-Gmeiner, Maria; Wotha, Brigitte, Wroblewski, Angela (2013): Considering Gender. Qualitätsvolle Evaluierung durch Einbeziehung der Geschlechterperspektive in Evaluationen, In: Zeitschrift für Evaluation, Jahrgang 12, Heft 2, 349-352. (Considering Gender. High-quality evaluation through integration of a gender perspective in evaluation)
(10.10.2013)
Peer Evaluation: Are Peers evaluators?
If yes, what kind of competences and training do they need?
Maria Gutknecht-Gmeiner has spent the past years investigating the evaluation competencies of “peers” who assume the role of external evaluators in peer evaluations/ peer reviews. In the last issued of the “Zeitschrift für Evaluation” an article on this topic was published:
Gutknecht-Gmeiner, Maria (2013): Experten, Gutachterinnen oder Dilettanten: Welche Evaluationskompetenz und Schulung benötigen Peers? In: Zeitschrift für Evaluation, Jahrgang 12, Heft 2, 235-256. (Experts, Reviewers or Amateurs: What Kind of Competence and Training in Evaluation Do Peers Need?)
Zeitschrift für Evaluation: Information on the volume can be found here
Download Abstract (English) here
(10.10.2013)
Presentation “Dealing with uncertainty – experiences with Developmental Evaluation”
Maria Gutknecht-Gmeiner put up for discussion her experiences with “Developmental Evaluation according to Patton as an approach for coping with uncertainty and complexity in evaluations”. The presentation took place in the session of the Topical Interest Group on Vocational Training during the yearly conference of the DeGEval on September 13, 2013.
Further information and the presentation are available (in German) here
(1.10.2013)
Seminar: Gender Mainstreaming of evaluations – how to “gender” evaluations
Maria Gutknecht-Gmeiner gave a seminar on gendering evaluations at the 16th Conference of the DeGEval in Munich on September 11, 2013.
Further information (in German) can be found here
(15.9.2013)
Internationalisation of Austrian schools
Maria Gutknecht-Gmeiner conducted an explorative study on the internationalisation of Austrians schools commissioned by the Austrian Ministry for Education, Arts and Culture. The study was carried out together with Katharina Novy in 2012/13.
Gutknecht-Gmeiner, Maria; Novy, Katharina (2013): Bericht zur Internationalisierung der österreichischen Schulen – eine qualitative Untersuchung, i.A. des Bundesministeriums für Unterricht, Kunst und Kultur, Wien.
A comprehensive report is available upon request: Contact
In addition, Maria Gutknecht-Gmeiner wrote an article on internationalisation for the anthology “Schule grenzenlos” (School without frontiers):
Gutknecht-Gmeiner, Maria (2013): Internationalisierung der österreichischen Schulen, In: Gesslbauer, Ernst; Großruck, Ursula; Siegele Petra (Hg.): Schule grenzenlos. Erfahrungen und Herausforderungen im 21. Jahrhundert, Innsbruck: Studienverlag, 36-48.
Information on the publication can be found here
(20.8.2013)
“Youth in action” Conference in Bonn, May 7-8, 2013: Workshop on Peer Learning
“Europe – A place of learning” was the title of a workshop which focused on “cross-border peer learning as an instrument to professionalise youth policy and youth work practitioners”. The workshop explored different perspectives and approaches to peer learning on the European level.
Maria Gutknecht-Gmeiner from IMPULSE gave an overview of cross-border Peer Learning, offering definitions, presenting formats and discussin challenges and opportunities. The presentation can be downloaded here
(30.5.2013)
News 2012
Evaluation and quality assurance in adult education
Introductory seminar to evaluation and quality assurance for the Master’s programme Adult Education, bifeb) Strobl
Contents: What is evaluation? What can it be used for? Who evaluates? What types of evaluation exist? What are important evaluation questions in adult education institutions? What are feasible and fun evaluation methods?
More information on the Master’s programme in German here
More information on the seminar available upon request: contact
(6.10.2012)
Evaluating the teaching and learning process, giving feedback and managing successful projects
Various seminars for schools offered through the KPH Wien, the University College of Teacher Education Vienna/Krems
More information (in German) here
(28.9.2012)
“Equity-focused evaluations” – online training course
An eight-week online training course on equity-focused evaluations is starting on Sept. 10, 2012. It is offered to evaluators around the world free of charge by UNICEF, Claremont Graduate University and IOCE, under the EvalPartners initiative, with support from The Rockefeller Foundation and in partnership with UN Women. More
(5.9.2012)
“Writing reports”
How can we write clear and concise reports, adhere to external requirements and still enjoy it?
Seminar for the IAB, Linz
(21.8.2012)
News 2011
Peer Review Impact Guidelines
The project Peer Review Impact (2009-2011) aimed to investigate the effects of Peer Review on institutions that had undergone a Peer Review. Based on the findings of this research, Peer Review Impact Guidelines were developed by a transnational expert team. The guidelines support vocational education and training institutions in their efforts to maximise the impact of Peer Review.
The Peer Review Impact Guidelines in English can be found here
Other language versions (including a German version) will be available soon.
(23.9.2011)
Computer-Science-Management
an innovative study programme in practice
Evaluation of Computer-Science-Management (CSM) at the Hertha Firnberg Schulen für Wirtschaft und Tourismus, Vienna, 2010/2011
IMPULSE conducted an evaluation of an innovative unique study programme combining education and training in business and services occupations with a focus on information technology and science. The programme was established at a traditionally “female” school in order to attract girls to the IT programme. Cooperation with Austria’s largest technological University of Applied Sciences was to ensure high-quality education and training in IT and and to acquaint students of secondary education and training with tertiary study programmes.
The aims of the evaluation commissioned by the Austrian Ministry of Education, Arts and Culture were to determine how CSM would succeed in practice, i.e. whether the underlying concept would qualify as suitable and capable of meeting expectations, and to document and analyse examples and recommendations for gender-sensitive instruction and school models.
The report in German language is available as well as an English summary:
Report Evaluation Computer-Science-Management 2010-11 (German, pdf 5,3 MB)
English Summary (pdf 82 KB)
(12.9.2011)
Effective training provision –
Planning, implementing and evaluating training successfully
Article in weiterbildung aktuell 2011/2012
Companies invest considerable funds in continuing training, expecting employees to become or remain up-to-date in their professional qualifications. So what do they have to consider when choosing a training offer or a trainer? The article gives answers to these and other questions relating to the planning, implementation and evalation of training in companies.
The article in German can be downloaded here
(15.8.2011)
Evaluation “Netbooks in schools 2010/2011”
After a first pilot in 2009/2010, the project “Netbooks in schools” has been extended to 24 schools comprising different school “types” – from elementary school to upper secondary education and training – from all over Austria. The aim of the project was to further develop the pedagogical potential of netbooks as versatile and mobile devices for enhancing learning in schools. Starting 2011/2012 the use of netbooks will be possible for a wide range of schools in Austria.
In the light of the intended roll-out of netbook use in schools, the Austrian Ministry of Education, Art and Culture (BMUKK) commissioned IMPULSE – Evaluation and Organisational Development to conduct an evaluation of the project in order to 1) come to a summative assessment of the experiences of the participating schools in the use of netbooks and 2) elaborate recommendations for the future broad deployment of netbooks in Austrian schools.
The report can be found here
Competencies of evaluators and didactics of evaluation
Presentation of the experiences of the Gesellschaft für Evaluation (DeGEval) concerning the definition of evaluator competencies and didactics of evaluation at the yearly conference of the Italian Evaluation Association on April 14, 2011 in Trento.
(14.4.2011)
Evaluation of Change Management
Advance notice
How can evaluation create additional benefits and complement traditional systems of reporting, controlling and consulting in enterprises and corporation? In which areas can evaluation be applied? Which methods are suitable? These and other related topics will be investigated in a presentation by Maria Gutknecht-Gmeiner. More (in German)
“3. Internationalen Symposiums für Berufs- und Betriebspädagogik” at the Johannes Kepler Universität Linz, Oct. 7, 2011
Further information is available here (in German)
(7.4.2011)
Evaluation of the project “ÖGB KulturlotsInnen”
“Art and culture for everybody!”
The project “ÖGB KulturlotsInnen” aims to create an “appetite for art and culture”. Through the use of the trade unions networks and by offering tailored programmes the participation of adults in Viennese cultural life is enhanced.
More (in German)
The unique project closes a gap in the promotion of art and culture for adults in employement. It was initiatied by the Verband Österreichischer Gewerkschaftlicher Bildung (VÖGB) and cofinanced by the city of Vienna. After a two-year implementation phase, the project underwent an external evaluation by “IMPULSE – Evaluation and Organisational Development” in order to reach a summative assessment of the project and to develop recommendations for its possible continuation.
The evaluation report can be found here (in German)
(30.3.2011)
Offers for Pedagocial Universities / Schools of Education
IMPULSE – Evaluation and Organisational Development provides special offers for pegagogical universities in the following areas: internal and external evaluation, quality management, education and training in evaluation and QM , Peer Review and international collaboration in evaluation.
The offers can be found here (in German so far)
(28.3.2011)
Didactics of education and training in evaluation
In the past two years, the working group on education and training in evaluation of the Evaluation Society (DeGeval) has developed recommendations for the didactics of education and training in evaluation. The recommendations were adopted by the members of the working group at the spring meeting in Marburg on March 25, 2011.
Didactics of education and training in evaluation
Recommendations of the working group on education and training in evaluation of the Evaluation Society – DeGEval e.V.
First edition (March 2011)
Authors: Maria Gutknecht-Gmeiner, Jan Hense, Stefanie Kihm, Katrin Lüth und Heike Steckhan
(38 pages) Further information on the activities of the working group on “Education and training in evaluation” of the DeGEval can be found here (in German) (25.3.2011)
Peer Review as an alternative to state-organised external evaluation?
Presentation and workshop at the yearly conference of the association of teachers at vocational schools of business administration in Nordrhein-Westfalen
March 18, 2011, Bochum
More (in German)
(18.3.2011)
First analysis report QALLL – Quality Assurance in Lifelong Learning
The Project QALLL – Quality Assurance in Lifelong Learning aims to analyse, assess and further disseminate the results and products of projects dealing with quality assurance from the programmes Leonardo da Vinci (vocational education and training) and Grundtvig (Adult education).
A first, quantitative Analysis of 80 completed projects was carried out by a team of 7 experts coordinated by Maria Gutknecht-Gmeiner in autumn 2010. The analysis report can be found here
Following activities of the project comprise a qualitative analysis of key issues in quality assurance of vocational education and training and adult education across Europe and the development of recommendations for future programmes. The project cooperates with EQAVET.
(20.1.2011)
Peer Review Impact – Evaluation Report
The impact of Peer Reviews on institutions of vocational education and training participating in the “Peer Review Projects” (2004 to 2009) has been investigated and analysed in an international study.
The evaluation report is available from IMPULSE – Evaluation and Organisational Development upon request.
(10.1.2011)
News 2010
Review: “Introduction to evaluation”
Increasing significance of evaluation in many realms of modern society have led to a efforts towards professionalisation also within the German-speaking evaluation community in the past 15 years: evaluation societies have been founded, evaluation standards adopted, journals established and education and training offers developed.
Two well-known experts of German evaluation have recently published an introduction into evaluation which delineates the above developments in the discipline and aims to impart basic knowledge on evaluation:
Reinhard Stockmann, Wolfgang Meyer: Evaluation. Eine Einführung. Verlag Barbara Budrich (Opaden; Farmington Hills, MI) 2010. 295 pp. ISBN 978-3-8252-8337-7. 19,90 EUR. UTB – 8337.
In her review of this book for “socialnet”, Maria Gutknecht-Gmeiner critically assesses its merits and traces underlying positions and demarcations of the authors: Who are the evaluators? Which scientific paradigms underlie evaluation? How are values treated?
The review in German language can be found here
(3.11.2010)
Didactics of evaluation 1
Recommendations for “Didactics of evaluation”
The working group on “Education and training in evaluation” of the Evaluation Society (DeGEval – Gesellschaft für Evaluation) has been working on concepts for the didactic design of educational offers in evaluation in the past two years.
At the yearly conference of the DeGEval in Luxembourg a paper with recommendations on the didactics of evaluation has been presented for discussion in a review version by the development team: “Didactics of evaluation – Recommendations for the design of educational offers for the acquisition of evaluation competence”.
The paper will be revised and shall be adopted by the working group in 2011.
Development team: Maria Gutknecht-Gmeiner (Impulse – Evaluation und Organisationsberatung), Jan Hense (Ludwig-Maximilians-Universität München), Stefanie Kihm (Universität des Saarlandes), Katrin Lüth (evalux – Agentur für Evaluation, Forschung und Beratung), Heike Steckhan (Universität Bonn)
A presentation of the current version in German can be found here
(17.9.2010)
Didactics of evaluation 2
Activating methods
At the DeGEval 2010 conference in Luxembourg three didactical methods were presented in a workshop on “Methods of evaluation didactics – activating teaching tolls for evaluation workshops and seminars”.
Maria Gutknecht-Gmeiner: Reflecting on tasks and roles of evaluators using metaphors. Handout (in German) available here
Wolfgang Beywl & Verena Friedrich: Chocolate exercise
Jan Hense: Role plays „Resistance of stakeholders” and „Giving feedback on negative evaluation results”
Additional informationen on the workshop in German can be found here
(17.9.2010)
Impact of Peer Review
“Meta-evaluation, use and impact of Peer Review in vocational education and training”
At the DeGEval 2010 conference in Luxembourg Maria Gutknecht-Gmeiner presented first results of a European study on the implementation of Peer Review in vocational education and training and the use of the results in the participating institutions.
Download presentation in German here
Further information on the session can be found here
(17.9.2010)
Introducing quality into education?
Current approaches to quality assurance under scrutiny
An article on quality of education and training was published in the 40th anniversary publication of the Austrian Institute for Research on Vocational Training (öibf) in June 2010.
Gutknecht-Gmeiner, Maria (2010): Wie kommt die Qualität in die Bildung? Aktuelle Ansätze zur Qualitätssicherung und Qualitätsentwicklung, In: Schlögl, Peter; Dér, Krisztina (2010): Berufsbildungsforschung. Alte und neue Fragen eines Forschungsfeldes, Festschriftanlässlich des 40-jährigen Bestehens des Österreichischen Instituts für Berufsbildungsforschung, Bielefeld: transcript Verlag, 194-207.
Further information on the publication here
(30.8.2010)
“Making technology more attractive for girls”
Qualitative study on the study programme “Computer-Science-Management”
At the second Austrian Conference for Research on Vocational Education and Training (2. Österreichische Berufsbildungsforschungskonferenz) Maria Gutknecht-Gmeiner presented the results of a study on demand for and feasibility of a special educational programme “Computer-Science-Management which combines education and training in business and IT at a school with a traditionally high enrolment of girls. Aim of this educational programme is to make training in information technology more attractive for girls.
Download of the presentation in German here
(9.7.2010)
Evaluation of the Lifelong Learning Programm
Interim Report for Austria
In spring 2010, Maria Gutknecht-Gmeiner, in cooperation with Silvia Weiß (ibw), conducted the interim evaluation of the EU Programm for Lifelong Learning in Austria.
Gutknecht-Gmeiner, Maria; Weiß, Silvia (2010): Länderbericht Österreich zur Zwischenevaluierung der Durchführung des EU-Programms Lebenslanges Lernen 2007-2013 im Zeitraum 2007-2009, im Auftrag des Bundesministeriums für Unterricht, Kunst und Kultur und des Bundesministeriums für Wissenschaft und Forschung, Wien. (Interim Evaluation of the Lifelong Learning Programme in Austria)
Download Report in German here
(30.6.2010)
