The Assessment of Teacher’s Instructional Quality in National Large-Scale Studies

Instructional design is of crucial significance in students’ educational progress. It is often possible to make connections between aspects of classroom management, student-orientation, or the level of stimulus with the learning progress in mathematics or German. However, measuring or assessing such qualitative characteristics remains a great challenge.

The project addressed the quality of student evaluations. Both established analytical models for the evaluation of nested verdicts were reviewed and frequently used survey instruments (items) were processed.

Hector

Since 2010 sixty Hector children academies have been created in the state of Baden-Württemberg with the financial funding of the Hector-Stiftung II in order to support highly gifted primary school students. The academic support and scientific monitoring is designed as a formative evaluation, so that vital findings of associated studies can be directly integrated into the work of the children academies.

The overall aim of the project is to examine to what extent the children academies contribute to a positive development of the talented and highly gifted primary school students. Furthermore the project aims at drawing generalizable findings that can later be used in the development and support of highly gifted children.

Mercator Institute for Literacy and Language Education

The Mercator Institute for Literacy and Language Education is an institute of the University of Cologne, initiated and funded by Stiftung Mercator. Its aim is to improve language education. To achieve this objective, it researches and develops innovative concepts, measures and tools for language education. It provides regional training for prospective teachers and national training for educators in nurseries, schools and adult education, and prepares academic findings specifically for decision-makers in educational policy, administration and practice. Through its research and the academic services it provides to language education in a multilingual society, the Mercator Institute helps create more equal opportunities in the education system.

Important work and services:

Research: The Mercator Institute for Language Support and German as a Second Language researches and develops concepts, instruments and measures for language education in a variety of projects. It conducts practical and application-oriented research on current and socially relevant issues of language education, from daycare to the transition to work. Its research covers the entire breadth and complexity of linguistic education: It focuses on the chain of effects from research to the training and continuing education of specialists and teachers, the framework conditions in the various educational institutions, implementation in educational practice, and the effects on children and young people. The research focuses on linguistic learning and development processes, classroom research, and teacher training and professionalization. The project teams use both qualitative and quantitative methods and are interdisciplinary. The institute also offers research-based services.

Development: The Mercator Institute advises educational actors on the basis of scientific findings relating to development and implementation processes, e.g. curricula or for the realization of language education concepts. In doing so, its objective is to help initiate systemic changes that will allow consistent language education standards to be maintained throughout all educational stages and institutions.

Professional Development: When it comes to staff development of educators in nursery schools, schools and adult education, the Mercator Institute cooperates with institutions at a regional and national level so as to develop and implement staff development concepts. Due to time constraints and the considerable need for continuing education, educators are increasingly calling for training courses that can be adapted to their individual schedules. This is why the Mercator Institute is also developing digital learning opportunities that combine independent learning with digital materials and face-to-face classes.

Transfer: There are many different stakeholders in language education: from the organizations that run child and youth welfare services, to nurseries and schools, and to ministries and authorities at the local and state level. Through its publications and events, the Mercator Institute promotes the transfer of good practice and scientific findings to education policymakers, education authorities and the educational institutions themselves. In addition, it is available to journalists as a point of contact for topics relating to language education. Current research into topical and socially relevant questions of language education is analysed and then made available in a variety of formats – from short basic knowledge and fact checks to detailed studies and expert reports.

Promotion of young researchers: At the Mercator Institute, there are various measures to promote young researchers. These include AcadeMI, a network of young researchers who meet in a self-coordinated manner and at regular intervals to promote peer exchange at the Mercator Institute and to support each other in their own qualification. Among other things, they organize workshops that all young researchers can profitably use for their work, and they are involved in the implementation of the annual Young Researchers Conference. Within the framework of AcadeMI, some staff members have joined together to form four internal research groups dealing with the topic of multilingualism and vocabulary. In addition, there is a mentoring program.

Research Data at the Mercator Institute:

The Research Database “Lernertexte” (FD-LEX) provides educational researchers access to over 6,200 texts by students of various ages that were collected in different writing projects. Transcripts and handwritten originals as well as anonymized metadata of the participants can be used for own research projects.

Leibniz Institute for Educational Trajectories (LIfBi)

The Leibniz Institute for Educational Trajectories (LIfBi) in Bamberg analyses educational processes from birth to old age. In order to promote longitudinal educational research in Germany, the LIfBi provides fundamental, nationally and internationally significant, research-based infrastructures for empirical educational research. One core of the institute is the National Educational Panel Study (NEPS), which is based at the LIfBi and brings together the expertise of a Germany-wide, interdisciplinary network of excellence. In addition to the ReGES and BildungswegeFlucht refugee studies, the LIfBi is also in charge of other major projects such as the Data Literacy monitoring study. This is based on their own research and development work, in particular the well-founded development of instruments and methods for longitudinal educational studies, from which other research projects also benefit.

Important work and services:

LIfBi runs the National Educational Panel Study (NEPS). The NEPS collects longitudinal data that are representative of Germany on the development of competencies, educational processes, educational decisions, and returns to education in formal, nonformal, and informal contexts throughout the whole life span.
NEPS data are made available to the national and international scientific community in the form of Scientific Use Files that can be accessed through various innovative ways using state-of-the-art technology. Interested data users are invited to attend regular user trainings.
NEPS data provide an empirical basis for research and offer a rich potential for analysis with regard to educational research and related disciplines (e.g., demography, educational science, economics, psychology, and sociology).
In addition to the standard documentation material for every single Scientific Use File, the NEPS Research Data Center offers a number of supplemental information and assisting tools for handling the NEPS data.
The Research Data Center LIfBi (RDC LIfBi) is primarily responsible for the user-friendly preparation of survey and test data of the National Educational Panel Study (NEPS) and its dissemination to the scientific community in the form of Scientific Use Files.
Third-party funded projects by LIfBi: LIfBi continually acquires new third-party funded projects.

Institute for Educational Quality Improvement (IQB)

The associated partner “Institute for Educational Quality Improvement (IQB)” is a research institute that supports the federal states of Germany in developing and monitoring the quality of primary and secondary education. As an academic institution of the German states, the IQB is an associated institute of Humboldt-Universität zu Berlin. The educational standards, which have been introduced by the Standing Conference of the Ministries of Education and Cultural Affairs of the federal states of Germany (Kultusministerkonferenz, KMK), form the basis of their work. These educational standards describe competencies students are expected to have developed at a certain point in their school career. The mission of the IQB is to monitor regularly to which degree these goals are met by schools in Germany (educational monitoring). Furthermore, the IQB supports the German states in implementing the educational standards, which are expected to serve as a central orienting framework for all actors in the educational system (implementation). In addition, the IQB is a research institute that conducts research projects in several areas of empirical educational research.

Important work and services:

Educational Monitoring and Implementation:

Under the auspices of the IQB, experienced teachers develop large collections of test items designed to measure competencies aligned with the educational standards.
The IQB develops proficiency level models as tools for interpreting test scores. These models provide a more concrete description of what students with a given test score are able to do.
The IQB conducts National Assessment Studies (Ländervergleiche) in order to determine the extent to which educational standards are being met in the 16 German federal states.
The IQB coordinates the development of items for state-wide comparison tests. These tests provide teachers with information on the strengths and weaknesses of their students with regard to the educational standards and are expected to promote the implementation of competence-oriented teaching.
In collaboration with experts in subject-matter education, the IQB publishes collections of annotated learning tasks which teachers may use in class.
The IQB coordinates the development of a pool of Abitur examination questions based on the educational standards.

Research – Some of the central research questions are:

How are students’ competencies structured and how can they be assessed?
Which factors influence educational success of children and adolescents?
How can methodological and statistical requirements of educational testing and large-scale assessments best be met?
Language skills are vital for school success. How can their development be measured and promoted?
To what extent and under which circumstances do teachers successfully adopt competence-oriented teaching methods?

Research Data Centre:

The Research Data Centre (FDZ) at IQB archives and documents data sets from national and international large-scale assessment studies and makes them accessible to researchers.

IPN – Leibniz Institute for Science and Mathematics Education

The Leibniz Institute for Science and Mathematics Education (IPN) was established in 1966 as a research center for science education. As an institute of the Leibniz Association, the IPN has a nationwide function. It is also affiliated to the University of Kiel. The department heads at the IPN hold professorships at the University of Kiel.

The institute’s mission is to advance science and mathematics education through its research. Therefore, research deals with the full range of issues concerning teaching and learning in the sciences and in mathematics inside and outside schools. The institute comprises six departments: Educational Research, Educational Assessment and Measurement, Biology Education, Chemistry Education, Mathematics Education, and Physics Education. Approximately 170 people make up the IPN staff; about 130 work as researchers, including 60 doctoral students. About 50% of the staff work on projects funded by different research foundations or clients.

The IPN’s work ranges across the entire field of science, mathematics, and technology education. The IPN concentrates on long-term and nationwide research projects, which cannot be covered by universities.

The IPN research program focuses on following areas:

Aims and models of mathematics and science education
Prerequisites for teaching and learning mathematics and science
Implementation and Evaluation of Concepts for Subject-Specific Teaching and Learning Processes in Mathematics and Science
Promotion of Mathematics and Science through Competitions and Supplementary Learning Opportunities
Educational Assessment and Measurement

Besides its research activities, the IPN offers different transfer activities: Coordination of national and international student competitions in biology, chemistry, physics and environmental issues, programs for teacher education, and publications on science and mathematics education addressing teachers and scientists.

Hector Research Institute of Education Sciences and Psychology (HIB)

The Hector Research Institute of Education Sciences and Psychology (HIB) is a research institute within the University of Tübingen. The institute focuses on individual, social and institutional determinants of learning and teaching processes. It takes an inter-disciplinary approach drawing from psychology, educational science and other disciplines. The Hector Research Institute is mainly funded by the Hector Foundation II. In addition, the state Baden-Württermberg funds the Tübingen Postdoctoral Academy for Research on Education (PACE).

Research at the institute can be grouped into the following six focus areas: (1) Educational Effectiveness, (2) Motivation, (3) Personality, (4) Potential Development and Giftedness, (5) Teaching Quality and Professional Competencies of Teachers as well as (6) Methodological Research. The institute played a leading role in the establishment of the LEAD Graduate School (Learning, Eduational Achievement and Life Course Development) which was funded by the Excellence Initiative via the German Research Foundation and has since developed into a flourishing research network.

Important work and services:

The Hector Research Institute is responsible for the multi-cohort longitudinal studies TOSCA, which examines educational biographies of high school students, and TRAIN, which analyses effect of learning environments in lower tracks in secondary education. With Professor Hasselhorn (DIPF) the institute also runs the scientific evaluation of the Hector Children’s Academies.
Regarding research on methods, the Hector Research Institute develops new statistical tools and strives to make them freely available as software (e.g. see here).
Within the National Education Panel Study (NEPS), the Hector Research Institute coordinates the focus area „From Upper Gymnasium Level and Transition to Higher Education, Vocational Training, or the Labor Market“ together with Professor Maaz (DIPF).

Leibniz Institute for Educational Media | Georg Eckert Institute (GEI)

The Leibniz Institute for Educational Media | Georg Eckert Institute (GEI) is a centre for research in educational media and is one of a kind in Germany. Its mission consists in carrying out practice-oriented, multidisciplinary research on textbooks and other school-based educational media, informed by the disciplines of history and cultural studies. The GEI also advises national and international stakeholders in educational policy and practice. The Institute coordinates and mediates debates and cooperations on international textbook-related matters. As a non-university establishment that both conducts research and makes research possible, the GEI sees itself as a research-oriented infrastructural institution. Its main attraction is the Research Library, with an internationally unique collection of textbooks from the areas of history, social studies/civics, geography and ethics/religion. With the information and communication portal Edumeres, the GEI provides virtual access to current information, publications and data from research on textbooks and educational media, organized into several modules.

Key research activities and services at the GEI:

With the advancement of digitalisation, education is undergoing a fundamental change. In response, the GEI has broadened its focus to include digital educational media. Research on knowledge and interpretation, on the societal dimension of media-based transformation, and on the use and adoption of new technology goes hand in hand with an expansion of the Research Library’s collection and the continuing digitisation of its holdings.
The GEI provides a diverse range of academic transfer and other services. It acts, for instance, as a coordinator for bilateral textbook commissions and projects and supplies consultation services on textbook-related matters to national and international education policymakers, practitioners and organisations. Additionally, it has served as a mediator in conflict-laden situations surrounding textbooks and provides support for the development of new textbooks and educational media.
The GEI’s research library, which holds a unique collection of history, geography, social studies and ethics/religious education textbooks from around the world, has been designated as a Special Subject Collection library by the German Research Foundation (DFG) and is the textbook centre of the Council of Europe.
The GEI runs the information and communication portal Edumeres.net, which acts as a virtual point of access to educational media research conducted from the perspective of historical and cultural studies.
The Institute has its own visiting professorship through the Georg Arnhold Program on Education for Sustainable Peace.

German Centre for Higher Education Research and Science Studies (DZHW)

DZHW provides applied empirical research in the field of higher education and science studies. As research institution of the Federal State and the Länder (Bund-Länder-Einrichtung) it works nationally and internationally as a partner of the scientific community and both higher education and science policies. Committed to politically independent and excellent research DZHW develops and elaborates innovative, socially and politically relevant issues regarding tertiary education and science. The research and service tasks of DZHW are organized in four research units:

Educational Careers and Graduate Employment
Longitudinal studies regarding educational and occupational trajectories, returns to education, in particular regarding decision to study, progress of studying, occupational and scientific careers.

Research System and Science Dynamics
Examinations of the development of the system of research and science in the international context and of the interactions of different structures of governance, funding, and promotion.

Governance in Higher Education and Science
Studies with respect to indicator systems of universities and scientific organizations, regarding the structures of governance of scientific continuing education at universities and examinations of structures and processes of governance with an organization-sociological perspective.

Research Infrastructure and Methods
Provision of research infrastructure, data sets, databases, buildup of the research data centre. The Research Data Centre for Higher Education and Science Studies (RDC-DZHW) archives quantitative and qualitative data from the field of higher education research and science studies and makes them available to researchers and teaching staff for secondary use.

Important work and services:

Longitudinal studies on persons with university entrance qualifications, students (e.g., social surveys of students since 1951, eurostudent), graduates, PhD students and PHD graduates;
NEPS 1: From Higher Education to the Labor Market (NEPS Starting Cohort 5 „First-Year Students“)
The DZHW is in the National Educational Panel Study (NEPS) responsible for the Starting Cohort 5 (First-Year Students) which following up a cohort of first-year students throughout their studies and into their careers. Particular attention will be paid to investigating educational decisions, the development of competencies, the returns to higher education, and the transition to the labor market;
NEPS 2: Panel of Teacher Education Students
The DZHW examines the study progress and success and the career start of teacher education students (additional sample of NEPS Starting Cohort 5) as well as potential differences in interests, occupational orientation and previous educational biographies compared to other students;
NEPS 3: Returns to Education Across the Life Course;
The DZHW examines as member of pillar 5 “Returns to Education Across the Life Course” of the National Educational Panel Study (NEPS) the non-monetary returns to education like health, deviance and social and political participation;
Educational monitoring and reporting (contributions to „National Report on Education” or ”National Report on Junior Scholars”;
Competence Centre for Bibliometrics for the evaluation of research performance;
RCD-DZHW with data sets of higher education research and science studies;
Services for Federal State and State (Länder) administrations, universities, commissions and boards, science organisations and researchers. More information can be found here (in German).

DIPF | Leibniz Institute for Research and Information in Education

The DIPF | Leibniz Institute for Research and Information in Education provides a central information infrastructure for and about education in Germany, expanding scientific foundations by conducting research in its own right. As a national centre for educational research, DIPF investigates education from systemic, institutional, individual and historical perspectives. Accordingly, it critically reflects on existing concepts for quality, governance and improvement. DIPF delivers theoretical, methodological and empirical contributions, connecting fundamental research with innovative developmental work and applications for the benefit of society. Owing to its diversity of disciplines, long-term experience in co-ordinating large-scale projects, national and international networking and positioning as a competence centre for knowledge communication and research on education, the Institute is particularly well equipped to react flexibly with respect to complex demands and diverse developments in education, and to provide incentives for the further development of education systems.

Important work and services:

DIPF holds a leading role in national educational reporting, commissioned by the federal government (Bund) and states (Länder).
DIPF continually takes on relevant tasks in the context of large-scale OECD studies on education (PISA, PIAAC).
In co-operation with Goethe-University Frankfurt and the Sigmund-Freud Institute (in German), DIPF runs the interdisciplinary centre for research on individual development processes in children and adaptive instructional design, IDeA.
The German Education Server is a central guide to education on the web. Jointly funded by the federal government (Bund) and the federal states (Länder), it is co-ordinated at DIPF.
The Research Data Centre for Education provides observation data (e.g., lesson videos) and interview data and related material. Questionnaires and individually indexed scales (item batteries) are also openly accessible. Registered users can download available anonymized transcripts, coding and descriptions
The Institute provides the German Education Portal (in German), the central access point to scientific information for educational researchers, educational scientists and pedagogical practice.
The interdisciplinary area of “Technology Based Assessment” is concerned with researching and developing new procedures in computer-based educational measurement.
The DIPF department “Research Library for the History of Education” is a centre for education historical research; it is the largest specialised pedagogical library in Germany.