← The Tertiary Education Strategy (TES) | Home Page | Finding Out More →

A baseline monitoring report

This is a baseline monitoring report. It provides a view of what is known now about the state of the tertiary education system at the start of the period of the TES and before the implementation of the tertiary education reforms. Therefore, the main focus of the report is on the 2002 academic year. In a number of areas, supporting information on trends over earlier years is also provided to show the direction and degree of change already underway. The report also provides an update on the implementation of the key policies that support the achievement of the TES as at the end of 2003.

This baseline report provides a basis against which change and progress related to the TES will be measured over the next five years. Following on from this report will be a series of annual reports that will monitor the overall progress of the tertiary education system against the TES.

The report is structured around the six strategies. For each strategy, there is a section on the expected change for the period of the TES, which is a summary of the key changes set out in the TES and priorities indicated in the Statement of Tertiary Education Priorities (STEP). This is followed by a summary of the baseline picture for 2002 and a discussion of the areas for further development of monitoring. Each section then provides more detailed data on key areas covered by or relevant to the strategy.

The currently available data and information can only provide a partial picture in relation to the TES. There are a number of areas where there will be a high reliance on new data sources, many of which will be generated through implementation of the tertiary education system reforms. There will also be areas where there will be a need to collect new information specifically for monitoring, particularly with regard to understanding the views of key stakeholders.

The purpose of monitoring

The purpose of monitoring the TES is to provide ongoing timely information on the progress of the tertiary education system against the TES and against the baseline postion set out in this report, which gives a view of the state of the sector before the reforms were implemented. Monitoring will help make sense of the extent to which the intended changes are happening, in which areas and to what degree.

There are two main audiences for this work. One is ministers and government, in terms of providing information on progress and highlighting any areas that may require further attention. The other is the tertiary education sector, associated government agencies and key stakeholders, in terms of providing information that can provide a broader context for policy development and sector planning processes.

Monitoring is focused on the tertiary education system as a whole; not on assessing the performance of individual organisations. It focuses on the overall patterns of change and response. However, it is specifically focused on the progress of the TES and is not intended to provide monitoring of all aspects of the tertiary education system. Monitoring and evaluation of specific policy and funding changes will be undertaken separately by the agencies responsible for implementation.

Monitoring will assess the progress of the system at several levels including:

• outcomes for learners, students, research users, Māori whānau, hapū and iwi, Pasifika communities, business and industry and the nation as a whole

• the degree of progress being made towards achieving the strategies, objectives and change messages and the current priorities expressed in the STEP

• the progress made in implementing specific policies and initiatives related to the TES.

Monitoring will also inform the development of the STEPs and of the next TES.

The challenge of monitoring

The TES sets a direction for the sector and has a focus on improving the ability of the sector to manage for outcomes. It does not set specific, measurable goals and targets. Much of the TES is aimed at shifting the attitudes, culture and focus of the sector. The key messages of the TES are summarised in the nine change messages. These are intrinsically difficult to measure.

A narrow focus on quantitative indicators could easily miss the real story. The system may be ‘scoring’ well but still missing the point of the TES – or the other way around.

The challenge of monitoring, therefore, will be to highlight the overall messages, not just report on indicators. This requires using a mix of quantitative data that can provide measures of change over time, balanced with qualitative information that can provide explanation of how and why change is or isn’t occurring. The monitoring will need to focus on system-wide indicators, but these are likely to be slow to respond in many areas. Therefore, there will also need to be information on examples of innovation and successful change. Some areas will require longer-term research beyond the scope of the monitoring work.

Even so, monitoring can only provide a partial and selective view of change across a system that is as complex as tertiary education.

Over the period of the TES, monitoring will shift in focus from summarising what is happening in relation to the TES, to making sense of how changes are contributing to the larger goals of the TES and the well-being of the country. It will also have a greater focus on progress against the change messages.

Evaluation of the TES

Monitoring will provide information and commentary on trends and changes in the system associated with areas of the TES. In addition to monitoring, an evaluation of the current TES is planned. The purpose of the evaluation will be to determine the significance, value and effectiveness of the current form of the TES. It will need to consider both the TES itself and the success with which it has been implemented. The evaluation will aim to improve understanding of areas such as:

• how well the TES has been implemented across the sector

• how effective the TES has been in focusing the sector on achieving outcomes and contributing to national goals

• the significant changes in the sector resulting from the TES.

The evaluation will help inform the development of the next TES.

Further areas for monitoring

In each of the strategy sections in this report there is a discussion of further areas for monitoring and ways in which new data and information will be used to fill out the picture. In general, monitoring will make the greatest possible use of existing data and new data that is being collected as part of policy and funding changes, or that is being developed for a range of purposes. Data will only be collected specifically for monitoring if it is essential to understanding progress and it would be difficult or costly to gather it through existing processes.

Significant areas of information that will be added to monitoring reports over the next one to two years include:

Information on qualification and course retention, completion and progression

The Ministry of Education has recently completed a longitudinal dataset of student enrolments and completions at qualification level, which provides a proper basis for calculating retention rates, completion rates and progression at qualification level. The work was completed concurrently with this report and information from it will be used in future monitoring reports. The full implementation of the National Student Index in 2003 will make it possible to analyse retention, completion and progression at both course and qualification level.

Information from TEO charters and profiles

The introduction of charters and profiles will, over time, provide comprehensive information on TEO plans and strategies, TEO engagement with key stakeholders, the development of collaborative initiatives and the performance of TEOs against their own targets. A key use of this material will be to assess the alignment of TEO plans to the TES and examine progress against the change messages.

Perspectives of key stakeholder groups on progress of TEOs

The information from charters and profiles will need to be compared with information from key stakeholders. This information will provide external views of TEO progress against the TES, the quality of relationships that have been developed by TEOs and perspectives on the relevance of tertiary education to student, community and industry needs.

Information from the PBRF

The PBRF provides new data on researchers and research in the tertiary education sector. It includes a census of research staff, which provides information on the characteristics of research staff overall. There is also new information on the quality of research from the peer review process.

Information from the Adult Literacy and Life Skills Survey

This survey will be conducted in 2004/05. It will not only provide new information about foundation competencies across the population, but also provide a number of insights into the relationship between foundation competencies, tertiary education and life outcomes.

Work will also be undertaken during 2004 to assess how best to develop information in a number of critical areas, including system-wide measures of various aspects of quality of education, information on graduate outcomes and improved information on the tertiary education sector workforce. In some areas, monitoring will be reliant on improvements to the quality of existing data.

← The Tertiary Education Strategy (TES) | Home Page | Finding Out More →

• Table of Contents

• The Baseline Picture

• Cross Strategy Indicators

• Raise foundation skills

• Skills for our knowledge society

• Strengthen research and knowledge creation

• Te rautaki mātauranga Māori

• Pacific people’s development and success

• Capability and quality

• Technical and Data Definitions