← Purpose | Home Page | Overall Findings →

Focus on TEO objectives

This analysis of profiles focuses on the objectives and performance measures set out in Part B of the profiles. This section of the profile articulates the key strategic actions and activities that the TEO is committing to over the period of the profile. It is considered that this section of the profile provides the strongest evidence of strategic change.

While TEIs are required to include a section in their profiles on alignment of their activity with the TES, this has not been used due to the wide variation in approach and standard. This has also been commented on in other research.¹ The requirements for ITOs combined the Part B objectives with alignment with the TES.

Focus on change

The analysis focuses on objectives which express a sense of change within the organisation. This has been interpreted from the wording of the objectives. Where objectives talk about maintaining or supporting current processes and/or achievement levels, they have been excluded.

The analysis does not attempt to measure or assess the extent of intended change or the state from which changes are being made.

Alignment with the TES

Only objectives with an identifiable alignment with an aspect of the TES are included in this analysis.

For the purposes of coding and analysis, the TES has been looked at in terms of the six strategies and areas of key change expressed within each strategy. The 35 objectives of the TES and nine overall key changes have not been used.

Due to limits of the coding process, each TEO objective has been coded to only one aspect of the TES, based on the researcher’s interpretation of ‘best fit’ of the objective. In many cases, TEOs have noted multiple connections of individual objectives to various parts of the TES. To have used all of these would have reduced the power of the analysis — in that almost everything can be related to everything. However, it would have been ideal in some parts of the analysis to have allowed more than one connection to be coded.

The change areas across the six strategies have also been mapped to the 2005 STEP to look at the extent to which 2006/08 profiles have responded to the messages in that document.

Appendix 1 sets out how the areas of analysis relate to the TES and the STEP.

Analysis

The analysis in this report looks at the objectives across sub-sectors in two ways:

• the extent to which planned change aligns with each strategy
• the types of actions and activities that make up the change.

The counts presented in the graphs are of the number of organisations with one or more change-focused objectives relating to the area in question. These counts are not adjusted for the number of objectives presented by an organisation in that area. This allows for the fact that one organisation may present the same change in one objective, while another may choose to present it in perhaps five separate objectives.

For some aspects of the TES, further analysis, using statistical methods, has been undertaken of the TEI data to look at the relationship between change-focused objectives in certain areas and the characteristics of the TEI. For example, to explore questions such as: does having a change-focused objective relating to the Pasifika strategy relate to having more or a higher proportion of Pasifika students in the institution?

This analysis has used student information and financial performance ratios. Two tests have been used — median-scores and rank-sums. Median-scores look at whether institutions with an objective in a specified area are more likely to be above or below the median on a particular characteristic. Rank-sums rank the institutions on the selected measure and then determine if having an objective in a specified area significantly affects the TEI’s place in the ranking.

The tests were run for 2005/07 profiles against 2004 student data and 2003 financial performance data; and 2006/08 profiles against 2005 student data and 2004 financial performance data. This would have been the latest data available at the time of writing the profiles. The analysis was run for the individual years and for the two years combined. The 90 percent confidence interval has been taken to indicate a statistically significant relationship. This takes into account the small sample size involved, particularly for individual year data (N=31). However, in many cases, the results have come out beyond the 95 or 99 percent confidence intervals. The p-values for all results are presented in Appendix 2.

Limitations

In addition to the points made above, and the earlier discussion on the nature of profiles, some limitations need to be noted:

• In the case of 2005/07 ITO profiles, the ITOs were provided with a standard template based on the STEP to fit their objectives into. While this provided a high level of consistency across ITO profiles, it also forced ITOs into presenting objectives within the STEP framework. There may be other aspects of the TES being addressed by the ITOs which were not included because they were not specifically identified in the template. There may also be areas of the template where ITOs included objectives which did not have a high level of organisational priority. In the 2006/08 ITO profiles there was more flexibility for ITOs to present their objectives in their own way.

• The analysis undertaken here is subject to the interpretation of the researcher of the statements made in the profile. A more in-depth knowledge of the situation of each organisation could lead to a different interpretation of the material.

• This analysis needs to be read alongside other evidence of change, including TEO annual reports and the annual TES monitoring reports.²

Areas for further work

This report is a starting place for the analysis of information in profiles. Some further extensions of the work could include:

• using qualitative research software to manage coding of objectives to multiple areas of the strategy

• extending the sample to include private training establishments and a larger group of ITOs

• looking at the reported achievement of objectives in profiles over time

• analysing objectives that do not align specifically with the TES

• analysing other parts of the profiles, such as the statements in Part A about planned significant changes

• undertaking an analysis of charters against the TES and profiles

• further developing the analysis of the relationship between expressions of change and various characteristics of the TEOs.

¹ See Gordon Paterson, David Mitchell et al, Engagement of Key Stakeholder Groups with the Tertiary Education Providers, Waikato Institute of Technology and Ministry of Education, 2006. (↑)

² Refer Ministry of Education, Baseline Monitoring Report — Tertiary Education Strategy 2002/07, April 2004; Monitoring Report 2004 — Tertiary Education Strategy 2002/07, April 2005; and The System in Change — Monitoring Report 2005, Tertiary Education Strategy, August 2006. (↑)

← Purpose | Home Page | Overall Findings →

• Contents

• Background
• Purpose
• Methodology

• Overall findings
• Alignment across the six strategies

• Foundation skills
• Skills for a knowledge society
• Research and knowledge creation
• Te rautaki mātauranga Māori
• Pacific Peoples’ development and success
• Capability and quality

*Appendix 1: Mapping of analysis areas to TES and STEP Δ

*Appendix 2: Results of statistical tests Δ