Youth Service Evaluation Snapshot

This snapshot presents findings from the evaluation of Youth Service in its first 18 months of operation. The findings consider:

young people’s participation in education, training or work-based learning, enrolment in budgeting and parenting activities, and achievement of NCEA qualifications
the impact of Youth Service on the main benefit receipt of Youth Payment (YP) and Young Parent Payment (YPP) participants. This is assessed using a comparison group drawn from young people who started a main benefit before the introduction of Youth Service.[1]

[1] This comparison group is made up of Independent Youth Benefit, Emergency Maintenance Allowance, and Domestic Purposes Benefit clients.

Youth Service Background

Youth Service is a new approach to working with vulnerable young people where MSD contracts a service provider to work actively and intensively with a young person

Youth Service was established in August 2012 to address the long-term impact early entrants to the benefit system have on overall benefit liability. Early entrants have a very high average liability of $109,000 per client. Youth Payment participants within the early entrants cohort have the highest liability at $185,000.

The aim of Youth Service is to connect young people with education and training as well as budgeting and parenting courses to equip them with skills and resilience to reduce their likelihood of requiring income support in the future.

The outcomes sought from Youth Service are that participating young people have a reduced risk of long-term benefit dependency, achieve at least an NCEA level 2 qualification (or equivalent), and have improved social outcomes for themselves and their children.

A strong focus on youth is required to reduce the number of people at risk of long-term benefit receipt

Although the number of new youth entrants to benefit each year is low, the most recent actuarial valuation shows that these participants account for a significant proportion of the liability over time. For instance, more than 70 per cent of the forward liability is in respect of people who first received a benefit before the age of 20 – indicating that many of these young people remain vulnerable to benefit dependency their whole lives.

Many young people enter Youth Service requiring intensive support

Many young people who enter Youth Service have serious or complex issues that require intensive support. For example, a review of Youth Payment grants by Work and Income found that 38 per cent of applicants were victims of domestic violence, 76 per cent suffered emotional neglect and 5 per cent were homeless or victims of sexual abuse.

Initial findings

Initial findings show that most young people in Youth Service are engaged in education, training and work-based learning activities, and are beginning to show signs of building independence from the benefit system:

there is evidence Youth Service is helping young people to work towards and gain an NCEA Level 2 qualification
early evidence suggests that Youth Service is helping YP participants spend less time on main benefit over the medium term, and that fewer young people are transitioning to a main working-age benefit.

Most young people in Youth Service are engaging in education, training and work-based learning activities

At the end of March 2014, four out of five young people enrolled in Youth Service were engaged in education, training or work-based learning in either a full-time or part-time capacity (72 per cent for YP; 79 per cent for YPP; and 64 per cent for NEET; Table 1 in Annex). [1]The majority of these young people were engaged in some form of education. The small number of participants who were not engaged in an education or training activity are likely to be new to the Service and not yet had an activity assigned to them.

Prior to the establishment of Youth Service many young people who received Work and Income assistance at age 16 or 17, such as the Independent Youth Benefit (IYB), were incentivised to look for employment as a first choice, accessing training or education only if they were unable to gain a full-time job. As a result, many of these young people gained employment in low-skilled industries that did not equip them with enough skills to prevent them requiring income support in the future, and IYB participants had one of the highest average benefit liabilities.

Given the opposite focus of Youth Service, we expect that more young people on benefit are now engaging in education or training instead of moving into employment in the short-term.[2]

It can take several months to engage a young person in an education or training activity, as many enter Youth Service disengaged from school and can have other complex issues

The longer a young person is enrolled in Youth Service, the more likely they are to be engaged in education, training, or workplace learning (Figure 1; Table 2 in Annex).

Many young people who enter Youth Service have experienced difficulties with mainstream education, are disengaged from school, and have serious or complex issues that require intensive support. As a result, it can often take several months before some participants are ready to enrol in an education or training activity, with providers and coaches instead working with the young person to stabilise their living situation or other factors in their life first. Correspondingly, engagement in education, training, or workplace learning increases with the length of time a young person is enrolled in Youth Service.

Under Youth Service, young people are also accessing budgeting and parenting support

Most young people undertake budgeting activities within their first five months of enrolment. Around 55 per cent of YP and YPP participants are engaged in budgeting activities five months after starting Youth Service (Figure 5 in Annex). Enrolment in this activity declines thereafter.

YPP participation in parenting programmes increases with the time a young parent spends in Youth Service. After one month’s duration, only nine per cent of YPP participants are enrolled in a parenting activity, compared to 50 per cent after 10 months (Figure 6 in Annex).

Early evidence suggests that Youth Payment participants spend less time on benefit under Youth Service

In the first six months after starting Youth Service, young people receiving YP initially spend slightly longer on benefit when compared to previous IYB recipients (Figure 7 in Annex). This effect is due to YP participants remaining on benefit while they complete their education or training.

However, after one year fewer Youth Payment recipients are receiving a main benefit relative to before the Youth Service was established (Figure 7 in Annex). These findings suggest that the Service is improving YP participant’s prospects of moving off benefit. We expect this is due to participants taking up employment or full-time tertiary training. Further analysis is required to confirm this.[3]

Fewer Youth Payment participants are transitioning to a working-age benefit

As a consequence of fewer YP participants receiving benefit after one year in Youth Service, the proportion of young people eligible for YP[4] who transition from a youth benefit onto a working-age main benefit has fallen by eight percentage points since Youth Service was implemented:

over the 12 months to March 2014, 30 per cent of YP participants transferred to a main working-age benefit
over the 12 months to March 2012, 38 per cent of Independent Youth Benefit participants transferred to a main working-age benefit.

It is too early to assess the impact of Youth Service for Young Parent Payment[5]

At present, the number of Young Parent Payment (YPP) participants is too small to observe a meaningful impact on benefit receipt (Figure 8 in Annex). Significant impacts are not anticipated until four to five years after teen parents start Youth Service, based on timeframes observed by the evaluation of the Training Incentive Allowance for sole parents. Because of their childcare responsibilities, we expect it is unlikely that many YPP participants can take full advantage of any gains in qualifications under Youth Service to move into employment until their children reach school age.

The Youth Service appears to be positively influencing young people’s ability to work towards and gain an NCEA Level 2 qualification

In the first 12 months of enrolment in Youth Service, YP and YPP participants are more likely to gain NCEA credits than similar young beneficiaries before Youth Service was established:

nearly two-thirds (63 per cent) of YP participants increased the number of credits they held towards an NCEA level qualification in their first 12 months of participating in Youth Service, compared to one quarter (24 per cent) of their comparison group (Figure 9 in Annex)
over two-fifths (43 per cent) of YPP participants increased their number of NCEA credits in their first 12 months, compared to one-fifth (20 per cent) of their comparison group (Figure 9 in Annex).

YP and YPP participants are more likely to meet the requirements[6] of NCEA Level 2 within their first 12 months in Youth Service

Within their first 12 months in Youth Service, 14 per cent of YP participants and 7 per cent of YPP participants met the requirements of NCEA Level 2. Over this period, both YP and YPP were more likely to meet the requirements for NCEA Level 2 relative to similar young beneficiaries before Youth Service was established (a nine percentage point gain for YP, and a two percentage point gain for YPP) (Figure 10 in Annex).

However 12 months after enrolment in the Service, YP and YPP participants remained less likely to have met the requirements of NCEA Level 2 than young people in the comparison group (Figure 10 in Annex). This is because, at the point of entry to Youth Service, YP and YPP participants were less likely to have met the requirements of an NCEA Level 2 qualification than their comparison group. There are two reasons for this:

the comparison group was selected without consideration of prior educational achievement, as NZQA qualifications data was not available at the time of the matching process
Youth Service targets young people who have lower levels of educational achievement.

As a result, the gains made by YP and YPP participants toward NCEA Level 2 under Youth Service have not been large enough to compensate for this difference (Figure 10 in Annex).

A more robust estimate that controls for differences in prior educational achievement will be developed for the 2015 evaluation update.

NEET[7] young people in Youth Service are achieving NCEA credits and qualifications

Of these young people:

half increased their number of NCEA credits within 12 months of starting Youth Service
15 per cent met the requirements of NCEA Level 2 within their first 12 months in the Service
17 per cent had already met the requirements of NCEA Level 2 prior to starting the Service.

[1] Some these young people may have been engaged in short courses, such as obtaining a driver’s license.

[2] We are unable to assess whether Youth Service has increased participation in education and training for young people on benefit, as education and training data were not collected for young people receiving benefit prior to the introduction of Youth Service.

[3] Analysis of employment outcomes will be able to be undertaken once MSD has completed linking with the Integrated Data Infrastructure (IDI) at Statistics New Zealand (within 12 months).

[4] Young people without children who are receiving a benefit from Work and Income.

[5] In addition, we are unable to assess the impact of Youth Service on the benefit recipient of NEET participants. Historical information on NEET young people who did not receive a benefit from Work and Income is not available for us to construct a comparison group.

[6] Data on NCEA qualifications was not available for this analysis. Instead we have counted young people as having ‘met the requirements’ for NCEA Level 2 when they had gained 80 NCEA Level 2 credits.

[7] The evaluation was able to match NZQA qualifications data for approximately 17,500 (89 per cent) of the 20,000 young people receiving Youth Service for NEET. Historical information on NEET young people who did not receive a benefit from Work and Income is not available for us to construct a comparison group for Youth Service: NEET educational achievement.

Limitations of the evaluation

There are limitations on this analysis around the comparison group used. In particular, the comparison group has been matched on a small number of participant characteristics[1] and are taken from participants starting youth benefits between 2008 and August 2012. As a result, we cannot rule out the possibility that reported impacts are due to uncontrolled differences in participant profiles (eg school achievement) or because of changing labour market conditions before and after the introduction of Youth Service. To eliminate these alternative explanations will require more rigorous analysis. A more robust estimate that better controls for differences in the participant profiles will be developed for the 2015 evaluation update.

In addition, a further limitation of the comparison group used to assess educational achievement outcomes is that it was selected without consideration of prior educational achievement. This was because the required NZQA qualifications data was not available at the time the matching process was carried out to select the comparison group. As a result, a higher proportion of young people in the comparison group had met the requirements for NCEA Level 2 at the point of entry to benefit than YP and YPP participants.

Historical information on NEET young people who did not receive a benefit from Work and Income is not available for us to construct a comparison group for NEET clients receiving Youth Service.

The total number of participants enrolled in the Youth Service generally declines across the 18-month period. In the case of YP participants, the total number of participants still enrolled in the Youth Service at the 18-month mark is approximately 400, compared to approximately 6,600 at the one-month mark. This means that small variations in activity enrolment will have a larger effect on the percentage enrolled at later months, compared to the earlier months. This effect is purely statistical in nature.

[1] These include whether they were a teen parent, time from starting a main benefit, and age at first benefit start date.

Annex

Table 1: Engagement in education, training and work-based learning activities (either full-time or part-time); current Youth Service participants at 31 March 2014

Activity	YP	YPP	NEET
Education	72%	79%	64%
Training	25%	17%	18%
Work-based learning	2%	1%	4%
Any education, training or work-based learning	88%	87%	77%
No education, training or work-based learning activity	12%	13%	23%

Note: Counts are based on data reported by Youth Service providers in the Activity Reporting Tool (ART) and include short courses, such as obtaining a driver’s license. Counts for Education, Training and Work-based Learning are not mutually exclusive: people may participate in more than one activity.