Building Research Strategies in Child Welfare: A Research and Evaluation Framework for Policy and Practice
On this Page:
Marie Connolly 1
Senior Lecturer
Director, Te Awatea Violence Research Centre
Department of Social Work
University of Canterbury
Abstract
While sound public policy development is built on rigorous research, research tends to occur in an ad hoc way, with little strategic attention being paid to identifying research information needs as they interact across a system of policy interest. The development of research strategies that address integrated information needs can help to ensure that the best possible advantage is gained from current and future research. This paper discusses the collaborative development of a research strategy that responds to the information needs of the statutory care system in New Zealand. The strategy addresses the contextual, operational, evaluative and developmental needs of the child welfare statutory care system. The paper discusses the way in which the research strategy was developed, broadly describes the menu of projects that emerged, and considers the implications of the strategy for social policy and practice development.
Introduction
In recent years, there has been a greater focus on developing research strategies that respond to the needs of children and families in New Zealand. Writers have identified the need for enhanced research activity, and have reinforced the importance of family policy that is informed by rigorous intersectoral or cross-portfolio research (Petrie and Wright 2001, Smithies and Bidrose 2000, The Royal Society of New Zealand 1998). Within the child welfare area of children in statutory care, the neglect of strategic research has been identified as an international issue (Courtney 2000, Wise 1999, Berridge 1997).
More generally, writers have called for major child welfare reform (Whittaker and Maluccio 2002). However, ideas of reform are problematic in countries where research is underdeveloped. For example, in both the United Kingdom and New Zealand, innovative child welfare legislation was introduced in 1989 - The Children Act in the United Kingdom, and the Children, Young Persons and Their Families Act in New Zealand. Yet despite their far-reaching implications, overall there has been little work done to more fully understand how the law has influenced the systems of statutory care in the United Kingdom (Berridge 1997), and the same can be said for New Zealand. Within these environments, there is inevitably a danger that policy can be developed without the influence of research, driven by policy and practice cultures alone (Shonkoff 2000). Berridge (1997) identifies a number of possible reasons for the neglect of research, including the unexceptional nature of foster care, lack of political imperative, and the undervaluing of foster care based on its gendered history - the care of children largely being a female activity. However, part of the difficulty may also rest in the complexity of the task of developing a research strategy that captures the broad-ranging concerns of the policy and practice area, effectively harnesses the research that has already been undertaken, and creates the opportunity to positively shape the care system of the future.
In 2002, the Department of Child, Youth and Family Services (CYF) in New Zealand commissioned the development of a research and evaluation strategy to test the effectiveness of the state as a provider of care. This paper discusses the development of the project, including the strategy's methodological approach and framework. It then explores some of the implications of the project for policy and practice development.
The Te Awatea Project Team
The development of the strategy was undertaken in conjunction with CYF by a project team within the Te Awatea Research Centre at the University of Canterbury. The multifaceted nature of the task required that the team be experienced in research (both qualitative and quantitative), policy, and practice within the child welfare area, and also have knowledge and expertise that reflected an understanding of New Zealand's cultural diversity. Sometimes a gulf exists between the world of academia and the world of organisational policy development that has the potential to create unhelpful partnerships and unsatisfactory outcomes. The shared agenda of the project, however, was recognised by CYF and the Te Awatea Project Team, and a commitment was made to ensure that full consultation occurred as the strategy developed. In this way, the transmission of ideas between the two institutions (the research institute of Te Awatea and the social policy domain of CYF) created a lively and productive environment within which issues could be openly debated (Shonkoff 2000).
Building The Strategic Framework
It was necessary to develop the strategy within a limited three-month time frame. This required a tightly structured process with a clearly identified methodology. The development of the strategy was influenced by the grounded theory approach to social research (Glaser and Strauss 1967, Strauss and Corbin 1990), and a model of analysis that was drawn from this approach (Connolly 2003a). A grounded theory approach to research has been defined as:
… theory that was derived from data, systematically gathered and analysed through the research process. In this method, data collection, analysis, and eventual theory stand in close relationship to one another. (Strauss and Corbin 1998:2)
Thus, the process is generally iterative, data collection and analysis being undertaken at the same time and continually being examined in relation to the other:
Throughout the research process, grounded theorists develop analytic interpretations of their data to focus further data collection, which they use in turn to inform and refine their developing theoretical analyses. (Charmaz 2003:250)
Figure 1 The Strategy Development Process
Source: Adapted from Connolly 2003a
The methodology used in the development of the research and evaluation strategy was modelled on this inductive grounded approach. The process began with a generative phase of expanding knowledge and increasing information. The information was then reduced into manageable components through a process of selective specification, followed by a developmental phase that created the domains of the strategy itself (see Figure 1).
Stage 1: The Generative Phase
The generative phase of the methodology focused on the expansion of information and knowledge relating to children in statutory care. Data was generated in three ways. Firstly, a literature review was undertaken. This provided important information about the diversity of factors that impinge on the welfare of children in care, the needs of families that interface with the care system, and the responsibilities of the state. Secondly, data was also generated by the development of a research report that outlined the historical and contemporary perspective of care provision in New Zealand. These two data sources informed the Project Team focus groups, which provided the third area of data generation. The Project Team members had expertise both in research methods and in child welfare policy and practice in New Zealand, and were encouraged to express multi-layered perspectives around a tightly defined topic.
From these three processes, broad research domains were developed (children, families, caregivers, social workers, external providers of care, and the wider professional and service network), and the following six key research questions emerged:
- How well does the state build resilience in those children and families involved in the care system?
- How well does the state facilitate the reintegration of children with their families?
- How well does the state manage transitions into, during and out of care?
- How well does the state form working alliances with the professional and services network?
- What type of care and caregiver produce the best results, for which children, and in what circumstances?
- What professional and organisational conditions produce the best results?
At the end of the generative stage, the project had accumulated a huge amount of information and many specific questions that related to the six broad research questions. At this point, the project team met with CYF to discuss the process so far, and to ensure that the research domains and the developing research questions captured the policy and practice concerns of the agency. We then moved into Stage 2 of the process, the selective phase.
Stage 2: The Selective Phase
The selective phase involved a process of reduction and specification. A subgroup of the project team looked for patterns and relationships between the specific questions we had generated, and noted points of difference. We found that questions relating to resilience, reintegration and transitions could be grouped together under the heading "Chronology of Care", but that care type, conditions of care and questions relating to future planning could not. By a process of further systemising connections, the framework of the strategy emerged: the Context of Care, the Chronology of Care, the Resources for Care, and the Development of Care.
The full project team came together a number of times during this process to examine the integrity of the emerging framework against the original data and research questions that were developed in the earlier stage. Once this was complete, further meetings were held with CYF to ensure that the framework was responding to the agency's policy and practice questions.
Stage 3: The Developmental Phase
It was agreed that the strategic framework responded to the diverse responsibilities of the state, including issues relating to current practice, such as setting standards of care, the effective operation of care systems, the development of support for those systems, and the evaluation of both the care system and its support networks. The strategy also responded to the developmental needs of the care system itself. Developmental responsibilities included the important challenge of shaping the care system of the future and developing the creative thinking around children and care.
While this evolution of care can (and often does) occur in an ad hoc way, it can also occur as a dynamic process of linking grass-roots initiatives with research-based strategies. This approach ensures that innovative and developmental initiatives are imbedded within a research programme that places the initiative within the context of national and international developments, and builds projects in ways that can be soundly evaluated and compared with existing systems.
It was clear from the information gained through the process of developing the research strategy that the reality of children and their care exists within a complex environment. The responsibility of the state with respect to children and their care is also complex, and researching the context of care, the realities of the care experience, and the developmental needs of the system is inevitably a multifaceted task. The strategy needed to respond to such complex needs and bridge the divide between qualitative and quantitative research, recognising the value of both, and promoting ways in which they could be used to respond to the range of questions confronting the state and its care responsibilities (Connolly 2002).
Influenced by Petrie and Wright (2001), the framework was then developed to include a menu of potential research projects. The projects combined small-scale, medium-scale and large-scale studies that offered short-term, medium-term and long-term results. In all, 21 projects were developed (see Figure 2).
Figure 2 The Strategic Framework and Project Development
The Context of Care
The Context of Care component of the strategy includes research proposals that respond to the broader information needs of the care system. For example, the promotion of research that contributes to the contextual knowledge-building needs of the care system includes research relating to children in changing families, how children cope through transitions, and the significance of attachment and resilience for children in the context of changing families. It is in this area, in particular, that the value of cross-portfolio, or intersectoral, research is most significant, since child welfare interests are generally supported by broader research relating to children and families. The Context of Care component also includes the development of databases that inform the care area, with respect to both establishing and maintaining a database of local and international research, and collecting New Zealand data relating to the care system.
The Chronology of Care
The Chronology of Care component relates directly to the core business of children in care, and as such is the centrepiece of the strategy. It includes studies that explore the experiences of children and adults who interface with the care system, and investigate the transitional issues confronting children and families involved in foster care, kinship care and residential care. In particular, projects focus on the transitions of care: entering care, living in care, and exiting from care.
The Resources for Care
The Resources for Care component of the strategy relates to the systems that support the state care of children. It includes a mix of research and evaluation projects that track care placements over time, evaluate care outcomes and compare differing systems of care. It also includes evaluative studies that investigate decision-making practices with respect to entry into care decisions, and exit from care decisions. It is clear from the literature review that questions remain regarding the over-inclusion (or under-inclusion) of children in the care system. Since professional decision-making is critical in these matters, the strategy has developed practice audits to illuminate national trends.
The Development of Care
The final component of the strategy responds to the developmental needs of the care system. It does this in three ways. Firstly, it identifies new initiatives that could reasonably contribute to the development of research and knowledge about the care system. By linking new initiatives with research-based strategies, the knowledge generated by the developmental pilots and projects are also used strategically within a research programme to further the knowledge base of care. Secondly, the strategy proposes the development of intensive specialist treatment pilots in the foster care and residential care areas. Thirdly, the strategy proposes a creative response to the future shaping of the care system by projects that are generative and solution-focused. They involve the use of action research to develop a 10-year plan of care and care management, and the use of meta-analysis to ensure that all existing data is considered in the development of care policy.
Discussion
The experience gained from the project described in this paper suggests that a team approach to the development of a research and evaluation strategy has much to recommend it. A large team can bring together diverse skills, knowledge and experience, which is particularly important when developing culturally responsive research approaches.
However, it is not necessarily enough merely to bring people together in a team. The equality of relationships is important. Power imbalances within the team may result in some team members becoming token contributors to the process. Control of the process can occur when expert knowledge creates interpretative dominance. For example, some team members may have cultural expertise, but may lack knowledge in research methodology and research processes. There may be a professional dominance over community expertise, or there may be a hierarchy of theoretical ideology. In such contexts, team members' contributions may be restricted, creating only an illusion of participation. Participatory teams function more effectively when relationships are equal and members are able to provide meaningful critique.
The qualitative nature of the methodology used in building the strategy had both advantages and disadvantages for the team. A significant advantage of the qualitative paradigm is its capacity to respond to both process and content. This is important when accessing team knowledge and expertise, and particularly so when teams reflect diverse interests. Qualitative methods are fundamentally interpretive and rely on the capacity to view social phenomena holistically and in context. As such, the adopted method suited the generative task of building the strategy.
However, in using a qualitative paradigm, the project was also exposed to some of the more difficult aspects of this type of approach. For example, as with qualitative research, the project generated voluminous data in the shape of hundreds of questions and ideas that needed to be organised in some way. Managing the information became a time-consuming and complex job requiring many hours of team effort. Nevertheless, as patterns developed from the information and as the relationships between ideas became more concrete, the rewards of seeing the strategy develop proved to be very satisfying for the team.
The methodology adopted in this project has been used to respond to the particular information needs of the statutory systems of care. More recently, the same methodological model has been used (successfully) to develop a research and evaluation strategy responding to the information needs of the solution-focused phase of child welfare, in particular, the Family Group Conference process directed by the Children, Young Persons and Their Families Act 1989 (Connolly 2003b). This suggests that the model may be usefully adapted to develop research and evaluation strategies across a range of policy interest areas, particularly when it is possible to delineate the subject area. For example:
- a Domestic Violence Act research and evaluation strategy
- a refugee resettlement research and evaluation strategy
- a mental health community care research and evaluation strategy
- a systems of elderly care research and evaluation strategy.
Developing a clearly defined methodology to interrogate the information needs of a research area, regardless of topic, encourages a more systematic design and instils confidence that the outcomes are comprehensive and robust. Research and evaluation strategies are useful wherever a coordinated approach to a policy area is required.
Developing research strategies within areas of policy interest provides a number of benefits. These strategies help to identify information gaps and crystallise policy and practice issues. They also provide a systematic pathway for research to inform policy and practice and, because they interrogate all aspects of the system, they offer the potential for more holistic responses to research development that are cognisant of policy and practice needs. In this sense, when undertaken collaboratively, they offer a vehicle through which the cultures of research, policy and practice can come together.
References
Berridge, D. (1997) Foster Care: A Research Review, The Stationery Office, London.
Charmaz, K. (2003) "Qualitative interviewing and grounded theory analysis" in J.A. Holstein and J.F. Gubrium (eds.) Inside Interviewing: New Lenses, New Concerns, pp.311-330.
Connolly, M. (2002) "Routes of knowledge toward informed practice: Differing paradigms in social work research" Social Work Review, 14(3):27-29.
Connolly, M. (2003a) "Qualitative analysis: A teaching tool for social work research" Qualitative Social Work: Research and Practice, 2(1).
Connolly, M. (2003b) Solution-Focused Practice and Family Group Decision-Making in Child Welfare: Research and Evaluation Strategy, Te Awatea Violence Research Centre, University of Canterbury, Christchurch.
Courtney, M.E. (2000) "Research needed to improve the prospects for children in out-of-home placement" Children and Youth Services Review, 22(9/10):743-761.
Glaser, B.G. and A.L. Strauss (1967) The Discovery of Grounded Theory, Aldine de Gruyter, Chicago.
Petrie, M. and J. Wright (2001) "A contribution to a research strategy on children's life outcomes" Social Policy Journal of New Zealand, 16:55-75.
Shonkoff, J.P. (2000) "Science, policy, and practice: Three cultures in search of a shared mission" Child Development, 71(1):181-187.
Smithies, R. and S. Bidrose (2000) "Debating a research agenda for children for the next five years" Social Policy Journal of New Zealand, 15:41-54.
Strauss, A. and J. Corbin (1998) Basics of Qualitative Research: Grounded Theory Procedures and Techniques, Sage Publications, Newbury Park, California.
The Royal Society of New Zealand (1998) "Family dynamics: A research strategy prepared for the Ministry of Research, Science and Technology" Wellington, December.
Whittaker, J.K. and A.N. Maluccio (2002) "Rethinking 'child placement': A reflective essay" Social Service Review (electronic version), 76(1):108.
Wise, Sarah (1999) The UK Looking After Children approach in Australia, Research Report no.2, Australian Institute of Family Studies, Melbourne.
1 Acknowledgements
This paper is based on a report commissioned by the Department of Child, Youth and Family Services to develop a research and evaluation strategy to test the effectiveness of the state as provider of care for children at risk. The strategy, Children in Statutory Care: Experiences and Outcomes, was developed by a project team at the Te Awatea Research Centre, University of Canterbury, in collaboration with the Department of Child, Youth and Family Services.
Correspondence
M. Connolly, Department of Social Work, University of Canterbury, P.O. Box 4800, Christchurch, New Zealand.
Email: marie.connolly@canterbury.ac.nz
