Acta Trop. Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, Mathai M, et al. 2003;19(4):613–23. In Nigeria, for example, despite the significant reduction in maternal and neonatal mortality since 2003 by 50 and 23 %, respectively, these still remain high at 545/100,000 and 37/1000 births, respectively, particularly in rural areas where most vulnerable groups reside [2, 3]. Fiona Keogh, Maria Pierce, Karen Neylon, Padraic Fleming, Intensive home care packages for people with dementia: a realist evaluation protocol, BMC Health Services Research, 10.1186/s12913-018-3630-8, 18, 1, (2018). 2012;18(2):176–91. Bhutta ZA, Memon ZA, Soofi S, Salat MS, Cousens S, Martines J. While the average number of PHC facilities in each Nigerian state is typically between 1000 and 4000, the SURE-P/MCH was implemented in clusters of 12 to 21 health centres within selected states. Adequate communication of results is an essential component of this study. studies, policy documents and so on) and so it is a form of secondary research. acceptability of the intervention by the communities and front-line service providers, appropriateness of the intervention design to the current context of PHC facilities in Anambra state and sustainability of changes achieved from the implementation of interventions in the longer term . The impact of the programme on key output and outcome indicators will be assessed through an interrupted time-series analysis (ITS) of monthly quantitative data from health information system and programme reports. Theory-driven forms of evaluation help understanding such complexity by studying how the different elements are intertwined  and recognising the role of context as a key influence in the production of outcomes . Accountability for Maternal, Newborn and Child Survival: An Update on Progress in Priority Countries. Introduction Significant evidence in the literature supports case management (CM) as an effective intervention to improve care for patients with complex healthcare needs. PubMed Central the Charter of Fundamental Rights of the EU) and international conventions (e.g. Evaluation of complex interventions requires a comprehensive understanding of intervention context, mechanisms and outcomes. This realist technique acknowledges that intervention programmes and policy changes do not necessarily work for everyone, since people are different and are embedded in different contexts. 2nd ed. BMJ. 2007;101(2):95–105. Specific methods of communicating research will include combinations of: Developing short and practical policy briefs to national and international policymakers and practitioners. Using IDIs, we will explore actors’ understandings and views about the intervention’s context and processes (or mechanisms), their expected and unexpected effects (outputs and outcomes) and effects of advocacy and lobbying in entrenching the MCH on the political agenda. The Logic Model Guidebook: Better Strategies for Great Results. Hum Resour Health. Findings from step 1 will be used to hypothesise the likely changes (interruptions in trends and levels) in the time series to make predictions to be verified during the ITS analyses. We realise, however, that implementation of a new programme can have an impact that goes beyond the direct programme beneficiaries and will consider these effects qualitatively through conducting IDIs with key actors referred to earlier. There are few published examples of realist synthesis. Google Scholar. Unnecessary collection of personal data will be avoided, and respondents will have the right to review outputs and withdraw consent. individual), meso (organisational) and macro (i.e. It explains when a realist impact evaluation may be most 2011;26 suppl 2:ii84–91. , Realist evaluation techniques recognise that there are many interwoven variables operative at different levels in society, thus this evaluation method suits complex social interventions, rather than traditional cause-effect, non-contextual methods of analysis. Two supplementary materials are included from this handbook. Health Res Policy Syst. Consumers stated and revealed preferences for community health workers and other strategies for the provision of timely and appropriate treatment of malaria in southeast Nigeria. Our study should improve understanding of the performance and functioning of complex system interventions involving both supply and demand sides. Implementing community-based perinatal care: results from a pilot study in rural Pakistan. It is based on the epistemological foundations of critical realism.Based on specific theories, realist evaluation provides an alternative lens to empiricist evaluation techniques for the study and understanding of programmes and policies. The data will be collected using three methods. Realist evaluation reports need to be developed in line with the realist nature of the evaluation. In the discussion, we also use to share some key initial methodological outputs which were produced since the start of the project . Google Scholar. Realist evaluation (RE) is a theory-driven evaluation method that is increasingly used for studying the implementation of complex interventions within health systems, including in low- and middle-income countries [23–25]. middle-range theories) that link intervention inputs to processes, outputs and outcomes within the context of Anambra state. reproductive and/or MCH, health workforce) and (b) a limited number (3–5 in total) of initial in-depth interviews (IDIs) with purposefully selected federal and state SURE-P programme managers. Springer Nature. neonatal and perinatal deaths); T This quantitative analysis interrelates with the qualitative IDIs mentioned earlier. California Privacy Statement, Witter S, Khadka S, Nath H, Tiwari S. The national free delivery policy in Nepal: early evidence of its effects on health facilities. 'Protocol based care' was envisioned by policy makers as a mechanism for delivering on the service improvement agenda in England. Preliminary evidence indicates that paying CCTs to pregnant mothers is linked to increase antenatal care visits and facility deliveries . realist evaluation covering the same kind of project or programme. This study will make an important and timely contribution to health systems strengthening in Nigeria. t Uzochukwu B, Obikeze E, Onwujekwe O, Onoka C, Griffiths U. The IWTs subsequently guided the identification of the specific information areas for the data collection and analysis. The hypotheses can also cover links within and between components of the conceptual framework (Fig. is a time series trend variable; I The making of Jamaica’s ‘National Policy for Persons with Disabilities 2000’: macro, meso and micro factors. We discussed the best course of action for our research with Nigerian health authorities and our funders. Ramsay CR, Matowe L, Grilli R, Grimshaw JM, Thomas RE. BMC Public Health. Implementation Science Jokhio AH, Winter HR, Cheng KK. The project will be implemented according to standard governance practices at the University of Leeds and University of Nigeria. A realist evaluation study protocol. Malar J. The study methodology will include three steps (Fig. This is known as the C-M-O configuration , and without accounting for all these dimensions, some aspects of the programme may go unrecorded, thus affecting the validity and reliability of results  and preventing replication . This handbook is available upon request. implications of staff absences due to training on availability and quality of MCH services) and any relations between the different outputs and between the different outcomes. A realist approach emphasises the contingent nature of programme outcomes and addresses questions about what works, in which setting, for whom, in what circumstances and why . However, to guide further development and implementation of CHW programmes in different contexts, it is necessary to better understand what makes CHW programmes successful in achieving desired outcomes and under what circumstances they succeed [11, 12]. The quality of government and programme indicators will first be assessed by comparing the data with relevant indicators from the demographic surveillance system, and we will adjust the HMIS/M&E variables accordingly in the case of major discrepancies. Country Implementation Plan for Prioritized Life-Saving Commodities for Women and Children. The study comprises three steps: (1) initial theory development; (2) theory validation and (3) theory refinement and development of lessons learned. Although we do not anticipate any further data collection for this stage, we do not see a linear progression between the steps: i.e. Green A, Thomas C, Newell JN, Baral SC, Byanjankar L. The contribution of volunteers to a successful community-orientated tuberculosis treatment centre in an urban setting in Nepal: A qualitative assessment of volunteers' roles and motivations. Ritchie J, Spencer L. Qualitative data analysis for applied policy research. CAS Resources Overview Realist Synthesis: An Introduction: This paper presents an introductory overview of realist synthesis as applied to the review of primary research on healthcare systems. Pawson R, Tilley N. Realist Evaluation. The qualitative evaluation proposed will be based on the document analysis of yearly ICP progress reports, selected case studies and focus group interviews with stakeholders. The parameter β0 represents the baseline level of the dependent variable; β1 represents the baseline trend in the dependent variable; β2 isolates any shift in the level (intercept) of the dependent variable following the introduction of the policy; β3 isolates any shift in the rate of change (slope) following the introduction of the policy and β4 represents the effect of a covariate. However, there is still little evidence about the facilitators and barriers to CM implementation in primary care setting. In addition to assessing whether the target indicators are achieved, we will identify how they are achieved (i.e. Geneva: WHO; 2007. Article 2011;6(1):11. Ebenso B, Manzano A, Uzochukwu B, Huss R, Etiaba E, Ensor T, et al. The two interventions (i.e. Options discussed included stopping or amending the research, as well as technical and political implications of each option from the different (policymakers’ and the funder’s) perspectives. The project uses an input-process-output-outcome continuum (see Fig. Realist impact evaluation draws on the realism school of philosophy. Health Policy Plan. autocorrelation), and important covariates will also be adjusted for, while multilevel methods will be used to address variation across health facilities. A realist evaluation provides an overall methodological framework for this multidisciplinary and mixed methods research, which will be undertaken in Anambra state. J Dev Econ. Hudson J, Lowe S. Understanding the Policy Process: Analysing Welfare Policy and Practice. A favourable policy environment within which the study is conducted will ensure the successful uptake of results into policy and practice. it is a form of primary research. 2011;38(2):65–76. CHW programmes are inherently complex, and their success is mediated by how the intervention is implemented within the health system context. Implicit in the figure are the intervention implementation outcomes, i.e. Subsidy Reinvestment and Empowerment (SURE) Programme. alternative respondents). This 5-year study is implemented in Anambra state, located in the southeast of Nigeria with a total of 4420 PHC facilities serving a population of 4.2 million. While studies have explored the effects of supply- and demand-side interventions separately [15, 18, 19], the combined effects of the two—such as the added value of CCTs within health programmes—are rarely assessed within the same intervention. is an error term. Cost Eff Resour Alloc. assessment of context will inform best practices in PHC staff performance management; assessment of the added value of CCTs will inform further demand-side financing schemes. This realist evaluation protocol sets out the approach that will be used to identify and explain which aspects of the programme are working, for whom and in what circumstances to produce the outcomes intended. London: SAGE Publications; 1997. statement and First, the policy environment in Nigeria within which the research is being undertaken is favourable to ensure a high-quality analysis, inform theoretical debate and ensure the uptake of results into policy and practice, as we found within our previous collaborative projects. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. An incremental cost-effectiveness analysis will be undertaken (cost per additional skilled delivery and associated care). Onwujekwe O, Uzochukwu B, Ojukwu J, Dike N, Shu E. Feasibility of a community health worker strategy for providing near and appropriate treatment of malaria in southeast Nigeria: An analysis of activities, costs and outcomes. All personal data will be coded, removed from the data for analysis and stored separately. The supply component also included infrastructure development, improving availability of supplies and medicines and activation of Ward Development Committees (WDCs). The challenge posed by dementia, which is the confluence of increasing life expectancy, population growth and the lack of cura-tive treatments, has been well documented [42, 54]. Lindsay B. Randomized controlled trials of socially complex nursing interventions: creating bias and unreliability? As new programmes emerge or existing programmes scale up, comparing the effectiveness of CHW programmes between different contexts becomes important. Part of 2008;6(1):20. At the outcome level, increased rates of facility delivery could help improve delivery outcomes, leading to a reduction in perinatal deaths. Its overall place in the toolkit of evaluation approaches is not yet fully established. In RE, researchers develop middle-range theories that take account of how Context (at micro, meso and macro levels) influences intervention processes or Mechanisms (e.g. organisations and their roles) and micro level (e.g. A detailed list of respondents will be developed within step 1, and snowballing will be used to identify any further informants. Only designated research staff will have access to the keys linking the data with the personal information. Realist evaluation provides an overall methodological approach for the project and will guide development, testing and refining of middle-range theories through the analysis of the relationships between the context (at macro, meso and micro levels), mechanisms and outcomes . Since December 2012, CCTs have been implemented in selected SURE-P/MCH sites in 9 of the 36 states of Nigeria (SURE-P/MCH + CCT). Wolff N. Randomised trials of socially complex interventions: promise or peril? The additional costs on the supply side are likely to be training midwives and CHWs, salaries, equipment and other supplies (Table 2). PubMed I Protocol for a rapid realist review July 2020 Informed consent will be obtained from all study participants, and in the case of refusal, alternative means of data collection will be explored (e.g. This paper should be of interest to researchers who are adapting and applying robust methodologies for assessing complex health systems interventions and policymakers and practitioners who are interested in commissioning, and engaging in, similar evaluations. This paper describes our longitudinal, mixed methods realist evaluation of HNA and care planning. Second, facility exit IDIs will be conducted with 20–25 purposefully identified users to explore in-depth their views. Working with International Medical Corps (IMC), an initial programme theory was elicited through literature and document reviews, semistructured interviews and focus groups with IMC programme managers and CHWs. Two aspects of the environment within which the study is being implemented are worth noting. It describes five key ideas which frame realism and their implications for impact evaluation. 2013;28(4):358–66.
2020 realist evaluation protocol