Evaluation Resources

This section includes a range of national and international resources including reports on monitoring and evaluation methodologies and strategies to promote and support best practice in suicide prevention programs and services as well as strengthening community engagement, Indigenous governance and self-determination.  It includes a range of culturally appropriate tools and techniques that are applicable to use in various settings including in remote, rural and urban contexts when conducting evaluations with Indigenous community controlled organisations. There are also a range of guidelines and resources to ensure mainstream services and organisations adhere to Indigenous best practice principles and guidelines. The resources encompass a practical, feasible, culturally responsive and user friendly approach to monitoring and evaluation in diverse Indigenous community contexts.

Fisher, S. (2012) Monitoring and evaluation methodologies for remote settings: A literature review conducted in 2010 Report to the Office of Indigenous Policy Coordination, Department of Families, Housing, Community Services and Indigenous Affairs Working Paper  http://www.nintione.com.au/resource/NW001_MonitoringAndEvaluationMethodologiesForRemoteSettings.pdf

This report was undertaken by Ninti-One and provides useful material for staff working on new initiatives in monitoring and evaluation. Drawing on a review of the literature this report provide an overall analysis and recommendations regarding the conduct of monitoring and evaluation in Indigenous remote communities.

Guenther, J, Galbraith, M 2014, Learning from evaluations of school-family strengthening programs: lessons for all

Abstract: For more than 10 years, a program called Families And Schools Together (FAST) has been run in schools across the Northern Territory. These programs have always had an evaluation component built in. However, over time, the evaluations have changed. Initially, they were summative, built around a quantitative psychometric tool (with a positivist research paradigm). The intent of the summative evaluation was to demonstrate the effectiveness of the program. However, as the program was rolled out in remote contexts, the need for adaptation was recognised. Changes were made but it was soon recognised that other outcomes, not captured or explored in the methodology, were emerging. After six years of working together, the evaluator and the program manager felt that it was time to explore effectiveness in a different way with a more qualitative evaluation process (based on naturalistic and pragmatic paradigms). The purpose of this paper is to share learnings from this experience with other program managers and evaluators. The presentation will be an opportunity for participants to engage in a discussion about monitoring and evaluation from a program evaluation perspective, particularly taking account of the complexities of the northern Australian context. This paper explores the six year learning journey through evaluation that the evaluator and program manager have undertaken. It describes tensions between the need for reliable and generalizable objective quantitative data and the need for authentic and credible data based on participant experience.

Maughan, C  2012, Monitoring and evaluating social impacts in Australia, Ninti One Limited

Abstract: Monitoring and evaluation (M&E) allows people to learn from past experiences, improve service delivery, plan and allocate resources and demonstrate results as part of accountability to stakeholders. M&E also assists in keeping projects on track, providing a basis for reassessing priorities and creating an evidence base for current and future projects There is a growing interest in the measurement of social impact across the business, government and nonprofit sectors. In recognising the role that non-profit organisations play in ‘enhancing the economic, social, cultural and environmental wellbeing of society’, the Australian Government has recently focused on improving the measurements of social impact This report describes the main monitoring and evaluating frameworks and methods used in Australia which include some assessment of social impact. Each of the following are discussed in terms of an overview of how it works (in terms of M&E), the benefits and limitations, examples of organisations using the technique and where to find further resources.

Rogers, Alison, Harrison, Nea, Puruntatameri, Therese, Puruntatameri, Alberta, Meredith, Joan, Dunne, Rachel 2018, Participatory evaluation is the sea eagle looking “long way wide eyed” https://journals.sagepub.com/doi/abs/10.1177/1035719X18778712

 Abstract: Participatory evaluation can be embedded in programs to support good governance and facilitate informed decision making in Aboriginal communities in remote and urban contexts. An Aboriginal Elder from the Tiwi Islands in the Northern Territory of Australia described participatory evaluation as a sea eagle looking “long way wide eyed.” The metaphor refers to the long-term and broad approach undertaken when a complex community development program used participatory processes to build evaluation capacity and solve problems. The evaluation approach ensured the program was inclusive, responsive, empowering, and resulted in direct benefits for the communities. This article addresses the lack of literature on applying developmental and empowerment evaluation approaches in practice by describing the methods, tools, and use of evaluation findings. The value of participating for the community members and partner organizations is shared and the benefits and implications for participants and the evaluator are discussed. The authors hope this article inspires practitioners and evaluators to consider participatory ways of working with communities to support community directed action and social change.