Background

The CBPATSISP Evaluation Framework builds on the ATSISPEP Evaluation Framework which was informed by a comprehensive national and international literature review, Indigenous community consultations 2009 – 2015 and Roundtable consultation findings, advice and direction, by an Indigenous expert leadership group, and the National Aboriginal and Torres Strait Islander Mental Health Indigenous Leadership (NATSIMHIL).

Importantly, the Evaluation Framework is underpinned by Indigenous understandings of the causes and solutions to suicide as well as evidence based national and international best practice research in Indigenous evaluation and suicide prevention. Furthermore, the Evaluation Framework reflects Indigenous Australian peoples’ holistic view of health, mental health, physical, cultural and spiritual health.

The evaluation methods and methodologies to assess the effectiveness and cultural appropriateness of prevention programs are designed to address a range of complex issues including social and historical determinants and inequities contributing to Indigenous suicide at individual, family and community levels and across diverse geographic contexts.

The Evaluation Framework serves two main purposes for different audiences:

  1. As a resource guide to support community planning, ownership and delivery and evaluation of best practice suicide prevention programs to reduce suicide and suicidal behaviours in specific communities and for specific population groups.
  2. It provides a set of principles and standards to review existing programs, services, training programs and resources to assess the extent to which they are both effective and culturally appropriate.

The Evaluation Framework comprises several critical components to evaluate key outcomes related to community based programs, universal, indicated and targeted programs, workforce initiatives, clinical services and organisational delivery of suicide prevention strategies and resources for different stakeholders.

The Evaluation Framework provides practical evaluation approaches, methodologies and resources for program stakeholders that include:

  • Methods and performance indicators to measure the effectiveness and cultural responsiveness of community-led or co-designed Indigenous suicide prevention programs, services and strategies which present a new way forward
  • Principles, standards and social and emotional wellbeing indicators developed on basis of the extensive community consultations and expert advice from the ATSISPEP National Advisory Group
  • A set of guidelines to integrate and implement Indigenous determined, culturally relevant, evidenced-based principles and indicators within programs, services and initiatives which aim to address suicide prevention

The Evaluation Framework builds on the ATSISPEP project foundations to ensure that assessments of best practice, promising practice, what works and what needs to happen are informed by Indigenous knowledge, perspectives and priorities.

The Evaluation Framework provides the basis to:

  • Assess the extent to which key principles and processes to enhance Indigenous self-determination, such as equal partnership, dual accountability and negotiation, are enacted in the development, implementation and evaluations of suicide prevention initiatives and interventions
  • Assess how various programs which aim to address risk factors strongly associated with suicide and self-harm in urban, rural and remote areas effect or contribute to social wellbeing outcomes
  • Assess the extent to which programs, policies and practices strengthen community and build capacity (in both Indigenous and broader societal contexts)
  • Provide a set of criteria by which to assess/critique the appropriateness and usefulness of existing social and performance indicators used by government and non-government agencies to measure the effectiveness of services, programs and interventions intended to reduce suicide and self-harm in diverse community and geographic contexts
  • Provide an inventory of different context-specific best practice examples to enhance program delivery in similar settings and more accurately inform policy decisions and resource allocations
  • Establish indicators to measure the extent to which Indigenous goals, terms of reference, equitable processes and outcomes are achieved

In summary, the Evaluation Framework provides a mechanism to monitor, evaluate and report on:

  • The effectiveness and cultural responsiveness of suicide prevention programs and services
  • Community, organisational and service level evaluations of outcomes of programs and services
  • Complex multilevel and integrated services strategies.

Rationale

The urgent need for successful initiatives to reduce the high rates of Indigenous suicide and self-harm among Aboriginal and Torres Strait Islander peoples in Australia is widely acknowledged by Governments and Aboriginal communities. Over the past 15 years a number of Australian studies have established the need for a culturally responsive evaluation framework (Walker & Scrine, 2016). All of these studies highlight the problems with applying conventional health and social indicators in assessing wellbeing and the need to take into account Indigenous community goals and social and community contexts in planning, implementing and evaluating community-based prevention efforts.

For instance, Walker et al. (2003) highlighted the need for:

…a substantial reframing of evaluation approaches to encompass these other policy parameters to gauge how well governments are successfully working towards Indigenous self-determination by maintaining partnerships, engaging in dual accountability and implementing effective program linkages.

The Evaluation Framework is used to formally assess the relevance, acceptability, effectiveness and culturally appropriateness of existing suicide prevention and early intervention initiatives, programs and services in Indigenous communities in Australia for inclusion on the CBPATSISP Clearinghouse Best Practice Programs and Services and Resources.

The Evaluation Framework also provides a practical planning and evaluation tool for communities and organisations to apply in their work in suicide prevention and for professional practitioners, program and service providers working in suicide prevention, interventions and clinical services.

Specific approaches to conducting and evaluating significant, complex multi-agency projects with Indigenous remote communities in Australia to bring about positive social and inter-cultural change are outlined by Kania and Kramer (2011). The key principles and values identified as crucial in these Australian studies are also echoed in international suicide literature and have informed the development of this Evaluation Framework.

Indigenous Goals, Principles and Rights

The Evaluation Framework encompasses the interrelationship of Indigenous research goals, principles and outcomes within a human rights framework. It also incorporates the key values and principles underpinning the CBPATSISP and is premised on the belief that Indigenous peoples have the right to self-determination and to freely pursue their economic, social and cultural development .

The Evaluation Framework provides the basis for discussions, decision-making and negotiation between Indigenous groups, program providers and evaluators about the ‘what’, ‘how’ and ‘why’ elements to support the effectiveness and culturally appropriateness of existing and/or new programs and services in reducing suicide and strengthening Indigenous social and emotional wellbeing.

Essential Criteria

The criteria for assessing relevant programs and services is based on extensive evidence that to be effective in reducing suicide and social and emotional wellbeing programs and services need to:

  • Assist in Indigenous capacity building
  • Prioritise Indigenous knowledge and experience
  • Respect cultural values
  • Recognise Indigenous rights and self-determination
  • Facilitate cultural strengthening
  • Facilitate and promote Indigenous leadership and governance
  • Foster genuine partnerships and community engagement
  • Promote healing

This conceptual framework is intended as both a process guide and a ‘cultural audit’ for applying Indigenous Evaluation principles and indicators to specific populations, issues, and community and organisational contexts.

Self-determination

Importantly, the Evaluation Framework is premised on the notion put forward by key Indigenous leaders that Indigenous self-determination is both a goal and a non-negotiable right – and as such, is a fundamental principle and criteria of all research concerning Indigenous peoples and a critical indicator of social and emotional wellbeing.

The Evaluation Framework will be used as a guide in reviewing existing programs, services, training programs and resources to assess the extent to which they are both effective and culturally appropriate.