Evaluation Guides

//Evaluation Guides
Evaluation Guides2018-10-30T04:14:05+00:00

Evaluation Guides

This section provides several guides for conducting evaluations:

Community Guide to Planning and Evaluating Community-Based Initiatives
Outlines a step by step process for planning, implementing and evaluating suicide prevention initiatives that empowering capacity building community strengthening.

Guide to Evaluating Organisations and Services
Includes questions and considerations for assessing key evaluation criteria including the effectiveness and cultural responsiveness of programs and services, continuous quality improvement and for making decisions around funding and developing programs.

Guide to Evaluating Clinical Services, Protocol and Practices (under development)
Outlines the key elements that need to be considered in providing clinically safe care, and ensuring effective, screening, diagnosis and treatment in a range of clinical settings to support Aboriginal and Torres Strait Islander people who have attempted suicide, experienced mental health issues or suicide ideation.

This section outlines steps for community groups, community controlled organisations and services to work in partnership to plan, implement and evaluate Suicide Prevention initiatives using Participatory Action Research (PAR) processes to ensure that programs are responsive to the local culture, context and specific issues.

PAR a key component in planning, developing and delivering effective community-led suicide prevention programs and services for Indigenous peoples. It involves: 1) developing a culturally safe environment for program participants; and 2) fostering trusting relationships between the community and stakeholders. Establishing trust and building respectful relationships and genuine partnerships is an important first step in gaining community support, facilitating community ownership and engagement and the broader community for, and uptake of, suicide prevention programs and services.

PAR provides a mechanism to integrate the cultural and social relevance for communities with rigour. The demonstrated benefits of using PAR include:

Strong engagement of the community by way of increased participant recruitment and program retention rates,

  • validated cultural measures,
  • reduced reporting bias,
  • enriched interpretation of research findings, and
  • increased translation of findings into action
  • greater ownership of the research and immediate transfer of knowledge into practice

Community development and PAR provide an important information resource and enriched evidence base for engaging in community-wide planning; identifying relevant programs and initiatives; and involving families and communities in developing, implementing and evaluating programs and services. These methods also enable national and state jurisdictions and funding bodies to support local, community-driven suicide prevention strategies and plans.

Key benefits of implementing the CBPATSISP Evaluation Framework using these culturally appropriate and responsive methods and engagement with the CBPATISP website include enabling communities to:

  • build a local and national evidence base drawing primarily on Indigenous knowledges, experiences and perspectives;
  • undertake comprehensive assessments of the overall comparative efficiency of initiatives;
  • produce information about ‘what works for whom and why’ to guide policy development and initiative and project implementation;
  • provide evidence of existing gaps to reform reporting and accountability requirements;
  • promotes learning by communities /organisations proposing or implementing projects;
  • document and share with stakeholders emerging understandings of barriers, enablers and effective strategies and good practices in suicide prevention, early intervention and postvention
  • provide compelling evidence of what does not work for Indigenous individuals, families and communities as well as what impacts adversely on Indigenous controlled services, training programs.

Evaluations that identify ‘what is working’ and how to enhance and continuously build on program and service outcomes can directly enable greater effectiveness of programs.

The main steps involved in establishing a culturally responsive evaluation process are outlined Figure 1. These processes provide an opportunity for community groups and community controlled organisations (often in partnership with program and service providers) to use community development and PAR approaches in the planning, implementation, monitoring and evaluation of suicide prevention activities.

These processes enable communities (and partnership agencies and services) to regard their community suicide prevention initiative as a unique opportunity to bring about profound and positive change for Indigenous communities.

The teams planning and conducting evaluations are encouraged to:

  • use a qualitative yarning approach involving Indigenous researchers/consultants for data collection, interpretation, analysis and reporting;
  • give priority to Indigenous perspectives, values, knowledge and experiences using in-depth unstructured; conversational interviews and facilitated focus group discussions.
  • collaborate with existing project partnerships using consultation processes and agreements established with Indigenous communities throughout the project development and implementation.

The actual specifics of the evaluation methodology and indicators, specific questions and data collection and communication reporting format need to be negotiated with Indigenous communities at an early stage, taking into account the local community contexts.

The key tasks for a culturally responsive community planning and evaluation process include:

  1. Establishing a local reference group with relevant community organisations and members who have relevant knowledge, skills and understandings to provide additional advice on the evaluation process;
  2. Negotiating to include appropriate qualitative and quantitative methodologies and reporting arrangements with all relevant stakeholders groups including Indigenous partner organisations, community leaders and stakeholders;
  3. Building a comprehensive understanding of the key issues and factors in the specific community contexts for specific populations of concern (eg. Children, young people; men; families, perinatal mental health; people in incarceration; young people in out of care; and LGBTQI or other vulnerable groups);
  4. Building an understanding of the risk and protective factors around suicide generally and in local contexts;
  5. Referring stakeholders to the CBPATSISP website resources and information to know what is available to assist communities and services;
  6. Reviewing and analysing relevant background reports and other relevant documentation regarding the project;
  7. Reviewing the literature relevant to building strong Indigenous families and communities;
  8. Reviewing the literature on ‘what works’ in suicide prevention and postvention;
  9. Consulting with relevant governmental staff and other stakeholders at the community level;
  10. Working collaboratively with groups to identify and conduct appropriate ‘information rich’ case studies using culturally responsive and methods (ie storytelling, yarning, focus groups etc);
  11. Disseminating draft findings and recommendations to community and relevant agencies using appropriate information formats for different audiences (ie community feedback sheets, project newsletters, fact sheets and policy briefs for policy and funding personnel). Figure 2 provides an overview of the steps involved in developing a community-initiated/led, whole of community suicide prevention initiative

DEVELOPING, IMPLEMENTING AND EVALUATING COMMUNITY SUICIDE PREVENTION

Using the CBPATSISP Evaluation Framework in developing, implementing and evaluating programs, services or specific initiatives (universal, targeted) teams need to consider the extent to which the community engagement and planning processes are:

  • culturally responsive and secure to support Indigenous peoples’ involvement and leadership in all phases of the initiative;
  • incorporate the key guiding principles; and
  • use local language in framing the evaluation.

The next section includes a set of suggested questions in each of the planning, implementation phases to support and assess the development, implementation and evaluation of intended outcomes.

Step 1 The first step for community researchers, groups, organisations (and services) involves: bringing community stakeholder together to share their stories; defining the issues surrounding suicide and self-harm or suicidal risk in their community; and identifying existing services or programs and their roles in suicide prevention and/or crisis intervention or postvention.

Considerations/Questions

  • What are the issues being faced by the community?
  • Why do the community members and stakeholders think these issues are happening?
  • What do people think needs to happen to address these issues?
  • What are the potential barriers and enablers for this to happen?
  • What does the new or existing program or initiative hope to achieve?
  • What services and agencies need to be involved?

Step 2 Draw a diagram to show the major relationships between groups and issues in the story; describe how activities relate to desired outcomes (eg. establish strong and secure early years programs; increase young people’s sense of identity; strengthen the role or fathers, mothers and families; provide hope for future; connect people to culture and country).

Step 3 Review relevant examples of community resources for planning and evaluation in suicide prevention listed in the CBPATSISP resources section and on the ATSISPEP website.

Step 4 Implement the evaluation plan

Step 5 Identify assumptions about relationships between activities and outcomes from an Indigenous community’ perspective, make sure all relevant groups and individuals are included in building the picture using culturally responsive and respectful processes and protocols.

  • How do these ideas/ perspectives match/align with existing government program goals and objectives and expected outcomes?

Develop the key elements the community or organisation wants to explore as the story unfolds in order to develop key questions and/or evaluation/value statements.

This phase involves drawing together sound/robust observations and interpretations of individual and communal experiences using a range of approaches to explore issues (and begin to gather both qualitative and quantitative information using various methods). It requires identifying the sources and timeframes for collecting data and developing a plan and set of tables to guide the implementation of the evaluation. Questions might include:

Questions
What are the key issues that matter to individuals, families and communities?

  • How can they be included in the program?
  • How can they be measured and costed?
  • Are local issues/ potential barriers taken into account?
  • Are all groups in the community involved?
  • Is/are local language/s recognised, are all family groups and language groups represented?
  • Are key people involved?

Consider issues of cultural and community appropriateness in collecting and interpreting information, using approaches that have been identified as appropriate and supporting Indigenous terms of reference including PAR, as well as assessment tools to measure empowerment and change such as Most Significant Change Technique (MSCT), appreciative inquiry, photovoice, art appreciation; bricolage; Growth and Empowerment Measure (GEM) (Haswell et al., 2013) and other community audit tools.

This step involves working with relevant local Aboriginal and Torres Strait Islander groups to assist in interpreting the formative information using appropriate techniques for analysing the qualitative and quantitative data (eg thematic narrative, NVivo).

PAR and Continuous Quality Improvement (CQI) can be applied involve using the knowledge and findings generated through the ongoing evaluation throughout the implementation to improve programs, services and practices. The same processes are used in the establishing the summative evaluation findings to meet the different needs of multiple audiences – including the community, policy makers, program and service providers and other relevant stakeholders and the wider community. In this instance the findings and recommendations can be used to support the continuation of funding and implementation of the programs.

This phase seeks to obtain data related to the following major themes:

Questions

  • What resources have been allocated towards the sustainability of project processes and activities – – now and in the future?
  • What strategies have been put in place to ensure sustainability, enhancement or further development of project outcomes?
  • What are the key features of the organisation that impact on success? Taking into account the extent to which these a) affect the success of the project and b) are affected by planning and implementing the project;
  • How effective are the partnerships to the success of the project, services? – the nature and activities of partnerships involved in the projects; the role in facilitating the formation of community partnerships and partnerships with community controlled organisations in the health, mental health and/or social and emotional wellbeing services sector; and partnerships between agencies.
  • What has changed as a result of the project? – positive, negative and unexpected outcomes, outcomes as they relate to the overall policy outcomes, participant satisfaction (individual, family and community level),
  • Why did it change as a result of the project? – participant perspectives regarding the role of the intervention program or service in achieving policy outcomes; using narrative storytelling, MSCT, photovoice and other culturally relevant data collection methods;
  • What activities were undertaken by projects? and their cultural relevance and importance in contributing to the outcomes;
  • What other external factors contributed to the outcomes? and importance of those other factors (e.g. other activities undertaken concurrently by the auspicing agency, previous activities, activities of other agencies);
  • What were the barriers and enablers to the success of the project? including features of a) the community (e.g. community readiness; level of community capacity; previous experience with relevant projects,), b) the local context local conditions; c) the project (e.g. its staffing, levels of cultural competency and previous experience, support from auspicing organisation, support for other agencies)
  • What are lessons learnt and next steps? using narrative storytelling, MSCT, photovoice and other culturally relevant data collection methods to consider what participants and personnel think could have been done different and more effectively or efficiently to achieve intended outcomes.
  • Other comments? – an opportunity for project participants to comment on anything else of importance that has not been collected in relation to the framework.

An important aspect of any evaluation is to celebrate the story of ‘what works’ with the community; as well as reflecting on and learning from those aspects of the story that don’t work and using the knowledge created through the evaluation to improve programs and services moving forward.

The evaluation can also be used to leverage more funding, and to identify what further research is needed to fill gaps in knowledge about suicide prevention in Indigenous contexts.

Examples of promising programs, initiatives and services identified as demonstrating best practice initially identified through the ATSISPEP have been further reviewed and are available on the CBPATSISP Clearinghouse.

In accordance with the Evaluation Framework principles, the first step in establishing strong healthy communities is to acknowledge and understand the devastating and enduring impacts of colonisation on Indigenous people’s contemporary lives. It also requires understanding the various pathways necessary for healing from historical trauma, using both cultural and contemporary understandings and processes to support social and emotional wellbeing. Establishing culturally responsive community, family and individual support systems and programs to promote pathways to recovery through dealing with loss, grief and disconnection, trauma and helplessness; and powerless and lack of control is essential to reduce mental health issues, high risk behaviours and suicide. This in turn requires:

  • self-determination and community governance;
  • reconnection and community life; and
  • restoration and community resilience.

Although there is considerable overlap within the three themes, it is important to address each one specifically as well as all three collectively to promote healing and recovery across generations. The interrelatedness of the ways in which the ongoing history of transgenerational trauma impacts on individual, family and community social and emotional wellbeing, and specific pathways to recovery, are depicted in Figure 24.2 below (Milroy, Dudgeon & Walker, 2014, p. 424). The diagram shows how each of the pathways to recovery outlines a conceptual framework and theory of change for program development and, in combination, provide a holistic framework to support community life and development, which include both the types of questions required to be addressed in developing, implementing and evaluating programs and services and the types of promising programs that are useful for community groups to consider in pursuing each of the respective pathways in re-establishing individual, family and community social and emotional wellbeing.

The figure depicts how each of the circles encompass the specific pathways to recovery related to each of three trauma themes. In combination they affirm that recognition of cultural strengths and facilitation of Indigenous ways of working, leadership, healing and empowerment is fundamental to promoting sustainable recovery. At the centre, each of the pathways of recovery come together to provide the holistic sense of health and social and emotional and wellbeing for individuals, family and community (Milroy et al., 2014).

It is now widely recognised that the adverse effects of trauma on individual, family and community wellbeing is ‘contagious’ and ‘traumatic events effect whole communities because a community is a dynamic system that acts and responds as a whole ‘body’ (Sheehan et al., p.50). Several studies confirm the need to focus on healing to address trauma in order to improve Indigenous health, mental health and wellbeing outcomes and prevent suicide (Atkinson et al., 2014; Milroy et al., 2014; Powell 2014; Anderson, Baum & Bentley 2007). These studies emphasise the need for health promotion or education programs in suicide awareness and prevention to focus on healing and empowerment to help people gain a greater level of control over their lives and circumstances (Atkinson et al., 2014; Dudgeon et al., 2010; Healing Foundation 2014; Powell et al., 2014). What these different programs confirm is the importance of governments recognising Indigenous peoples as capable and efficacious individuals; as resilient, creative, adaptive with cultural knowledge and wisdom to determine their own solutions.

There are a range of validated high-quality resources that are valuable for local mental health and social and emotional wellbeing workers addressing transgenerational trauma available on the CBPATSISP website and the Healing Foundation website:

  • ‘What’s up with My mob?’

Community members, practitioners and service providers engaging in suicide prevention will find a range of culturally responsive strategies, principles and practices to strengthen Indigenous mental health and social and emotional and wellbeing in the following resources

Organisations need to be resourced to develop their capacity for undertaking evaluation, continuous quality improvement (CQI) processes; (action research) through: building an evidence base, fostering and, informing best practice in suicide prevention and early intervention within the public health sector.

This section includes evidence based, principles, processes and resources for Aboriginal controlled and government and non-government organisations and services to use in planning, implementing and evaluating the effectiveness and culturally responsiveness of suicide prevention and early intervention initiatives.

All suicide prevention and early intervention initiatives, programs and service delivery models implemented in Indigenous contexts need a sound theoretical base, appropriate conceptual frameworks, program logic and social indicators and performance measures to assess the effectiveness of programs, practices and services and delivery systems which are designed to simultaneously work with Indigenous children, families and communities;

  • strengthen the capacity, leadership and empowerment;
  • support their cultural and social and emotional wellbeing, and
  • address the social and economic determinants that are key risk factors.

Planning

A clear understanding of the extent of issues
The population they are intending to ‘target’ or work with
A sound theory and logic for the program service or initiative they are evaluating

  • Step 1: Identify and prioritise issues for action
  • Step 2: Understand the issue
  • Step 3: Plan your program or service
  • Step 4: Implement and adapt your program or service
  • Step 5: Evaluate your program or service

What follows then is a set of questions or criteria to measure the key elements and principles identified as essential for partner organisations, program and service providers and policy personnel to determine whether programs and services have:

  • Increased the accessibility of services for people at risk of suicide or self-harm.
  • Improved the cultural responsiveness and effectiveness of service delivery.
  • Improved outcomes and influenced positive changes in the social determinants that contribute to poor mental health and wellbeing outcomes, high risk factors and suicidal behaviours, and suicide.

Broader questions also need to be answered to assess the efficacy, integrity and effectiveness of programs, services and strategies designed to reduce Indigenous suicide and address the associated risk and protective factors in complex and diverse circumstances. The key elements that are operationalised include: adopting a social determinants approach; assessing initiatives to address racism; embedding cultural responsiveness and cultural safety; ensuring dual accountability and genuine partnership; facilitating Indigenous governance and self-determination; incorporating human rights; facilitating empowered and socially sustainable communities; strengthening protective factors; supporting the healing of trauma, grief and loss; establishing healthy communities; developing/identifying resources for individual community healing and recovery; developing/identifying workforce training and resources; assessing program sustainability.

Communities and organisations need to have the capacity to evaluate programs to enhance their understanding of ‘what works’, under what conditions and why in particular contexts (Baillie et al, 2013; O’Donoghue et al., 2014; Schierhout et al., 2013). Most organisations would benefit from having the support (by way of resources, knowledge and understandings) to undertake cycles of ongoing review and reflection. Continuous Quality Improvement (CQI) allows program and service providers to change things that are not working throughout their implementation, and to embed and document things that are working. Adopting a PAR approach, which is closely associated with CQI, enables organisations and services to engage in, and benefit from, action learning processes involving a process of critical reflection and a cycle of action, observation, reflection and planning to ensure program and service delivery is continuously improved. The value of measuring performance is in both monitoring progress towards achievement of agreed outcomes and in providing feedback on effective design and implementation of interventions. This continuous cycle of monitoring, review and refinement is illustrated in the Figure below extracted from the Australian Health Department website below.
The Plan, Do, Check, Act Cycle

There are a number of reoccurring themes around the support that organisations and programs need to ensure sustainable, culturally appropriate and effective service and program delivery – these themes reinforce the principles underpinning the Evaluation Framework. These principles include: community negotiation, community engagement, and strengthening community capacity and community control and self-determination. These elements are critical to ensure organisational cultural competence and cultural safety in a deep and sustained way. The review of evaluations of existing programs such as the Yirriman and Mt Theo programs highlight many of these important effective characteristics of developing and implementing suicide prevention programs through Indigenous community controlled organisations.

The One21seventy Health promotion CQI tools are an evidence-based resource package for use in Indigenous community settings. Health promotion tools and resources for the prevention of Indigenous suicide, self-harm and high risk behaviours can be successfully used by multidisciplinary teams with limited health promotion experience. The tools are accessible from https://www.menzies.edu.au/page/Research/Centres_initiatives_and_projects/One21Seventy/.

The CQI tools enable communities and service providers to:

  • systematically describe and assess how well activities and projects align with good practice
  • assess how well organisational systems are functioning to support health promotion
  • plan how to improve systems that support good health promotion practice.

Incorporating Cultural Indicators in Evaluation
The Evaluation Framework aims to ensure Indigenous cultural values, knowledges and terms of reference are respected, recognised and incorporated into all heath, mental health and social and emotional wellbeing and other associated programs and services, including universal, targeted and indicated strategies to reduce suicide.

The use of PAR combined with Aboriginal-led and negotiated CQI in evaluating organisations and services ensures that Indigenous voices are given primacy and authority in determining the most appropriate social and performance indicators to measure improvements in processes and outcomes; and, restoring/maintaining Indigenous cultural protocols, practices, priorities and aspirations for the future.

There is growing evidence of the impacts of racism on health and social and emotional wellbeing outcomes and the subsequent association with poor mental health, suicide and self-harm. The experience of racism and social exclusion felt by Indigenous children at school, by Indigenous people in the workplace and when accessing health, education and social services has been shown to adversely impact on wellbeing outcomes and contribute to depression, despair and anxiety. Both the ATSISPEP consultations and the literature review findings confirm that Indigenous people continue to experience institutional racism (implicit or explicit, indirect or direct, unconscious or conscious) through the language, practices and processes of social policies and programs which impact upon every aspect of their lives. These findings highlight the need for health and mental health stakeholders to address racism, discrimination and marginalisation and enhance cultural competence in programs, services and initiatives to support suicide prevention and interventions. All communities and community controlled partner organisations have the right to expect state and national government organisations and services to address issues of racism and discrimination as part of their core business.

Institutional racism needs to be addressed at three levels:

  1. individual (changing individual attitudes and behaviours);
  2. intra-organisational (fostering a culturally inclusive culture, and positive relationships between staff and organisational policies and procedures); and,
  3. system-wide: (including strategies to increase the awareness, commitment, willingness and resources to implement system level policies).

Some questions community organisations and service providers may consider include:

Questions

  • Are there recruitment and processes in place to encourage cultural affirmation?
  • Are there induction practices in place that promote a culturally safe work and service environment?
  • Is there a reconciliation action plan in place developed with local Indigenous groups?
  • Are there courses to support cultural leaning in the workplace and in service provision?
  • Do staff understand what it means to enact the guiding principles?
  • Are the guiding principles to support social and emotional wellbeing of Indigenous individuals, families and communities consciously enacted?
  • Are there mechanisms and resources to embed anti-racism and cultural competence in individual and organisational practice?

Monitoring and evaluation of changes can occur through self-administered organisational audits by staff and community groups intended to benefit, such as online surveys , or interviews and focus groups. Evidence of the effectiveness of strategies might include improved awareness, knowledge, skills, capacity, networks, and reduced racial tension experienced by consumers, clients and community. Skilled, culturally aware leadership and clear positive communication about the importance of addressing race-based discrimination and its impacts is critical to support individual and organisational level change and reduce any resistance.

Approaches to implementing organisational accountability and development, diversity training and measures to increase positive inter-group contact can be found in the resources. Ideally, resources will need to be developed by and provided to workplaces on the basis of their specific needs and contexts of each organisation. Encouraging staff to participate in cultural competence and reflective learning courses is also important.

Cultural Competence – Transforming Policy, Services, Programs and Practice. Walker R, Schultz C & Sonn C. 2014, Chapter 12 in Dudgeon P, Milroy H & Working Together: Aboriginal and Torres Strait Islander Mental Health and Wellbeing Principles and Practice (2014). Canberra: Australian Government Department of the Prime Minister and Cabinet, 2nd edition. pp 195-220.

The VicHealth’ Building on Our Strengths Framework identified five key ways in which organisations can play a key and important role in reducing race-based discrimination:

  1. implementing organisational accountability/development
  2. diversity training
  3. resource development and provision
  4. serving as sites for positive inter-group contact (Paradies et al., 2009; VicHealth, 2009)
  5. serving as ‘role models’ in anti-discrimination and pro-diversity practice for other organisations (Trennery, Franklin & Paradies, 2012).

Ensuring Genuine Partnership and Dual Accountability
Ideas of dual accountability recognise that Indigenous organisations are accountable for the efficient use of funds in achieving program goals. At the same time, Indigenous people have fundamental rights to have access to funds and services which can contribute to their social and economic wellbeing and on their terms of reference. An extensive review by Dudgeon, Walker et al., (2014) found compelling evidence that both mainstream and Indigenous-specific programs and services that adhere to the Closing the Gap service-delivery principles of engagement, sustainability, access, integration and accountability are more effective than those that do not. The Closing the Gap service delivery principles are:

  • Indigenous engagement: Engagement with Indigenous men, women and children and communities should be central to the design and delivery of programs and services.
  • Sustainability: Programs and services should be directed and resourced over an adequate period of time to meet the COAG targets.
  • Access: Programs and services should be physically and culturally accessible to Indigenous people recognising the diversity of urban, regional and remote needs.
  • Integration: There should be collaboration between and within Governments at all levels and their agencies to effectively coordinate programs and services.
  • Accountability: Programs and services should have regular and transparent performance monitoring, review and evaluation.

In the spirit of genuine partnership, community controlled organisations are encouraged to ensure government and non-government organisations are held accountable to answer the following questions:

Questions

  • Do existing state and national government suicide prevention and intervention policies, programs and services have strategies to:
  • protect and enhance the rights of Indigenous Australians to live and develop according to Indigenous terms of reference, values and needs?
  • achieve equitable social and economic and wellbeing outcomes?

Such accountability questions need to be included in all evaluations. The Indigenous evaluation framework help to inform government and organisations responses to these questions.

Facilitating Self-determination

Indigenous self-determination requires genuine partnerships between Indigenous people, governments and non-government agencies together with a commitment to dual accountability. The notion of partnership along with self-determination is deeply embedded in human rights and is a key principle underlying the Evaluation Framework. Historically Indigenous peoples have been self-determining – defining their own being through control over all aspects of their lives, including ceremonies, spiritual practices, medicine, social relationships, management of land, law, and economic activities (Howitt 2013). There is substantial evidence to confirm that ‘being self-determining’ is a key protective factor in achieving suicide prevention outcomes). Having a sense of control over one’s situation is crucial to Indigenous social and emotional wellbeing.

The extent to which a program, service or initiative supports self-determination may be measured by evidence of how they support: Indigenous cultural connectedness, language, spirituality and social and emotional wellbeing; Indigenous people contributing equally to decisions about practices and standards throughout all phases of the development, implementation and evaluation of programs and services.

Questions

  • What processes and mechanisms are in place to ensure Indigenous people have been able to make a genuine contribution at all phases of the program/service development, implementation and evaluation?
  • Does the implementation of strategies, programs and services support Indigenous rights and values?
  • Does the program or initiative enable Indigenous people to have a sense of control over their lives?
  • Is this reflected in the indicators for the evaluation?
  • Does the program or activity support elements that contribute to an individual, family or community’s sense of being self-determining such cultural ceremonies and spiritual practices, social relationships, and economic activities?

Incorporating Human Rights for Equity and Sustainability

Several studies emphasise the importance of incorporating human and Indigenous rights principles into evaluation and policy frameworks to improve Indigenous health and social and emotional wellbeing and achieve an equitable and sustainable future for Indigenous peoples. Recognition of human rights requires a greater commitment by governments to uphold Indigenous people’s individual rights to health, as well as their collective right to maintain and use their own health systems and practices in pursuit of their right to health. The recognition of human and Indigenous rights are higher order goals which are fundamental to the achievement of self-determination and equity), and ultimately to strengthening social and emotional wellbeing and reducing poor health outcomes including suicide.

Questions/Considerations

Importantly, a rights-based framework can also serve as a set of standards that can be used to assess the extent to which governments policies, services and programs:

  • acknowledge Indigenous people as the traditional owners of the land and their connection with the land
  • support and encourage connections and mutual respect between Indigenous and other Australians
  • provide services based on understanding and respect for Indigenous culture
  • develop and implement policies at the direction of the Indigenous peoples to redress past wrongs and support them to achieve the outcomes they desire.
  • recognise and work to address the continuing oppression, dispossession and disadvantage perpetrated on Indigenous people
  • integrate Indigenous culture into all educational, cultural and ceremonial activities to strengthen relationships, understanding and recognition to provide beneficial outcomes to the whole community and reduce discrimination in the long term
  • provide education and information about the Indigenous culture and history of the local area across the whole community to foster understanding and respect maximise opportunities for Indigenous people to participate in local decisions.

While the notion of capacity building is firmly embedded in government policy frameworks, there is a need for a shift in government thinking about program implementation and evaluation approaches that strengthen individual, family and community capacity.

Strengthening capacity is a two -way reciprocal process involving all stakeholders and especially those intended to benefit from the programs and services being planned, implemented and evaluated. All stakeholders need information to understand the different types and purposes of an evaluation, as well as familiarity with the concepts and language of community planning and evaluation. The CBPATSISP Clearinghouse website has links to a range of useful and culturally relevant resources and toolkits developed to assist evaluators in conducting empowering, participatory community evaluation included on government and community services websites on suicide prevention.

This section addresses specifically the evaluation of clinical services, protocols and practices. This section is still under development. For now, links have been provided to professional practice guidelines of health and allied health professions.

Recommendations from representative groups for clients/consumers, carers and their families have been integrated within each of the three guideline documents which can be downloaded from the Mental Health Standing Committee website at: http://www.health.gov.au/mhsc.

The discipline-specific guidelines that may apply include, but are not limited to, the following:

For Psychologists
From the Australian Psychological Society:
• Code of Ethics, Available from:
http://www.psychology.org.au/Assets/Files/NewCode160807WEB.pdf.

• Ethical and practice guidelines and procedures. Available for members only from: http://www.psychology.org.au/about/ethics/

For Social Workers
From the Australian Association of Social Workers:
• Australian Social Work Education and Accreditation Standards (2012) Available from: http://www.aasw.asn.au/document/item/100

• AASW Practice Standards for Mental Health Social Workers (2008) Available from: http://www.aasw.asn.au/document/item/17

For Occupational Therapists
From the Australian Association of Occupational Therapists:
• Code of Ethics (revised 2001). Available from: http://otaus.com.au/sitebuilder/about/knowledge/asset/files/1/codeofethics.pdf
• Australian Minimum Competency Standards for New Graduate Occupational Therapists 2010. From the Occupational Therapy Board of Australia.
• Code of conduct for registered health practitioners (2012). Available from: http://www.occupationaltherapyboard.gov.au/Codes-Guidelines.aspx

For Psychiatrists
From the Royal Australian and New Zealand College of Psychiatrists
• position statements, clinical practice guidelines, ethical guidelines, Code of conduct and Code of ethics are available from:
http://www.ranzcp.org/Resources/StatementsGuidelines/Practice-Guidelines.aspx

For Nurses
From the Australian College of Mental Health Nurses:
• Standards of Practice for Australian Mental Health Nurses: 2010. Available from: http://www.acmhn.org/images/stories/About-Us/standards_2010_web.pdf
From the Nursing and Midwifery Board of Australia:
• Code of Ethics for Nurses in Australia (2008)
• Code of Professional Conduct for Nurses in Australia (2008)
From the National Nursing Competency Standards for the Registered and Enrolled Nurses. Available from:
http://www.nursingmidwiferyboard.gov.au/Codes-Guidelines-Statements/Codes-Guidelines.aspx

From the Congress of Aboriginal and Torres Strait Islander Nursing and Midwives (CATSINaM)
• National Summit on Cultural Safety in Nursing and Midwifery: Background paper for all mental health and health practitioner s/organisations link
From the National Aboriginal Community Controlled Health Organisation 2011,
• Creating the NACCHO Cultural Safety Training Standards and Assessment Process: a background paper, NACCHO, Canberra, http://www.naccho.org.au/promote-health/cultural-safety/
This background paper provides a comprehensive overview of concepts and meanings related to cultural safety and cultural safety training, including: cultural safety and respect as a human right, racism and cultural safety, good practice in cultural safety training. It also outlines a number of organisation strategies to achieve improved cultural safety at an institutional and individual level.

For General Practitioners
Australian Indigenous Doctors Association
• Position paper: Cultural Safety for Aboriginal and Torres Strait Islander doctors, medical students and patients, 2013, AIDA, Canberra.
AIDA’s suggest that strengthening cultural safety is a shared responsibility between educational institutions, national peak bodies and health services, as well as government and individual non-Indigenous medical practitioners

Resources

Ngarngadji! Listen/ understand! Improving Care for Aboriginal and Torres Strait Islander Patients (ICAP) Resource Kit