Abstract: When individuals and partners baulk at the thought of being evaluated, they may have formed an expectation that the proposed ‘review and evaluation’ is masquerading as an exercise in criticism and passing judgment, ‘blaming and shaming’ and issuing penalties. Such negative connotations are not conducive to building strong, open and resilient partnerships. They make the work of the partnership broker harder. The author argues that when reviews are carefully thought out and managed, capitalizing on Utilization (User) Focused Evaluation approaches and taking a progressive approach to review design, they can in fact be turned into an important trust building mechanism in the partnership cycle. This in turn helps create learning partnerships, which would sustain future rigorous and thought provoking evaluations.
Progressive review and evaluation as a trust building mechanism in partnerships
by Julie Mundy
“What if we have failed?” These words represent a common underlying fear for many organizations and individuals when they think about a program evaluation and review. In the case of cross sector partnerships, this fear can be exacerbated by the thought of being held responsible for decisions or actions that other partners may have or have not taken in addition to one’s own areas of direct accountability. Although sharing risk and responsibility, and maximizing mutuality, is at the heart of successful partnerships, it also represents one of its biggest challenges.
Other amplifiers of anxiety in partnership reviews can be linked to the different objectives, cultures, organizational approaches and attitudes partners may have towards evaluation. In new partnerships, where trust and familiarity between partners is not yet fully developed, this apprehension can be marked. Given such a climate, careful consideration of the design and conduct of a review process is critical to the well being of the partnership.
If real or imagined fears lead to the review process being mishandled, through neglect rather than by design, working relationships between partners which may have been painstakingly developed over time can become irreparably damaged in the evaluation process. If there is blame-laying, resentment, frustration and disengagement, then trust – the fundamental building block of resilient partnerships – can be destroyed. How then, can a partnership broker assist partners to overcome these very natural fears and to build, rather than undermine, trust?
Solomon and Flores[1] talk about the necessary conditions for building authentic trust[2] in relationships, such as self-scrutiny, caring about the long term relationship (in addition to the outcome), negotiation and mutual understanding, making and honoring one’s own commitments, and a keen awareness of and willingness to assume responsibility for risks and liabilities. Authentic trust is what leads to productive and effective organizational relationships[3].
Taking these factors into account, a partnership review process which is consciously designed and managed, can become not only a positive experience, but one which, over time, has the potential to contribute to trust building and strengthening relationships between partners. This is true even when (or sometimes, particularly when) some of the findings of the review may be challenging: ‘going through the trenches’ of a tough (though well-managed and facilitated) review process together, can help build a shared experience, understanding, and trust. Symbiotically, the greater the trust between partners, the more able that partnership is to support and benefit from highly rigorous evaluation.
In order to achieve this outcome, partnerships and their brokers should consider adopting a thoughtful and progressive approach to review, which can evolve with the partnership itself, over time. In all stages, it is recommend that highly participatory, utilization-focused evaluation[4] (U-FE) approaches (see Box 1) are employed in reviewing partnerships, in order to support and not undermine the partnership approach. The latter is a risk if external, ‘independent’ (traditional) or rigid evaluation methodologies which do not take account of the interplay of partner relations, are adopted, particularly in reviewing the partnership itself[5].
Box 1 : Patten’s Utilization-Focused EvaluationPatten’s Utilization-Focused Evaluation (U-FE) maintains that an evaluation should be judged on its usefulness to intended users, and should therefore be planned and conducted in ways which enhance the likely utilization of both findings and of the process itself to inform decisions and improve performance. UFE has two key pillars: firstly, the primary intended users of the evaluation must be clearly identified and actively involved from the very start of the evaluation process. Secondly, evaluators must ensure all other decisions made about the evaluation process, are guided by the intended users. The main role of the evaluator/broker is to facilitate decision making and ownership amongst those who will use the findings of the evaluation, not to make decisions separate to intended users. U-FE does not proscribe research methods or tools, but rather describes the approach to be taken in designing evaluation. U-FE original simple 5 step process: Step 1 – Identify primary intended users Step 2 – Gain commitment to U-FE and focus the evaluation Step 3 – Decide on evaluation options (including methodology) Step 4 – Analyze and interpret finding and reach conclusions Step 5 – Disseminate evaluation findings Patten has subsequently developed an excellent and very user-friendly 12-step UEF Checklist which is freely available at: http://www.wmich.edu/evalctr/archive_checklists/ufe.pdf |
Unlike traditional methods, Utilization-Focused Evaluation maintains that the intended end-users of the information generated by an evaluation should be involved in the design and conduct of any evaluation at the outset to ensure its relevance and usefulness. If they are engaged fully in the process, then they are more likely to find the results useful and hence take action. This approach positions evaluation squarely as a learning mechanism, where the learning is as much acquired by the key players (or ‘intended users’) from the process of evaluation as it is from the end results.
Involving partners in the entire review process also has the added benefit of building the internal capacity and more importantly, the confidence, of partners to participate and make informed decisions about partnership reviews and evaluations. Over time, this can help build the partnership’s sustainability and reduce its reliance on external facilitators and brokers. However in order to achieve this end-goal, it is proposed brokers consider a staged approach to reviews:
Stage 1 – Emerging partnerships: Exploring and Learning
Where partnerships are newly formed and still in the early stages of exploring how they will work with the partners themselves and what the partnership can achieve, it makes sense for partners, with the assistance of a partnership broker or evaluation facilitator (critically, they must be experienced in and understand partnerships), to co-design and jointly administer an internal review process, including developing the review questions together. Ideally, this will build on a jointly developed ‘theory of change’ which the partnership may have developed at the commencement of its work, and which should include partnership as well as project indicators.
At this stage, the partners may agree to restrict the scope of the review to assessing the wellbeing, effectiveness and efficiency of the immediate partnership group relationship, behaviors, processes and governance, as well as activity outputs (although in larger and more complex partnerships this may be handled separately). The assistance of an external broker, experienced in partnership reviews, to assist the partners in the design and to facilitate (not conduct) the review process can help navigate any difficult discussions, and assist by modeling appropriate behaviors. The presence of an independent/external broker can also free partners to talk about and examine sensitive issues, which in the early stages of a partnership; they may not feel otherwise able to do.
In designing a review process in these early stages, it may well be appropriate to acknowledge that a valid, and even primary, objective of the first review process is to help build trust, transparency, accountability and effective working relationships between partners. The review process also presents partners with the opportunity to practice problem-solving constructively together – a vital partnership skill for ‘when things go wrong’. It is entirely reasonable therefore, with these objectives in mind, for partners to decide initially to limit the scope of their review. Restricting the review group to those most closely involved at this stage, can provide confidence and freedom to partners to have genuine and important discussions, which may not be achieved with a wider group. (see Box 2 for an example of this approach in action, and a subsequent ‘stepping up’ as trust and confidence between partners grew).
Another critical factor in terms of trust building at the early review stage is for partners and brokers/facilitator to follow through after the review process to ensure that lessons learned and actions agreed during the review process are implemented and integrated into the partnership’s way of working, thus reducing the risk of recurrence of the same issues year after year. This is an important consideration for trust building and for valuing the review process (‘we did what we said we were going to do’). When a utilization-focused approach is used, there is much more likely to be buy-in and implementation of review outcomes.
Box 2: An example of a staged review process:A cross-sector partnership between a government agency and a peak body, had previously worked together in a fairly transactional, prescriptive and inequitable relationship. With genuine intent on both sides, the partners decided to embark on a dramatically different partnership approach, in order to reflect their growing engagement and to work towards innovative and creative higher level outcomes. With the broker’s input, they adopted a staged approach to review and evaluation of the relationship and its outcomes in order to build and strengthen relationships, a sense of equity, understanding and trust. In the first year, by agreement, the partners limited the scope of the review to initially seeking feedback from the immediate core partnership group. A joint workshop was held to facilitate discussion of the key review questions and to agree actions, particularly focusing on what they intended to do differently. This reflected a shared, genuine commitment by both partners to be part of a ‘learning partnership’. In the following year, the partners felt their relationship was resilient enough (and that partnership activities had been implemented sufficiently) to extend the evaluation scope to include those internal stakeholders impacted by the partnership (still within their respective organizations but considerably beyond the core group). At this stage, while an external broker still assisted in the review process, the discussions about lessons learned were much more free flowing and only required ‘light touch’ facilitation. This was also assisted by no changeover in personnel in the key group, who were increasingly comfortable with each other. In the third year of the evaluation process, the partners were intending to extend their evaluation reach to assess the impact of their partnership on external stakeholders as well: something which would not have been possible, or desirable in the first year of the partnership. Importantly, and as a real indicator of the success of the partnership, while seeking external support in gathering the information from stakeholders, they felt confident enough to facilitate the review findings’ discussions themselves. |
Stage 2: Growing partnerships – Building capacity
Once one review cycle has been managed effectively (it may take more than one where new partners have subsequently joined, for example), the partners have established routine working relations, and have had the opportunity to problem solve together constructively in the ‘safe environment’ provided by the broker, confidence and curiosity usually replace fear in the lead up to the annual review. The broker’s focus at this stage then can be on building on the gains made at the last review – particularly around constructive problem solving with challenging review findings. At the start of the review, a quick appraisal of what learnings have occurred and which changes have been implemented as a result of the previous review process can also help to build confidence and engagement. At this stage, brokers should be encouraging and supporting any internal brokers (formal or informal) within the partnership to step up and start to take the lead on the review processes: building the confidence and capacity of partners to conduct their own reviews, rather than relying on external support, is key to the ongoing health and sustainability of the partnership.
Stage 3: Established partnership – challenging the status quo
By the time a partnership has matured, and the partnership is capable of managing its own review processes, the role of the external broker in the review process changes again. At this point, real value can be added by brokers willing to help partnerships to push the boundaries, really look at innovation and social impact of the work they are doing together. While the early years of working together in partnership are about establishing and embedding processes, protocols, working relationships and protocols, building trust and mutual respect, once this has been achieved and maintained, partners may well feel empowered and confident enough in each other to exploit the value-add of working in partnership, that is, the potential for social innovation, creativity and change. At this stage, apart from gathering information on the partnership’s impact from external stakeholders (and hopefully end-users), review processes may include ‘dreaming’ or ‘stretch’ sessions as part of the process, looking at what they have achieved together, what appetite is there for going beyond and so on? The fact that partners have an established track record together means that the ‘dreaming’ has a greater chance of becoming reality.
Summary of progressive approach to partnership reviews:
| Stage of Partnership | Emerging Partnerships | Growing Partnerships | Established Partnerships |
| Review Cycle | 1st and possibly 2nd cycle or where there is a change in membership of partners | 2nd – 3rd cycle | 3rd cycle and beyond |
| Review Objectives (sub-text) |
|
|
|
| Review lessons |
|
As for Emerging Partnerships, plus sharing partnership learnings throughout partner organizations (beyond core partnership group) | Extending lessons learned beyond the partnership via disseminating at public forums, networks and public documentation |
| Broker’s Role |
|
|
|
Conclusion
With careful design and consideration, taking the time to embark on a progressive approach to partnership review and evaluation will ultimately reward the partnership, and replace the fear of failure with a genuine engagement and openness to learning through the review process. It does this by contributing to the development of authentic trust between partners, as they learn to constructively and collaboratively problem solve together in the interests of the partnership instead of reverting to blame behaviors when tough review findings are encountered. Importantly, if partners are involved in the review processes, from start to finish, they are also more likely to follow through and implement any review findings, and so the cycle of review-trust-learning continues. This can assist partnerships develop a genuine culture of learning which in turn may result in more efficient and effective partnerships. Where authentic trust exists alongside a learning culture within partnerships, conditions are ripe to support the likelihood of joint risk-taking, creativity and innovation. Imagine the potential of a fearless partnership!
Author
Julie Mundy is a partnerships specialist with more than 20 years experience as an internal broker and practitioner in the international development sector across the Asia-Pacific and in Africa. As an independent external broker for the last 4 years, Julie has been involved in a designing and facilitating a wide range of complex partnership reviews with bilateral donors, governments and the community sectors. Now based in Melbourne, Australia, but working internationally, Julie is also a PBA Authorized Lead Trainer, PBAS mentor and a founding member of the PBA Training and Evaluation Communities of Practice.
[1] RC Solomon & F Flores (2001) Building Trust in Business, Politics, Relationships and Life, Oxford University Press.
[2]Solomons and Flores (ibid) define authentic trust as being reflective and honest’ open to possibilities of betrayal and can cope with such situations; focused on relationships rather than single transactions and outcomes; not naïve or self-destructive, self-confident, self-scrutinizes, is articulated, demands reasons for trust, is chosen and maintained with effort, develops through relations with others, involves choices, is a matter of commitment and integrity, cannot be taken for granted and is conditional.
[3] BJ Starnes, SA Truhon & V McCarthy, (2010) A Primer on Organizational Trust: How Trust Influences Organizational Effectiveness and How Leaders Can Build Employer-Employee Relationships Based on Authentic Trust, ASQ: Milwaukee
[4] MQ Patton (2008) Utilization-Focused Evaluation: 4th Edition, Thousand Oaks, Ca, Sage Publications.
[5] It is recognized that some partners (bilateral donors in particular) may have stringent internal requirements for ‘independent evaluations’ of program activities, outputs and outcomes. These services may also have to be put out to tender. Where this is the case, brokers should encourage partners to include experience of working in a partnership context and an understanding of a partnership approach, as well as a utilization-focused evaluation approach, as part of the scope of services. Equally, the review of the partnership itself (as opposed to the partnership activities) can be separated out and should, where at all possible, be managed and conducted by the partners themselves, with the assistance of a broker as needed, as suggested in this paper.
