While there is now a mandate that many, if not most, federal and state initiatives use collaboration as both a method of governance and as a mode of decision-making, there is little empirical evidence documenting the influence of collaboration on concrete program outcomes. In fact, several reviews in the public health arena have found that fewer than 20% of all collaborative initiatives produce outcomes that equal or better than those initiatives carried out by a single agency.
This does not mean that we should abandon collaboration. But it does mean that we must get much more precise about what makes for successful collaboration. Two known features of highly successful collaboratives are the presence of an open and credible process and the presence of leadership oriented towards fostering and securing that process. An open and credible process has five critical features: 1) It includes all relevant stakeholders (but does not over-include by inviting participants who may represent an important sector but who do not work directly in agencies or with communities directly relevant to the initiative). 2) All stakeholders are treated equally (measured not in formal avenues for participation, but the degree of influence each member perceives he or she has). 3) The process is perceived by stakeholders as authentic (measured by the perception that the process can issue binding decisions and is not a legitimation ritual to satisfy some mandated ideal). 4) It focuses on the root problem. 5) It can be revised when the application of some norm, no matter how well-intentioned, produces unwelcome effects on the process.
We need to develop better instruments for measuring these qualities and, more importantly, we need to develop sophisticated modeling techniques that can identify just how these features are related, or not, to concrete program outcomes. These outcomes should be focus on the ability of the program to reach and retain those it serves, that it is delivered with fidelity, and that it produces demonstrable improvements in well-being. Research starting to do this has been published recently (Evaluation Review, Oct 2008). Along with the President’s commitment to funding evidence-based health and education initiatives, there should be an equally strong commitment to making sure that we know if collaboration has any demonstrable effect on how these programs perform, and just what features of collaboration are associated with these outcomes.
Comments (3)
Well, yeah. It's new so data ain't there yet.
The effect of collaboration on complex public policy disputes has received quite a lot of study over the years. Recently the US Institute for Environmental Conflict Resolution has been compiling and analyzing info from their case-load. I don't know how much they have on their website, but if it interests you take a look at ecr.gov
This is a very important suggestion and one that will be very challenging, though necessary to implement. There is a real need for more empirical information on the outcomes and impacts of collaborative engagement and public dispute resolution. We need to emphasize performance as well as principles; effectiveness and efficiency as well as fair and open democratic standards of deliberation. Until we have a way to measure this, we will not be able to improve and optimize our investments in collaboration overtime.
USIECR spearheaded a multi-agency evaluation effort and as Nic suggested there is some information on ecr.gov. Their evaluation findings will be published in the Conflict Resolution Quarterly this October.