Comparison With Other Approaches

The Evidence Network's methodology is the key component of overall evaluations of innovation enablers because it provides for direct evidence of innovation enabler impact on client or member firms - evidence that innovation enablers are fulfilling their missions.

There are, however, other ways to evaluate innovation enablers.  Innovation enablers themselves or consultants acting on their behalf may undertake client satisfaction surveys or may collect data on capacity measures such as funds raised, funds dispersed, number of members, number of clients, and number and types of services.  Innovation enablers that provide technology-related services may measure publications, citations, or patents.  Enablers may also use approaches such as Kaplan and Norton's Balanced Scorecard that measure organizational learning.

Academics also contribute to the evaluation of innovation enabler performance.  Using a case study approach researchers have evaluated the impact of organizations such as SEMATECH (Grindley, Mowery & Silverman, 1994)¹ , and using economic approaches researchers have measured the performance, or effect of the activities of the US Advanced Technology Program (Feldman & Kelley, 2006 ²; Sakakibara & Branstetter, 2003 ³), the Advanced Technology Development Center of the Georgia Institute of Technology (Rothaermel & Thursby, 2005 ⁴ ), US regional institutes (McEvily & Zaheer, 1999 ⁵ ), Japanese research consortia (Branstetter Sakakibara, 2002 ⁶), Canadian industry associations (Dalziel, 2006 ⁷ ), US small firm manufacturing networks (Human & Provan, 1997 ⁸ ), Swedish (Löfsten & Lindelöf, 2002 ⁹) and Chinese (Yu & Heshmati, 2007 ¹⁰) science parks, US (Di Gregorio & Shane, 2003 ¹¹) and European (Kaufmann & Tödtling, 2001 ¹² ) university technology transfer offices, and international standards organizations (Rosenkopf, Metiu & George, 2001 ¹³; Rysman & Simcoe 2006 ¹⁴). 

While these methods contribute a great deal to our understanding of the impact of innovation enablers, The Evidence Network's approach is unique in its ability to provide evidence of innovation enabler impact that is both reliable and systematic while at the same time providing in-depth data on the quality of organizational processes.  Our method is generalizable across all types of innovation enablers and provides timely feedback using scales that are suitable for recurring assessments.  As shown in the following table, The Evidence Network's approach is the only approach that achieves all of these benefits.

References

1. Grindley, P. Mowery, D.C., & Silverman, B.  1994.  SEMATECH and collaborative research design:  Lessons in the design of high-technology consortia.  Journal of Policy Analysis and Management, 13:  723-758.
2. Feldman, M.P. & Kelley, M.R.  2006.  The ex-ante assessment of knowledge spillovers:  Government R&D Policy, economic incentives and private firm behavior.  Research Policy, 35:  1509-1521.
3. Sakakibara, M. & Branstetter, L.G.  2003.  Measuring the impact of US research consortia.  Managerial and Decision Economics, 24:  51-69.
4. Rothaermel, F.T. & Thursy, M.  2005.  University-incubator firm knowledge flows:  assessing their impact on incubator firm performance.  Research Policy, 34:  305-320.
5. McEvily, B. & Zaheer, A.  1999.  Bridging ties:  A source of firm heterogeneity in competitive capabilities.  Strategic Management Journal, 20:  1133-1156.
6. Branstetter, L.G. & Sakakibara, M.  2002.  When do research consortia work well and why?  Evidence from Japanese panel data.  American Economic Review, 92:  143-159.
7. Dalziel, M.  2006.  The impact of industry associations.  Innovation:  Management, Policy & Practice, 8:  296-306.
8. Human, S.E. & Provan, K.G.  1997.  An emergent theory of structure and outcomes in small-firm strategic manufacturing networks.  Academy of Management Journal, 40:  368-403.  
9. Löfsten, H. & Lindelöf, P.  2002.  Science parks and the growth of new technology-based firms-academic-industry links, innovation and markets.  Research Policy, 31:  859-876.
10. Yu, Z. & Heshmati, A.  2007.  Growth and performance of science parks in China.  In Heshmati, Almas (Ed.) Recent Developments in the Chinese Economy.  pp. 55-82.
11. Di Gregorio, D. & Shane, S.  2003.  Why do some universities generate more start-ups than others?  Research Policy, 32:  209-227.
12. Kaufmann, A. & Tödtling, F.  2001.  Science-industry interaction in the process of innovation:  The importance of boundary-crossing between systems.  Research Policy, 30:  791-804.
13. Rosenkopf, L., Metiu, A. & George, V.P.  2001.  From the bottom up?  Technical committee activity and alliance formation.  Administrative Science Quarterly, 46:  748-772.
14. Rysman, M. & Simcoe, T.  2006.  Patents and the performance of voluntary standard setting organizations.  NET Institute Working Paper No. 05-22. Available at SSRN: http://ssrn.com/abstract=851245 [Accessed January 12th, 2009].