How do you measure innovation?

A framework for measuring organizational performance in complex organizations

Measuring value is a tricky undertaking, especially in complex organizations: stakeholders will have different, sometimes conflicting, views and multiple perspectives and factors that need to be considered. In new research published in Accounting, Organizations and Society, Veronica Casarin, Assistant Professor of Accounting and Management Control at ESSEC, proposes a theoretical model for accounting for and mediating the multiple and conflicting values inherent to complex organizations. She studied the case of technology incubation sites, which provide startups with resources to grow. A central concern for technology incubators relates to the production and assessment of innovation, most notably in the area of life sciences.

The value of innovation can take many forms: social, economic, developmental, financial… making it complicated to provide one sole definition of value. Previous research (1) looked at the related issue of company valuation and how different financial analysts make distinctive sets of socio-cognitive associations between different “categories”, “analogies” and “metrics” to come up with their estimates of a company’s value. These associations are called “calculative frames”.

Dr. Casarin broadens the applicability of the calculative frame construct from the context of mere company valuation to performance measurement in complex organizations. The idea is to leverage its merit in conceptualizing compromise in sites of conflicting values. To this aim, she reframes the “analogy” and “category” components as “root metaphors” and “principles of worth”, respectively. Ultimately, this recast offers a way to interpret the technology incubator’s activities and how their multiple values are mediated by accounting calculations, i.e. performance metrics.

There are several root metaphors in the case of technology incubators: for example, they are an organization dedicated to common welfare (as opposed to profit). But they can also be seen as investors in a more capitalist context, clashing with the former metaphor. When this kind of clash occurs, it can lead to difficulty sorting incubators into one single form of value. In turn, these different metaphors of incubation are consequential for the choice of performance metrics to be used in assessing value. Thus, the third component of the calculative frame, i.e. “key metrics”, plays an important role in materializing value, in making it visible. For example, for a technology incubator, this could include the number of jobs and companies created or the amount of external funds attracted. By taking these three elements together, Dr. Casarin reconciles socio-cognitive elements (root metaphors and principles of worth) and calculative elements (key metrics) and jointly applies them to shed light on how compromise forms when multiple values clash with each other.

The case of technology incubators

Dr. Casarin developed her calculative frame model drawing on data from seven technology incubators in the UK. These incubators had different structures, but were primarily companies limited by guarantees, meaning that any profit generated must be put toward their primary mission as opposed to shared among members; only one was “limited by shares”, meaning that the profit could be shared among shareholders. They also received funding from different sources, meaning they had different types of stakeholders. In turn, these different structures can have an impact on how incubator performance is measured.

To understand incubator performance measurement practices, Dr. Casarin conducted a series of in-depth interviews with incubator managers, startup founders, scientists, and institutional investors. She also analyzed internal documents for some of the incubators, including budget documents, financial plans, finance and strategy reports, and audit reports.

Analyzing value

Four calculative frames emerged from the data analysis: common welfare, financial sustainability, investment, and innovation quality.

Within the common welfare frame, incubators are understood as a “municipality”: they create jobs for the local economy and, as such, add value. This can help generate wealth for the community as well, and return the taxpayers’ investment with this added value. Dr. Casarin finds that this value is typically calculated by looking at “key regional indicators”: the number of jobs created and of assisted startups, and how the incubators hit targets on these indicators. For example, one incubator, which received funding from the European Union, created over 1600 jobs in a decade, having set the objective of creating 1000, thereby surpassing their target for job creation.

The financial sustainability frame looks at the incubator as a “business”, meaning that it must act like a self-sufficient entity and maintain financial health over time. Key metrics for this framework include rental revenues, operating costs, and growth.

The investment frame treats the incubator as an “investor”, meaning they can create their own vehicle for providing funds and then invest in incubated startups. Key metrics here are return on investment, exit valuations, and financial milestones.

Finally, there is the innovation quality frame, where the incubator is a “scientific laboratory”. Technology incubators can be based in universities and engage in producing cutting-edge knowledge and scientific research. It can be hard to quantify innovation quality, but Dr. Casarin finds that typical measures relate to intellectual property production, which includes patents and copyrights.

Mediating conflict with accounting tools

Sometimes these frames can collide, like in the case of the financial sustainability and innovation quality frames. Scientific innovation can be a risky move financially, and so introducing stable streams of income, like rental income from “fee-for-service” type of tenants, can be financially savvy. However, these frames lead to conflicting priorities when it comes to allocating resources. Dr. Casarin finds that looking at accounting tools such as budgets and performance measurement indicators can help understand how compromise is made between the two conflicting frames. Similarly, compromises can be made between the other frames, such as looking at how surplus is used to mediate the conflict between financial sustainability and common welfare.

Takeaways

Taken together, Dr. Casarin’s work helps provide an innovative framework, i.e. calculative frames, for interpreting how multiple, and sometimes conflicting, values in organizations are held together in compromise. The use of calculative frames and introducing accounting calculations as “mediators” between these frames points toward a novel role of performance measurement. Traditionally, accounting tools such as performance measures are seen to operate as “ammunition machines” (2) or support dominant positions in organizations. In her work, Dr. Casarin advances that accounting tools can operate as more inclusive devices for mediating multiple values, for pacifying conflicts by creating compromise solutions.

Since modern day organizations have multiple and often conflicting demands from their stakeholders, with different concepts of what “value” means, this is a useful tool for understanding how organizations quantify value, especially in innovation, and how compromise among multiple values can be reached.

Further reading

Casarin, V. (2023). Calculative frames, compromising metrics, and the multiple values of innovation: The case of technology incubation in the UK. Accounting, Organizations and Society, 111, 101479.

References

1. Beunza, D., & Garud, R. (2007). Calculators, lemmings or frame-makers? The intermediary role of securities analysts. The Sociological Review, 55(2_suppl), 13-39.

2. Burchell, S., Clubb, C., Hopwood, A., Hughes, J., & Nahapiet, J. (1980). The roles of accounting in organizations and society. Accounting, Organizations and Society, 5(1), 5-27.