An area I have been interested in investigating further is metrics that can show the performance of a research organization. I put a lot of credence in the  expression, “If you can’t measure it you can’t improve it”.  As such metrics to measure the performance of a research and development organization would give the organization a better idea of where hold-ups are in the product development cycle.

One idea I would like to see implemented, which isn’t really a metric based system,but would give an organization an early warning system for projects would be a prediction market setup. In James Surowiecki’s book The Wisdom of Crowds and in case-studies of both Best Buy and Google the utility of prediction markets has been shown for use both in predicting events like sales as well as a project dashboard which can indicate a problem is occurring well before an official status report communicates the problem to the appropriate decision makers.  The reason I say this isn’t a metric based system is because while individuals participate in the prediction market each uses different metrics and bits of knowledge to make a call on a particular project rather than everyone collecting the same metric.

A prediction market works similar to a stock market where you can buy or short a particular “stock”. That stock might be “Best Buy will open a store in Alaska by Jun 2015”. Market participants can then buy or short the stock thus giving a probability of perceived success.  The beauty of a prediction market is the ability to aggregate large amounts of unfiltered data and communicate those results to key decision makers on a real-time basis.

If anyone has examples of this used at various companies I would be really interested in hearing about it. As always thoughts and comments are welcome!