The data is coming, the data is coming!

Today, professional management is increasingly about obtaining and utilizing mountains of data. Business analytics and big data are the buzz words of the day. Managers believe all good things flow from good data.

Originally published in the March issue of Value Chain.


While getting and understand good data are laudable goals, it is not always possible to make decisions on good data. Sometimes we need to just trust ourselves and make decisions based on our own understanding and experience - whether we have data or not. And that’s okay.

Often, the issue is not whether or not we can make good decisions without good data, but whether or not we can sell those decisions to the rest of the decision’s stakeholders in the organization. How difficult a decision is perceived to be, the availability of data, and how much experience we have facing these types of decisions, all play roles in deciding how difficult it will be to get people onboard.

Decisions. Decisions. Decisions.
Some types of problems are widely known to be too difficult to solve. Any problem related to people and people management easily falls into this category. We don’t worry about building flawless policies and procedures around people management, because everyone involved already believes perfection is impossible. Any problem tamped-down today is just going to pop up again tomorrow. We don’t ever “solve” these recurring problems. We “manage” them on an ongoing basis.

Other problems we believe should be solvable if we only had the right data. We handle these problems through estimation. If it is clear that we aren’t going to be able to gather the data we want, we estimate what the data is likely to be and make decisions based on our guesses. This is not optimal, of course, but because it is as good as anyone can do, there is no fault in using this approach. Everyone understands that it is the best we can do.

Finally, there is a class of problems that seem to be easily solvable, but for some crazy reason just aren’t. Year after year we work on them and wonder why our results are alway sub-par. It turns out there are many problems we believe should be solvable but, in reality, are too difficult to solve. They are not just intractable for us, they are intractable for anyone.

The “traveling salesman problem”, first formulated formally in 1930, is just such a problem. “Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?” Can’t be that tough, can it? Similar problems include picking up tennis balls on a tennis court, utilizing key assets like satellites, running delivery routes in a distribution schedule, etc.

It turns out there is often no optimal route or that finding one is so computationally difficult that even when using the world’s most powerful supercomputer, the sun would burn out before we could find it. Mathematicians and computer scientists have devised a variety of algorithms to attack the problem, some seeking actual answers, some involving relaxing constraints to make the problem easier. Solving similar, simpler problems has enormous value. We can determine the upper and lower bounds of the answer, for example. We can determine the nature of the answer and potential places where our thinking is just too fuzzy.

Cognitive scientists have found that humans, however, can estimate very, very good answers to traveling salesman-type problems in a matter of minutes. The answers humans give are not perfect, of course, but they are goodenough, even competitive with those given by rigorous computation.

If the data comes, great.
If it doesn’t, that’s also fine.

Not all problems need to be solved all the time.


Rodney J. Johnson

Rodney J. Johnson is President of Erudite Risk and Co-Founder of the KBLA. He has lived in Asia for most of his adult life, but still longs for good Mexican food.


Have questions about the KBLA? Email us at Ready to join?