(Page 2 of 3)
But complexity makes people nervous. Its not part of our basic education. By definition, complex problems are hard or even impossible to understand. BUT is understanding really necessary to progress? Surprisingly, no.
1. Knowledge-less-ness: We can find solutions to problems that we do not (and can not) “understand”
There is a field of math/computer science known as Computability Theory (I hovered here as an undergrad). It’s the study of how to solve logical and mathematical puzzles, how to classify these puzzles into different types, based on their difficulty, and how to rank solutions by how efficient they are. Solutions usually involve clever methods for changing a hard problem to look like an easier problem.
There is a class of problems, however, known as NP-Complete. (You do not care what that means for the purposes of this discussion, but Wikipedia has a decent definition if you really must.) Let’s just say that these problems are the ones where you can’t find a nice tidy algorithm for efficiently solving the problem.
One of the canonical examples of NP-Complete problems is known as the Traveling Salesman problem. [You can tell I took a math course in college because I can say “canonical example.”] The traveling salesman problem goes like this. A salesman has to visit each of a list of cities exactly once. What is the shortest path he can take through them?
It turns out that it is impossible to simplify this problem using traditional analytic methods, and the only way to find out the optimal answer is to try every possible combination of cities, check how long it takes, and pick the shortest. This is no big deal if the number of cities is 5 or 25, but the effort required to solve the problem grows exponentially as the number of cities grow. We have no clever solution.
So I’m eyeball deep in this stuff, and I read an article — a little “Computer Recreations” column in Scientific American by A K Dewdney, describing Flibs. It was a mind-altering, possibly life-altering article.
Flibs solve the traveling salesman problem. Easily.
A Flib is a random list of the cities and their distances. What you do with them is cool. You generate a bunch of flibs. And you can check how long the distance is if you visited the cities in the order listed in the Flib. And then you can take the best 10 or 25 percent of the Flibs and exchange their front and back ends — like a gene might do — the first 10 cities of Flib 1, and the last 20 cities from Flib 2, etc. And you score this new generation. Or you can create some random “point” mutations, or other schemes for mixing up the good performers, score them again and voila: within a very few generations, you have a very good answer. Is it optimal? It's very close, and takes a tiny fraction of the effort to get there.
The Flibs solution did not require a clever trick based on insight into the intrinsic properties of the problem (graphs). It solved the problem in spite of the fact that it was impossible.
Why does this matter? Genetic Algorithms are an example of how networks of agents can solve impossible problems.
Most business and social problems are “impossible” — they were always difficult but the pace, competition, expectations and interconnectedness of things has changed the game. But that does not mean that we cannot face them — if we get over the fact that we can’t “know” the answer, but we can discover it.
It suggests that our newly networked workforce may be able to tackle some impossible problems. We know they can — we see it, but we can’t really describe it, and we’re struggling to understand how to promote it.
2. The Bar problem: We can develop stupid, but correct, solutions to impossible problems
I spent some time in the 90’s actually getting paid to experiment with agents and complex adaptive systems. We did clever little custom apps for FedEx, AT&T and the government.
Agents, in this sense, are just little objects with a little bit of state information and some rules they can follow. They are the simplest possible things. [If you’re worried you don’t "get" them, its only because you aren’t used to things being so simple. It's a common problem in math — we seek meaning where there is none, and think we’re stupid because we don’t get it. Or maybe that’s just me. Moving on…] So agents are little objects, consisting of a few numbers and a few rules of the “if agent 5 is blue, turn red” sort.