Coding in the Age of Machine Learning
by Imran Haider on August 11, 2017
Industrial Revolution automated labor work. Information Revolution automated the mental work. Machine Learning Revolution will automate the automation itself. —Pedro Domingos
The magic of computers has always been in their programmability. You write a set of instructions and use them again and again to perform certain tasks. Inputs utilized are data and algorithms or programs (a program is just an algorithm written in a specific programming language). And output is the desired result. This works great until you realize that you can't distill everything into a set of instructions. For example you can't teach a computer to cook in a particular way. A computer does not understand a recipe because a recipe is not a set of instructions. A recipe cannot be a set of instructions because computer has no idea what a spoon is. It will also not be able to determine what sugar is exactly. Humans know these specifics because of our observation and experience, but a computer does not have either of these capacities. Driving a car is another example. You can't code the entire process of driving because it is a learned behavior we acquire through observation and repetition. We learn how to avoid dangers on the road by studying a manual and also through hands-on practice.
As an alternative example, computers excel at face recognition and this is something you can’t code. In a world without machine learning you would have to write a program for every uploaded photo for billions of people. This proves to be an immensely challenging task, because not only is every face fundamentally different, but so is every occurrence of that face. What machine learning does is to it take data (lots of photos) and a desired result (show that this is “John” and this is “Mary”) and it then creates an algorithm on its own. And then it starts applying that algorithm to all the photos. Since algorithm is computer written, it actually modifies this on the fly whenever there is a mistake in face detection. The result is an algorithm that improves as data increases. This is unlike traditional programming where more data means less effectiveness of the original algorithm.
Two things are of note here. First, machine learning is dependent on data, large sets of data. Less data means less effectiveness of the underlying algorithm. Second, the automation process for automating a program means that initially, hundreds of lines of code can easily become millions of lines of code in no time. This is such a massive complexity killer. A million lines of code might be a programmer's life's work. But for a machine learning algorithm it's only a matter of hours, if not minutes provided you have enough data.
Our lives are learning algorithms. We learn something and that learning becomes part of our life algorithm. Using that learning algorithm trajectory we build a greater knowledge platform. Often, the more knowledge we gain, the better our life algorithm becomes.
Historically we have taught skills to computers in a “do this and do that” format. And what they learn is effective, however, as soon as they performed the task, they forget about it. For example if you ask a computer to add 2+2, you will have to code the algorithm. And no matter how many times it performed that task, the fact that 2+2=4 never got added to its algorithm. Because it never had one of its own. Machine learning enables that. Since the algorithm is now devised by computer itself, it only needs to perform 2+2=4 once. After that it becomes part of its knowledge base.
It's clear that role of a computer programmer is going to change dramatically because of machine learning. A useful way to understand that change will be farmer and manufacturer analogy. From The Master Algorithm:
The power of machine learning is perhaps best explained by a low-tech analogy: farming. In an industrial society, goods are made in factories, which means that engineers have to figure out exactly how to assemble them from their parts, how to make those parts, and so on—all the way to raw materials. It’s a lot of work. Computers are the most complex goods ever invented, and designing them, the factories that make them, and the programs that run on them is a ton of work. But there’s another, much older way in which we can get some of the things we need: by letting nature make them. In farming, we plant the seeds, make sure they have enough water and nutrients, and reap the grown crops. Why can’t technology be more like this? It can, and that’s the promise of machine learning. Learning algorithms are the seeds, data is the soil, and the learned programs are the grown plants. The machine-learning expert is like a farmer, sowing the seeds, irrigating and fertilizing the soil, and keeping an eye on the health of the crop but otherwise staying out of the way.
