This article is the first in a series that is designed to help you understand machine learning so that you can either get interested enough to create your own solutions, or at least understand it enough to understand what a specialist is building for you.
We live in an age of data.
We now produce, on a daily basis, more data than we produced in a whole year only ten years ago. Not only do we produce more data, we also store this data, thanks to how cheap storage has become. These two changes: the way we generate data, and the way we now store data has made it possible for advances in an old field — Machine Learning.
Machine learning is a sub-field of artificial intelligence. It is concerned with the creation of computer programs. There is one difference. Unlike when you write regular programs, when you write machine learning software, you leave out some things. We will give those things a name in a moment.
Let us imagine that you have a friend who is an accountant and sells mineral water, and said friend likes to keep records. Let us also assume that she sells at different locations and every time your friend makes a sale, she stores the following information: date, time, quantity, location, amount, temperature, humidity.
The first question that comes to mind is: why should you (or anyone else for that matter) be interested in your friend’s data?
You should be interested in your friend’s data because you can learn trends from this data. You can learn for instance, whether there is a relationship between the quantity of sales and the location where the sales were made. We call this kind of relationship a correlation. You can also look at the data and infer whether there is a correlation between sales and temperature, or sales and humidity, or maybe even both temperature and humidity.
Now, at this point, I hope that you are beginning to get ideas.
But this is not yet Machine Learning.
Now, let us say that your friend has run this business for five years, and has kept meticulous records. Now, assume your friend wants to adopt a business model called Just-In-Time, in which your friend will stock just the minimum required quantity of raw materials. In order to do that, your friend needs to know (or predict) future sales.
That prediction is Machine Learning.
The image above shows weights. Some individuals spend a lot of time training to lift weights. Without similar training, it would be impossible for us to walk up and perform the same feat of lifting those weights.
Remember how I said you do not write a complete program when Machine Learning is involved? The parts of the program that you leave out are called weights! So instead of giving the program explicit weights, you train your computer program to find the right weights implicitly. Of course you need to have as much mental ability as the man in the picture has muscular ability, but that is a story for another day.
The secret ingredient in machine learning is all of that data that gets collected and stored on a daily basis.
It turns out that data can tell you a lot of things many of which are things we can learn from.
In the case of the example above, from the data that your friend has kept, you’ll need to write and train a program that will learn to predict sales by learning the required weights. We call this type of problem a regression.
So, let's summarize.
Given enough data, you should be able to come up with a computer program that can give you information about future sales. That is what machine learning is.
Now a sentence or two about training. The data you get from your friend is used for training. The amount of information is important.
If the essence of the data you get from your friend is to predict future sales, and the historical data you get from your friend has those past sales values (these values are called target variables), then we say that you have a supervised learning problem.
If, on the other hand, your friend wants you to look at the data and pick out some relationship without any target variables, or labels, we call that an unsupervised learning problem. In this case, what you will be doing is separating the records into two or more categories.
We will discuss supervised and unsupervised learning in future posts, but for now, here is a quick summary of what you can do with supervised learning.
Your friend might want you to predict future sales. Remember that is what we call regression.
Your friend might have employed a new book keeper who stops recording the location of sales, and your friend wants you to figure out where each sale was made. We call that a classification.
Your friend might come to you and say: I have a new book keeper and these records don’t look right. I think something smells fishy!
Smelling out fishy transactions is an unsupervised learning problem called anomaly detection.
Your friend might want you to put the transactions into groups so that junior managers can be assigned to them. We call that clustering.
Machine Learning is all around us. Examples include smart assistants like Siri in iPhone and Google Assistant on Android, smart thermostats for regulating climate conditions in homes, offices and hotels, and self-driving cars.
When you upload a photo on Facebook and it automatically identifies people and tags them, that is an advanced version of machine learning.
Facebook even recently announced that they are rolling out intervention for people with suicidal tendencies. That is to say, they can tell from a person’s posts if that person is depressed to the point of committing suicide. That is really important.
In some parts of the world, you can buy an item by taking a picture with your phone and sending it instead of describing what you want. It is not humans that look at the pictures, but computers that read it and figure out what you want.
When you receive an email, your provider automatically filters out spam. This is another example of machine learning where past emails, and emails that you mark as spam, are used to train a spam classifier.
There are other more advanced uses of data. Can you think of any uses on your own? If you do, please leave a note. For now, thanks for reading and look out for the next installment of this series.
I have a free course that teaches you Linear Regression without going into the Math. Visit https://www.thas.dev to get started.
This article was originally published on my Medium blog.