不是技术也能看懂云计算，大数据，人工智能

Today I’m going to talk about these three topics, one is cloud computing, one is big data, and one is artificial intelligence. Why should I talk about these three things? Because these three things are very popular now, they seem to have a relationship with each other. Generally speaking, big data is mentioned when talking about cloud computing, big data is also mentioned when talking about artificial intelligence, and cloud computing is also mentioned when talking about artificial intelligence. Therefore, it feels that they are complementary and inseparable, if they are non-technical personnel, it may be difficult to understand the relationship between the three, so it is necessary to explain.

First of all, let’s talk about cloud computing, the initial goal of cloud computing is to manage resources, mainly in three aspects: computing resources, network resources and storage resources.

And if the physical devices are ready in advance, the virtualization software to virtualize a computer is very fast and can be solved in a few minutes. So if you want to create a computer on any cloud, it will come out in a few minutes. That’s why.

But there are still a lot of sentimental people in this world, especially among programmers, what do sentimental people like to do? Open source. A lot of software in the world has closed source and open source, and the source is the source code. That is to say, a certain software is well done and everyone likes to use it. The code of this software is closed and only my company knows, but no one else knows. If other people want to use this software, they have to pay me. This is called closed source. But there are always some big bulls in the world who don’t like to let the family make money. Daniel feels that you can and I can develop this technology, and I can develop it. When I develop it, I will share the code with everyone for free. Anyone in the world can use it, and all people can enjoy the benefits. This is called open source.

Why does Amazon want to be a public cloud? We know that Amazon used to be a relatively large foreign e-commerce business, and when it does e-commerce, it will certainly encounter a scene similar to double 11, and at some point everyone rushes forward to buy things. When everyone is rushing to buy things, there is a special need for the time and space flexibility of the cloud. Because it can not always prepare all the resources, that is too wasteful. But we can’t be unprepared for anything, watching so many users who want to buy things fail to register on Singles’ Day. So when you need Singles Day, create a large number of virtual computers to support e-commerce applications, and then release these resources to do something else after Singles Day. So Amazon needs a cloud platform.

Although the way of scripting can solve the problem of deploying one’s own application, it varies from environment to environment, and a script often runs correctly in one environment, but not in another.

After having the container, all the goods are packed together, and the size of the container is all the same, so every time you change the ship, a box can be moved as a whole, and the hour level can be completed. the crew no longer have to go ashore for a long time.

The so-called mirror image is the moment you weld the container and save the state of the container, just as Sun WuKong said: “fixed”, the container is fixed at that moment, and then save the state of this moment into a series of files. The format of these files is standard, and anyone who sees them can restore the fixed moment at that time. The process of restoring the image to the runtime (that is, the process of reading the image file and restoring that moment) is the process of running the container.

At first, big data was not big. How much data do you imagine there was? Now everyone goes to read e-books and read news on the Internet. when we were born in the 1980s, the amount of information was not so large, so we just read books and newspapers. How many words are there in a week’s newspapers? if you are not in a big city, the library of an ordinary school has only a few bookshelves. Later, with the advent of information, there will be more and more information.

The first step is called data collection. First of all, there must be data. there are two ways to collect data. the first way is to take it. Professionally speaking, it is called crawling or crawling. For example, a search engine does this. It downloads all the information on the Internet to its data center, and then you can search it. For example, when you go to search, the result will be a list. Why is this list in the search engine company? it is because he has taken down all this data, ah, but if you click on a link, the site will no longer be in the search engine company. For example, there is a news on Sina. If you search it with Baidu, when you don’t click, that page is in the Baidu data center, and the page that comes out is in the data center of Sina. Another way is push, there are many terminals can help me collect data, such as Xiaomi bracelet, you can upload your daily running data, heartbeat data, sleep data to the data center.

The second step is the transmission of data. Generally, it will be carried out through the queue, because the amount of data is so large that the data must be processed to be useful, but the system can not handle it, so we have to queue up and deal with it slowly.

The third step is the storage of data. Now data is money, and mastering data is equivalent to mastering money. How else would the website know what you want to buy? It is because it has your historical transaction data, this information can not be given to others, it is very valuable, so it needs to be stored.

The fourth step is data processing and analysis. The data stored above is the original data, the original data is mostly disorganized, there are a lot of junk data in it, so it needs to be cleaned and filtered to get some high-quality data. For high-quality data, we can analyze the data, so as to classify the data, or find the relationship between the data and get knowledge. For example, the rumored story of beer and diapers in Wal-Mart is that through the analysis of people’s purchase data, it is found that when men usually buy diapers, they will buy beer at the same time, thus discovering the relationship between beer and diapers. Gain knowledge, and then apply it to practice, make beer and diaper counters very close, and gain wisdom.

When the amount of data is very small, a few machines can solve it. Slowly, when the amount of data is getting larger and larger, the best server can not solve the problem, what do you want to do? To gather the power of many machines, we all work together to get this done, and everyone gathers firewood and the fire is high.

For data transmission, a queue in memory is bound to be overwhelmed by a large amount of data, so a distributed queue based on hard disk is generated, so that the queue can be transmitted by multiple machines at the same time, no matter how much data you have, as long as I have enough queues and the pipes are thick enough to hold up.

For data storage, the file system of one machine must not fit, so a large distributed file system is needed to do this, typing the hard drives of multiple machines into a large file system.

So big data platform, what is called big data, to put it bluntly, is that a machine can not be finished, we all do it together. With the increasing amount of data, many small companies need to deal with quite a lot of data. what if these small companies don’t have so many machines?

Although there is big data, people’s desires cannot always be satisfied. Although there is a search engine in the big data platform, I found something I wanted as soon as I searched it. But there is also such a situation, what I want will not be searched, can not be expressed, and the search is not what I want. For example, music software recommends a song, this song I have not heard, of course, do not know the name, also can not search, but the software recommended to me, I do like, this is what search can not do. When people use this app, they will find that the machine knows what I want, rather than searching in the machine when I want it. This machine really understands me like my friends, which means something of artificial intelligence.

People have been thinking about this for a long time. At first, people imagined that if there was a wall and there was a machine behind it, if I spoke to it, it would respond to me. If I couldn’t feel whether it was a man or a machine, then it would really be an artificial intelligence thing.

How can we do this? People think: first of all, I want to tell the computer about human reasoning ability. You see what is important to people, and what is the difference between people and animals, that is, they can reason. It would be nice if I told the machine about my reasoning ability, and the machine would be able to reason out the corresponding answer according to your question. In fact, people slowly enable machines to do some things, such as proving mathematical formulas. This is a very surprising process, and the machine can prove the mathematical formula. But slowly found that this result is not so surprising, because we found a problem, the mathematical formula is very rigorous, the reasoning process is also very rigorous, and the mathematical formula is easy to express with the machine, the program is also relatively easy to express. However, human language is not so simple, for example, tonight, you and your girlfriend date, your girlfriend said: if you come early, I did not come, you wait, if I come early, you do not come, you wait. This machine is more difficult to understand, but everyone knows it, so you dare not be late for a date with your girlfriend.

Therefore, it is not enough to tell the machine strict reasoning, but also to tell the machine some knowledge. But the knowledge of this matter, ordinary people may not be able to do, perhaps experts can, such as language experts, or experts in the field of finance and economics. Can the knowledge in the field of language and finance be expressed as strictly as a mathematical formula? For example, language experts may sum up the grammatical rules of subject-predicate-object-definite adverbial complement, which must be followed by a predicate and an object. Will these be summed up and strictly expressed soon? Later found that this is not good, too difficult to sum up, the language expression is ever-changing. Take the example of subject, predicate and object. Very often, the predicate is omitted in the spoken language, and others ask, “who are you?” I replied: I am Liu Chao. But you can’t require you to speak standard written language to the machine during speech semantic recognition, which is still not smart enough, as Luo Yonghao said in a speech. “Please call so-and-so for me, this is a very embarrassing thing,” as Luo Yonghao said in a speech.

This stage of artificial intelligence is called expert system. The expert system is not easy to succeed, on the one hand, the knowledge is difficult to summarize, on the other hand, the summarized knowledge is difficult to teach to the computer. Because you are still confused, you seem to feel regular, but how can you teach the computer by programming if you can’t say it?

So people thought, it seems that the machine is a completely different species from the human, just let the machine learn by itself. How do machines learn? Since the statistical ability of machines is so strong, based on statistical learning, we must be able to find certain rules in a large number of numbers.

What if we write a random string of numbers and then take a word from adjectives, nouns and verbs in numerical order?

Don’t you feel something? Of course, the real statistics-based learning algorithm is much more complex than this simple statistics.

However, statistical learning is easier to understand simple relevance, for example, one word and another word always appear together, the two words should be related, but can not express complex correlation, and the formula of statistical methods is often very complex, in order to simplify the calculation, we often make various independent assumptions to reduce the difficulty of formula calculation, but in real life, independent events are relatively few.

The human brain does not store a large number of rules, nor record a large number of statistical data, but through the trigger of neurons, each neuron has input from other neurons, when it receives the input, it will produce an output to stimulate other neurons, so a large number of neurons react with each other, and finally form the result of various outputs. For example, when people see a beautiful woman’s pupil dilated, it is not the brain that makes a regular judgment according to the proportion of the body, nor does it count all the beautiful women they have seen in their lives, but neurons trigger from the retina to the brain and back to the pupil. In this process, it is difficult to sum up what role each neuron played in the final result, but it did.

This neuron has input and output, and the relationship between input and output is represented by a formula. The input affects the output according to the degree of importance (weight).

If the function represents the law, it also means that the law, no matter how wonderful and incomprehensible, can be expressed by a large number of neurons, through a large number of weights.

We treat each neuron as an individual engaged in economic activity in society. So the neural network is equivalent to the whole economy and society, and each neuron has the right to adjust the input of the society and make corresponding outputs, such as wages rising, vegetable prices rising, stocks falling, what should I do and how to spend my own money. Is there no pattern in this? There must be, but what is the specific rule? But it’s hard to tell.

The economy based on expert system belongs to planned economy, and the expression of the whole economic law does not want to be shown through the independent decision of each economic individual, but hopes to be summed up through the strategically advantageous position and foresight of experts. Experts will never know which street in which city lacks a sweet tofu pudding seller. So experts say that how much steel and steamed bread should be produced is often a big gap from the real needs of the people’s life, and even if the whole plan is written in hundreds of pages, it will not be able to express the small rules hidden in the people’s life.

Microeconomics based on neural network is the most accurate expression of the whole economic law, everyone makes their own adjustments to the input from the society, and the adjustment will also be fed back to the society as an input. Imagine the subtle fluctuation curve of the stock market, which is the result of the continuous trading of each individual, and there is no unified law to follow. And everyone makes independent decisions according to the input of the whole society, when some factors have been trained many times, they will also form a macro statistical law, which is what macroeconomics can see. For example, every time a large number of money is issued, house prices will eventually rise, and after many training, people will learn.

However, the neural network contains so many nodes, each node contains very many parameters, the total number of parameters is really too large, the amount of calculation required is really too large, but it does not matter, we have big data platform, we can gather the strength of multiple machines to calculate together, in order to get the desired results in a limited time.

Artificial intelligence can do many things, such as identifying spam, identifying pornographic and violent words and pictures, and so on. This has also gone through three stages. The first stage relies on black and white lists of keywords and filtering techniques, including which words are yellow or violent words. As there are more and more network languages and words are constantly changing, it is a bit hard to keep updating this thesaurus. In the second stage, based on some new algorithms, such as Bayesian filtering, you don’t care what the Bayesian algorithm is, but you should have heard the name, this is a probability-based algorithm. The third stage is to carry out more accurate user portrait and text understanding and image understanding based on big data and artificial intelligence.