强人工智能必经之路？知识图谱超详细总结，快速入门KG首选(万字长文，值得收藏)

Models that rely on machine learning and deep learning algorithms try to describe the “truth” behind the world through data statistics from the point of view of relevance and probability, but it is still a long way from the real “artificial intelligence”. The “knowledge graph ” is more like machine intelligence that can analyze and reason like human beings.

This article explains the knowledge related to the knowledge graph in an easy-to-understand way, and describes in detail how to build the knowledge graph from scratch through the financial risk control case . As well as the steps to be experienced in the process and the problems to be considered at each stage.*

In order to better understand the knowledge graph, we must first understand the difference between information and knowledge.

Then understand the concept of knowledge graph from different perspectives.

And now a lot of data are connected to each other, if you want to analyze the value of these links, knowledge graph can be an effective tool. With the advent of the era of the Internet of everything, the information contained in the link is bound to play a greater value, which is the main reason why the knowledge graph has developed so rapidly in recent years.

At present, there is no unified definition of knowledge graph in academic circles. It is clearly described in the documents released by Google that knowledge graph is a technical method of using graph model to describe the relationship between knowledge and modeling everything in the world.

Specifically, knowledge graph is a general formal description framework of semantic knowledge, which uses nodes to represent semantic symbols and edges to represent the relationship between semantics.

Many scenes in the real world are very suitable to be expressed by knowledge graph. For example, in a social network graph, we can have both “human” and “corporate” entities. The relationship between people can be either “friend” or “colleague”. The relationship between a person and a company can be a “current” or “former” relationship. Similarly, a risk control knowledge graph can include “phone” and “company” entities, and the relationship between phone and phone can be “call” relationship.

Next, it explains in detail the three elements that make up the knowledge graph, including: * entity * , * relationship * and * attribute * .

The image above is a simple attribute diagram.

*The architecture of knowledge graph mainly includes its own logical structure and architecture. *

The construction of knowledge graph is the basis of subsequent application, and the premise of construction is to extract data from different data sources. For vertical knowledge graphs, * their data sources mainly come from two channels: one is the data of the business itself, which is usually contained in the company’s database tables and stored in a structured way; the other is the publicly crawled data on the network, which is usually unstructured data stored in the form of web pages. *

The difficulty of information extraction is to deal with unstructured data. In the process of constructing similar maps, natural language processing techniques are mainly involved in the following aspects:

The construction and application of large-scale knowledge base needs the support of a variety of natural language processing technologies. Through the technology of * knowledge extraction * , knowledge elements such as entities, relationships and attributes can be extracted from some open semi-structured and unstructured data. Through * knowledge fusion * , the ambiguity between the referential items such as entities, relations, attributes and factual objects can be eliminated, and a high-quality knowledge base can be formed. * knowledge reasoning * is to further mine the hidden knowledge on the basis of the existing knowledge base, so as to enrich and expand the knowledge base. The comprehensive vector formed by distributed knowledge representation is of great significance to the construction, reasoning, fusion and application of knowledge base.

Knowledge extraction is mainly oriented to open linked data, and the available knowledge units are extracted by automatic technology. The knowledge unit mainly includes three knowledge elements: * entity (extension of concept) * , * relation * and * attribute * . And on this basis, a series of high-quality fact expressions are formed, which lays the foundation for the construction of the upper pattern layer. There are three main tasks of knowledge extraction:

In recent years, the representation learning technology represented by deep learning has made important progress, which can express the semantic information of entities as dense low-dimensional real-valued vectors, and then efficiently calculate entities, relationships and their complex semantic associations in low-dimensional space, which is of great significance for the construction, reasoning, fusion and application of knowledge base.

Because there are a wide range of knowledge sources in the knowledge graph, there are some problems, such as the quality of knowledge is different, knowledge repetition from different data sources, knowledge association is not clear enough, so it is necessary to carry out knowledge fusion. Knowledge fusion is a high-level knowledge organization, which enables knowledge from different knowledge sources to carry out heterogeneous data integration, disambiguation, processing, reasoning verification, updating and other steps under the same framework. To achieve the integration of data, information, methods, experience and human ideas, to form a high-quality knowledge base.

Among them, knowledge updating is an important part. Human cognitive ability, knowledge reserve and business needs will all increase over time. Therefore, the content of knowledge graph also needs to keep pace with the times. * whether it is general knowledge graph or industry knowledge graph, they all need to be iterated and updated constantly to expand existing knowledge and add new knowledge.*

Third, the construction process and design of knowledge graph.

First of all, we still have to set a specific problem, so that the whole design has a clear purpose. In this chapter, we mainly take * financial risk control * as an example to describe the construction process of knowledge graph.

The construction of a complete knowledge graph includes the following steps:

Let’s follow this process to talk about what needs to be done and what needs to be thought about in each step.

In this process, the core is actually the design of the map, because once the map is designed, it will become our follow-up “brain”, good or bad design will directly affect the future application. This is similar to building database tables, once the design is not reasonable, there will be a lot of problems. To a large extent, designing a knowledge graph depends on the understanding of the business and the prospect of the future business.

At present, the problem to be solved is how to determine the fraud risk of a person’s applicant by technical means.

The next step is to identify the data source and do the necessary data preprocessing. For data sources, we need to consider the following:

What needs to be noted here is that not all data related to anti-fraud must enter the knowledge graph, and some of the decision-making principles of this part will be introduced in more detail in the following section.

For anti-fraud, there are several data sources that we can easily imagine, including the basic information of * users, behavior data, operator data, e-commerce data, blacklists, public information on the network, and so on. Assuming that we already have a list of data sources, the next step is to see which data needs further processing, for example, for unstructured data, we more or less need to use technologies related to natural language processing. The basic information filled in by users is basically stored in the business table, except for individual fields that need further processing, many fields can be directly used for modeling or added to the knowledge graph system. For behavioral data, we need to go through some simple processing and extract valid information such as “how long the user stays on a page” and so on. For the web page data open on the network, some technologies related to information extraction are needed.*

For example, for the basic information of the user, we probably need to do the following. On the one hand, user information such as name, age, education and other fields can be extracted and used directly from the structured database. But on the other hand, for the company name filled in, we may need to do further processing. For example, some users fill in “Beijing greedy Technology Co., Ltd.” and others fill in “Beijing Wangjing greedy Technology Co., Ltd.”. In fact, they all point to the same company. Therefore, at this time, we need to do the company name alignment, the technical details can refer to the entity alignment technology mentioned earlier.

In the design of knowledge graph, we will certainly face the following common problems:

Based on these common problems, we abstract a series of design principles from previous design experience. These design principles are similar to the paradigm in the traditional database design to guide the relevant personnel to design a more reasonable knowledge graph system while ensuring the efficiency of the system.

The above is summed up from previous experience and may not be completely accurate, but it can at least reflect the holes that we avoid when designing knowledge graphs. Next, I will explain to you around each point.

Using the above principles, we can design a graph in the field of financial anti-fraud. Finally, a simplified graph is shown below. Of course, the map in practical application is much more complicated than this!

After building the knowledge graph, we should use it to solve specific problems. For the knowledge graph of financial risk control, the first task is to mine the hidden fraud risk in the relational network. From an algorithm point of view, there are * two different scenarios * : one is * rule-based * , and the other is * probability-based * . The former depends on the experience of experts, while the latter depends on data-driven.

First of all, let’s look at several rule-based applications, which are * inconsistency verification * , * rule-based feature extraction * , * pattern-based judgment * .

In order to judge the risks in the relationship network, a simple way is to do inconsistency verification, that is, to identify potential contradictions through some rules. These rules are artificially defined in advance, so some business knowledge is needed in the matter of designing rules. For example, in the following picture, both Li Ming and Li Fei indicate the same company phone number, but actually judge from the database that they actually work in different companies, which is a contradiction. As a matter of fact, there can be many similar rules, which are not listed here.

We can also extract some features from the knowledge graph based on rules, and these features are generally based on depth-based searches such as 2-degree, 3-degree or even higher dimensions. For example, we can ask the question: “what is the relationship between the borrower and the two?” From the picture, we can easily observe that the borrower is Li Fei’s father, and Li Ming is Li Fei’s friend. After these features are extracted, they can be used as the input of the risk model. I would like to make it clear here that if the features do not involve in-depth relationships, in fact, traditional relational databases are sufficient to meet the needs.

For example, we can also find the strongly connected graph from the knowledge graph, mark it, and then do further risk analysis. Strongly connected graph means that each node can reach other points through a certain path, which shows that there is a strong relationship between these nodes.

In addition to rule-based methods, probability and statistical methods can also be used, such as * community mining * , * tag propagation * , * clustering * and so on.

Because community mining is based on probability methodology, the advantage is that there is no need to define rules artificially, especially for a large relational network, defining rules itself is a very complicated thing.

Compared with the regular methodology, the disadvantage of the probability-based approach is that it requires enough data. If the amount of data is small and the whole graph is sparse, the rule-based method can become our first choice. Especially for the financial sector, there are fewer data labels, which is the main reason why rule-based methodology is still more widely used in the financial field.

All of the above analyses are based on a static relationship graph. The so-called static relationship graph means that we do not consider the change of the graph structure itself with time, but focus on the current knowledge graph structure. However, we also know that the structure of the map changes over time, and these changes themselves can be associated with risk.

Knowledge graph is a challenging and interesting field. As long as there is a correct practical application scenario, the value of knowledge graph is worth looking forward to. In the near future, knowledge graph technology will be widely used in various fields.

And knowledge graph is a relatively new tool, its main function is to analyze the relationship, especially the depth relationship. So in business, first of all to ensure its necessity, in fact, many problems can be solved in a non-knowledge graph way.

One of the most important roles in the field of knowledge graph is knowledge reasoning. And the reasoning of knowledge is the only way to strong artificial intelligence.