However, for many people, artificial intelligence is still a relatively "advanced" technology. But even the most advanced technologies start from basic principles. There are 10 well-known algorithms in the field of artificial intelligence. Their principles are simple, they were discovered and applied a long time ago, you may have even learned them in middle school, and they are extremely common in life.
1
Linear regression is perhaps the most popular machine learning algorithm. The goal of linear regression is to find a straight line that fits the data points in a scatter plot as closely as possible. It attempts to represent the independent variable (x-values) and the numerical outcome (y-values) by fitting the linear equation to the data. This line can then be used to predict future values! The most commonly used technique in this algorithm is the least squares method. This method calculates the best-fit line that minimizes the vertical distance to each data point on the line. The total distance is the sum of the squares of the vertical distances (green lines) to all data points. The idea is to fit the model by minimizing this squared error or distance.
For example, simple linear regression has one independent variable (x-axis) and one dependent variable (y-axis), used to predict next year's house price increase, or the sales volume of new products in the next quarter. It sounds simple, but the difficulty of linear regression algorithms lies not in obtaining the predicted value, but in how to make it more accurate. Countless engineers have devoted their youth and hair to that potentially minuscule number.
2
Logistic regression is similar to linear regression, but it only yields two values. If linear regression predicts an open-ended value, logistic regression is more like answering a yes-or-no question. The Y value in the logistic function ranges from 0 to 1, representing a probability value. The logistic function is typically S-shaped, with the curve dividing the graph into two regions, making it suitable for classification tasks.
For example, the logistic regression curve above shows the relationship between the probability of passing an exam and study time, and can be used to predict whether or not one will pass the exam. Logistic regression is frequently used by e-commerce or food delivery platforms to predict users' purchasing preferences for product categories.
3
If linear and logistic regression both conclude a task within a single round, then decision trees represent a multi-step process. They are also used in regression and classification tasks, but the scenarios are typically more complex and specific. For a simple example, consider a teacher with a class of students. Who are the good students? Simply judging them based on a score of 90 seems too simplistic; we can't rely solely on grades. Therefore, for students scoring below 90, we can discuss their homework, attendance, and participation in class separately.
The above is an example of a decision tree, where each branching circle is called a node. At each node, we ask questions about the data based on the available features. The left and right branches represent possible answers. The final node (i.e., the leaf node) corresponds to a predicted value. The importance of each feature is determined using a top-down approach. The higher the node, the more important its attribute. For example, in the example above, the teacher considers attendance more important than homework, so the attendance node is higher, and naturally, the score node is higher as well.
4
Naive Bayes is based on Bayes' theorem, which states the conditional relationship between two classes. It measures the probability of each class, and the conditional probability of each class gives the value of x. This algorithm is used for classification problems and produces a binary "yes/no" result. See the equation below.
Naive Bayes classifiers are a popular statistical technique, with a classic application in filtering spam. Of course, I bet 80% of people didn't understand the above paragraph. (This 80% figure is just my guess, but intuitively it's a Bayesian calculation.) To explain Bayes' theorem in non-technical terms, it means using the probability of B occurring under condition A to deduce the probability of A occurring under condition B. For example, if a kitten likes you and there's an a% chance it will roll over on its back in front of you, what's the probability that the kitten likes you if it rolls over on its back? Of course, solving this directly is like guessing, so we need to introduce other data, such as the kitten liking you, having a b% chance of nuzzling you, and a c% probability of purring. So how do we know the probability that the kitten likes us? We can calculate this from the probabilities of rolling over, nuzzling, and purring using Bayes' theorem.
5
Support Vector Machine (SVM) is a supervised algorithm for classification problems. SVM attempts to draw two lines between data points with the maximum margin between them. To do this, we represent data items as points in n-dimensional space, where n is the number of input features. Based on this, SVM finds an optimal boundary, called a hyperplane, that best separates the possible outputs by class labels. The distance between the hyperplane and the nearest class point is called the margin. The optimal hyperplane has the largest boundary to classify points, thus maximizing the distance between the nearest data point and both classes.
Therefore, the problem that support vector machines aim to solve is how to differentiate a large amount of data. Its main applications include character recognition, facial recognition, text classification, and other recognition methods.
6
The K-Nearest Neighbors (KNN) algorithm is quite simple. KNN classifies objects by searching for the K most similar instances (K neighbors) across the entire training set and assigning a common output variable to all K instances. The choice of K is crucial: small values can lead to a lot of noise and inaccurate results, while large values are impractical. It is most commonly used for classification, but it is also suitable for regression problems. The distance used to evaluate the similarity between instances can be Euclidean distance, Manhattan distance, or Minkowski distance. Euclidean distance is the ordinary straight-line distance between two points. It is actually the square root of the sum of the squares of the differences in the point coordinates.
KNN Classification Example: KNN theory is simple and easy to implement, and can be used for text classification, pattern recognition, cluster analysis, etc.
7
K-means clusters data by classifying it. For example, this algorithm can be used to group users based on their purchase history. It finds K clusters in the dataset. K-means is used for unsupervised learning, so we only need training data X and the number of clusters K we want to identify. The algorithm iteratively assigns each data point to one of K groups based on its features. It selects K points for each K-cluster (called a centroid). New data points are added to the cluster with the nearest centroid based on similarity. This process continues until the centroids stop changing.
In everyday life, the K-means plays an important role in fraud detection and is widely used in the fields of automotive, health insurance, and insurance fraud detection.
8
Random Forest is a very popular ensemble machine learning algorithm. The basic idea behind this algorithm is that the opinions of many people are more accurate than those of an individual. In Random Forest, we use decision tree ensembles (see Decision Tree).
(a) During training, each decision tree is constructed based on the bootstrap samples of the training set. (b) During classification, the decision of the input instance is made based on majority voting. Random forests have broad application prospects, from marketing to healthcare insurance. They can be used for marketing simulation modeling, statistical analysis of customer acquisition, retention, and churn, as well as for predicting disease risk and patient susceptibility.
9
The sheer volume of data we can capture today has made machine learning problems far more complex. This means training is extremely slow, and finding a good solution is incredibly difficult. This problem is often referred to as the "curse of dimensionality." Dimensionality reduction attempts to address this issue by combining specific features into higher-level features without losing the most important information. Principal Component Analysis (PCA) is the most popular dimensionality reduction technique. PCA reduces the dimensionality of a dataset by compressing it into a lower-dimensional line or hyperplane/subspace. This preserves as many salient features as possible from the original data.
An example of dimensionality reduction can be achieved by approximating all data points to a straight line.
10
Artificial Neural Networks (ANNs) can handle large and complex machine learning tasks. A neural network is essentially a set of interconnected layers of edges and nodes with weights, called neurons. Multiple hidden layers can be inserted between the input and output layers. Artificial neural networks use two hidden layers. In addition, deep learning is required. The working principle of an artificial neural network is similar to the structure of the brain. A set of neurons is assigned random weights to determine how the neurons process the input data. The neural network learns the relationship between inputs and outputs by training it on the input data. During the training phase, the system has access to the correct answers. If the network cannot accurately identify the input, the system adjusts the weights. After sufficient training, it will consistently identify the correct patterns.
Each circular node represents an artificial neuron, and the arrows indicate connections from the output of one artificial neuron to the input of another. Image recognition is a well-known application of neural networks. Now, you have a basic understanding of the most popular artificial intelligence algorithms and some knowledge of their practical applications.
