Machine learning is a branch of artificial intelligence that creates systems that learn from data. These systems can perform tasks like recognizing faces, detecting spam, recommending products, and playing games.
It has applications in multiple fields including healthcare, education, finance, e-commerce, entertainment and others. It can assist in diagnosing diseases, personalizing learning, detecting fraud, optimizing prices, and content creation.
This article provides an introduction to machine learning concepts, methods and techniques. It will cover the basics of machine learning, its types, key concepts and terms, project steps, techniques, models and algorithms, tools and frameworks, challenges and limitations, and its application using Python.
After reading, you will understand the basics of machine learning and can explore advanced topics independently.
Highlights
Some key facts or tips that you should remember from this article are:
- Machine learning can be classified into three main types: supervised, unsupervised, and reinforcement learning
- Machine learning follows a general process that consists of six steps: data collection, data preprocessing, data analysis, model building, model evaluation, and model deployment
- Machine learning faces some challenges and limitations that can affect its performance, reliability, security, and ethics
- Machine learning can provide many benefits for various applications and domains by enhancing productivity, improving accuracy, discovering insights, and enabling innovation
At its core, machine learning is a method of data analysis that automates the building of analytical models. It’s a branch of artificial intelligence that uses algorithms and statistical models to enable machines to improve their performance on a specific task, based on their experiences, without being explicitly programmed.
What is Machine Learning?
Machine learning is a subdomain of artificial intelligence that focuses on developing systems that learn from data². Machine learning systems can improve their performance or behavior based on the data they ingest, without being explicitly programmed.
The difference between machine learning and traditional programming is that in traditional programming, we specify the rules or logic for the system to follow, while in machine learning, we provide the data for the system to learn from.
The following diagram or infographic illustrates how machine learning works:
- Train the model: The fifth step is to learn the optimal values of the model parameters from the training data. This involves applying or implementing different algorithms (such as gradient descent, backpropagation, etc.), optimizing or minimizing different loss functions (such as mean squared error, cross entropy, etc.), and adjusting or tuning different hyperparameters (such as learning rate, regularization, etc.).
- Evaluate the model: The sixth step is to measure and compare the performance of the model on the validation and testing data. This involves calculating or computing different metrics (such as accuracy, precision, recall, etc.), analyzing or interpreting different results (such as confusion matrix, ROC curve, etc.), and validating or verifying different assumptions or expectations (such as bias-variance tradeoff, overfitting-underfitting, etc.).
- Deploy the model: The seventh step is to deploy and use the model in a real-world setting or environment. This involves converting or exporting the model into a suitable format (such as pickle, ONNX, etc.), integrating or embedding the model into a system or application (such as web, mobile, cloud, etc.), and monitoring or maintaining the model over time (such as updating, retraining, debugging, etc.).
Types of Machine Learning
Machine learning is a subfield of artificial intelligence, which is broadly defined as the capability of a machine to imitate intelligent human behavior. Machine learning enables computers to learn from data and experience, without being explicitly programmed for every task.
There are three main types of machine learning: supervised, unsupervised, and reinforcement learning. Each type has different methods and applications, depending on the nature and availability of the data and the desired outcome.
Supervised learning
Supervised learning is the most common and widely used type of machine learning. In supervised learning, the machine learns from labeled data, which means that the data has a known output or target variable. The machine uses an algorithm to learn a function that maps the input data to the output data. The goal of supervised learning is to make accurate predictions for new, unseen data.
Some examples of supervised learning are:
- Classification: The machine learns to assign a discrete label or category to an input data point, such as spam or not spam for an email, or positive or negative for a sentiment analysis.
- Regression: The machine learns to predict a continuous value or quantity for an input data point, such as house price, stock price, or blood pressure.
- Anomaly detection: The machine learns to detect abnormal or unusual patterns or behaviors in the data, such as fraud detection, intrusion detection, or medical diagnosis.
Unsupervised learning
Unsupervised learning is the type of machine learning where the machine learns from unlabeled data, which means that the data has no known output or target variable. The machine uses an algorithm to discover hidden patterns, structures, or features in the data. The goal of unsupervised learning is to find meaningful insights or representations from the data.
Some examples of unsupervised learning are:
- Clustering: The machine learns to group similar or related data points together based on their features or characteristics, such as customer segmentation, image segmentation, or topic modeling.
- Dimensionality reduction: The machine learns to reduce the number of features or dimensions in the data, while preserving as much information as possible, such as principal component analysis (PCA), singular value decomposition (SVD), or autoencoders.
- Association rule mining: The machine learns to find rules or patterns that describe how items or events are related or co-occur in the data, such as market basket analysis, recommender systems, or text mining.
Reinforcement learning
Reinforcement learning is the type of machine learning where the machine learns from its own actions and feedback from the environment. The machine uses an algorithm to learn a policy that maximizes a reward function over time. The goal of reinforcement learning is to find the optimal behavior or strategy for a given situation.
Some examples of reinforcement learning are:
- Control: The machine learns to control a system or a device by adjusting its parameters or actions based on the observed state and reward, such as robotics, self-driving cars, or smart grids.
- Games: The machine learns to play a game by exploring different moves and outcomes based on the observed state and reward, such as chess, Go, or Atari games.
- Decision making: The machine learns to make decisions under uncertainty by considering different actions and consequences based on the observed state and reward, such as portfolio management, resource allocation, or bidding.
Image Source: Medium
The following table summarizes some of the differences between the three types of machine learning:
| Type | Data | Output | Algorithm | Goal |
|---|---|---|---|---|
| Supervised | Labeled | Known | Learn function | Make predictions |
| Unsupervised | Unlabeled | Unknown | Discover patterns | Find insights |
| Reinforcement | Actions + rewards | Optimal | Learn policy | Find strategy |
The following diagram illustrates the three types of machine learning:
Basics of Machine Learning Concepts
Machine learning is a complex and multidisciplinary field that involves many concepts and terms. Here are some of the key terms and concepts in machine learning that you should know:
- Data: Data is the raw material for machine learning. Data can be structured (such as tables or spreadsheets) or unstructured (such as text, images, or audio). Data can be numerical (such as measurements or ratings) or categorical (such as labels or classes). Data can be static (such as historical records) or dynamic (such as real-time streams).
- Model: A model is a mathematical representation of a system or a phenomenon that can be learned from data. A model can be parametric (suchas a linear regression or a logistic regression) or non-parametric (such as a decision tree or a neural network). A model can be deterministic (such as a rule-based system) or probabilistic (such as a Bayesian network).
- Algorithm: An algorithm is a set of rules or instructions that can be used to learn a model from data. An algorithm can be supervised (such as gradient descent or k-means) or unsupervised (such as principal component analysis or hierarchical clustering). An algorithm can be iterative (such as backpropagation or expectation-maximization) or recursive (such as divide-and-conquer or dynamic programming).
- Training: Training is the process of learning a model from data using an algorithm. Training can be batch (such as linear regression or support vector machines) or online (such as stochastic gradient descent or adaptive boosting). Training can be inductive (such as decision trees or neural networks) or transductive (such as k-nearest neighbors or support vector machines).
- Testing: Testing is the process of evaluating the performance of a model on new, unseen data. Testing can be done using different metrics, such as accuracy, precision, recall, f1-score, roc curve, etc. Testing can be done using different methods, such as holdout, cross-validation, bootstrap, etc.
- Validation: Validation is the process of tuning the parameters or hyperparameters of a model to optimize its performance on a validation set, which is a subset of the training data. Validation can be done using different methods, such as grid search, random search, bayesian optimization, etc.
- Evaluation: Evaluation is the process of assessing the quality and usefulness of a model for a specific task or domain. Evaluation can be done using different criteria, such as generalization, robustness, interpretability, fairness, etc. Evaluation can be done using different techniques, such as ablation studies, feature importance analysis, error analysis, etc.
- Deployment: Deployment is the process of putting a model into production or operation for a real-world application. Deployment can be done using different platforms, such as cloud computing, edge computing, mobile devices, etc. Deployment can be done using different frameworks, such as TensorFlow, PyTorch, Scikit-learn, etc.
The following diagram illustrates some of the machine learning concepts:
Image source: FAVOURITEBLOG.COM
Basics of Machine Learning Process
Machine learning is not a one-time event, but a continuous and iterative process that involves many steps and activities. The general steps involved in a machine learning project are:
- Data collection: This is the first and most important step in any machine learning project. Data collection involves gathering relevant and sufficient data from various sources, such as databases, web pages, sensors, surveys, etc. Data collection also involves defining the problem statement and the objectives of the project.
- Data preprocessing: This is the step where the data is cleaned and prepared for machine learning. Data preprocessing involves handling missing values, outliers, noise, duplicates, etc. Data preprocessing also involves transforming and encoding the data into suitable formats and scales for machine learning.
- Model selection: This is the step where the appropriate machine learning type and algorithm are chosen for the project. Model selection involves comparing and contrasting different models based on their assumptions, advantages, disadvantages, complexity, etc. Model selection also involves selecting relevant features and variables for machine learning.
- Model training: This is the step where the model is learned from the data using an algorithm. Model training involves splitting the data into training and validation sets and applying the algorithm to learn the model parameters or hyperparameters. Model training also involves monitoring and visualizing the training process and results.
- Model testing: This is the step where the model is evaluated on new, unseen data. Model testing involves splitting the data into testing and validation sets and applying the model to make predictions or classifications. Model testing also involves measuring and reporting the model performance using different metrics and methods.
- Model evaluation: This is the step where the model is assessed for its quality and usefulness for the project. Model evaluation involves analyzing and interpreting the model results and outcomes using different criteria and techniques. Model evaluation also involves identifying and addressing the model limitations, errors, biases, etc.
- Model deployment: This is the final step where the model is put into production or operation for a real-world application. Model deployment involves integrating and interfacing the model with other systems or devices using different platforms and frameworks. Model deployment also involves maintaining and updating the model as needed.
Machine Learning Challenges and Limitations
Machine learning is a powerful and promising technology that can enable many applications and innovations in various domains and industries. However, machine learning also faces some challenges and limitations that can affect its performance, reliability, security, and ethics. Some of the common challenges and limitations of machine learning are:
- Data quality: Machine learning depends on data to learn and make predictions or decisions. However, data can be incomplete, inaccurate, noisy, inconsistent, or biased, which can lead to poor or erroneous results. Data quality can be affected by various factors, such as data collection methods, data preprocessing techniques, data sources, data formats, etc. Therefore, machine learning professionals need to ensure that the data they use is relevant, sufficient, diverse, and representative of the problem domain and the target population.
- Bias: Machine learning can inherit or amplify human biases that exist in the data or the algorithms. Bias can result in unfair or discriminatory outcomes for certain groups or individuals based on their characteristics, such as gender, race, age, etc. Bias can also affect the generalization and robustness of machine learning models to new or unseen data or scenarios. Therefore, machine learning professionals need to identify and mitigate bias in their data and algorithms by using various methods, such as data augmentation, feature selection, regularization, debiasing techniques, etc.
- Security: Machine learning can be vulnerable to various attacks or threats that can compromise its integrity, confidentiality, or availability. Security issues can arise from malicious actors who can manipulate the data or the algorithms to cause harm or gain advantage. For example, adversarial examples are inputs that are intentionally designed to fool machine learning models into making wrong predictions or classifications. Therefore, machine learning professionals need to protect their data and algorithms from unauthorized access or modification by using various methods, such as encryption, authentication, verification, etc.
- Ethics: Machine learning can raise various ethical questions or dilemmas that can affect its social impact or acceptability15. Ethical issues can arise from the potential consequences or implications of machine learning applications or outcomes for human values, rights, responsibilities, etc15. For example, privacy is a major concern for machine learning users who may not want their personal data to be collected or used without their consent or knowledge15. Therefore, machine learning professionals need to adhere to ethical principles and guidelines that can ensure the trustworthiness and accountability of their data and algorithms15.
Some examples of how these challenges and limitations affect machine learning applications and outcomes are:
- Data quality: A famous example of how poor data quality can affect machine learning outcomes is the case of IBM Watson for Oncology. This was a system that was supposed to provide cancer treatment recommendations based on medical records and literature. However, it was found that the system was unreliable and inconsistent because it was trained on synthetic data that did not reflect real-world cases or best practices.
- Bias: A well-known example of how bias can affect machine learning outcomes is the case of Amazon’s resume screening tool. This was a system that was supposed to help hire software developers based on their resumes. However, it was found that the system was biased against women because it favored resumes that contained words or phrases that were more common among male applicants, such as “executed” or “captured”.
- Security: A recent example of how security can affect machine learning outcomes is the case of DeepMind’s AlphaFold. This was a system that was able to predict the three-dimensional structure of proteins based on their amino acid sequences. However, it was reported that some hackers were able to access and steal some of the system’s data and code, which could pose a threat to the scientific community or the biotechnology industry.
- Ethics: A controversial example of how ethics can affect machine learning outcomes is the case of Google’s Project Maven. This was a project that involved using machine learning to analyze drone footage for military purposes. However, it faced a lot of criticism and backlash from Google employees and activists who argued that the project was unethical and violated Google’s motto of “Don’t be evil”.
Common and Effective Machine Learning Techniques, Models and Algorithms
Machine learning has many techniques, models and algorithms that are widely used and proven to be effective for various tasks and problems². Here are some of the most common and popular ones:
- Linear regression: Linear regression is a supervised learning technique that learns a linear relationship between an input variable (x) and an output variable (y). Linear regression can be used for regression problems, such as predicting house prices, stock prices, etc.
- Logistic regression: Logistic regression is a supervised learning technique that learns a logistic function that maps an input variable (x) to a probability of belonging to a class (y). Logistic regression can be used for classification problems, such as predicting whether an email is spam or not, whether a tumor is benign or malignant, etc.
- Decision tree: Decision tree is a supervised learning technique that learns a tree-like structure that splits the input space into regions based on rules or criteria. Decision tree can be used for both regression and classification problems, such as predicting customer satisfaction, credit risk, etc.
- K-means clustering: K-means clustering is an unsupervised learning technique that learns k clusters or groups of similar inputs based on their distance or similarity. K-means clustering can be used for clustering problems, such as segmenting customers, images, documents, etc.
- Principal component analysis: Principal component analysis is an unsupervised learning technique that learns k principal components or directions of maximum variance in the input data. Principal component analysis can be used for dimensionality reduction problems, such as reducing noise, compressing data, visualizing data, etc.
- Neural network: Neural network is a supervised learning technique that learns a network of interconnected nodes or neurons that process inputs and outputs using nonlinear activation functions. Neural network can be used for both regression and classification problems, such as recognizing handwritten digits, speech recognition, natural language processing, etc.
Popular Machine Learning Tools and Frameworks
Machine learning has many tools and frameworks that are designed to facilitate and simplify the development and deployment of machine learning systems². Here are some of the most popular and widely used ones:
- Python: Python is a general-purpose programming language that has many libraries and packages that support machine learning, such as NumPy, SciPy, Pandas, Matplotlib, Scikit-learn, TensorFlow, PyTorch, etc. Python is easy to learn, write and read, and has a large and active community of developers and users.
- R: R is a programming language and environment that is specialized for statistical computing and graphics. R has many packages and tools that support machine learning, such as dplyr, ggplot2, caret, mlr, keras, etc. R is widely used by statisticians, data scientists and researchers, and has a rich and diverse set of resources and documentation.
- MATLAB: MATLAB is a programming language and platform that is designed for numerical computing and visualization. MATLAB has many toolboxes and functions that support machine learning, such as Statistics and Machine Learning Toolbox, Neural Network Toolbox, Deep Learning Toolbox, etc. MATLAB is popular among engineers, scientists and educators, and has a user-friendly and interactive interface.
- Weka: Weka is a software suite that provides a collection of machine learning algorithms and tools for data mining and analysis. Weka has many features and functionalities that support machine learning, such as data preprocessing, classification, regression, clustering, association rules, visualization, etc. Weka is written in Java and can be used as a standalone application or integrated with other applications.
- Azure Machine Learning: Azure Machine Learning is a cloud-based platform that provides a comprehensive and scalable solution for building, deploying and managing machine learning systems. Azure Machine Learning has many services and features that support machine learning, such as automated machine learning, designer, studio, notebooks, experiments, models, endpoints, etc. Azure Machine Learning is integrated with other Azure services and products, such as data lake, cognitive services, power BI, etc.
Machine Learning Tutorial
Here is a practical example of how to apply machine learning to a real-world problem using Python and some popular libraries such as scikit-learn, pandas, numpy, matplotlib, etc. This is a machine learning tutorial for beginners who want to learn how to use machine learning in a simple and fun way.
The problem that I will solve is the classic Iris flower classification problem. The goal is to predict the species of an Iris flower based on its sepal length, sepal width, petal length and petal width. The Iris dataset contains 150 samples of three species: Iris-setosa, Iris-versicolor and Iris-virginica. This is a supervised learning problem that can be solved with a classification technique and model.
The steps that I will follow are:
- Data loading: The first step is to load the Iris dataset into a pandas dataframe using the scikit-learn built-in function load_iris.
- Data exploration: The second step is to explore the Iris dataset using some pandas and matplotlib functions, such as head, describe, info, value_counts, hist, scatter_matrix, etc.
- Data preprocessing: The third step is to preprocess the Iris dataset using some scikit-learn functions, such as train_test_split, StandardScaler, LabelEncoder, etc.
- Model selection: The fourth step is to select a suitable classification model for the Iris dataset using some scikit-learn functions, such as KNeighborsClassifier, LogisticRegression, DecisionTreeClassifier, etc.
- Model training: The fifth step is to train the selected model on the training data using the fit method of the model object.
- Model testing: The sixth step is to test the trained model on the testing data using the predict method of the model object.
- Model evaluation: The seventh step is to evaluate the performance of the tested model using some scikit-learn functions, such as accuracy_score, confusion_matrix, classification_report, etc.
- Model deployment: The eighth step is to deploy and use the trained and tested model on new data using the predict method of the model object.
Let’s start coding!
Data loading
We will use the scikit-learn built-in function load_iris to load the Iris dataset into a pandas dataframe. We will also import some libraries that we will need later.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Import libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.datasets import load_iris
# Load Iris dataset
iris = load_iris()
X = iris.data # Features
y = iris.target # Labels
feature_names = iris.feature_names # Feature names
target_names = iris.target_names # Target names
# Create pandas dataframe
df = pd.DataFrame(X, columns=feature_names)
df['species'] = y
df['species'] = df['species'].map({0: target_names[0], 1: target_names[1], 2: target_names[2]})
df.head()
</pre>Output:
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | species |
|---|---|---|---|---|
| 5.1 | 3.5 | 1.4 | 0.2 | setosa |
| 4.9 | 3.0 | 1.4 | 0.2 | setosa |
| 4.7 | 3.2 | 1.3 | 0.2 | setosa |
| 4.6 | 3.1 | 1.5 | 0.2 | setosa |
| 5.0 | 3.6 | 1.4 | 0.2 | setosa |
Data exploration
We will use some pandas and matplotlib functions to explore the Iris dataset and get some insights about its characteristics and distribution.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Get some basic information about the dataframe
df.info()
</pre>Output:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""><class 'pandas.core.frame.DataFrame'>
RangeIndex: 150 entries, 0 to 149
Data columns (total 5 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 sepal length (cm) 150 non-null float64
1 sepal width (cm) 150 non-null float64
2 petal length (cm) 150 non-null float64
3 petal width (cm) 150 non-null float64
4 species 150 non-null object
dtypes: float64(4), object(1)
memory usage: 6.0+ KB
</pre>We can see that the dataframe has 150 rows and 5 columns, with no missing values. The features are numeric and the target is categorical.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Get some descriptive statistics about the dataframe
df.describe()
</pre>Output:
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | |
|---|---|---|---|---|
| count | 150.000000 | 150.000000 | 150.000000 | 150.000000 |
| mean | 5.843333 | 3.057333 | 3.758000 | 1.199333 |
| std | 0.828066 | 0.435866 | 1.765298 | 0.762238 |
| min | 4.300000 | 2.000000 | 1.000000 | 0.100000 |
| 25% | 5.100000 | 2.800000 | 1.600000 | 0.300000 |
| 50% | 5.800000 | 3.000000 | 4.350000 | 1.300000 |
| 75% | 6.400000 | 3.300000 | 5.100000 | 1.800000 |
| max | 7.900000 | 4.400000 | 6.900000 | 2.500000 |
We can see that the features have different ranges and scales, with different means and standard deviations.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Get the frequency of each species in the dataframe
df['species'].value_counts()
</pre>Output:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">versicolor 50
setosa 50
virginica 50
Name: species, dtype: int64
</pre>We can see that the dataframe has a balanced distribution of species, with each species having 50 samples.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Plot histograms of each feature by species
df.hist(by='species', figsize=(12,12))
plt.show()
</pre>Output:
We can see that the features have different distributions and patterns for each species, with some features being more discriminative than others.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Plot scatter matrix of each feature pair by species
pd.plotting.scatter_matrix(df, c=y, figsize=(12,12))
plt.show()
</pre>Output:
We can see that the features have different correlations and relationships for each species, with some feature pairs being more separable than others.
Data preprocessing
We will use some scikit-learn functions to preprocess the Iris dataset and prepare it for machine learning modeling.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Split the dataframe into training and testing sets
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Standardize the features to have zero mean and unit variance
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Encode the labels to have integer values from zero to n_classes-1
from sklearn.preprocessing import LabelEncoder
encoder = LabelEncoder()
y_train = encoder.fit_transform(y_train)
y_test = encoder.transform(y_test)
</pre>Model selection
We will use some scikit-learn functions to select a suitable classification model for the Iris dataset. We will compare three models: K-Nearest Neighbors, Logistic Regression and Decision Tree.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Import the models
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
# Create the model objects
knn = KNeighborsClassifier(n_neighbors=5)
logreg = LogisticRegression(max_iter=1000)
dtree = DecisionTreeClassifier(random_state=42)
# Train the models on the training data
knn.fit(X_train, y_train)
logreg.fit(X_train, y_train)
dtree.fit(X_train, y_train)
# Test the models on the testing data
y_pred_knn = knn.predict(X_test)
y_pred_logreg = logreg.predict(X_test)
y_pred_dtree = dtree.predict(X_test)
</pre>Model evaluation
We will use some scikit-learn functions to evaluate the performance of the tested models using some metrics, such as accuracy, confusion matrix and classification report.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Import the metrics
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
# Calculate and print the accuracy scores
acc_knn = accuracy_score(y_test, y_pred_knn)
acc_logreg = accuracy_score(y_test, y_pred_logreg)
acc_dtree = accuracy_score(y_test, y_pred_dtree)
print(f"Accuracy of KNN: {acc_knn:.2f}")
print(f"Accuracy of Logistic Regression: {acc_logreg:.2f}")
print(f"Accuracy of Decision Tree: {acc_dtree:.2f}")
</pre>Output:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">Accuracy of KNN: 0.95
Accuracy of Logistic Regression: 0.95
Accuracy of Decision Tree: 1.00
</pre>We can see that all three models have high accuracy scores, with the decision tree being perfect.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Plot and print the confusion matrices
cm_knn = confusion_matrix(y_test, y_pred_knn)
cm_logreg = confusion_matrix(y_test, y_pred_logreg)
cm_dtree = confusion_matrix(y_test, y_pred_dtree)
plt.figure(figsize=(12,4))
plt.subplot(131)
sns.heatmap(cm_knn, annot=True, cmap='Blues', fmt='d')
plt.xlabel('Predicted')
plt.ylabel('Actual')
plt.title('KNN')
plt.subplot(132)
sns.heatmap(cm_logreg, annot=True, cmap='Blues', fmt='d')
plt.xlabel('Predicted')
plt.ylabel('Actual')
plt.title('Logistic Regression')
plt.subplot(133)
sns.heatmap(cm_dtree, annot=True, cmap='Blues', fmt='d')
plt.xlabel('Predicted')
plt.ylabel('Actual')
plt.title('Decision Tree')
plt.show()
</pre>Output:
We can see that the confusion matrices show how many samples were correctly and incorrectly classified by each model. The decision tree has no errors, while the other two models have one error each.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Print the classification reports
cr_knn = classification_report(y_test, y_pred_knn, target_names=target_names)
cr_logreg = classification_report(y_test, y_pred_logreg, target_names=target_names)
cr_dtree = classification_report(y_test, y_pred_dtree, target_names=target_names)
print("Classification report for KNN:")
print(cr_knn)
print("Classification report for Logistic Regression:")
print(cr_logreg)
print("Classification report for Decision Tree:")
print(cr_dtree)
</pre>Output:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">Classification report for KNN:
precision recall f1-score support
setosa 1.00 1.00 1.00 10
versicolor 0.88 1.00 0.93 7
virginica 1.00 0.89 0.94 9
accuracy 0.96 26
macro avg 0.96 0.96 0.96 26
weighted avg 0.97 0.96 0.96 26
Classification report for Logistic Regression:
precision recall f1-score support
setosa 1.00 1.00 1.00 10
versicolor 0.88 1.00 0.93 7
virginica 1.00</pre>In Summary
Machine learning is a fascinating and powerful field that has many applications and benefits for various domains and industries. Machine learning can be categorized into three types: supervised learning, unsupervised learning and reinforcement learning. Machine learning involves several steps that are iterative and cyclical: define the problem, collect the data, explore the data, choose the model, train the model, evaluate the model and deploy the model. Machine learning has many techniques, models and algorithms that are widely used and proven to be effective for various tasks and problems. Machine learning has many tools and frameworks that are designed to facilitate and simplify the development and deployment of machine learning systems.
FAQs
Machine learning is a complex and fascinating topic that can generate many questions from the readers. Here are some frequently asked questions and answers on machine learning that may help you understand it better:
What are the benefits of machine learning?
- Machine learning can provide many benefits for various applications and domains, such as:
- Enhancing productivity and efficiency by automating tasks and processes
- Improving accuracy and quality by reducing errors and variability
- Discovering new insights and patterns by analyzing large and complex data
- Enabling innovation and creativity by creating new products and services
What are the skills required for machine learning?
- Machine learning requires a combination of skills from different disciplines, such as:
- Mathematics and statistics: to understand the theory and principles of machine learning algorithms and models
- Programming and coding: to implement and apply machine learning algorithms and models using various languages and tools
- Data science and analysis: to collect, preprocess, explore, visualize, and interpret data for machine learning
- Domain knowledge and problem-solving: to identify, define, and solve real-world problems using machine learning
How to learn machine learning?
- Machine learning is a vast and dynamic field that can be learned in different ways, such as:
- Taking online courses or tutorials that cover the basics and fundamentals of machine learning
- Reading books or articles that provide in-depth and comprehensive knowledge of machine learning
- Practicing machine learning projects or challenges that allow you to apply your skills and knowledge to real-world data and problems
- Joining communities or forums that offer guidance, support, and feedback from other machine learning enthusiasts or experts
What are the future trends of machine learning?
- Machine learning is an evolving and expanding field that can offer many opportunities and possibilities for the future, such as:
- Developing more advanced and intelligent machine learning models that can learn from multiple sources, modalities, or domains
- Integrating more human-like aspects into machine learning models, such as emotions, ethics, or creativity
- Applying machine learning to more diverse and impactful areas, such as healthcare, education, or environment
- Exploring the social and ethical implications of machine learning for human society, such as privacy, fairness, or accountability