Supervised Learning is a Machine Learning task where a function is programmed to predict the next value without being explicitly coded for it.

In other words, it involves training a machine on a labeled dataset, where a function maps inputs to outputs based on examples in the training data. It is a task-driven technique that enables the machine to predict the next value.

A supervised learning algorithm analyzes the data and produces an inferred function that fits the desired task. This inferred function may or may not be used for mapping new examples. In supervised learning, we focus on creating a mapping from input to the required output.

If the output we are looking for is categorical (e.g., determining whether someone will buy a computer at an electronics shop or whether a patient has a particular disease), the supervised learning problem is categorized as a classification problem. In cases where the output depends on a single algorithm, it’s essential to evaluate when the algorithm fails to deliver the desired result. By analyzing the true and false outcomes, we can refine the training data with new, categorized, real-world data to achieve better results.

On the other hand, if the task involves predicting continuous outcomes, such as determining whether it will rain tomorrow, this type of problem is referred to as a regression problem.

Steps to Solve a Supervised Learning Problem

To solve a supervised learning problem, the following steps are typically followed:

  1. Determine the Training Examples
    Begin by identifying the training examples. For instance, in the examples of buyers or weather forecast data, decide what kind of data will be used as the training set.
  2. Use a Real-World Training Set
    Select a training set that closely resembles the real-world use of the function. This includes gathering corresponding input and output data.
  3. Determine the Input Feature Representation
    Define how the input features will represent the learned function. The accuracy of the learned function heavily depends on the input objects. Transform the input objects into feature vectors that include multiple descriptive features of the objects.
  4. Select the Structure and Learning Algorithm
    Choose the structure of the model and the learning algorithm. Examples include decision trees, support vector machines, etc.
  • Select the Structure and Learning Algorithm
    Choose the structure of the model and the learning algorithm. Examples include decision trees, support vector machines, etc.
  • Run the Algorithm on the Training Dataset
    Execute the algorithm on the training dataset. Many learning algorithms require tuning control parameters, which can be optimized by evaluating performance on a validation set derived from the training data.
  • Evaluate the Accuracy of the Learned Function
    Finally, assess the accuracy of the function or algorithm using a separate dataset that is distinct from the training set.

    • Algorithm Choice

    Supervised learning offers numerous algorithms, each suited to specific tasks. Each algorithm comes with its own strengths and weaknesses, and there is no universal algorithm that works for all supervised learning problems. The selection of an algorithm depends on factors such as:


      • Bias-Variance Tradeoff

      • Function Complexity and Amount of Training Data

      • Dimensionality of the Input Space

      • Noise in the Output Values

    Some of the most widely used supervised learning algorithms include:

      • Naïve Bayes

      • Neural Networks

      • Similarity Learning

      • Support Vector Machines

      • Linear Regression

      • Logistic Regression

      • Linear Discriminant Analysis

      • Decision Trees

      • K-Nearest Neighbor Algorithm

    Supervised Learning Mathematical Approach

    Given: –


      • A set of input features X1, X2 ………, Xn

      • A target features Y

      • A set of training examples where the values for the input and target features are given for every single example

      • A new example, where only the values for the input features are given

    Predict the values for the target for the new example

      • Classification when Y is discrete

      • Regression when Y is continuous

    Classification


    Example – Credit Scoring
    Differentiating between high and low-risking assets from their income and savings

    Regression

    Example – Price of 2nd hand cars
    X : Car attributes
    Y : Price
    y = g (x, θ)
    g() model,
    θ parameters


    Real-World Example: Classifying Buyers

    In the following graphical representation of buyers, there are two possibilities:

      1. The person will buy the computer.

      1. The person will not buy the computer.

    In this case, we need to train our task algorithm in such a way that it can classify who may or may not buy the computer from the shop. These possibilities depend on various factors such as income, savings, budget, environmental conditions, and even natural causes.


    To solve this problem, we can use multiple classifiers and select the one best suited for the desired situation and real-world possibilities to predict whether a customer will buy a computer or not.

    Example 1: Basic Classifier

    In the first example, a simple linear decision boundary is drawn:

      • Everything to the left of the line is classified as “will not buy a computer.”

        • Everything to the right of the line is classified as “will buy a computer.”
          •  

      Here, the red points to the left of the line represent non-buyers, while the points to the right represent buyers.


      Performance Analysis:

            • Accuracy: The accuracy of this classifier is low because the decision boundary is overly simplistic and doesn’t account for complex patterns in the data.

            • Speed: The classifier is fast and computationally efficient.

      Example 2: Optimized Classifier

      In this example, the decision boundary is tilted or adjusted in such a way that it better separates the buyers and non-buyers.

            • This adjustment ensures that the maximum number of buyers and non-buyers are correctly classified by the algorithm.

      Performance Analysis:

            • Accuracy: The accuracy is significantly higher as the classifier can now better distinguish between the two groups.

            • Speed: Although this classifier may be slightly slower than the first due to the increased complexity, it is still efficient and more reliable in real-world scenarios.

      Conclusion

      By comparing the two classifiers, it is clear that optimizing the decision boundary can significantly improve accuracy and reliability while maintaining practical computational efficiency. Choosing the right classifier depends on the specific requirements of the task and the real-world conditions influencing customer behavior.

      Further Exploration:

      As machine learning continues to advance, exploring various algorithms and techniques for supervised learning becomes crucial. Understanding the limitations and strengths of each method ensures the best choice for a given problem. Supervised learning remains a powerful tool for real-world applications in fields like finance, healthcare, and retail.

      Additional Resources: