You can use facial recognition to unlock your phone. This process involves your device addressing two key challenges: generalisation and discrimination. Your iPhone must differentiate your face from those of others, even though faces share similar features (discrimination). Additionally, it has to recognise your face despite variations in lighting, angle, facial hair, and other factors (generalisation). How do contemporary AI systems tackle these issues in recognising patterns?
To excel in the pattern recognition task, you must construct a network of neurons where an input pattern enters from one side and passes through various layers of neurons, ultimately producing an output on the opposite end of the network. By modifying the connection weights between neurons, you can enable the network to execute a range of operations on its input. If you adjust the weights correctly, you can train an algorithm to accurately recognise an input pattern by the time it reaches the end of the network. Changing the weights in one manner may allow it to identify faces, while adjusting them differently could enable it to detect smells. The challenging aspect lies in teaching the network how to learn the appropriate weights, a topic we will explore in future blog posts. Today, let’s consider the earliest attempts to model artificial networks that mimic the mechanical functionalities of biological neurons. As we have already discussed, AI researchers and practitioners have succeeded in imitating these mechanisms but have failed to address the chemistry behind how biological neurons function.
When did scientists begin utilising weight-based functions to develop AIs?
One of the earliest efforts was carried out by neurophysiologist Warren McCulloch and logician Walter Pitts in 1943. In their research, they outlined a mathematical concept of a neuron as a weighted sum of inputs, processed through a transition function to generate an output. They referred to it as a perceptron. To put it simply, a perceptron is a miniature model of a brain cell (neuron) that assists a computer in making basic decisions, such as responding with “yes” or “no.”
How does a perceptron “decide” whether something is a YES or a NO?
A neuron takes in multiple inputs, which could include data from sensors or numerical values representing various characteristics of a problem, such as the colour or weight of a fruit. Each input is assigned a weight that indicates its significance to the neuron. The outcome, known as the weighted sum, is then processed through an activation function. You can think of this function as a guideline that determines: “If the sum reaches a certain threshold, respond with YES. If not, respond with NO.”
In math terms, this is shown as:
output=f(∑wixi+b)
where xi are the inputs, wi are the weights, b is the bias, and f is the activation function.
Picture yourself consulting three friends about whether you should take up piano lessons. You hold one friend’s opinion in higher regard than the others (that’s the weight), and you also factor in your intuition (the bias). You combine all your friends’ advice along with your gut feeling. If the sum exceeds the threshold you’ve set, the answer is a YES; if not, it’s a NO. The perceptron generates an output (0 for NO and 1 for YES) to arrive at a decision. It improves over time by adjusting its weights and bias to enhance its accuracy.
In this blog, we’ve previously covered SNARC (Stochastic Neural Analogue Reinforcement Calculator), which Marvin Minsky created in 1951 – 1952. The SNARC machine functioned as a perceptron.
In biological systems, there are numerous inputs (dendrites) and a single output (axon). A biological neuron transmits electrical signals through the axon based on the signals received through the dendrites. In this context, AI researchers covered the mechanics of inputs leading to an output quite well. If only learning decision-making in such systems were as simple as it appears at first sight!