Introduction to Soft Computing - Presentation

Introduction to Soft Computing - Presentation

• 1.
  • The transformativefield that bridges the gap between traditional computing and human-like intelligence—Soft Computing. • Soft computing is not just a technology; it's a paradigm shift in how we approach problem-solving in a world that is often uncertain, imprecise, and ambiguous. • Unlike traditional, or "hard computing," which relies on precise algorithms and binary logic, soft computing mimics the remarkable flexibility and adaptability of the human brain. • It thrives on approximations, probabilities, and the ability to learn and adapt to changing environments.
• 2.
  Soft Computing HardComputing Deals with approximate models Deals with exact models Tolerant to uncertainty and imprecision Requires precise inputs and outputs Flexible and adaptive Rigid and deterministic Example: Neural Networks, Fuzzy Logic Example: Binary Logic, Arithmetic Tolerance for imprecision and uncertainty Flexibility and adaptability are critical strengths of soft computing. These characteristics allow systems to adjust to new, dynamic, or unforeseen conditions and continue functioning effectively. In contrast to hard computing, which relies on rigid, rule-based systems, soft computing methods can evolve, learn, and respond to changes in their environment.
• 3.
  Soft Computing • SoftComputing is a consortium of methodologies that work synergistically to solve real-world problems that are hard to model mathematically • "Soft computing is an approach to computing which parallels the remarkable ability of the human mind to reason and learn in an environment of uncertainty and imprecision.“ - By Lotfi A. Zadeh (Father of Fuzzy Logic) • Soft computing is an umbrella term for solving problems using approximate models and solutions, as opposed to precise and exact methods used in hard computing • Soft computing is a consortium of computing techniques aimed at building intelligent, adaptive, and robust systems for solving complex real-life problems where traditional algorithms struggle.
• 4.
  • soft computingbrings together a powerful set of methodologies, including: • Fuzzy Logic, which allows machines to think in shades of gray, rather than black and white. • Neural Networks, inspired by the human brain, which learn patterns and make predictions based on experience. • Genetic Algorithms, which mimic the principles of evolution to find optimal solutions. • Probabilistic Reasoning, which tackles uncertainty with statistical tools. Fuzzy Logic: Deals with uncertainty and imprecision.Neural Networks: Learns from data and recognizes patterns.Genetic Algorithms: Evolves solutions to optimization problems.Use Relatable Examples:Fuzzy Logic: Controlling AC temperature based on "warm," "hot," or "cold."Neural Networks: Recognizing a friend’s face, even with new glasses.Genetic Algorithms: Finding the fastest route on Google Maps by evolving routes.
• 5.
  Neural Networks • Definition: Neuralnetworks are computational systems inspired by the human brain. They consist of interconnected nodes (neurons) that process data and learn patterns. • A neural network is a processing device , either an algorithm or an actual h/w, whose design was inspired by the design and functioning of animal brains and components. • Neural networks have the ability to learn by example, makes them very flexible and powerful.
• 6.
  Artificial Neural Networks •Definition: Artificial Neural Network may be defined as an information –processing model that inspired by the way biological nervous systems, such as the brain, process information. • Model replicate basic functions of brain. Key element is the novel structure of information processing system. • Advantages of Neural Networks • Adaptive Learning: An ANN is endowed with the ability to learn how to do tasks based on the data given for training or initial experience • Self organization: An ANN can create its own organization or representation of the information it receives during learning time. • Real time operation: ANN computations carried in parallel. Special h/w devices are being designed and manufactured to take advantage of this capability of ANN. • Fault tolerance via redundant information coding: Partial destruction of a neural network leads to the corresponding degradation performance. How ever, some network capabilities may be retained even after major network damage.
• 7.
  • Applications: •Medical Diagnosis:Predicting diseases based on symptoms. •Finance: Stock market prediction, risk assessment. •Manufacturing: Quality control, production optimization. •Robotics: Adaptive motion planning. •Transportation: Route optimization, self-driving cars.
• 8.
  • Fuzzy Logic: Fuzzy Logic is a computational approach based on the principles of reasoning with imprecision and vagueness. Unlike traditional binary logic, which deals strictly with true (1) or false (0), fuzzy logic allows for degrees of truth, representing real-world situations more effectively. • Fuzziness: Deals with uncertainty and imprecision. • For example, terms like "hot" or "cold" are subjective and vary in degree. • Membership Functions: Represent how much a value belongs to a fuzzy set. • For instance, "temperature = 30°C" might belong 70% to the "hot" set and 30% to the "warm" set. • Fuzzy Sets: Unlike crisp sets, where elements either belong or do not belong, fuzzy sets allow partial membership.
• 9.
  Genetic Algorithm (GA) •Definition: A Genetic Algorithm (GA) is a search and optimization technique inspired by the process of natural selection and genetics. It is part of evolutionary computing and works by mimicking biological evolution to solve complex problems. • Natural Selection: The fittest individuals are more likely to survive and reproduce. • Crossover: Combines parts of two parent solutions to create offspring. • Mutation: Introduces random changes to solutions for diversity. • Fitness Function: Measures how good a solution is for the given problem. • Optimization: Solve complex problems where traditional methods fail. • Adaptability: Explore large solution spaces effectively. • Robustness: Handle noisy, incomplete, or non-linear data.
• 10.
  Hybrid Systems Hybrid systemsin soft computing combine two or more computational techniques (such as Fuzzy Logic, Neural Networks, and Genetic Algorithms) to leverage the strengths of each method while mitigating their individual weaknesses. These systems are designed to solve complex, real-world problems where a single technique may not be sufficient. • Neuro-Fuzzy Systems: Combines Neural Networks (learning ability) with Fuzzy Logic (handling imprecision). Determine parameters by processing data samples • Example: Adaptive Neuro-Fuzzy Inference System (ANFIS) for control systems like automatic braking in vehicles. • Fuzzy-Genetic Systems: Uses Genetic Algorithms for optimizing fuzzy system parameters. • Example: Tuning the membership functions of a fuzzy controller for robotics. • Neuro-Genetic Systems: Employs Genetic Algorithms to optimize Neural Network architectures or weights. (learning rate, momentum rate, tolerance level, no. of hidden layers etc) • Example: Selecting the best neural network structure for image recognition tasks.
• 11.
  HARD COMPUTING SOFTCOMPUTING
• 12.
  Artificial Neural Network •Artificial Neural Networks are mathematical models designed to simulate the behavior of biological neural networks. They consist of layers of interconnected artificial neurons. • Artificial Neural Network may be defined as an information –processing model that inspired by the way biological nervous systems, such as the brain, process information. • Model replicate basic functions of brain. Key element is the novel structure of information processing system • ANNs process large no. of highly interconnected large processing elements called nodes or units or neurons
• 13.
  Biological Neural Network •Biologicalnervous system is the most important part of many living things, in particular, human beings. •There is a part called brain at the center of human nervous system. •In fact, any biological nervous system consists of a large number of interconnected processing units called neurons. •Each neuron is approximately 10µm long and they can operate in parallel. •Typically, a human brain consists of approximately 1011 neurons communicating with each other with the help of electrical impulses.
• 14.
  • Soma orcell body-where cell nucleus is located • Dendrites- A bush of very thin fibre where the nerve is connected to the cell body • Axon- A long cylindrical fibre which carries the impulses of the neuron. • Synapse : It is a junction where axon makes contact with the dendrites of neighboring dendrites. Schematic diagram of a biological neuron
• 15.
  • Dendrites aretree like networks made of nerve fiber connected to the cell body. • An Axon is a single, long connection extending from the cell body and carrying signals from the neuron. • The end of axon splits into fine strands. • It is found that each strand terminated into small bulb like organs called as synapse. • It is through synapse that the neuron introduces its signals to other nearby neurons. • The receiving ends of these synapses on the nearby neurons can be found both on the dendrites and on the cell body. • There are approximately 104 synapses per neuron in the human body. • Electric impulse is passed between synapse and dendrites. It is a chemical process which results in increase/decrease in the electric potential inside the body of the receiving cell. If the electric potential reaches a threshold value, receiving cell fires & pulse / action potential of fixed strength and duration is send through the axon to synaptic junction of the cell. After fire, cell has to wait for a period called refractory period.
• 16.
  •There is achemical in each neuron called neurotransmitter. • Neurotransmitters are chemical messengers in the brain. They are released by neurons to transmit signals to other neurons, muscles, or glands. Examples include dopamine, serotonin, and acetylcholine. •A signal (also called sense) is transmitted across neurons by this chemical. • Neurons communicate via synaptic transmission, where an electrical signal (action potential) triggers the release of neurotransmitters. These neurotransmitters cross the synaptic gap between neurons, allowing the signal to propagate •That is, all inputs from other neuron arrive to a neurons through dendrites. • Dendrites are branch-like structures that extend from the neuron’s cell body (soma). They receive input signals (via neurotransmitters) from other neurons' axons. This is the starting point of signal processing in the neuron •These signals are accumulated at the synapse of the neuron and then serve as the output to be transmitted through the neuron. • At the synapse, neurotransmitters released by the presynaptic neuron bind to receptors on the postsynaptic neuron. This triggers changes in the postsynaptic neuron's membrane potential, integrating all incoming signals.
• 17.
  •An action mayproduce an electrical impulse, which usually lasts for about a millisecond. • The electrical impulse referred to here is the action potential. When the sum of all incoming signals (excitatory and inhibitory) exceeds a specific threshold at the soma, the neuron generates an action potential, lasting ~1 millisecond. This is the electrical signal transmitted along the axon. •Note that this pulse generated due to an incoming signal and all signal may not produce pulses in axon unless it crosses a threshold value. • Neurons follow the all-or-nothing principle. Not all incoming signals are strong enough to generate an action potential. Only when the summed signals at the soma exceed a threshold value does the neuron "fire" and transmit a signal down its axon. •Also, note that an action signal in axon of a neuron is commutative signals arrive at dendrites which summed up at soma. • Commutative nature: This means that the order of signals arriving at the dendrites does not matter; their total effect is considered at the soma. Summing up at the soma: The neuron processes incoming signals by summing up excitatory (positive) and inhibitory (negative) inputs. If the net signal surpasses the threshold, the neuron generates an action potential.
• 18.
  The biological basisof signal transmission in neurons, highlighting critical concepts like: • Chemical communication via neurotransmitters. • Integration of signals at the soma. • The generation of action potentials based on a threshold mechanism. This process is often the inspiration for Artificial Neural Networks (ANNs) in machine learning, where: • Inputs to a neuron represent dendrites. • A summation function in ANNs mimics the integration of signals. • A threshold activation function in ANNs is inspired by the biological firing threshold.
• 19.
  •In fact, thehuman brain is a highly complex structure viewed as a massive, highly interconnected network of simple processing elements called neurons. •Artificial neural networks (ANNs) or simply we refer it as neural network (NNs), which are simplified models (i.e. imitations) of the biological nervous system, and obviously, therefore, have been motivated by the kind of computing performed by the human brain. •The behavior of a biolgical neural network can be captured by a simple model called artificial neural network.
• 20.
  Biological neuron Artificial neuron Cell Neuron DendritesWeights or interconnections Soma Net input Axon Output Mathematical model of artificial neuron In this model net input is calculated as 𝑦𝑖𝑛 = 𝑥1𝑤1 + 𝑥2𝑤2 + + ⋯ 𝑥𝑛𝑤𝑛 = n ∑ 𝑥𝑖𝑤𝑖 𝑖=1 Where, i represents ith processing element. The activation function applied over it to calculate the output. The weight represents the strength of synapses connecting the input and output.
• 21.
  x1 w1 x2 w2 w3 x3 xn wn
• 22.
  •a neuron isa part of an interconnected network of nervous system and serves the following. • Compute input signals • Transportation of signals (at a very high speed) • Storage of information • Perception, automatic training and learning •Truly, every component of the model (i.e. artificial neuron) bears a direct analogy to that of a biological neuron. It is this model which forms the basis of neural network (i.e. artificial neural network).
• 23.
  ….. x1 w1 x2 w2 w3 x3 Xn wn input weight Summation unit Thresholdunit output Here, x1, x2, · · · , xn are the n inputs to the artificial neuron. w1, w2, · · · , wn are weights attached to the input links.
• 24.
  Note that, abiological neuron receives all inputs through the dendrites, sums them and produces an output if the sum is greater than a threshold value. The input signals are passed on to the cell body through the synapse, which may accelerate or retard an arriving signal. It is this acceleration or retardation of the input signals that is modeled by the weights. An effective synapse, which transmits a stronger signal will have a correspondingly larger weights while a weak synapse will have smaller weights. Thus, weights here are multiplicative factors of the inputs to account for the strength of the synapse.
• 25.
  • An artificialneural network (ANN) is an efficient information processing system which resembles the characteristics of biological neural network. • ANNs contain large number of highly interconnected processing elements called nodes or neurons or units. • Each neuron is connected with other by connection link and each connection link is associated with weights which contain information about the input signal. • This information is used by neuron net to solve a particular problem. ANNs have ability to learn, recall and generalize training pattern or data similar to that of human brain. The ANN processing elements called neurons or artificial neurons
• 26.
  • Each neuronhas an internal state of its own, called activation or activity level of neuron which is the function of the inputs the neuron receives. • The activation signal of a neuron is transmitted to other neurons. A neuron can send only one signal at a time which can be transmitted to several neurons. • Consider the figure, here X1 and X2 are input neurons, Y is the output neuron W1 and W2 are the weights net input is calculated as • 𝑦𝑖𝑛 = 𝑥1𝑤1 + 𝑥2𝑤2 • where x1 and x2 are the activation of the input neurons X1 and X2, i.e., is the output of the input signals. The output y of the output neuron Y can be obtained by applying activations over the net input. Architecture of a simple artificial neuron net
• 27.
  • 𝑦 =( 𝑓 𝑦𝑖𝑛) • Output = Function (net input calculated) • The function to be applied over the net input is called activation function. The net input calculation is similar to the calculation of output of a pure linear straight line equation y=mx Neural net of pure linear equation
• 28.
  The weight involvein the ANN is equivalent to the slope of the straight line.
• 29.
  Hence, the totalinput say I received by the soma of the artificial neuron is I = w1x1 + w2x2 + · · · + wnxn = Σ n i=1 wi xi To generate the final output y , the sum is passed to a filter φ called transfer function, which releases the output. That is, y = φ(I) ….. x1 x2 xn w1 w2 w3 x3 wn input weight Summation unit Threshold unit output I Ø(I) y
• 30.
  •A very commonlyknown transfer function is the thresholding function. •In this thresholding function, sum (i.e. I) is compared with a threshold value θ. •If the value of I is greater than θ, then the output is 1 else it is 0 (this is just like a simple linear filter). •In other words, y = φ( Σ n i=1 wi xi − θ) where φ(I) = 1 , if I > θ 0 , if I ≤ θ Such a Φ is called step function (also known as Heaviside function).
• 31.
  • For thenetwork shown in the figure , calculate the net input to the output neuron where [𝑥1 , 𝑥2 , 𝑥3 ]= [ 0.3, 0.5, 0.6] [𝑤1 , 𝑤2 , w3 ]=[ 0.2, 0.1, -0.3] 𝑦𝑖𝑛 = -0.07 In this model net input is calculated as 𝑦𝑖𝑛 = 𝑥1𝑤1 + 𝑥2𝑤2 + + ⋯ 𝑥𝑛𝑤𝑛 = n ∑ 𝑥𝑖𝑤𝑖 𝑖=1
• 32.
  Term Brain Computer SpeedExecution time is few milliseconds Execution time is few nano seconds Processing Perform massive parallel operations simultaneously Perform several parallel operations simultaneously. It is faster the biological neuron Size and complexity Number of Neuron is 1011 and number of interconnections is 1015 . So complexity of brain is higher than computer It depends on the chosen application and network designer. Storage capacity  Information is stored in interconnections or in synapse strength.  New information is stored without destroying old one.  Sometimes fails to recollect information  Stored in continuous memory location.  Overloading may destroy older locations.  Can be easily retrieved Comparison between Biological neuron and Artificial neuron
• 33.
  Term Brain Computer Tolerance Fault tolerant  Store and retrieve information even interconnections fails  Accept redundancies  No fault tolerance  Information corrupted if the network connections disconnected.  No redundancies Control mechanism Depends on active chemicals and neuron connections are strong or weak CPU Control mechanism is very simple
• 34.
  • Characteristics ofANN: • It is a neurally implemented mathematical model • Large number of processing elements called neurons exists here. • Interconnections with weighted linkage hold informative knowledge. • Input signals arrive at processing elements through connections and connecting weights. • Processing elements can learn, recall and generalize from the given data. • Computational power is determined by the collective behavior of neurons. ANN is a connection models, parallel distributed processing models, self- organizing systems, neuro-computing systems and neuro - morphic system
• 35.
  Evolution of neuralnetworks Year Neural network Designer Description 1943 McCulloch and Pitts neuron McCulloch and Pitts Arrangement of neurons is combination of logic gate. Unique feature is thresh hold 1949 Hebb network Hebb If two neurons are active, then their connection strengths should be increased. 1958,1959,1962, 1988, Perceptron Frank Rosenblatt, Block, Minsky and Papert Here the weights on the connection path can be adjusted. 1960 Adaline Widrow and Hoff Here the weights are adjusted to reduce the difference between the net input to the output unit and the desired output. 1972 Kohonen self- organizing feature map Kohonen Inputs are clustered to obtain a fired output neuron. 1982, 1984, 1985, 1986, 1987 Hopfield network John Hopfield and Tank Based on fixed weights. Can act as associative memory nets
• 36.
  Year Neural networkDesigner Description 1986 Back propagation network Rumelhart,Hinton and Williams  Multilayered  Error propagated backward from output to the hidden units 1988 Counter propagation network Grossberg Similar to kohonen network. 1987-1990 Adaptive resonance Theory(ART) Carpenter and Grossberg Designed for binary and analog inputs. 1988 Radial basis function network Broomhead and Lowe Resemble back propagation network, but activation function used is Gaussian function. 1988 Neo cognitron Fukushima For character recognition.
• 37.
  Basic models ofartificial neural networks • Models are based on three entities • The model’s synaptic interconnections. • The training or learning rules adopted for updating and adjusting the connection weights. • Their activation functions Connections • The arrangement of neurons to form layers and the connection pattern formed within and between layers is called the network architecture. There exist five basic types of connection architecture. • They are: • Single layer feed forward network • Multilayer feed-forward network • Single node with its own feedback • Single-layer recurrent network • Multilayer recurrent network
• 38.
  • Feed forwardnetwork: If no neuron in the output layer is an input to a node in the same layer / proceeding layer. • Feedback network: If outputs are directed back as input to the processing elements in the same layer/proceeding layer. • Lateral feedback: If the output is directed back to the input of the same layer. • Recurrent networks: Are networks with feedback networks with closed loop.
• 39.
  Single layer feedforward network Layer is formed by taking processing elements and combining it with other processing elements. Input and output are linked with each other Inputs are connected to the processing nodes with various weights, resulting in series of outputs one per node. When a layer of processing nodes is formed, the inputs can be connected to these nodes with various weights, resulting in a series of outputs, one per node. This is called single layer feed forward network.
• 40.
  Multilayer feed-forward network Thisnetwork is formed by the interconnection of several layers. Input layer receives input and buffers input signal. Output layer generated output. Layer between input and output is called hidden layer. Hidden layer is internal to the network. There are Zero to several hidden layers in a network. More the hidden layer more is the complexity of network, but efficient output is produced.
• 41.
  Single node withits own feedback It is a simple recurrent neural network having a single neuron with feedback to itself
• 42.
  Single layer recurrentnetwork • A single layer network with feedback from output can be directed to processing element itself or to other processing element/both.
• 43.
  Multilayer recurrent network Processingelement output can be directed back to the nodes in the preceding layer, forming a multilayer recurrent network.
• 44.
  Maxnet –competitive interconnectionshaving fixed weights.
• 45.
  On-center-off-surround/lateral inhibition structure– each processing neuron receives two different classes of inputs- “excitatory” input from nearby processing elements & “inhibitory” elements from more distantly located processing elements. This type of interconnection is shown below
• 46.
  Learning • Learning orTraining is the process by means of which a neural network adapts itself to a stimulus by making proper parameter adjustments, resulting in the production of desired response. • Two broad kinds of learning in ANNs is: • Parameter learning – updates connecting weights in a neural net. • Structure learning – focus on change in the network. • Apart from these, learning in ANN is classified into three categories as • Supervised learning • Unsupervised learning • Reinforcement learning
• 47.
  Supervised learning: wherea model learns to map inputs to outputs based on labeled training data. It operates under the guidance of a "supervisor," which is the labeled dataset, containing input-output pairs. In ANN, each input vector requires a corresponding target vector, which represents the desired output. The input vector along with target vector is called training pair. Input vector results in output vector. The actual output vector is compared with desired output vector. If there is a difference means an error signal is generated by the network. It is used for adjustment of weights until actual output matches desired output.
• 48.
  Unsupervised learning where themodel learns patterns and structures from data without labeled outputs. Unlike supervised learning, it deals with datasets that contain only inputs and no corresponding target labels. The network itself discover patterns, regularities, features/ categories from the input data and relations for the input data over the output. Exact clusters are formed by discovering similarities & dissimilarities so called as self – organizing.
• 49.
  Reinforcement Learning (RL)is a type of learning where an agent learns to make decisions by interacting with an environment to maximize a reward signal. It is inspired by the way humans and animals learn through trial and error. It is similar to supervised learning. Learning based on critic information is called reinforcement learning & the feedback sent is called reinforcement signal. The network receives some feedback from the environment. Feedback is only evaluative.
• 50.
  Activation Functions Activation functionis applied over the net input to calculate the output of an ANN. Information processing of processing element has two major parts: input and output. An integration function (f) is associated with input of processing element.
• 51.
  Identity function: Itis a linear function which is defined as 𝑓( ) = 𝑥 𝑥 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑥 - The output is same as the input. Binary step function: This function can be defined as Where, θ represents threshold value. It is used in single layer nets to convert the net input to an output that is binary (0 or 1). Bipolar step function: This function can be defined as Where, θ represents threshold value. It is used in single layer nets to convert the net input to an output that is bipolar (+1 or -1).
• 52.
  Sigmoid function: Itis used in Back propagation nets. Two types: a) Binary sigmoid function: It is also termed as logistic sigmoid function or unipolar sigmoid function. It is defined as • where, λ represents steepness parameter. The derivative of this function is 𝑓′( ) = ( )[1 − ( )] 𝑥 𝜆𝑓 𝑥 𝑓 𝑥 The range of sigmoid function is 0 to 1.
• 53.
  b) Bipolar sigmoidfunction: It is defined as -1 = where, λ represents steepness parameter and the sigmoid range is between -1 and +1. The derivative of this function is 𝑓′( ) = /2[1+ ( )][1 − ( )] 𝑥 𝜆 𝑓 𝑥 𝑓 𝑥 It is closely related to hyperbolic tangent function, which is written as h(x)= h(x)= • The derivative of the hyperbolic tangent function is ℎ′ ( ) = [1 + ( )][1 − 𝑥 ℎ 𝑥 ( )] ℎ 𝑥
• 55.
  Ramp function: Theramp function is defined as Important Terminologies of ANNs Weights: Weight matrix or connection matrix Bias: A constant value added to the weighted sum of inputs before applying the activation function. • Equation: y=f(∑wixi+b) • Positive bias and negative bias
• 56.
  Threshold :A valuethat determines whether a neuron activates. • Example:If the output is greater than the threshold, the neuron fires. • Relation to Activation Functions: Threshold is often integrated within activation functions like step or sigmoid functions. • Learning Rate: denoted by α. It is used to control the amount of weight adjustment at each step of training. The learning rate, ranging from 0 to 1, determines the rate of learning at each time step. • Momentum Factor: Convergence is made faster if a momentum factor is added to the weight updation process. This is generally done in the back propagation network. If momentum has to be used, the weights from one or more previous training patterns must be saved. Momentum helps the net in reasonably large weight adjustments until the corrections are in the same general direction for several patterns.
• 57.
  Vigilance parameter: The vigilanceparameter is denoted by “ρ”. It is generally used in adaptive resonance theory (ART) network. The vigilance parameter is used to control the degree of similarity required for patterns to be assigned to the same cluster unit. The choice of vigilance parameter ranges approximately from 0.7 to 1 to perform useful work in controlling the number of clusters.
• 58.
  McCulloch and PittsNeuron • It is discovered in 1943 and usually called as M-P neuron. • M-P neurons are connected by directed weighted paths. • Activation of M-P neurons is binary (i.e) at any time step the neuron may fire or may not fire. • Weights associated with communication links may be excitatory (wgts are positive)/inhibitory (wgts are negative). • Threshold plays major role here. There is a fixed threshold for each neuron and if the net input to the neuron is greater than the threshold then the neuron fires. • They are widely used in logic functions. A simple M-P neuron is shown in the figure. It is excitatory with weight w (w>0) / inhibitory with weight –p (p<0). • The M-P neuron has no particular training algorithm. An analysis is performed to determine the weights and the threshold. It is used as a building block where any function or phenomenon is modeled based on a logic function.
• 59.
  In the figure, inputs from x1 to xn possess excitatory weighted connection and Xn+1 to Xn+m has inhibitory weighted interconnections 𝑓 (𝑥)= {1𝑖𝑓 𝑦 𝑖𝑛≥𝜃 0𝑖𝑓 𝑦 𝑖𝑛< 𝜃 The firing of neuron is based on threshold, activation function is defined as Output will fire if it receives “k” or more excitatory inputs but no inhibitory inputs where kw ≥ θ>(k-1) w For inhibition to be absolute, the threshold with the activation function should satisfy the following condition: θ > nw –p
• 60.
  Hebb network • DonaldHebb stated in 1949 that “In brain, the learning is performed by the change in the synaptic gap”. • When an axon of cell A is near enough to excite cell B, and repeatedly or permanently takes place in firing it, some growth process or metabolic change takes place in one or both the cells such that A’s efficiency, as one of the cells firing B, is increased. • According to Hebb rule, the weight vector is found to increase proportionately to the product of the input and the learning signal. • In Hebb learning, two interconnected neurons are ‘on’ simultaneously. The weight update in Hebb rule is given by wi(new) = wi (old)+ xi y • Hebbs network is suited more for bipolar data. If binary data is used, the weight updation formula cannot distinguish two conditions namely: • A training pair in which an input unit is “on” and the target value is “off”. • A training pair in which both the input unit and the target value is “off”.