Adaptive resonance networks are a form of artificial neural networks based on the theory of adaptive resonance by Stephen Grossberg and Gale Carpenter. Includes training models with and without a teacher , which are used in solving problems of pattern recognition and prediction.
General Description
The basic idea is that pattern recognition is the result of downward expectations and upward sensory information. Moreover, downward expectations take the form of remembered prototypes or samples, which are then compared with the actually observed properties of the object. This comparison underlies the measure of categorization. When the difference between the expectation and the observed does not exceed a certain threshold (“vigilance”), the observed object is considered to belong to a certain category. Thus, the system offers a solution to the problem of plasticity / stability, that is, the problem of acquiring new knowledge without violating the existing one.
Training
The standard AR system is a model of learning without a teacher. As a rule, it consists of comparison fields and recognition fields composed of neurons, as well as a vigilance parameter and a reset module. A vector of numbers is fed to the input of the comparison field, for which the corresponding neuron in the recognition field is determined, that is, one whose weights are most similar to the input vector. Each neuron of the recognition field inhibits other neurons from this field (the strength of the action is proportional to the degree of correspondence).
After classifying the input, the reset module compares the degree of matching during recognition with the alert parameter. If the threshold is overcome, training takes place: the weights of the victorious neuron are adjusted to the values of the input vector. If the threshold has not been overcome, then the winning neuron is suppressed and the search procedure is started. During this procedure, the recognition neurons are switched off one by one using the reset function until the alert threshold is reached. On each search cycle, the most active recognizing neuron is selected, and is turned off if activation does not reach the alert threshold. The vigilance parameter has a significant impact on the system: high values create highly detailed memory (many small categories), while small values create more general images (fewer larger categories).
Varieties
ART1 [1] [2] is the simplest variety that accepts only binary values as input.
ART2 [3] - provides support for continuous values.
ART2-A
ART3 [4] is a physiologically more realistic version of ART2. Models the mediator regulation of synaptic activity .
Fuzzy ART [5] - a variant of the model using the principles of fuzzy logic .
ARTMAP (Predictive ART)
Fuzzy ARTMAP
Distributed ARTMAP
ARTMAP-IC
Default ARTMAP
Criticism
It was noticed that in ART1 and Fuzzy ART the results are highly dependent on the order of presentation of the training sample. The effect can be reduced by lowering the coefficient of learning speed, however, it manifests itself regardless of the size of the training sample. Therefore, the results of ART1 and Fuzzy ART models are not consistent estimates in terms of mathematical statistics.
Notes
- ↑ Carpenter, GA & Grossberg, S. (2003), Adaptive Resonance Theory Archived May 19, 2006 by Wayback Machine , In Michael A. Arbib (Ed.), The Handbook of Brain Theory and Neural Networks, Second Edition (pp. 87-90). Cambridge, MA: MIT Press
- ↑ Grossberg, S. (1987), Competitive learning: From interactive activation to adaptive resonance Archived September 7, 2006 at Wayback Machine , Cognitive Science (Publication) , 11, 23-63
- ↑ Carpenter, GA & Grossberg, S. (1987), ART 2: Self-organization of stable category recognition codes for analog input patterns Archived September 4, 2006 at Wayback Machine , Applied Optics , 26 (23), 4919-4930
- ↑ Carpenter, GA & Grossberg, S. (1990), ART 3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures Archived September 6, 2006 at Wayback Machine , Neural Networks (Publication) , 3, 129–152
- ↑ Carpenter, GA, Grossberg, S., & Rosen, DB (1991b), Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system Archived May 19, 2006 on Wayback Machine , Neural Networks (Publication) , 4, 759—771
Links
- Stephen Grossberg 's website .
- ART's implementation for unsupervised learning (ART 1, ART 2A, ART 2A-C and ART distance) can be found at https://web.archive.org/web/20120109162743/http://users.visualserver.org/xhudik/ art
- [1] Summary of the ART algorithm