Artificial Neural Networks/Self-Organizing Maps

Self-Organizing Maps

Self-organizing maps (SOM), sometimes called Kohonen SOM after their creator, are used with unsupervised learning. SOM are modeled on biological neural networks, where groups of neurons appear to self organize into specific regions with common functionality.

Neuron Regions

Different regions of the SOM network are trained to be detectors for distinct features from the input set. Initial network weights are either set randomly, or are based off the eigenvectors of the input space. The Euclidean distance from each input sample to the weight vector of each neuron is computed, and the neuron whose weight vector is most similar to the input is declared the best match unit (BMU). The update formula is given as:

$w_{j}[n+1]=w_{j}[n]+\Theta [j,n]\alpha [n](x[n]-w_{j}[n])$

Here, w is the weight vector at time n. α is a monotonically decreasing function that ensures the learning rate will decrease over time. x is the input vector, and Θ[j, n] is a measure of the distance between the BMU and neuron j at iteration n. As can be seen from this algorithm, the amount by which the neuron weight vectors change is based on the distance from the BMU, and the amount of time. Decreasing the potential for change over time helps to reduce volatility during training, and helps to ensure that the network converges.