are more alike than members of di erent groups The aim of a clustering algorithm is therefore to construct clusters of similar input vectors (patterns), where similarity is usually measured in terms of Euclidean distance LVQ-I performs such clustering The training process of LVQ-I to construct clusters is based on competition Referring to Figure 41, each output unit ok represents a single cluster The competition is among the cluster output units During training, the cluster unit whose weight vector is the closest to the current input pattern is declared as the winner The corresponding weight vector and that of neighboring units are then adjusted to better resemble the input pattern The closeness of an input pattern to a weight vector is usually measured using the Euclidean distance The weight update is given as uki (t) = (t)[zi,p uki (t 1)] if k k,p (t) 0 otherwise (419)

where (t) is a decaying learning rate, and k,p (t) is the set of neighbors of the winning cluster unit ok for pattern p It is, of course, not strictly necessary that LVQ-I makes use of a neighborhood function, thereby updating only the weights of the winning output unit

Figure 42 Learning Vector Quantizer to Illustrate Clustering An illustration of clustering, as done by LVQ-I, is given in Figure 42 The input space, de ned by two input units z1 and z2 , is represented in Figure 42(a), while Figure 42(b) illustrates the LVQ-I network architecture required to form the clusters Note that although only three classes exist, four output units are necessary one for each cluster Less output units will lead to errors since patterns of di erent classes will be grouped in the same cluster, while too many clusters may cause over tting For the problem illustrated in Figure 42(a), an additional cluster unit may cause a separate cluster to learn the single in cluster 4 The Kohonen LVQ-I algorithm is summarized in Algorithm 42 For the LVQ-I, weights are either initialized to random values, sampled from a uniform distribution, or by

44 Learning Vector Quantizer-I Algorithm 42 Learning Vector Quantizer-I Training Algorithm Initialize the network weights, the learning rate, and the neighborhood radius; while stopping condition(s) not true do for each pattern p do Compute the Euclidean distance, dk,p , between input vector zp and each weight vector uk = (uk1 , uk2 , , uKI ) as

dk,p (zp , uk ) =

(zi,p uki )2

(420)

Find the output unit ok for which the distance dk,p is the smallest; Update all the weights for the neighborhood k,p using equation (419); end Update the learning rate; Reduce the neighborhood radius at speci ed learning iterations; end

taking the rst input patterns as the initial weight vectors For the example in Figure 42(b), the latter will result in the weights u11 = z1,1 , u12 = z2,1 , u21 = z1,2 , u22 = z2,2 , etc Stopping conditions may be a maximum number of epochs is reached, stop when weight adjustments are su ciently small, a small enough quantization error has been reached, where the quantization error is de ned as PT 2 p=1 ||zp uk ||2 (421) QT = PT One problem that may occur in LVQ networks is that one cluster unit may dominate as the winning cluster unit The danger of such a scenario is that most patterns will be in one cluster To prevent one output unit from dominating, a conscience factor is incorporated in a function to determine the winning output unit The conscience factor penalizes an output for winning too many times The activation value of output units is calculated using ok,p = where 1 for min k {dk,p (zp , uk ) bk (t)} 0 otherwise 1 bk (t) = ( gk (t)) I gk (t) = gk (t 1) + (ok,p gk (t 1)) (422)

(423) (424)