cluster analysis(1).pptxbfdhdhhthjhfghhj

Copyright © 2010 Pearson Education, Inc. 20-1
Chapter Twenty
Cluster Analysis

Chapter Outline
1) Overview
2) Basic Concept
3) Statistics Associated with Cluster Analysis
4) Conducting Cluster Analysis
i. Formulating the Problem
ii. Selecting a Distance or Similarity Measure
iii. Selecting a Clustering Procedure
iv. Deciding on the Number of Clusters
v. Interpreting and Profiling the Clusters
vi. Assessing Reliability and Validity

Chapter Outline
5) Applications of Nonhierarchical
Clustering
6) Clustering Variables
7) Summary

Cluster Analysis
• Cluster analysis is a class of techniques used to classify
objects or cases into relatively homogeneous groups called
clusters. Objects in each cluster tend to be similar to each
other and dissimilar to objects in the other clusters.
Cluster analysis is also called classification analysis, or
numerical taxonomy.
• Both cluster analysis and discriminant analysis are
concerned with classification. However, discriminant
analysis requires prior knowledge of the cluster or group
membership for each object or case included, to develop
the classification rule. In contrast, in cluster analysis there
is no a priori information about the group or cluster
membership for any of the objects. Groups or clusters are
suggested by the data, not defined a priori.

An Ideal Clustering Situation
Variable 2(price sensitivity)
Variable
1(quality
conciousness)
Fig. 20.1
One respondent in one cluster

A Practical Clustering Situation
X
Variable 2
Variable
1
Fig. 20.2

Statistics Associated with Cluster Analysis
• Agglomeration schedule. An agglomeration schedule
gives information on the objects or cases being combined at
each stage of a hierarchical clustering process.
• Cluster centroid. The cluster centroid is the mean values
of the variables for all the cases or objects in a particular
cluster.
• Cluster centers. The cluster centers are the initial starting
points in nonhierarchical clustering. Clusters are built
around these centers, or seeds.
• Cluster membership. Cluster membership indicates the
cluster to which each object or case belongs.

• Dendrogram. A dendrogram, or tree graph, is a graphical
device for displaying clustering results. Vertical lines
represent clusters that are joined together. The position of
the line on the scale indicates the distances at which
clusters were joined. The dendrogram is read from left to
right. Figure 20.8 is a dendrogram.
• Distances between cluster centers. These distances
indicate how separated the individual pairs of clusters are.
Clusters that are widely separated are distinct, and
therefore desirable.

• Icicle plot. An icicle plot is a graphical display of
clustering results, so called because it resembles
a row of icicles hanging from the eaves of a
house. The columns correspond to the objects
being clustered, and the rows correspond to the
number of clusters. An icicle plot is read from
bottom to top. Figure 20.7 is an icicle plot.
• Similarity/distance coefficient matrix. A
similarity/distance coefficient matrix is a lower-
triangle matrix containing pairwise distances
between objects or cases.

Conducting Cluster Analysis
Formulate the Problem
Assess the Validity of Clustering
Select a Distance Measure
Select a Clustering Procedure
Decide on the Number of Clusters
Interpret and Profile Clusters
Fig. 20.3

Attitudinal Data For Clustering
Case No. V1 V2 V3 V4 V5 V6
1 6 4 7 3 2 3
2 2 3 1 4 5 4
3 7 2 6 4 1 3
4 4 6 4 5 3 6
5 1 3 2 2 6 4
6 6 4 6 3 3 4
7 5 3 6 3 3 4
8 7 3 7 4 1 4
9 2 4 3 3 6 3
10 3 5 3 6 4 6
11 1 3 2 3 5 3
12 5 4 5 4 2 4
13 2 2 1 5 4 4
14 4 6 4 6 4 7
15 6 5 4 2 1 4
16 3 5 4 6 4 7
17 4 4 7 2 2 5
18 3 7 2 6 4 3
19 4 6 3 7 2 7
20 2 3 2 4 7 2
Table
20.1

Conducting Cluster Analysis:
Formulate the Problem
• Perhaps the most important part of formulating the
clustering problem is selecting the variables on which
the clustering is based.
• Inclusion of even one or two irrelevant variables may
distort an otherwise useful clustering solution.
• Basically, the set of variables selected should describe
the similarity between objects in terms that are relevant
to the marketing research problem.
• The variables should be selected based on past research,
theory, or a consideration of the hypotheses being
tested. In exploratory research, the researcher should
exercise judgment and intuition.

Select a Distance or Similarity Measure
• The most commonly used measure of similarity is the Euclidean
distance or its square. The Euclidean distance is the square root of
the sum of the squared differences in values for each variable. Other
distance measures are also available. The city-block or Manhattan
distance between two objects is the sum of the absolute differences
in values for each variable. The Chebychev distance between two
objects is the maximum absolute difference in values for any
variable.
• If the variables are measured in vastly different units, the clustering
solution will be influenced by the units of measurement. In these
cases, before clustering respondents, we must standardize the data
by rescaling each variable to have a mean of zero and a standard
deviation of unity. It is also desirable to eliminate outliers (cases
with atypical values).
• Use of different distance measures may lead to different clustering
results. Hence, it is advisable to use different measures and compare
the results.

A Classification of Clustering Procedures
Fig. 20.4
Nonhierarchical
Hierarchical
Agglomerative Divisive
Sequential
Threshold
Parallel
Threshold
Optimizing
Partitioning
Linkage
Methods
Variance
Methods
Centroid
Methods
Ward’s
Method
Single
Linkage
Complete
Linkage
Average
Linkage
Other
Two-Step
Clustering Procedures

Select a Clustering Procedure–Hierarchical
• Hierarchical clustering is characterized by the
development of a hierarchy or tree-like structure.
Hierarchical methods can be agglomerative or divisive.
• Agglomerative clustering starts with each object in a
separate cluster. Clusters are formed by grouping objects
into bigger and bigger clusters. This process is continued
until all objects are members of a single cluster.
• Divisive clustering starts with all the objects grouped in a
single cluster. Clusters are divided or split until each object
is in a separate cluster.
• Agglomerative methods are commonly used in marketing
research. They consist of linkage methods, error sums of
squares or variance methods, and centroid methods.

Select a Clustering Procedure – Linkage Method
• The single linkage method is based on minimum distance,
or the nearest neighbor rule. At every stage, the distance
between two clusters is the distance between their two closest
points (see Figure 20.5).
• The complete linkage method is similar to single linkage,
except that it is based on the maximum distance or the
furthest neighbor approach. In complete linkage, the distance
between two clusters is calculated as the distance between
their two furthest points.
• The average linkage method works similarly. However, in
this method, the distance between two clusters is defined as
the average of the distances between all pairs of objects,
where one member of the pair is from each of the clusters
(Figure 20.5).

Linkage Methods of Clustering
Fig. 20.5 Single Linkage
Minimum Distance
Complete Linkage
Maximum
Distance
Average Linkage
Average Distance
Cluster 1 Cluster 2
Cluster 1 Cluster 2
Cluster 1 Cluster 2

Select a Clustering Procedure – Variance Method
• The variance methods attempt to generate clusters to
minimize the within-cluster variance.
• A commonly used variance method is the Ward's procedure.
For each cluster, the means for all the variables are computed.
Then, for each object, the squared Euclidean distance to the
cluster means is calculated (Figure 20.6). These distances are
summed for all the objects. At each stage, the two clusters
with the smallest increase in the overall sum of squares within
cluster distances are combined.
• In the centroid methods, the distance between two clusters
is the distance between their centroids (means for all the
variables), as shown in Figure 20.6. Every time objects are
grouped, a new centroid is computed.
• Of the hierarchical methods, average linkage and Ward's
methods have been shown to perform better than the other
procedures.

Other Agglomerative Clustering Methods
Ward’s Procedure
Centroid Method
Fig. 20.6

Select a Clustering Procedure – Nonhierarchical
• The nonhierarchical clustering methods are frequently
referred to as k-means clustering. These methods include
sequential threshold, parallel threshold, and optimizing
partitioning.
• In the sequential threshold method, a cluster center is
selected and all objects within a prespecified threshold value
from the center are grouped together. Then a new cluster
center or seed is selected, and the process is repeated for the
unclustered points. Once an object is clustered with a seed, it
is no longer considered for clustering with subsequent seeds.
• The parallel threshold method operates similarly, except that
several cluster centers are selected simultaneously and objects
within the threshold level are grouped with the nearest center.
• The optimizing partitioning method differs from the two
threshold procedures in that objects can later be reassigned to
clusters to optimize an overall criterion, such as average within
cluster distance for a given number of clusters.

Select a Clustering Procedure
• It has been suggested that the hierarchical and
nonhierarchical methods be used in tandem. First, an initial
clustering solution is obtained using a hierarchical
procedure, such as average linkage or Ward's. The number
of clusters and cluster centroids so obtained are used as
inputs to the optimizing partitioning method.
• Choice of a clustering method and choice of a distance
measure are interrelated. For example, squared Euclidean
distances should be used with the Ward's and centroid
methods. Several nonhierarchical procedures also use
squared Euclidean distances.

Results of Hierarchical Clustering
Stage cluster
Clusters combined first appears
Stage Cluster 1 Cluster 2 Coefficient Cluster 1 Cluster 2 Next stage
1 14 16 1.000000 0 0 6
2 6 7 2.000000 0 0 7
3 2 13 3.500000 0 0 15
4 5 11 5.000000 0 0 11
5 3 8 6.500000 0 0 16
6 10 14 8.160000 0 1 9
7 6 12 10.166667 2 0 10
8 9 20 13.000000 0 0 11
9 4 10 15.583000 0 6 12
10 1 6 18.500000 6 7 13
11 5 9 23.000000 4 8 15
12 4 19 27.750000 9 0 17
13 1 17 33.100000 10 0 14
14 1 15 41.333000 13 0 16
15 2 5 51.833000 3 11 18
16 1 3 64.500000 14 5 19
17 4 18 79.667000 12 0 18
18 2 4 172.662000 15 17 19
19 1 2 328.600000 16 18 0
Agglomeration Schedule Using Ward’s Procedure
Table 20.2

Results of Hierarchical Clustering
Number of Clusters
Label case 4 3 2
1 1 1 1
2 2 2 2
3 1 1 1
4 3 3 2
5 2 2 2
6 1 1 1
7 1 1 1
8 1 1 1
9 2 2 2
10 3 3 2
11 2 2 2
12 1 1 1
13 2 2 2
14 3 3 2
15 1 1 1
16 3 3 2
17 1 1 1
18 4 3 2
19 3 3 2
20 2 2 2
Cluster Membership of Cases Using Ward’s Procedure
Table
20.2,
cont.

Vertical Icicle Plot Using Ward’s Method
Fig. 20.7

Fig. 20.8
Dendrogram Using Ward’s Method

Decide on the Number of Clusters
• Theoretical, conceptual, or practical considerations may
suggest a certain number of clusters.
• In hierarchical clustering, the distances at which clusters
are combined can be used as criteria. This information can
be obtained from the agglomeration schedule or from the
dendrogram.
• In nonhierarchical clustering, the ratio of total within-group
variance to between-group variance can be plotted against
the number of clusters. The point at which an elbow or a
sharp bend occurs indicates an appropriate number of
clusters.
• The relative sizes of the clusters should be meaningful.

Interpreting and Profiling the Clusters
• Interpreting and profiling clusters involves
examining the cluster centroids. The
centroids enable us to describe each cluster
by assigning it a name or label.
• It is often helpful to profile the clusters in
terms of variables that were not used for
clustering. These may include demographic,
psychographic, product usage, media usage,
or other variables.

Cluster Centroids
Table 20.3
Cluster V1 V2 V3 V4 V5 V6
1 5.750 3.625 6.000 3.125 1.750 3.875
2 1.667 3.000 1.833 3.500 5.500 3.333
3 3.500 5.833 3.333 6.000 3.500 6.000
Means of Variables

Assess Reliability and Validity
1. Perform cluster analysis on the same data using different
distance measures. Compare the results across measures to
determine the stability of the solutions.
2. Use different methods of clustering and compare the results.
3. Split the data randomly into halves. Perform clustering
separately on each half. Compare cluster centroids across the
two subsamples.
4. Delete variables randomly. Perform clustering based on the
reduced set of variables. Compare the results with those
obtained by clustering based on the entire set of variables.
5. In nonhierarchical clustering, the solution may depend on the
order of cases in the data set. Make multiple runs using
different order of cases until the solution stabilizes.

Table 20.4
Results of Nonhierarchical Clustering
Initial Cluster Centers
4 2 7
6 3 2
3 2 6
7 4 4
2 7 1
7 2 3
V1
V2
V3
V4
V5
V6
1 2 3
Cluster
Convergence achieved due to no or small distance
change. The maximum distance by which any center
has changed is 0.000. The current iteration is 2. The
minimum distance between initial centers is 7.746.
a.
Iteration History
a
2.154 2.102 2.550
0.000 0.000 0.000
Iteration
1
2
1 2 3
Change in Cluster Centers

Table 20.4 cont.
Cluster Membership
3 1.414
2 1.323
3 2.550
1 1.404
2 1.848
3 1.225
3 1.500
3 2.121
2 1.756
1 1.143
2 1.041
3 1.581
2 2.598
1 1.404
3 2.828
1 1.624
3 2.598
1 3.555
1 2.154
2 2.102
Case Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Cluster Distance

Table 20.4, cont.
Final Cluster Centers
Cluster
1 2 3
V1
V2
V3
V4
V5
V6
4
6
3
6
4
6
2
3
2
4
6
3
6
4
6
3
2
4
Distances between Final Cluster Centers
1 2 3
1
2
3
5.568
5.698
5.568
6.928
5.698
6.928
Cluster

ANOVA
29.108 2 0.608 17 47.888 0.000
13.546 2 0.630 17 21.505 0.000
31.392 2 0.833 17 37.670 0.000
15.713 2 0.728 17 21.585 0.000
22.537 2 0.816 17 27.614 0.000
12.171 2 1.071 17 11.363 0.001
V1
V2
V3
V4
V5
V6
Mean Square df
Cluster
Mean Square df
Error
F Sig.
The F tests should be used only for descriptive purposes because the clusters have been
chosen to maximize the differences among cases in different clusters. The observed
significance levels are not corrected for this, and thus cannot be interpreted as tests of the
hypothesis that the cluster means are equal.
Number of Cases in each Cluster
6.000
6.000
8.000
20.000
0.000
1
2
3
Cluster
Valid
Missing
Table 20.4, cont.

Results of Two-Step Clustering
Table 20.5 Auto-Clustering
Number of Clusters
Akaike's
Information
Criterion (AIC)
AIC
Change(a)
Ratio of AIC
Changes(b)
Ratio of
Distance
Measures(c)
1 104.140
2 101.171 -2.969 1.000 .847
3 97.594 -3.577 1.205 1.583
4 116.896 19.302 -6.502 2.115
5 138.230 21.335 -7.187 1.222
6 158.586 20.355 -6.857 1.021
7 179.340 20.755 -6.991 1.224
8 201.628 22.288 -7.508 1.006
9 224.055 22.426 -7.555 1.111
10 246.522 22.467 -7.568 1.588
11 269.570 23.048 -7.764 1.001
12 292.718 23.148 -7.798 1.055
13 316.120 23.402 -7.883 1.002
14 339.223 23.103 -7.782 1.044
15 362.650 23.427 -7.892 1.004
a The changes are from the previous number of clusters in the table.
b The ratios of changes are relative to the change for the two cluster solution.
c The ratios of distance measures are based on the current number of clusters
against the previous number of clusters.

Cluster Distribution
Table 20.5, cont.
N
% of
Combined % of Total
1 6 30.0% 30.0%
2 6 30.0% 30.0%
3 8 40.0% 40.0%
Cluster
Combined 20 100.0% 100.0%
Total 20 100.0%

Cluster Profiles
Table 20.5, cont.
Fun Bad for Budget Eating Out
Mean Std. Deviation Mean Std. Deviation Mean Std. Deviation
1 1.67 .516 3.00 .632 1.83 .753
2 3.50 .548 5.83 .753 3.33 .816
3 5.75 1.035 3.63 .916 6.00 1.069
Cluster
Combined 3.85 1.899 4.10 1.410 3.95 2.012
Best Buys Don't Care Compare Prices
Mean Std. Deviation Mean Std. Deviation Mean Std. Deviation
3.50 1.049 5.50 1.049 3.33 .816
6.00 .632 3.50 .837 6.00 1.549
3.13 .835 1.88 .835 3.88 .641
4.10 1.518 3.45 1.761 4.35 1.496

Clustering Variables
• In this instance, the units used for analysis are the variables,
and the distance measures are computed for all pairs of
variables.
• Hierarchical clustering of variables can aid in the identification
of unique variables, or variables that make a unique
contribution to the data.
• Clustering can also be used to reduce the number of
variables. Associated with each cluster is a linear combination
of the variables in the cluster, called the cluster component.
A large set of variables can often be replaced by the set of
cluster components with little loss of information. However, a
given number of cluster components does not generally
explain as much variance as the same number of principal
components.

SPSS Windows
To select this procedure using SPSS for Windows,
click:
Analyze>Classify>Hierarchical Cluster …
Analyze>Classify>K-Means Cluster …
Analyze>Classify>Two-Step Cluster 

SPSS Windows: Hierarchical Clustering
1. Select ANALYZE from the SPSS menu bar.
2. Click CLASSIFY and then HIERARCHICAL CLUSTER.
3. Move “Fun [v1],” “Bad for Budget [v2],” “Eating Out [v3],” “Best Buys [v4],”
“Don’t Care [v5],” and “Compare Prices [v6]” into the VARIABLES box.
4. In the CLUSTER box, check CASES (default option). In the DISPLAY box,
check STATISTICS and PLOTS (default options).
5. Click on STATISTICS. In the pop-up window, check AGGLOMERATION
SCHEDULE. In the CLUSTER MEMBERSHIP box, check RANGE OF
SOLUTIONS. Then, for MINIMUM NUMBER OF CLUSTERS, enter 2 and for
MAXIMUM NUMBER OF CLUSTERS, enter 4. Click CONTINUE.
6. Click on PLOTS. In the pop-up window, check DENDROGRAM. In the
ICICLE box, check ALL CLUSTERS (default). In the ORIENTATION box,
check VERTICAL. Click CONTINUE.
7. Click on METHOD. For CLUSTER METHOD, select WARD’S METHOD. In the
MEASURE box, check INTERVAL and select SQUARED EUCLIDEAN
DISTANCE. Click CONTINUE.
8. Click OK.

SPSS Windows: K-Means Clustering
2. Click CLASSIFY and then K-MEANS CLUSTER.
3. Move “Fun [v1],” “Bad for Budget [v2],” “Eating Out [v3],”
“Best Buys [v4],” “Don’t Care [v5],” and “Compare Prices
[v6]” into the VARIABLES box.
4. For NUMBER OF CLUSTER, select 3.
5. Click on OPTIONS. In the pop-up window, in the
STATISTICS box, check INITIAL CLUSTER CENTERS and
CLUSTER INFORMATION FOR EACH CASE. Click CONTINUE.
6. Click OK.

SPSS Windows: Two-Step Clustering
2. Click CLASSIFY and then TWO-STEP CLUSTER.
3. Move “Fun [v1],” “Bad for Budget [v2],” “Eating Out [v3],”
“Best Buys [v4],” “Don’t Care [v5],” and “Compare Prices
[v6]” into the CONTINUOUS VARIABLES box.
4. For DISTANCE MEASURE, select EUCLIDEAN.
5. For NUMBER OF CLUSTER, select DETERMINE
AUTOMATICALLY.
6. For CLUSTERING CRITERION, select AKAIKE’S
INFORMATION CRITERION (AIC).
7. Click OK.

SAS Learning Edition
To select this procedure using SAS Learning
Edition, click:
Analyze>Classify>Cluster Analysis

SAS Learning Edition: Hierarchical Clustering
1. Select ANALYZE from the SAS Learning Edition
menu bar.
2. Select Multivariate>Cluster Analysis.
3. Move V1-V6 to the Analysis variables task role.
4. Click Cluster and select Ward’s minimum
variance method under Cluster method.
5. Click Results and select Simple summary
statistics.
6. Click Run.

SAS Learning Edition Windows:
K-Means Clustering
1. Select ANALYZE from the SAS Learning Edition
menu bar.
2. Select Multivariate>Cluster Analysis.
3. Move V1-V6 to the Analysis variables task role.
4. Click Cluster and select K-means algorithm as the
cluster method and 3 for the Maximum number of
clusters.
5. Click Run.

All rights reserved. No part of this publication may be
reproduced, stored in a retrieval system, or transmitted, in
any form or by any means, electronic, mechanical,
photocopying, recording, or otherwise, without the prior
written permission of the publisher. Printed in the United
States of America.
Copyright © 2010 Pearson Education, Inc.

cluster analysis(1).pptxbfdhdhhthjhfghhj

More Related Content

Similar to cluster analysis(1).pptxbfdhdhhthjhfghhj

Recently uploaded

cluster analysis(1).pptxbfdhdhhthjhfghhj