kmeans elbow method
Find k for kmeans using the elbow method?
The KMeans algorithm can cluster observed data. But how many clusters (k) are there?
The technique to determine K, the number of clusters, is called the elbow method.
With a bit of fantasy, you can see an elbow in the chart below.
- values for K on the horizontal axis
- the distortion on the Y axis (the values calculated with the cost function).
This results in:
When K increases, the centroids are closer to the clusters centroids.
The improvements will decline, at some point rapidly, creating the elbow shape.
That point is the optimal value for K. In the image above, K=3.
The example code below creates finds the optimal value for k.
# clustering dataset