Playground: Chapter 14
Advanced Clustering
This page practises the ideas of Chapter 14 in four parts. Practise the chapter works through the book’s exercises, with starter code, hints, and solutions. Go further goes beyond the book, with simulations and real datasets. Check your understanding asks short questions whose answers open with a click. Do it yourself gives open tasks with an empty code space and no answers: you decide what counts as a group, as you will in your own thesis.
In the exercises below, replace each ______ with your own code and press Run Code.
Practise the chapter
These are the exercises at the end of Chapter 14, with the same numbers.
Exercise 1: Four mixture profiles
Fit a Gaussian mixture model with 4 components; describe the four profiles and compare with Chapter 9’s k-means.
Set n_components=4.
gmm4 = GaussianMixture(n_components=4, covariance_type='full', random_state=123).fit(profile_data)The four profiles split the overloaded profile: one group remains heavily overloaded, while a small subgroup with extremely high caffeine consumption and very short sleep forms its own cluster. The four-cluster solution shows strong agreement with Chapter 9’s 4-cluster k-means (ARI \(\approx 0.73\)).
Exercise 2: Clear members
Count the students with a membership probability above 0.95 for their profile, and report the share of the sample.
Take the maximum across each student row using axis=1.
certainty = probs.max(axis=1)About 72% of students belong to their assigned profile with greater than 95% certainty. The remaining 28% fall in the overlapping zones between profiles.
Exercise 3: DBSCAN with different eps
Run DBSCAN on the profile data with eps of 1.2, 1.6, 2.0, and 2.4 (and min_samples = 8), and observe the number of clusters and noise points.
Pass min_samples=8.
db = DBSCAN(eps=eps, min_samples=8).fit(profile_data)At eps = 1.2, the points are fragmented with hundreds labeled as noise. At eps = 2.0, DBSCAN recognizes the student body as one continuous cloud with 13 isolated outliers (noise).
Exercise 4: The profiles without the noise students
Leave out the DBSCAN noise students, fit the mixture model again, and describe whether the profiles change.
Filter out noise points where labels_ != -1.
keep = db.labels_ != -1The resulting profiles are virtually identical (ARI \(\approx 0.85\)). Dropping the 13 noise points slightly lowers the overloaded profile’s caffeine average, confirming that the profiles are robust and not driven by a few extreme outliers.
Exercise 5: Mixture model against hierarchical clustering
Compare the three-cluster mixture model with Ward’s hierarchical clustering cut into three clusters.
Calculate the adjusted Rand index with adjusted_rand_score.
ari_hier = adjusted_rand_score(profiles['gmm_cluster'], hierarchical)The adjusted Rand index is approximately 0.63, reflecting substantial agreement. The differences occur along the boundaries between the disengaged and balanced clusters.
Exercise 6: Profiles and final GPA
Compare the three profiles on final GPA.
The balanced profile finishes with the highest GPA (~3.27), followed by the overloaded profile (~3.07), while the disengaged profile finishes lowest (~2.99). Overloaded students study the most yet earn lower GPAs than balanced students due to sleep deficit and high stress.
Exercise 7: A gap between the groups
In the simulation, move the long group upwards to \(y = 3.0\) so that an empty gap separates it from the round group.
Test with long_y = 3.0.
for long_y in [1.4, 3.0]:Without a gap, DBSCAN treats the touching groups as one continuous cluster (ARI \(\approx 0.01\)). With a gap at \(y = 3.0\), DBSCAN isolates the two distinct clusters perfectly (ARI \(\approx 0.98\)).
Go further
These exercises go beyond the book.
Exercise 8: Membership probabilities by hand
Calculate the Old Faithful membership probability by hand using the normal density.
The mean for the long wait group is 80.1, and the denominator is short + long_.
long_ = 0.64 * norm.pdf(wait, loc=80.1, scale=5.9)
p_short = short / (short + long_)A 50-minute wait has \(P(\text{short}) = 1.0\), while an 85-minute wait has \(P(\text{short}) = 0.0\). A 67-minute wait has \(P(\text{short}) \approx 0.42\): although 67 is closer to 54.6 than to 80.1, long waits are nearly twice as common (prior weight 0.64 vs 0.36).
Exercise 9: Groups of different sizes and densities
Compare k-means, DBSCAN, and GMM on groups with varying densities.
DBSCAN struggles when groups have very different densities, as a single eps cannot fit both tight and loose clusters. GMM accommodates unequal variances easily, scoring high ARI (\(\approx 0.94\)).
Exercise 10: DBSCAN on Old Faithful
Run DBSCAN on scaled Old Faithful waiting and eruption durations.
At eps = 0.3, DBSCAN separates the short-waiting eruptions (~54 min) from the long-waiting eruptions (~80 min), flagging only a few noise points.
Exercise 11: Earthquakes near Fiji
Cluster earthquake locations near Fiji.
Both methods separate the eastern and western bands along the ocean trenches (ARI \(\approx 0.94\)). DBSCAN naturally follows the non-spherical curved contours of the trenches and detects isolated noise tremors.
Check your understanding
Answer each question in your own words first, then click to see a model answer.
1. What are the three definitions of a group used by k-means, mixture models, and DBSCAN?
k-means defines a group around a common geometric centre (round, equal-sized clusters). GMM defines a group as a multivariate probability distribution (allowing soft probabilistic membership and ellipsoidal geometry). DBSCAN defines a group as a continuous dense region separated by sparse areas (allowing arbitrary geometric shapes and outlier noise).
2. What is the difference between hard and soft membership?
Hard assignment strictly places each observation into one cluster with 100% certainty. Soft membership provides a probability distribution across all clusters, allowing researchers to identify boundary cases and evaluate classification certainty.
3. What does BIC compare when choosing the number of clusters in a mixture model?
BIC balances the model’s log-likelihood fit against the number of parameters required. Extra components and unconstrained covariance matrices fit training points better but increase model complexity.
4. Why did DBSCAN find only one cluster of students in the chapter?
Because the graduate students form a continuous multidimensional distribution without low-density chasms separating them. DBSCAN bridges regions of high density, treating the entire student body as a single connected cloud while detecting extreme outliers as noise.
5. How can a clustering be judged when there is no answer key?
Through a convergence of evidence: internal compactness/separation metrics (silhouette, BIC), cross-method stability (adjusted Rand index across k-means, GMM, Ward), bootstrap resampling stability, and external validity against substantive outcomes not used during clustering (such as dropout rates or GPA).
Do it yourself
These tasks have no starter code and no answers. Write your own code, as you would for a thesis. The datasets profiles, profile_data, and all clustering packages are loaded.
1. Choose a different subset of student variables (e.g., sleep, caffeine, support, wellbeing). Cluster with k-means and DBSCAN, justify your settings, and evaluate noise.
2. Check the stability of three k-means profiles by drawing 20 bootstrap samples and computing the adjusted Rand index against the full-sample clustering.
3. Write the methods paragraph of a thesis detailing your clustering design and validation decisions.
Work on your own computer
The project contains the datasets and all exercises in Python (exercises.py and solutions.py).
- Download chapter14.zip, unzip it, and open the folder in your favourite Python editor.