[1] 0.01801949
[1] 1.866025
[1] 1.755929
[1] 1.773948
November 17 + 19 + 24, 2025
Grouping or categorizing observational units (objects) without any pre-assigned labels or scores (no outcome information!)
Latent Dirichlet Allocation: Topic Modeling of TSL Articles
Network & Clustering: Characters in ‘Love Actually’
Using absolute distance doesn’t fix things.
The Hamming distance across the two DNA strands is 7.
dist function in RThe function dist in R calculates the distances given above.
Comparison of string distance metrics from https://www.kdnuggets.com/2019/01/comparison-text-distance-metrics.html.
is a set of nested clusters that are organized as a tree. Note that objects that belong to a child cluster also belong to the parent cluster.
Agglomerative methods start with each object (e.g., gene, penguin, etc.) in its own group. Groups are merged until all objects are together in one group.
Divisive methods start with all objects in one group and break up the groups sequentially until all objects are individuals.
Single Linkage algorithm defines the distance between groups as that of the closest pair of individuals.
Complete Linkage algorithm defines the distance between groups as that of the farthest pair of individuals.
Average Linkage algorithm defines the distance between groups as the average of the distances between all pairs of individuals across the groups.
of Single Linkage Agglomerative Hierarchical Clustering
| A | B | C | D | E | |
|---|---|---|---|---|---|
| A | 0 | ||||
| B | 0.2 | 0 | |||
| C | 0.6 | 0.5 | 0 | ||
| D | 1 | 0.9 | 0.4 | 0 | |
| E | 0.9 | 0.8 | 0.5 | 0.3 | 0 |
see class notes to walk through the process.
strengths
shortcomings
══ Workflow ════════════════════════════════════════════════════════════════════
Preprocessor: Recipe
Model: hier_clust()
── Preprocessor ────────────────────────────────────────────────────────────────
2 Recipe Steps
• step_rm()
• step_normalize()
── Model ───────────────────────────────────────────────────────────────────────
Hierarchical Clustering Specification (partition)
Main Arguments:
num_clusters = 3
linkage_method = average
Computational engine: stats
List of 7
$ cluster_names : Factor w/ 3 levels "Cluster_1","Cluster_2",..: 1 2 3
$ centroids : tibble [3 × 4] (S3: tbl_df/tbl/data.frame)
..$ bill_length_mm : num [1:3] -0.369 0.603 2.253
..$ bill_depth_mm : num [1:3] 0.617 -1.123 -0.368
..$ flipper_length_mm: num [1:3] -0.65 1.13 2.05
..$ body_mass_g : num [1:3] -0.612 1.06 1.977
$ n_members : int [1:3] 219 119 4
$ sse_within_total_total: num [1:3] 283.19 111.49 2.01
$ sse_total : num 656
$ orig_labels : NULL
$ cluster_assignments : Factor w/ 3 levels "Cluster_1","Cluster_2",..: 1 1 1 1 1 1 1 1 1 1 ...
# A tibble: 342 × 1
.pred_cluster
<fct>
1 Cluster_1
2 Cluster_1
3 Cluster_1
4 Cluster_1
5 Cluster_1
6 Cluster_1
7 Cluster_1
8 Cluster_1
9 Cluster_1
10 Cluster_1
# ℹ 332 more rows
It’s important to note that there is no guarantee that predict() on the training data will produce the same results as extract_cluster_assignments(). The process by which clusters are created during the agglomerations results in a particular partition; but if a training observation is treated as new data, it is predicted in the same manner as truly new information.
penguins |>
drop_na(bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g) |>
select(island) |>
cbind(cluster = cutree(penguin_hclust, k = 3) ) |>
table() cluster
island 1 2 3
Biscoe 44 119 4
Dream 124 0 0
Torgersen 51 0 0
penguins |>
drop_na(bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g) |>
select(species) |>
cbind(cluster = cutree(penguin_hclust, k = 3) ) |>
table() cluster
species 1 2 3
Adelie 151 0 0
Chinstrap 68 0 0
Gentoo 0 119 4
Artwork by (allison_horst?).
https://www.naftaliharris.com/blog/visualizing-k-means-clustering/
If a point is “closer” to a different center, moving it will lower the objective function.
Averages minimize squared differences, so taking the new average will result in a lower objective function.
If a point is equidistant from two clusters, the point won’t move.
The algorithm must converge in finite number of steps because there are finitely many points.
strengths
shortcomings
══ Workflow ════════════════════════════════════════════════════════════════════
Preprocessor: Recipe
Model: k_means()
── Preprocessor ────────────────────────────────────────────────────────────────
2 Recipe Steps
• step_rm()
• step_normalize()
── Model ───────────────────────────────────────────────────────────────────────
K Means Cluster Specification (partition)
Main Arguments:
num_clusters = 3
Engine-Specific Arguments:
initializer = random
Computational engine: ClusterR
List of 7
$ cluster_names : Factor w/ 3 levels "Cluster_1","Cluster_2",..: 1 2 3
$ centroids : tibble [3 × 4] (S3: tbl_df/tbl/data.frame)
..$ bill_length_mm : num [1:3] -1.047 0.66 0.656
..$ bill_depth_mm : num [1:3] 0.486 0.816 -1.098
..$ flipper_length_mm: num [1:3] -0.89 -0.286 1.157
..$ body_mass_g : num [1:3] -0.769 -0.374 1.09
$ n_members : int [1:3] 132 87 123
$ sse_within_total_total: num [1:3] 122 113 143
$ sse_total : num 1364
$ orig_labels : int [1:342] 1 1 1 1 1 1 1 1 2 1 ...
$ cluster_assignments : Factor w/ 3 levels "Cluster_1","Cluster_2",..: 1 1 1 1 1 1 1 1 2 1 ...
Find the observations (data values!)
Important:
Randomly assign a number, from 1 to
Iterate until the cluster assignments stop changing:
strengths
shortcomings
Silhouette Width (use
Elbow plot (use
Consider observation
Note that if
We are looking for a large silhouette width.