Modules
1. What is a Neuron?
2. Neural Networks
3. How Training Works
4. Word Embeddings
5. Attention Mechanism
6. The Transformer
7. How LLMs Work
8. RAG Pipelines
9. Building AI Agents
10. Prompt Engineering
11. Linear Regression
12. Logistic Regression
13. Decision Trees
14. Random Forest
15. K-Means Clustering
16. Cosine Similarity
17. Types of AI & AI Agents
18. Tokenization in NLP
19. Fine-Tuning LLMs
20. Evaluating LLMs
21. Vector Databases Explained
22. Diffusion Models & Image Generation
23. Generative Adversarial Networks (GANs)
24. Reinforcement Learning Basics
25. RLHF: Reinforcement Learning from Human Feedback
26. Convolutional Neural Networks (CNNs)
27. RNNs and LSTMs
28. Overfitting & Regularization
29. Hyperparameter Tuning
30. Support Vector Machines (SVM)
31. Naive Bayes Classifier
32. PCA & Dimensionality Reduction
33. Model Context Protocol (MCP)
34. Why LLMs Hallucinate
35. Context Windows & Long Context
36. Multimodal AI (Vision + Language)
37. AI Ethics & Bias in Machine Learning
38. Classifier Evaluation Metrics: Precision, Recall, ROC & F1
39. Bayesian Inference & MAP Estimation
40. Backpropagation Explained
41. Autoencoders & Variational Autoencoders
42. NLP Text Representation: N-grams, Bag-of-Words & TF-IDF
43. Markov Decision Processes & the Bellman Equation
44. Data Preprocessing & Cleaning for Machine Learning
45. Feature Engineering for Machine Learning
46. ML Frameworks Compared: TensorFlow, PyTorch & JAX
47. Data Selection & Active Learning
48. ML Hardware Acceleration: GPUs, TPUs & Accelerators
49. ML Benchmarking Methodology
50. Responsible AI Engineering: Safety & Robustness
51. DBSCAN & Hierarchical Clustering
52. ML Pipelines: Chaining Preprocessing and Models
53. History & Evolution of AI Systems
54. ML System Design Fundamentals
55. The Machine Learning Workflow & Lifecycle
56. Data Engineering for Machine Learning
57. Model Compression: Quantization, Pruning & Distillation
58. Model Serving & Inference Systems
59. MLOps: Monitoring & Maintaining Models in Production
60. The Turing Test & AI Evaluation
61. Heuristics & Search in AI
62. Genetic Algorithms & Evolutionary Computation
63. Knowledge Representation & Semantic Networks
64. Expert Systems
65. AI and Games: How AlphaZero Works
66. AI in Bioinformatics
67. Symbolic vs. Connectionist AI
68. Constraint Satisfaction Problems
69. Fuzzy Logic Systems
70. Multivariate Regression Analysis
71. Non-Linear Least Squares Fitting
72. k-Nearest Neighbors (kNN)
73. Ridge & Lasso Regression
74. Polynomial Regression
75. Boosting Algorithms: AdaBoost, Gradient Boosting & XGBoost
76. Ensemble Modeling: Bagging, Boosting & Stacking Compared
77. Gaussian Mixture Models
78. Linear Discriminant Analysis (LDA)
79. Anomaly Detection in Machine Learning
80. Time Series Forecasting with Machine Learning
81. Survival Analysis for Machine Learning
82. Multicollinearity & Variance Inflation Factor (VIF)
83. Cross-Validation Strategies
84. Statistical Hypothesis Testing for ML Model Comparison
85. Activation Functions Deep Dive: ReLU, ELU, Sigmoid, Softmax & Swish
86. Neural Network Hyperparameters Explained
87. The Multilayer Perceptron (MLP)
88. CNN Building Blocks: Convolution & Pooling Layers
89. Famous CNN Architectures: VGG, ResNet & Beyond
90. GRUs vs LSTMs
91. Transformer-XL & Reformer: Handling Longer Sequences
92. NLU vs NLG: Understanding vs Generating Language
93. The Pre-Transformer Embedding Era: ELMo, ULMFit & BERT
94. Deep Reinforcement Learning (DRL) Overview
95. Policy Gradient Methods
96. Actor-Critic Methods
97. Self-Supervised Learning
98. Transfer Learning
99. Batch Normalization & Layer Normalization
100. ML Deployment Paradigms: Cloud, Edge & Mobile
101. Edge AI & TinyML
102. System Balance: Matching ML Workloads to Hardware
103. Neural Computation Fundamentals
104. Network Architecture Design Tradeoffs
105. Distributed Training: Data & Model Parallelism
106. Federated Learning
107. ML Reproducibility Engineering
108. Feature Stores Explained
109. Model Registry & Versioning
110. A/B Testing ML Models in Production
111. Shadow Deployment & Canary Releases for ML
112. ML Cost Optimization
113. ONNX & Model Interoperability
114. Model Explainability: SHAP & LIME
115. ML System Observability
116. Synthetic Data Generation
117. Data Augmentation Techniques
118. ML Security: Adversarial Attacks
119. Privacy-Preserving Machine Learning
120. Chain-of-Thought Reasoning
121. Mixture of Experts (MoE)
122. Speculative Decoding
123. Constitutional AI & Alignment Techniques
124. Tool-Use Patterns in LLM Agents
125. Advanced RAG Patterns: Hybrid Search & Re-ranking
126. Semantic Search vs Keyword Search
127. Fine-Tuning Embedding Models
128. Prompt Injection Security
129. AI Red-Teaming
130. Speech Recognition (ASR)
131. Text-to-Speech (TTS)
132. Computer Vision: Object Detection
133. Image Segmentation
134. OCR & Document AI
135. NumPy Fundamentals for Machine Learning
136. Pandas for Data Science
137. Data Visualization for ML with Matplotlib
138. Model Interpretability: Feature Importance Methods
139. ML Experiment Tracking
140. AutoML Explained
141. Handling Imbalanced Data: SMOTE & Resampling Techniques
142. Multi-Label Classification
143. Multi-Task Learning
144. Zero-Shot & Few-Shot Learning
145. Curriculum Learning
146. Model Calibration
147. Snapshot & Neural Network Ensembles
148. Knowledge Graphs in AI
149. Jupyter Notebooks for Machine Learning
150. Linear Algebra for Machine Learning
151. Calculus for Machine Learning
152. Probability Theory for Machine Learning
153. Information Theory: Entropy, Cross-Entropy & KL Divergence
154. Optimization Algorithms: SGD, Adam & RMSprop Compared
155. Convex vs. Non-Convex Optimization
156. Lagrange Multipliers & Constrained Optimization
157. Monte Carlo Methods in Machine Learning
158. Numerical Stability in Machine Learning
159. The Central Limit Theorem for Machine Learning
160. Bootstrapping & Resampling Methods
161. Bayesian vs. Frequentist Statistics
162. Confidence Intervals for ML Metrics
163. Power Analysis & Sample Size Calculation
164. A/A Testing & Experiment Validity Checks
165. The Adam Optimizer Explained
166. Learning Rate Schedules: Warmup, Cosine Annealing & Step Decay
167. Gradient Clipping
168. Mixed Precision Training
169. Weight Initialization Strategies
170. Association Rule Mining & the Apriori Algorithm
171. Collaborative Filtering for Recommender Systems
172. Matrix Factorization for Recommender Systems
173. Content-Based Recommender Systems
174. The Cold Start Problem in Recommender Systems
175. Spectral Clustering
176. Mean Shift Clustering
177. Local Outlier Factor (LOF)
178. t-SNE Deep Dive
179. UMAP Deep Dive
180. Graph Neural Networks (GNNs)
181. Siamese Networks
182. One-Shot Learning & Meta-Learning (MAML)
183. Neural Ordinary Differential Equations (Neural ODEs)
184. Normalizing Flows
185. Energy-Based Models
186. Restricted Boltzmann Machines (RBMs)
187. Self-Organizing Maps (SOMs)
188. Capsule Networks
189. Spiking Neural Networks (SNNs)
190. Part-of-Speech Tagging & Named Entity Recognition
191. Dependency Parsing: Representing Sentence Structure
192. Coreference Resolution: Tracking Who's Who in Text
193. Text Summarization: Extractive vs. Abstractive
194. Machine Translation: From Rule-Based to Neural
195. Language Model Perplexity: What It Measures
196. Subword Tokenization Deep Dive: BPE, WordPiece & Unigram
197. Byte-Level Tokenization: Solving the Unknown-Token Problem
198. Instruction Tuning Datasets: Building FLAN & Alpaca-Style Data
199. RLHF Reward Hacking: When Models Game the Reward Model
200. Image Classification Metrics: Top-1 & Top-5 Accuracy
201. Super-Resolution with GANs
202. Neural Style Transfer
203. Human Pose Estimation
204. Face Recognition Systems
205. Video Action Recognition
206. Point Clouds & 3D Vision
207. SLAM Basics (Simultaneous Localization and Mapping)
208. Vision Data Augmentation Deep Dive
209. Image Captioning
210. Multi-Agent Reinforcement Learning
211. Inverse Reinforcement Learning
212. Imitation Learning
213. Reward Shaping in Reinforcement Learning
214. The Sparse Rewards Problem in RL
215. Sim-to-Real Transfer in Robotics
216. Offline Reinforcement Learning
217. Curiosity-Driven Exploration
218. Hierarchical Reinforcement Learning
219. Model-Based Reinforcement Learning
220. The ReAct Agent Pattern
221. Plan-and-Execute Agents
222. Multi-Agent Orchestration Patterns
223. Agent Memory Architectures
224. Agent Evaluation Methodology
225. Prompt Caching & Context Caching
226. Structured Outputs & JSON Mode
227. Vector Database Comparison Deep Dive
228. LoRA Merging & Model Merging
229. Quantization Formats: GGUF, AWQ & GPTQ Compared
230. AI Regulation: The EU AI Act & Risk-Based Frameworks
231. AI & Copyright: Open Questions
232. The Environmental Impact of AI
233. AI and the Future of Work
234. Learning to Rank for Search
235. Fraud Detection Systems with Machine Learning
236. Credit Scoring Models
237. Demand Forecasting with Machine Learning
238. Small Language Models (SLMs) vs. LLMs
239. On-Device LLMs
240. The Hugging Face Ecosystem Overview
241. LangChain vs. LlamaIndex
242. Agent Framework Comparison: AutoGPT, CrewAI & LangGraph
243. Feature Crossing for Recommenders
244. A/B Testing ML-Powered Products
245. Isolation Forest Deep Dive
246. OPTICS Clustering
247. Affinity Propagation Clustering
248. Decision Stumps & Weak Learners
249. Hopfield Networks
Share Guide
AI & MLDeep
Advanced

OPTICS Clustering

11 min read

Learn
Deep Reading
Estimated 11 mins
Prereq
Advanced
Requires advanced math/coding
Interactive
Live Simulator
Contains interactive viz

DBSCAN's blind spot: one density for the whole dataset

DBSCAN clusters points using two fixed parameters: eps (the neighborhood radius) and min_samples (how many neighbors make a point "dense"). This works well when all real clusters in the data have roughly the same density. But real data often has clusters of very different densities — a tight cluster of 50 points packed into a small region, and a loose cluster of 50 points spread across a much larger one.

With a single global eps, DBSCAN cannot handle both well simultaneously: an eps small enough to resolve the tight cluster will fail to connect the loose cluster's points at all, while an eps large enough to connect the loose cluster will merge the tight cluster into its neighbors. OPTICS (Ordering Points To Identify the Clustering Structure) was designed specifically to fix this.

Reachability distance: the core new idea

OPTICS doesn't partition data directly into clusters. Instead, it computes an ordering of all points and, for each point, a reachability distance — roughly, how far you'd have to extend a neighborhood radius to density-reach that point from the nearest already-processed dense point. Formally, the reachability distance of point *p* from point *o* is the larger of: the core distance of *o* (the radius needed for *o* to have min_samples neighbors), and the actual distance between *o* and *p*.

OPTICS then processes points in order of increasing reachability distance, similar to Prim's algorithm for minimum spanning trees, building an ordering where points in the same dense region end up adjacent to each other in the sequence.

The reachability plot: clusters as valleys

The output of OPTICS is best understood visually: plot each point's reachability distance against its position in the computed ordering. Dense clusters show up as valleys — runs of points with low reachability distance, because they're close to their neighbors in the ordering. Gaps between clusters show up as peaks — sharp spikes in reachability distance where you jump from one dense region to a different one.

Crucially, valleys of different depths represent clusters of different densities, all visible in the same plot. You can extract flat clusters from this structure using a variable threshold, or an automated method like the xi extraction algorithm, without having to pick one global density cutoff in advance.

python

Note

You can recover a DBSCAN-equivalent clustering from an OPTICS ordering by cutting the reachability plot at any fixed height — that's literally what a single global eps would do. OPTICS is strictly more informative: it computes the full ordering once, and you can extract clusterings at multiple density thresholds afterward without rerunning the core algorithm, which is useful for exploring data where the right density cutoff isn't known in advance.

DBSCAN vs. OPTICS

DimensionDBSCANOPTICS
Density assumptionSingle global density thresholdHandles varying density clusters
OutputDirect cluster labelsAn ordering + reachability plot, clusters extracted after
Parameterseps, min_samples (eps is sensitive)min_samples, xi (no fixed eps required)
Computational costLower — one pass with fixed epsHigher — effectively explores a range of eps values
Best forRoughly uniform-density clustersMixed-density real-world data

Note

While OPTICS removes the need to pick a single eps, min_samples still strongly shapes the reachability plot — too small and noise looks like clusters, too large and small genuine clusters get smoothed away or marked as noise. Try a few values and inspect the resulting reachability plot visually rather than trusting the first result.

Contrast: how K-Means handles the same data (for comparison)

📊 K-Means Clustering Simulator

Interactive
Points: 0Iteration: 0Inertia: N/A

What's next

OPTICS generalizes density-based clustering, but another family of algorithms takes a completely different route to the same goal — letting data points "vote" for their own exemplars through message passing. The next lesson covers Affinity Propagation.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Need custom AI or MarTech setup? Let's build together.