DSA Notes
HubAlgorithm vs Data Structure — Key Differences, Definitions, and How They Work TogetherApplications of DSA — How Data Structures and Algorithms Power Real-World SoftwareTime and Space Complexity — A Complete BeginnerWhat is DSA? — A Complete BeginnerWhy Learn DSA? — Career Benefits, Real-World Impact, and Long-Term ValueAsymptotic Analysis — Complete Framework with Recurrence Relations and Master TheoremBest, Average, and Worst Case Analysis — A Practical Guide for Every AlgorithmBig O Notation — Complete Guide with Formal Definition, Rules, and ExamplesBig Omega Notation — Lower Bounds, Best Case, and Formal DefinitionBig Theta (Θ) Notation — Tight Bounds and Exact Growth RatesSpace Complexity — Memory Analysis, Auxiliary Space, and Time-Space TradeoffsTime Complexity Examples — 25 Worked Problems from Basic to AdvancedArray Interview Problems — 15 Classic Problems with Complete SolutionsArray Operations — Every Operation Explained with Code, Complexity, and Edge CasesArray Rotation — Algorithms, Techniques, and Interview ProblemsArray Searching — Linear Search, Binary Search, and Advanced TechniquesArrays — The Complete Introduction to the Most Fundamental Data StructureDynamic Arrays — How Python Lists and ArrayLists Work InternallyMultidimensional Arrays — 2D Arrays, Matrices, and Memory LayoutSliding Window Technique — Complete Guide with 10 Classic ProblemsAnagram Problems — Detection, Grouping, and Sliding Window VariationsPalindrome Problems — Check, Longest, Count, and Expand Around CenterPattern Searching — KMP, Rabin-Karp, and Boyer-Moore AlgorithmsString Interview Problems — 10 Classic Problems with Complete SolutionsString Matching — Longest Common Subsequence, Edit Distance, and Z-AlgorithmString Operations — Complete Reference with Complexity and ImplementationStrings — Complete Introduction to String Data Structure and OperationsCircular Linked List — Structure, Operations, and ApplicationsDoubly Linked List — Complete Implementation with Bidirectional OperationsLinked List Interview Problems — 12 Classic Problems with Complete SolutionsLinked Lists — A Complete Introduction with Memory Model and TypesLinked List Operations — Every Technique with Code and ComplexityReverse Linked List — All Approaches from Basic to AdvancedSingly Linked List — Complete Implementation with All OperationsBalanced Parentheses — Stack-Based Validation with All VariantsInfix, Prefix, and Postfix Expressions — Stack-Based Conversion and EvaluationStack Implementation — Three Complete Implementations with Edge CasesStack Interview Problems — 10 Classic Problems with Complete SolutionsStack — Complete Introduction to LIFO Data StructureStack Using Arrays — Memory Layout, Capacity Management, and PerformanceStack Using Linked List — Implementation, Analysis, and When to Use ItCircular Queue — The Ring Buffer Explained with ImplementationDeque — Double-Ended Queue with Complete Implementation and ApplicationsPriority Queue — Complete Guide from Concept to ImplementationQueue Applications — BFS, Level-Order Traversal, 01-BFS, and MoreQueue Implementation — Array, Linked List, and Two-Stack ApproachesQueue Interview Problems — 10 Classic Problems with Complete SolutionsQueue — Complete Introduction to the FIFO Data StructureAVL Tree - Self-Balancing Binary Search TreeBinary Search Tree (BST)Binary Tree - Structure and OperationsFenwick Tree (Binary Indexed Tree)Red-Black Tree - Another Balanced BSTSegment Tree - Range Query Data StructureTree Interview Problems CollectionTree Data Structure FundamentalsTree Traversals - Inorder, Preorder, Postorder, Level OrderHeap — Complete Introduction to the Priority-Based Tree Data StructureHeap Operations — Complete Reference with Complexity AnalysisHeap Problems — 8 Classic Interview Problems with Complete SolutionsMax Heap — Complete Implementation and ApplicationsMin Heap — Complete Implementation with All OperationsPriority Queue Using Heap — Implementation, Applications, and PatternsBellman-Ford Algorithm — Shortest Path with Negative Weights and Cycle DetectionBFS Algorithm — Breadth-First Search with All Applications and VariantsDFS Algorithm — Depth-First Search with Cycle Detection and All ApplicationsDijkstraFloyd-Warshall Algorithm — All-Pairs Shortest Paths with Complete AnalysisGraph Interview Problems — 12 Classic Problems with Complete SolutionsGraph — Complete Introduction to the Most Powerful Data StructureGraph Representation — Adjacency List, Matrix, Edge List with Complete AnalysisKruskalMinimum Spanning Tree — Definition, Properties, and ApplicationsPrimShortest Path Algorithms — Complete Comparison and Selection GuideTopological Sorting — DFS and BFS (KahnUnion-Find (DSU) in Graphs — Connected Components and Dynamic ConnectivitySeparate Chaining — Complete Implementation and AnalysisCollision Handling — Chaining vs Open Addressing Complete ComparisonHash Functions — Design, Properties, and Common ImplementationsHashing — Complete Introduction to Hash Tables and Hash FunctionsHashing Problems — 10 Classic Interview Problems with Complete SolutionsOpen Addressing — Linear Probing, Quadratic Probing, and Double HashingBubble Sort — Complete Analysis with Optimizations and When to Use ItCounting Sort — O(n+k) Sorting That Breaks the Comparison Lower BoundHeap Sort — Guaranteed O(n log n) with O(1) SpaceInsertion Sort — The O(n) Best Case Sort That Powers PythonMerge Sort — Complete Guide to the Guaranteed O(n log n) Stable SortQuick Sort — The Fastest Practical Sorting Algorithm with Complete AnalysisRadix Sort — O(nk) Digit-by-Digit Sorting for Large Integer RangesSelection Sort — Complete Analysis with Swaps Minimization PropertySorting Algorithm Comparison — The Complete Decision GuideSorting — Complete Introduction to Sorting AlgorithmsBinary Search — Complete Guide with All Variants and Proof of CorrectnessInterpolation Search — O(log log n) Search for Uniformly Distributed DataLinear Search — Complete Guide with All Variants and ApplicationsSearch Algorithm Comparison — Choosing the Right Search for Every SituationSearching Interview Problems — 10 Classic Problems with Complete SolutionsTernary Search — Finding Maximum/Minimum of Unimodal FunctionsBacktracking Introduction — The Recursive Exploration with UndoRecursion Basics — The Complete BeginnerRecursion Problems — 12 Classic Problems from Basic to AdvancedRecursion Tree — Visualizing Recursion for Complexity AnalysisRecursion vs Iteration — When to Use Each and How to ConvertTail Recursion — Definition, Optimization, and Converting to IterationDP Interview Problems — 12 Classic Problems with Pattern Recognition GuideDynamic Programming — Complete Introduction with Properties and Problem IdentificationFibonacci DP — From Exponential to Linear with Memoization and Matrix ExponentiationKnapsack Problem — 0/1, Unbounded, and Fractional with Complete AnalysisLongest Common Subsequence — Complete Guide with DP Table and String VariantsLongest Increasing Subsequence — O(n log n) with Patience SortingMatrix Chain Multiplication — Interval DP with Complete AnalysisMemoization — Top-Down Dynamic Programming with CachingTabulation — Bottom-Up Dynamic Programming with Complete Table BuildingActivity Selection Problem — Greedy Scheduling with Proof of CorrectnessFractional Knapsack — Greedy by Value Density with ProofGreedy Interview Problems — 10 Classic Problems with Complete SolutionsGreedy Algorithms — Complete Introduction with Proof TechniquesHuffman Coding — Optimal Prefix-Free Data CompressionJob Sequencing Problem — Greedy Scheduling with DeadlinesBacktracking — The Complete Guide to Recursive Exploration with UndoBacktracking Problems — 12 Classic Interview Problems with Complete SolutionsN-Queens Problem — Complete Solution with Backtracking and OptimizationRat in a Maze — Backtracking Path Finding with All SolutionsSubset Generation — All Subsets, Combinations, and Partitions with BacktrackingSudoku Solver — Backtracking with Constraint PropagationCodeforces Guide — Complete Strategy to Reach 1800+ RatingCommon CP Techniques — Essential Algorithms Every Competitive Programmer Must KnowContest Strategies — How to Perform Your Best in Competitive Programming ContestsCompetitive Programming — Complete Introduction for BeginnersFast Input/Output — Why I/O Speed Matters and Complete Templates for C++ and PythonLeetCode Guide — Complete Strategy to Crack FAANG InterviewsBit Manipulation — Complete Guide with All Tricks and Interview ProblemsBloom Filter — Space-Efficient Probabilistic Membership TestingComputational Geometry — Points, Lines, Convex Hull, and Geometric AlgorithmsDisjoint Set Union (DSU) — Union-Find with Path Compression and Union by RankRandomized Algorithms — Las Vegas, Monte Carlo, and Probabilistic TechniquesSkip List — The Probabilistic Data Structure that Competes with Balanced BSTsSuffix Array — Construction, LCP Array, and String Search ApplicationsTrie Data Structure — Complete Guide with Implementation and ApplicationsAutocomplete System Using TrieLibrary Management System Using DSARoute Finder Using GraphsMini Search Engine ProjectSocial Network Analysis Using GraphsEssential Coding Patterns for InterviewsDSA Cheatsheet: Quick ReferenceEssential DSA Interview QuestionsFAANG DSA Sheet: Company-Specific Patterns and ProblemsProblem Solving Strategies for Coding InterviewsComplexity CheatsheetImportant Algorithms SummaryBest Practice Platforms for DSARecommended Books for DSA
Complete guide to open addressing for hash collision resolution — linear probing with clustering analysis, quadratic probing, double hashing, tombstone deletion, the DELETED sentinel pattern, performance at different load factors, and Python dict
Open Addressing Fundamentals
In open addressing, all elements are stored in the hash table array itself — no external linked lists or trees. When a collision occurs, the algorithm probes a sequence of alternative slots until finding an empty one.
| Insert key=k1 (hash=3): slot 3 is empty | place at 3 |
| Insert key=k2 (hash=3): slot 3 occupied | probe slot 4 → empty → place at 4 |
| Insert key=k3 (hash=3): probe 3 | 4→5 → empty → place at 5 |
| Table | [_, _, _, k1, k2, k3, _] |
The probe sequence determines which slots are tried in order.
Linear Probing — Clustering Analysis
The major weakness of linear probing is primary clustering: long runs of occupied consecutive slots form, making probing slow.
Table (showing probe lengths for each element)
[3, 1, 4, 2, 5, 1, _, _, 1, 2, _]
↑ long run here
When inserting a new key that hashes to index 0
Must probe through the entire cluster: 0,1,2,3,4,5 = 6 probes!
The cluster grows longer with each insertion into or near it.
Knuth's analysis — expected probes for linear probing at load factor α:
| Successful lookup | (1 + 1/(1-α)²) / 2 ≈ 1/(2(1-α)) |
| Unsuccessful lookup | (1 + 1/(1-α)²) / 2 ≈ 1/(2(1-α))² |
| At α = 0.5 | ~2.5 probes (acceptable) |
| At α = 0.7 | ~5.8 probes (degrading) |
| At α = 0.9 | ~50 probes (terrible) |
Quadratic Probing
Example
class QuadraticProbingMap:
"""Probe sequence: h(k)+0², h(k)+1², h(k)+2², h(k)+3², ..."""
def __init__(self, capacity=17): # Prime capacity for better coverage
self.capacity = capacity
self.keys = [None] * capacity
self.values = [None] * capacity
def _probe(self, key):
start = hash(key) % self.capacity
for i in range(self.capacity):
yield (start + i * i) % self.capacity
def put(self, key, value):
for idx in self._probe(key):
if self.keys[idx] is None:
self.keys[idx] = key
self.values[idx] = value
return
if self.keys[idx] == key:
self.values[idx] = value
return
raise RuntimeError("Table full or probe cycle")
Quadratic probing reduces primary clustering by jumping farther and farther. However, it introduces secondary clustering: all keys with the same initial hash follow the same probe sequence.
Important constraint: With quadratic probing, not all slots are necessarily reached. To guarantee every slot is probed, the table size must be a prime and load factor < 0.5.
Double Hashing
Example
class DoubleHashingMap:
"""
Probe sequence: h1(k), h1(k)+h2(k), h1(k)+2*h2(k), ...
Each key has a unique step size h2(k), eliminating secondary clustering.
"""
def __init__(self, capacity=17): # Prime
self.capacity = capacity
self.keys = [None] * capacity
self.values = [None] * capacity
self.DELETED = object()
self.count = 0
def _h1(self, key):
return hash(key) % self.capacity
def _h2(self, key):
# Must be: 1 ≤ h2 ≤ capacity-1 and coprime with capacity
# For prime capacity: any 1..capacity-1 is coprime with capacity
return 1 + (hash(key) % (self.capacity - 1))
def _probe(self, key):
h1 = self._h1(key)
h2 = self._h2(key)
for i in range(self.capacity):
yield (h1 + i * h2) % self.capacity
def put(self, key, value):
if self.count / self.capacity > 0.5:
self._rehash()
first_deleted_idx = None
for idx in self._probe(key):
if self.keys[idx] is None:
ins = first_deleted_idx if first_deleted_idx is not None else idx
self.keys[ins] = key
self.values[ins] = value
self.count += 1
return
if self.keys[idx] is self.DELETED:
if first_deleted_idx is None:
first_deleted_idx = idx
elif self.keys[idx] == key:
self.values[idx] = value
return
def get(self, key, default=None):
for idx in self._probe(key):
if self.keys[idx] is None: return default
if self.keys[idx] != self.DELETED and self.keys[idx] == key:
return self.values[idx]
return default
def _rehash(self):
old_keys, old_vals = self.keys, self.values
self.capacity = next_prime(self.capacity * 2)
self.keys = [None] * self.capacity
self.values = [None] * self.capacity
self.count = 0
for k, v in zip(old_keys, old_vals):
if k is not None and k is not self.DELETED:
self.put(k, v)
def next_prime(n):
"""Find next prime ≥ n."""
def is_prime(x):
if x < 2: return False
for i in range(2, int(x**0.5)+1):
if x % i == 0: return False
return True
while not is_prime(n): n += 1
return n
Double hashing distribution:
- Each key has a unique probe sequence (determined by h2)
- No clustering of any kind
- Uniformly distributed probing
- Best theoretical performance among open addressing methods
Probe Sequence Comparison
| Method | Probe Sequence | Clustering | Table Size Required |
|---|---|---|---|
| Linear | h+0, h+1, h+2, ... | Primary | Any |
| Quadratic | h+0, h+1, h+4, h+9 | Secondary | Prime, α<0.5 |
| Double Hash | h, h+h2, h+2h2, ... | None | Prime |
Expected Probes at Different Load Factors
Example
Load factor (α) → Expected probes for unsuccessful search:
Linear Quadratic Double Hash
α = 0.1 1.06 1.06 1.06
α = 0.3 1.31 1.21 1.19
α = 0.5 2.50 1.88 1.50 (= 1/(1-α))
α = 0.7 5.08 2.79 2.00
α = 0.9 50.5 5.38 5.26
At α = 0.5, double hashing is optimal: exactly 1/(1-α) = 2.0 probes. Linear probing degrades much faster due to clustering.
Python Dict's Probing Strategy
Python's dict uses a perturbation-based scheme that's more complex than simple linear probing:
python example
# Simplified version of Python's probe sequence
def python_like_probe(key, size):
idx = hash(key) % size
perturb = hash(key)
while True:
yield idx
perturb >>= 5
idx = (idx * 5 + perturb + 1) % sizeThis achieves good distribution while being cache-friendly and fast for the common case where the first probe succeeds.
Summary
| Method | Avg probes (α=0.5) | Avg probes (α=0.8) | Clustering | Memory |
|---|---|---|---|---|
| Linear Probing | ~2.5 | ~8.0 | Primary | O(1) per entry |
| Quadratic | ~1.9 | ~4.0 | Secondary | O(1) per entry |
| Double Hashing | ~1.5 | ~2.5 | None | O(1) per entry |
Open addressing advantages over chaining:
- Better cache performance (all data in one array)
- Lower memory overhead (no pointer per entry)
- Better for small key-value pairs that fit in cache lines
Open addressing disadvantages:
- Load factor must stay below ~0.7
- Deletion requires tombstones (complicates logic)
- Clustering degrades performance as table fills
*Next Lesson: Hashing Problems*
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Open Addressing — Linear Probing, Quadratic Probing, and Double Hashing.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Data Structures & Algorithms topic.
Search Terms
data-structures-algorithms, data structures & algorithms, data, structures, algorithms, hashing, open, addressing
Related Data Structures & Algorithms Topics
Continue learning this concept
HashingSeparate Chaining — Complete Implementation and AnalysisMaster separate chaining for hash collision resolution — complete implementation using linked lists, array-backed lists, and balanced BSTs as chain structures, formal average-case analysis, Java HashMapHashingCollision Handling — Chaining vs Open Addressing Complete ComparisonMaster all collision resolution strategies for hash tables — separate chaining with linked lists, open addressing (linear probing, quadratic probing, double hashing), comparison of all methods, load factor effects, and when to use each approach.HashingHash Functions — Design, Properties, and Common ImplementationsA complete guide to hash functions — the mathematical properties of a good hash function, common hashing methods (division, multiplication, universal hashing), hash functions for strings and objects, cryptographic vs non-cryptographic hashes, and PythonHashingHashing — Complete Introduction to Hash Tables and Hash FunctionsA complete introduction to hashing — what a hash table is, how hash functions work, why hash tables achieve O(1) average operations, the load factor, the difference between hash maps and hash sets, and real-world applications of hashing in databases, security, and caching.HashingHashing Problems — 10 Classic Interview Problems with Complete SolutionsSolve 10 classic hashing interview problems — two sum, group anagrams, longest consecutive sequence, subarray sum equals k, top k frequent elements, valid anagram, isomorphic strings, word pattern, 4-sum count, and design a hashmap from scratch.Advanced TopicsBit Manipulation — Complete Guide with All Tricks and Interview ProblemsMaster bit manipulation completely — bitwise operators with truth tables, bit shifting, common tricks for setting/clearing/toggling bits, XOR magic properties, counting set bits (Brian Kernighan), power of two checks, swap without temp variable, and 15 classic interview problems solved with bit manipulation.