DSA Notes
HubAlgorithm vs Data Structure — Key Differences, Definitions, and How They Work TogetherApplications of DSA — How Data Structures and Algorithms Power Real-World SoftwareTime and Space Complexity — A Complete BeginnerWhat is DSA? — A Complete BeginnerWhy Learn DSA? — Career Benefits, Real-World Impact, and Long-Term ValueAsymptotic Analysis — Complete Framework with Recurrence Relations and Master TheoremBest, Average, and Worst Case Analysis — A Practical Guide for Every AlgorithmBig O Notation — Complete Guide with Formal Definition, Rules, and ExamplesBig Omega Notation — Lower Bounds, Best Case, and Formal DefinitionBig Theta (Θ) Notation — Tight Bounds and Exact Growth RatesSpace Complexity — Memory Analysis, Auxiliary Space, and Time-Space TradeoffsTime Complexity Examples — 25 Worked Problems from Basic to AdvancedArray Interview Problems — 15 Classic Problems with Complete SolutionsArray Operations — Every Operation Explained with Code, Complexity, and Edge CasesArray Rotation — Algorithms, Techniques, and Interview ProblemsArray Searching — Linear Search, Binary Search, and Advanced TechniquesArrays — The Complete Introduction to the Most Fundamental Data StructureDynamic Arrays — How Python Lists and ArrayLists Work InternallyMultidimensional Arrays — 2D Arrays, Matrices, and Memory LayoutSliding Window Technique — Complete Guide with 10 Classic ProblemsAnagram Problems — Detection, Grouping, and Sliding Window VariationsPalindrome Problems — Check, Longest, Count, and Expand Around CenterPattern Searching — KMP, Rabin-Karp, and Boyer-Moore AlgorithmsString Interview Problems — 10 Classic Problems with Complete SolutionsString Matching — Longest Common Subsequence, Edit Distance, and Z-AlgorithmString Operations — Complete Reference with Complexity and ImplementationStrings — Complete Introduction to String Data Structure and OperationsCircular Linked List — Structure, Operations, and ApplicationsDoubly Linked List — Complete Implementation with Bidirectional OperationsLinked List Interview Problems — 12 Classic Problems with Complete SolutionsLinked Lists — A Complete Introduction with Memory Model and TypesLinked List Operations — Every Technique with Code and ComplexityReverse Linked List — All Approaches from Basic to AdvancedSingly Linked List — Complete Implementation with All OperationsBalanced Parentheses — Stack-Based Validation with All VariantsInfix, Prefix, and Postfix Expressions — Stack-Based Conversion and EvaluationStack Implementation — Three Complete Implementations with Edge CasesStack Interview Problems — 10 Classic Problems with Complete SolutionsStack — Complete Introduction to LIFO Data StructureStack Using Arrays — Memory Layout, Capacity Management, and PerformanceStack Using Linked List — Implementation, Analysis, and When to Use ItCircular Queue — The Ring Buffer Explained with ImplementationDeque — Double-Ended Queue with Complete Implementation and ApplicationsPriority Queue — Complete Guide from Concept to ImplementationQueue Applications — BFS, Level-Order Traversal, 01-BFS, and MoreQueue Implementation — Array, Linked List, and Two-Stack ApproachesQueue Interview Problems — 10 Classic Problems with Complete SolutionsQueue — Complete Introduction to the FIFO Data StructureAVL Tree - Self-Balancing Binary Search TreeBinary Search Tree (BST)Binary Tree - Structure and OperationsFenwick Tree (Binary Indexed Tree)Red-Black Tree - Another Balanced BSTSegment Tree - Range Query Data StructureTree Interview Problems CollectionTree Data Structure FundamentalsTree Traversals - Inorder, Preorder, Postorder, Level OrderHeap — Complete Introduction to the Priority-Based Tree Data StructureHeap Operations — Complete Reference with Complexity AnalysisHeap Problems — 8 Classic Interview Problems with Complete SolutionsMax Heap — Complete Implementation and ApplicationsMin Heap — Complete Implementation with All OperationsPriority Queue Using Heap — Implementation, Applications, and PatternsBellman-Ford Algorithm — Shortest Path with Negative Weights and Cycle DetectionBFS Algorithm — Breadth-First Search with All Applications and VariantsDFS Algorithm — Depth-First Search with Cycle Detection and All ApplicationsDijkstraFloyd-Warshall Algorithm — All-Pairs Shortest Paths with Complete AnalysisGraph Interview Problems — 12 Classic Problems with Complete SolutionsGraph — Complete Introduction to the Most Powerful Data StructureGraph Representation — Adjacency List, Matrix, Edge List with Complete AnalysisKruskalMinimum Spanning Tree — Definition, Properties, and ApplicationsPrimShortest Path Algorithms — Complete Comparison and Selection GuideTopological Sorting — DFS and BFS (KahnUnion-Find (DSU) in Graphs — Connected Components and Dynamic ConnectivitySeparate Chaining — Complete Implementation and AnalysisCollision Handling — Chaining vs Open Addressing Complete ComparisonHash Functions — Design, Properties, and Common ImplementationsHashing — Complete Introduction to Hash Tables and Hash FunctionsHashing Problems — 10 Classic Interview Problems with Complete SolutionsOpen Addressing — Linear Probing, Quadratic Probing, and Double HashingBubble Sort — Complete Analysis with Optimizations and When to Use ItCounting Sort — O(n+k) Sorting That Breaks the Comparison Lower BoundHeap Sort — Guaranteed O(n log n) with O(1) SpaceInsertion Sort — The O(n) Best Case Sort That Powers PythonMerge Sort — Complete Guide to the Guaranteed O(n log n) Stable SortQuick Sort — The Fastest Practical Sorting Algorithm with Complete AnalysisRadix Sort — O(nk) Digit-by-Digit Sorting for Large Integer RangesSelection Sort — Complete Analysis with Swaps Minimization PropertySorting Algorithm Comparison — The Complete Decision GuideSorting — Complete Introduction to Sorting AlgorithmsBinary Search — Complete Guide with All Variants and Proof of CorrectnessInterpolation Search — O(log log n) Search for Uniformly Distributed DataLinear Search — Complete Guide with All Variants and ApplicationsSearch Algorithm Comparison — Choosing the Right Search for Every SituationSearching Interview Problems — 10 Classic Problems with Complete SolutionsTernary Search — Finding Maximum/Minimum of Unimodal FunctionsBacktracking Introduction — The Recursive Exploration with UndoRecursion Basics — The Complete BeginnerRecursion Problems — 12 Classic Problems from Basic to AdvancedRecursion Tree — Visualizing Recursion for Complexity AnalysisRecursion vs Iteration — When to Use Each and How to ConvertTail Recursion — Definition, Optimization, and Converting to IterationDP Interview Problems — 12 Classic Problems with Pattern Recognition GuideDynamic Programming — Complete Introduction with Properties and Problem IdentificationFibonacci DP — From Exponential to Linear with Memoization and Matrix ExponentiationKnapsack Problem — 0/1, Unbounded, and Fractional with Complete AnalysisLongest Common Subsequence — Complete Guide with DP Table and String VariantsLongest Increasing Subsequence — O(n log n) with Patience SortingMatrix Chain Multiplication — Interval DP with Complete AnalysisMemoization — Top-Down Dynamic Programming with CachingTabulation — Bottom-Up Dynamic Programming with Complete Table BuildingActivity Selection Problem — Greedy Scheduling with Proof of CorrectnessFractional Knapsack — Greedy by Value Density with ProofGreedy Interview Problems — 10 Classic Problems with Complete SolutionsGreedy Algorithms — Complete Introduction with Proof TechniquesHuffman Coding — Optimal Prefix-Free Data CompressionJob Sequencing Problem — Greedy Scheduling with DeadlinesBacktracking — The Complete Guide to Recursive Exploration with UndoBacktracking Problems — 12 Classic Interview Problems with Complete SolutionsN-Queens Problem — Complete Solution with Backtracking and OptimizationRat in a Maze — Backtracking Path Finding with All SolutionsSubset Generation — All Subsets, Combinations, and Partitions with BacktrackingSudoku Solver — Backtracking with Constraint PropagationCodeforces Guide — Complete Strategy to Reach 1800+ RatingCommon CP Techniques — Essential Algorithms Every Competitive Programmer Must KnowContest Strategies — How to Perform Your Best in Competitive Programming ContestsCompetitive Programming — Complete Introduction for BeginnersFast Input/Output — Why I/O Speed Matters and Complete Templates for C++ and PythonLeetCode Guide — Complete Strategy to Crack FAANG InterviewsBit Manipulation — Complete Guide with All Tricks and Interview ProblemsBloom Filter — Space-Efficient Probabilistic Membership TestingComputational Geometry — Points, Lines, Convex Hull, and Geometric AlgorithmsDisjoint Set Union (DSU) — Union-Find with Path Compression and Union by RankRandomized Algorithms — Las Vegas, Monte Carlo, and Probabilistic TechniquesSkip List — The Probabilistic Data Structure that Competes with Balanced BSTsSuffix Array — Construction, LCP Array, and String Search ApplicationsTrie Data Structure — Complete Guide with Implementation and ApplicationsAutocomplete System Using TrieLibrary Management System Using DSARoute Finder Using GraphsMini Search Engine ProjectSocial Network Analysis Using GraphsEssential Coding Patterns for InterviewsDSA Cheatsheet: Quick ReferenceEssential DSA Interview QuestionsFAANG DSA Sheet: Company-Specific Patterns and ProblemsProblem Solving Strategies for Coding InterviewsComplexity CheatsheetImportant Algorithms SummaryBest Practice Platforms for DSARecommended Books for DSA
Master dynamic programming from scratch — what DP really is, the two essential properties (overlapping subproblems and optimal substructure), how to identify a DP problem, the four-step DP recipe, comparison with divide-and-conquer and greedy, and a taxonomy of DP problem types.
What Is Dynamic Programming?
Dynamic Programming (DP) is an optimization technique that solves complex problems by breaking them into simpler overlapping subproblems, solving each subproblem only once, and storing the result to avoid redundant computation.
The name is somewhat misleading — it has nothing to do with "dynamic" in the programming sense. Richard Bellman coined the term in the 1950s, partly because "dynamic" sounded impressive and would secure funding from the US Department of Defense (which was paying for his research). The "programming" refers to planning or scheduling, not computer programming.
At its core, DP is about a single powerful idea: "If you've already solved a subproblem, don't solve it again."
DP vs Divide-and-Conquer
Both break problems into subproblems and combine solutions. The key difference:
| Divide-and-Conquer | Dynamic Programming | |
|---|---|---|
| Subproblems | Independent | Overlapping |
| Storage needed | No (each solved once) | Yes (cache results) |
| Classic examples | Merge Sort, Binary Search | Fibonacci, Knapsack |
| When to use | Independent partitions | Repeated subproblems |
Merge Sort divides an array into two halves — the left half and right half don't share elements. They're computed once each and combined. No overlap, no memoization needed.
Fibonacci needs fib(3) to compute both fib(4) and fib(5). Without storing fib(3), it gets recomputed exponentially many times.
DP vs Greedy
Both solve optimization problems. The difference is in how they make decisions:
| Greedy | Dynamic Programming | |
|---|---|---|
| Decision making | One locally-optimal choice at a time | Consider all choices, pick best overall |
| Revisit decisions | No — irrevocable | No revisits needed (but evaluates all) |
| Correctness guarantee | Only if greedy-choice property holds | Always correct if substructure holds |
| Speed | Usually faster | Usually slower but more general |
| Example | Activity selection, Huffman | 0/1 Knapsack, Edit Distance |
When to use greedy: If a locally optimal choice always leads to a globally optimal solution (greedy-choice property proven).
When to use DP: When locally optimal choices can lead to globally suboptimal results.
| Coin change | denominations = [1, 5, 6], target = 10 |
| Greedy: Take 6 first | remaining 4 → take 1, 1, 1, 1 = 5 coins |
| DP | Take 5 + 5 = 2 coins ← OPTIMAL |
The Four-Step DP Recipe
Every DP solution follows the same design process:
Step 1 — Define the State
A state completely describes a subproblem. The state tells you: "Given this information, what is the answer?"
| Fibonacci | state = the number n |
| 0/1 Knapsack | state = (item index, remaining capacity) |
| Longest Common Subsequence | state = (position in s1, position in s2) |
| Climbing Stairs | state = current step number |
Step 2 — Define the Recurrence (Transition)
How do smaller states combine to form the answer for a larger state?
Fibonacci
dp[n] = dp[n-1] + dp[n-2]
0/1 Knapsack (item i has weight w_i, value v_i):
dp[i][w] = max(
dp[i-1][w], # Don't take item i
dp[i-1][w - w_i] + v_i # Take item i (if w >= w_i)
)
LCS
if s1[i-1] == s2[j-1]:
dp[i][j] = dp[i-1][j-1] + 1
else:
dp[i][j] = max(dp[i-1][j], dp[i][j-1])
Step 3 — Define the Base Cases
What are the trivially known values that don't depend on any other state?
| Fibonacci | dp[0] = 0, dp[1] = 1 |
| Knapsack: dp[0][w] = 0 (no items | value 0) |
| dp[i][0] = 0 (capacity 0 | value 0) |
| LCS | dp[i][0] = 0 (empty s2) |
Step 4 — Determine the Final Answer
Which state contains the answer to the original problem?
| Fibonacci | dp[n] |
| Knapsack | dp[n][W] (n items, capacity W) |
| LCS | dp[m][n] (lengths of both strings) |
Two Approaches: Memoization vs Tabulation
Memoization (Top-Down DP)
Start with the original problem, recurse toward base cases, and cache results.
Example
def fibonacci_memo(n, memo={}):
"""Memoization: recursive with cache."""
if n in memo: return memo[n] # Cached result
if n <= 1: return n # Base case
memo[n] = fibonacci_memo(n-1, memo) + fibonacci_memo(n-2, memo)
return memo[n]
# Or using Python's built-in cache:
from functools import lru_cache
@lru_cache(maxsize=None)
def fibonacci(n):
if n <= 1: return n
return fibonacci(n-1) + fibonacci(n-2)
Pros of memoization:
- Natural to implement (just add cache to recursion)
- Only computes states that are actually needed
- Handles irregular state spaces
Cons:
- Recursion overhead
- Risk of stack overflow for large n
Tabulation (Bottom-Up DP)
Build the solution table starting from base cases and work toward the final answer.
Example
def fibonacci_tab(n):
"""Tabulation: iterative, bottom-up."""
if n <= 1: return n
dp = [0] * (n + 1)
dp[1] = 1
for i in range(2, n + 1):
dp[i] = dp[i-1] + dp[i-2]
return dp[n]
# Space-optimized (only need last 2 values):
def fibonacci_optimal(n):
if n <= 1: return n
prev, curr = 0, 1
for _ in range(2, n+1):
prev, curr = curr, prev + curr
return curr
Pros of tabulation:
- No recursion overhead
- No stack overflow risk
- Often more space-efficient (sliding window over table)
Identifying DP Problems
These signals in a problem statement strongly suggest DP:
Signal 1 — "Count the number of ways"
Example
"How many ways can you make change for n cents?"
"How many distinct paths are there from top-left to bottom-right?"
"How many unique BSTs can you build with n nodes?"
→ DP: count[state] = sum of counts from smaller states
Signal 2 — "Find the minimum/maximum"
Example
"Minimum number of coins to make amount n"
"Maximum profit from stock trades"
"Minimum edit distance between two strings"
→ DP: optimal[state] = best of choices from smaller states
Signal 3 — "Is it possible to..."
Example
"Can you partition the array into two equal-sum subsets?"
"Can you reach the last index?"
→ DP: possible[state] = true if any path leads to it
Signal 4 — Optimal choices depend on future consequences
Example
If making a greedy choice now can hurt you later,
DP explores all choices and picks the globally best one.
→ Coin change, knapsack, matrix chain multiplication
DP Problem Taxonomy
| Category | Classic Problems |
|---|---|
| 1D Linear DP | Fibonacci, Climbing Stairs, House Robber, Jump Game |
| 2D Grid DP | Unique Paths, Minimum Path Sum, Dungeon Game |
| String DP | LCS, LIS, Edit Distance, Palindrome Partitioning |
| Knapsack variants | 0/1 Knapsack, Unbounded Knapsack, Subset Sum |
| Interval DP | Matrix Chain Multiplication, Burst Balloons, Palindrome DP |
| Tree DP | House Robber III, Max Path Sum in Tree |
| Bitmask DP | Traveling Salesman, Assignment Problem |
| Digit DP | Count numbers with property in range [a, b] |
Simple First Example — Climbing Stairs
Example
def climb_stairs(n):
"""
Count ways to climb n stairs, taking 1 or 2 steps at a time.
State: dp[i] = number of ways to reach step i
Recurrence: dp[i] = dp[i-1] + dp[i-2] (arrive from step i-1 or i-2)
Base cases: dp[1] = 1, dp[2] = 2
Answer: dp[n]
"""
if n <= 2: return n
prev2, prev1 = 1, 2
for _ in range(3, n+1):
prev2, prev1 = prev1, prev1 + prev2
return prev1
print(climb_stairs(5)) # 8 ways
print(climb_stairs(10)) # 89 ways
This is structurally identical to Fibonacci — a perfect first DP problem.
Summary
Dynamic programming is the technique of caching solutions to overlapping subproblems to avoid recomputation:
Two required properties:
- Overlapping subproblems — same subproblem computed multiple times
- Optimal substructure — optimal solution built from optimal sub-solutions
The four-step recipe:
- Define state — what information describes a subproblem?
- Define recurrence — how do smaller states combine?
- Define base cases — what trivial states do you know?
- Find the answer — which state is the final answer?
Two implementation styles:
- Memoization (top-down): recursion + cache
- Tabulation (bottom-up): iteration, build table from base cases
DP feels difficult at first because identifying the right state definition is non-obvious. With practice, pattern recognition develops quickly. The next lessons cover memoization and tabulation in depth, then build through increasingly complex DP patterns.
*Next Lesson: Memoization*
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Dynamic Programming — Complete Introduction with Properties and Problem Identification.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Data Structures & Algorithms topic.
Search Terms
data-structures-algorithms, data structures & algorithms, data, structures, algorithms, dynamic, programming, introduction
Related Data Structures & Algorithms Topics
Continue learning this concept
Dynamic ProgrammingDP Interview Problems — 12 Classic Problems with Pattern Recognition GuideSolve 12 classic DP interview problems — maximum product subarray, decode ways, jump game II, unique paths with obstacles, minimum path sum, coin change II, target sum, longest palindromic substring, word break, stock problems, regular expression matching, and wildcard matching.Dynamic ProgrammingFibonacci DP — From Exponential to Linear with Memoization and Matrix ExponentiationMaster Fibonacci and Fibonacci-like problems with dynamic programming — naive recursion O(2^n), memoization O(n), tabulation O(n), space-optimized O(1) space, matrix exponentiation O(log n), and classic DP problems that reduce to Fibonacci variants like climbing stairs, house robber, and tile tiling.Dynamic ProgrammingKnapsack Problem — 0/1, Unbounded, and Fractional with Complete AnalysisMaster all knapsack variants — 0/1 knapsack with 2D and 1D DP tables, unbounded knapsack with reusable items, fractional knapsack with greedy, subset sum, partition equal subset sum, and the general framework for recognizing knapsack problems in interviews.Dynamic ProgrammingLongest Common Subsequence — Complete Guide with DP Table and String VariantsMaster Longest Common Subsequence (LCS) completely — definition with clear examples, 2D DP table construction, string reconstruction, space optimization, and all important variants including Longest Common Substring, Edit Distance, Shortest Common Supersequence, and real-world applications.Dynamic ProgrammingLongest Increasing Subsequence — O(n log n) with Patience SortingMaster Longest Increasing Subsequence (LIS) completely — the O(n²) DP solution, the O(n log n) patience sorting algorithm, reconstructing the actual LIS, variants including longest non-decreasing subsequence and bitonic subsequence, and applications in scheduling and sequence analysis.Dynamic ProgrammingMatrix Chain Multiplication — Interval DP with Complete AnalysisMaster matrix chain multiplication as the gateway to interval DP — why multiplication order affects cost, the O(n³) interval DP solution, building the DP table from small intervals to large, reconstructing optimal parenthesization, and other interval DP problems like burst balloons and palindrome partitioning.