CSE-250 Fall 2022 - Section B - Inductive Proofs, Divide and Conquer

Inductive Proofs, Divide and Conquer

CSE-250 Fall 2022 - Section B

Sept 26, 2022

Textbook: Ch. 15

Warm up...

Fibonacci

What's the complexity? (in terms of n)


    def fibb(n: Int): Long =
      if(n < 2){ 1 }
      else { fibb(n-1) + fibb(n-2) }

Fibonacci

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n < 2\\ T(n-1) + T(n-2) + \Theta(1) & \textbf{otherwise} \end{cases}$$

Test Hypothesis: $T(n) \in O(2^n)$

Divide and Conquer

Remember the Towers of Hanoi...

You can move $n$ blocks, if you know how to move $n-1$ blocks
You can move $n-1$ blocks, if you know how to move $n-2$ blocks
You can move $n-2$ blocks, if you know how to move $n-3$ blocks
You can move $n-3$ blocks, if you know how to move $n-4$ blocks

...

You can always move $1$ block

Divide and Conquer

To solve the problem at $n$:

Divide the problem into a problem at $n-1$
Conquer the size $n-1$ problems
Combine the size $n-1$ solutions

Merge Sort

Input: An array with elements in unknown order.

Output: An array with elements in sorted order.

Merge Sort

Observation: Merging two sorted arrays can be done in $O(n)$.

Merge Sort


  def merge[A: Ordering](left: Seq[A], right: Seq[A]): Seq[A] = {
    val output = ArrayBuffer[A]()
  
    val leftItems = left.iterator.buffered
    val rightItems = right.iterator.buffered
  
    while(leftItems.hasNext || rightItems.hasNext) {
      if(!left.hasNext)          { output.append(right.next) }
      else if(!right.hasNext)    { output.append(left.next) }
      else if(Ordering[A].lt( left.head, right.head ))
                                 { output.append(left.next) }
      else                       { output.append(right.next) }
    }
    output.toSeq
  }

Merge Sort

Each time though loop advances either left or right.

Total Runtime: $\Theta(|\texttt{left}| + |\texttt{right}|)$

Merge Sort

Observation: Merging two sorted arrays can be done in $O(n)$.

Idea: Split the input in half, sort each half, and merge.

Merge Sort

... but how do you sort each half?

Merge Sort

Divide and Conquer

To solve the problem at $n$:

Divide the problem into two problems of size $\frac{n}{2}$
Conquer the size $\frac{n}{2}$ problems
Combine the size $\frac{n}{2}$ solutions

Merge Sort

If the sequence has $1$ or $0$ values: Sorted!
If the sequence has $n > 1$ values:
1. Sort values $1$ to $\left\lfloor\frac{n}{2}\right\rfloor-1$
2. Sort values $\left\lfloor\frac{n}{2}\right\rfloor$ to $n$
3. Merge sorted halves

Merge Sort


    def sort[A: Ordering](data: Seq[A]): Seq[A] = 
    {
      if(data.length <= 1) { return data }
      else {
        val (left, right) = data.splitAt(data.length / 2)
        return merge(
          sort(left),
          sort(right)
        )
      }
    }

Divide and Conquer

If we solve a problem of size $n$ by:

Dividing it into $a$ sub-problems
...where each problem is of size $\frac{n}{b}$ (usually $b=a$)
...and stop recurring at $n \leq c$
...and the cost of dividing is $D(n)$
...and the cost of combining is $C(n)$

The total cost will be: $$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq c \\ a\cdot T(\frac{n}{b}) + D(n) + C(n) & \textbf{otherwise} \end{cases}$$

Merge Sort

Divide: Split the sequence in half: $D(n) = \Theta(n)$ (can do in $\Theta(1)$)
Conquer: Sort left and right halves: $a = 2$, $b = 2$, $c = 1$
Combine: Merge halves together: $C(n) = \Theta(n)$

Merge Sort

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + \Theta(1) + \Theta(n) & \textbf{otherwise} \end{cases}$$

How can we find a closed-form hypothesis?

Idea: Draw out the cost of each level of recursion.

Merge Sort: Recursion Tree

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + \Theta(1) + \Theta(n) & \textbf{otherwise} \end{cases}$$

Each node of the tree shows $D(n) + C(n)$

Merge Sort: Recursion Tree

At level $i$ there are $2^i$ tasks, each with runtime $\Theta(\frac{n}{2^i})$,
and there are $\log(n)$ levels.

Merge Sort: Recursion Tree

At level $i$ there are $2^i$ tasks, each with runtime $\Theta(\frac{n}{2^i})$,
and there are $\log(n)$ levels.

$\sum_{i=0}^{\log(n)}$ $\sum_{j=1}^{2^i}$ $\Theta(\frac{n}{2^i})$

Merge Sort: Recursion Tree

$$\sum_{i=0}^{\log(n)} \sum_{j=1}^{2^i} \Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} (2^i+1-1)\Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} 2^i\Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} \Theta(n)$$

$$(\log(n) - 0 + 1) \Theta(n)$$

$$\Theta(n\log(n)) + \Theta(n)$$

$$\Theta(n\log(n))$$

Merge Sort: Proof By Induction

Now use induction to prove that there is a $c, n_0$
such that $T(n) \leq c \cdot n\log(n)$ for any $n > n_0$

$$T(n) = \begin{cases} c_0 & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + c_1 + c_2\cdot n & \textbf{otherwise} \end{cases}$$

Merge Sort: Proof By Induction

Base Case: $T(1) \leq c \cdot 1$

$$c_0 \leq c$$

True for any $c > c_0$

Merge Sort: Proof By Induction

Assume: $T(\frac{n}{2}) \leq c \frac{n}{2} \log\left(\frac{n}{2}\right)$

Show: $T(n) \leq c n \log\left(n\right)$

$$2\cdot T(\frac{n}{2}) + c_1 + c_2 n \leq c n \log(n)$$

By the assumption and transitivity, showing the following inequality suffices:
$$2 c \frac{n}{2} \log\left(\frac{n}{2}\right) + c_1 + c_2 n \leq c n \log(n)$$

$$c n \log(n) - c n \log(2) + c_1 + c_2 n \leq c n \log(n)$$

$$c_1 + c_2 n \leq c n \log(2)$$

$$\frac{c_1}{n \log(2)} + \frac{c_2}{\log(2)} \leq c$$

True for any $n_0 \geq \frac{c_1}{\log(2)}$ and $c > \frac{c_2}{\log(2)}+1$

Next time...

Quick Sort
Average Runtime