CSE-250 Fall 2022 - Section B - Inductive Proofs, Divide and Conquer

### Inductive Proofs, Divide and Conquer

Sept 26, 2022

Warm up...

#### Fibonacci

What's the complexity? (in terms of n)


def fibb(n: Int): Long =
if(n < 2){ 1 }
else { fibb(n-1) + fibb(n-2) }


#### Fibonacci

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n < 2\\ T(n-1) + T(n-2) + \Theta(1) & \textbf{otherwise} \end{cases}$$

Test Hypothesis: $T(n) \in O(2^n)$

#### Divide and Conquer

Remember the Towers of Hanoi...

1. You can move $n$ blocks, if you know how to move $n-1$ blocks
2. You can move $n-1$ blocks, if you know how to move $n-2$ blocks
3. You can move $n-2$ blocks, if you know how to move $n-3$ blocks
4. You can move $n-3$ blocks, if you know how to move $n-4$ blocks

...

• You can always move $1$ block

#### Divide and Conquer

To solve the problem at $n$:

• Divide the problem into a problem at $n-1$
• Conquer the size $n-1$ problems
• Combine the size $n-1$ solutions

#### Merge Sort

Input: An array with elements in unknown order.

Output: An array with elements in sorted order.

#### Merge Sort

Observation: Merging two sorted arrays can be done in $O(n)$.

#### Merge Sort


def merge[A: Ordering](left: Seq[A], right: Seq[A]): Seq[A] = {
val output = ArrayBuffer[A]()

val leftItems = left.iterator.buffered
val rightItems = right.iterator.buffered

while(leftItems.hasNext || rightItems.hasNext) {
if(!left.hasNext)          { output.append(right.next) }
else if(!right.hasNext)    { output.append(left.next) }
{ output.append(left.next) }
else                       { output.append(right.next) }
}
output.toSeq
}


#### Merge Sort

Each time though loop advances either left or right.

Total Runtime: $\Theta(|\texttt{left}| + |\texttt{right}|)$

#### Merge Sort

Observation: Merging two sorted arrays can be done in $O(n)$.

Idea: Split the input in half, sort each half, and merge.

#### Merge Sort

... but how do you sort each half?

#### Divide and Conquer

To solve the problem at $n$:

• Divide the problem into two problems of size $\frac{n}{2}$
• Conquer the size $\frac{n}{2}$ problems
• Combine the size $\frac{n}{2}$ solutions

#### Merge Sort

1. If the sequence has $1$ or $0$ values: Sorted!
2. If the sequence has $n > 1$ values:
1. Sort values $1$ to $\left\lfloor\frac{n}{2}\right\rfloor-1$
2. Sort values $\left\lfloor\frac{n}{2}\right\rfloor$ to $n$
3. Merge sorted halves

#### Merge Sort


def sort[A: Ordering](data: Seq[A]): Seq[A] =
{
if(data.length <= 1) { return data }
else {
val (left, right) = data.splitAt(data.length / 2)
return merge(
sort(left),
sort(right)
)
}
}


#### Divide and Conquer

If we solve a problem of size $n$ by:

• Dividing it into $a$ sub-problems
• ...where each problem is of size $\frac{n}{b}$ (usually $b=a$)
• ...and stop recurring at $n \leq c$
• ...and the cost of dividing is $D(n)$
• ...and the cost of combining is $C(n)$

The total cost will be: $$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq c \\ a\cdot T(\frac{n}{b}) + D(n) + C(n) & \textbf{otherwise} \end{cases}$$

#### Merge Sort

Divide: Split the sequence in half
$D(n) = \Theta(n)$ (can do in $\Theta(1)$)
Conquer: Sort left and right halves
$a = 2$, $b = 2$, $c = 1$
Combine: Merge halves together
$C(n) = \Theta(n)$

#### Merge Sort

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + \Theta(1) + \Theta(n) & \textbf{otherwise} \end{cases}$$

How can we find a closed-form hypothesis?

Idea: Draw out the cost of each level of recursion.

#### Merge Sort: Recursion Tree

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + \Theta(1) + \Theta(n) & \textbf{otherwise} \end{cases}$$

Each node of the tree shows $D(n) + C(n)$

#### Merge Sort: Recursion Tree

At level $i$ there are $2^i$ tasks, each with runtime $\Theta(\frac{n}{2^i})$,
and there are $\log(n)$ levels.

#### Merge Sort: Recursion Tree

At level $i$ there are $2^i$ tasks, each with runtime $\Theta(\frac{n}{2^i})$,
and there are $\log(n)$ levels.

$\sum_{i=0}^{\log(n)}$ $\sum_{j=1}^{2^i}$ $\Theta(\frac{n}{2^i})$

#### Merge Sort: Recursion Tree

$$\sum_{i=0}^{\log(n)} \sum_{j=1}^{2^i} \Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} (2^i+1-1)\Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} 2^i\Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} \Theta(n)$$

$$(\log(n) - 0 + 1) \Theta(n)$$

$$\Theta(n\log(n)) + \Theta(n)$$

$$\Theta(n\log(n))$$

#### Merge Sort: Proof By Induction

Now use induction to prove that there is a $c, n_0$
such that $T(n) \leq c \cdot n\log(n)$ for any $n > n_0$

$$T(n) = \begin{cases} c_0 & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + c_1 + c_2\cdot n & \textbf{otherwise} \end{cases}$$

#### Merge Sort: Proof By Induction

Base Case: $T(1) \leq c \cdot 1$

$$c_0 \leq c$$

True for any $c > c_0$

#### Merge Sort: Proof By Induction

Assume: $T(\frac{n}{2}) \leq c \frac{n}{2} \log\left(\frac{n}{2}\right)$

Show: $T(n) \leq c n \log\left(n\right)$

$$2\cdot T(\frac{n}{2}) + c_1 + c_2 n \leq c n \log(n)$$

By the assumption and transitivity, showing the following inequality suffices:
$$2 c \frac{n}{2} \log\left(\frac{n}{2}\right) + c_1 + c_2 n \leq c n \log(n)$$

$$c n \log(n) - c n \log(2) + c_1 + c_2 n \leq c n \log(n)$$

$$c_1 + c_2 n \leq c n \log(2)$$

$$\frac{c_1}{n \log(2)} + \frac{c_2}{\log(2)} \leq c$$

True for any $n_0 \geq \frac{c_1}{\log(2)}$ and $c > \frac{c_2}{\log(2)}+1$

#### Next time...

• Quick Sort
• Average Runtime