CSE-250 Fall 2022 - Section B - Average Runtime, Stacks, Queues

### Average Runtime, Stacks, Queues

Sept 30, 2022

#### QuickSort

1. Pick a value at random as a $pivot$.
2. Swap values until the array is subdivided into...
• $low$: array elements that are $\leq pivot$
• $pivot$
• $high$: array elements that are $> pivot$
3. Recursively sort $low$
4. Recursively sort $high$

What's the worst-case runtime?

$$T_{quicksort}(n) \in O(n^2)$$

... but this isn't really representative of typical behavior

Let's talk probabilities

If $X$ represents a random (numerical) outcome, the average over all possibilities is the expectation of $X$, or $E[X]$

$$E[X] = \sum_{i} P_i \cdot X_i$$

1. $E[X + Y] = E[X] + E[Y]$ (always)
2. $E[X \cdot Y] = E[X] \cdot E[Y]$ (If $X$ and $Y$ are independent)
$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ T(0) + T(n-1) + \Theta(n) & \textbf{if } n > 1 \wedge X = 1\\ T(1) + T(n-2) + \Theta(n) & \textbf{if } n > 1 \wedge X = 2\\ T(2) + T(n-3) + \Theta(n) & \textbf{if } n > 1 \wedge X = 3\\ ..\\ T(n-2) + T(1) + \Theta(n) & \textbf{if } n > 1 \wedge X = n-1\\ T(n-1) + T(0) + \Theta(n) & \textbf{if } n > 1 \wedge X = n\\ \end{cases}$$

We pick the $X$th largest element as a pivot

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ E[T(X-1) + T(n-X)] + \Theta(n) & \textbf{otherwise} \end{cases}$$

We pick the $X$th largest element as a pivot

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ E[T(X-1)] + E[T(n-X)] + \Theta(n) & \textbf{otherwise} \end{cases}$$

$$E[T(X-1)]$$

$= \sum_{i=1}^{n} P_i \cdot T(X_i-1)$

$= \sum_{i=1}^{n} \frac{1}{n} \cdot T(i-1)$ ($T(0)$ up to $T(n-1)$)

$= \sum_{i=1}^{n} \frac{1}{n} \cdot T(n-i)$ ($T(n-1)$ down to $T(0)$)

$= E[T(n-X)]$

We pick the $X$th largest element as a pivot

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ 2 E[T(X-1)] + \Theta(n) & \textbf{otherwise} \end{cases}$$

Each $T(X-1)$ is independent.

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ 2 \left(\sum_{i=1}^{n} \frac{1}{n} E[T(i-1)] \right) + \Theta(n) & \textbf{otherwise} \end{cases}$$

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ \frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + \Theta(n) & \textbf{otherwise} \end{cases}$$

Back to induction...

Hypothesis: $E[T(n)] \in O(n \log(n))$

Base Case: $E[T(1)] \leq c(1 \log(1))$

$$E[T(1)] \leq c \cdot (1 \cdot 0)$$

$$E[T(1)] \not \leq 0$$

Base Case (take Two): $E[T(2)] \leq c(2 \log(2))$

$$2\cdot E_i[T(i-1)] + 2c_1 \leq 2c$$

$$2\cdot \left(\frac{1}{2}T(0) + \frac{1}{2}T(1)\right) + 2c_1 \leq 2c$$

$$T(0) + T(1) + 2c_1 \leq 2c$$

$$2c_0 + 2c_1 \leq 2c$$

True for any $c \geq c_0 + c_1$

Assume: $E[T(n')] \leq c(n' \log(n'))$ for all $n' < n$

Show: $E[T(n)] \leq c(n \log(n))$

$$\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + c_1 \leq c n \log(n)$$

$$\frac{2}{n}\left(\sum_{i=0}^{n-1} c i \log(i) \right) + c_1 \leq c n \log(n)$$

$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$

$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$

$$c\frac{2 \log(n)}{n}\left(\sum_{i=0}^{n-1} i \right) + c_1 \leq c n \log(n)$$

$$c\frac{2 \log(n)}{n}\left( \frac{(n-1)(n-1+1)}{2}\right) + c_1 \leq c n \log(n)$$

$$c\frac{\log(n)}{n}\left(n^2 - n\right) + c_1 \leq c n \log(n)$$

$$cn\log(n) - c\log(n) + c_1 \leq c n \log(n)$$

$$c_1 \leq c\log(n)$$

$E[T_{quicksort}(n)] = O(n\log(n))$

So is Quicksort $O(n\log(n))$? No!

#### What guarantees do you get?

$f(n)$ is a Tight Bound
The algorithm always runs in exactly $cf(n)$ steps
$f(n)$ is a Worst-Case Bound
The algorithm always runs at-most $cf(n)$ steps
$f(n)$ is an Amortized Worst-Case Bound
$n$ invocations of the algorithm always run in $cnf(n)$ steps
$f(n)$ is an Average Bound
ðŸ¤·

#### mutable.Seq

mutable.IndexedSeq (e.g., Array)
Efficient apply(), update()
mutable.Buffer (e.g., ArrayBuffer, ListBuffer)
Efficient apply(), update(), append()
mutable.Stack
Efficient push(), pop(), top()
mutable.Queue

#### Stacks

A stack of objects on top of one another.

Push
Put a new object on top of the stack
Pop
Remove the object on top of the stack
Top
Peek at what's on top of the stack

#### Stacks in Practice

• Storing function variables in a "call stack"
• Certain types of parsers ("context free")
• Backtracking search
• Reversing sequences

#### Stacks


trait Stack[A] {
def push(element: A): Unit
def top: A
def pop: A
}


#### ListStack


class ListStack[A] extends Stack[A] {

def push(element: A): Unit =
_store.prepend(element)

def top: A =

def pop: A =
_store.remove(0)
}


What's the runtime?

#### ArrayBufferStack


class ArrayBufferStack[A] extends Stack[A] {
val _store = new ArrayBuffer()

def push(element: A): Unit =
_store.append(element)

def top: A =
_store.last

def pop: A =
_store.remove(store.length-1)
}


What's the runtime?

#### Stack

Scala's Stack implementation is based on ArrayBuffer (ArrayDequeue); Keeping memory together is worth the overhead of amortized $O(1)$.

#### Queue

Outside of the US, "queueing" is lining up.

Enqueue
Put a new object at the end of the queue
Dequeue
Remove the next object in the queue
Peek at the next object in the queue

#### Queues vs Stacks

Queue
First in, First out (FIFO)
Stack
Last in, First Out (LIFO / FILO)

#### Queues in Practice

• Delivering network packets, emails, twokstagrams
• Scheduling CPU Cycles

#### Queues


trait Queue[A] {
def enqueue(element: A): Unit
def dequeue: A
}


#### ListQueue


class ListQueue[A] extends Queue[A] {

def enqueue(element: A): Unit =
_store.append(element)