CSE-250 Fall 2022 - Section B - Average Runtime, Stacks, Queues

Average Runtime, Stacks, Queues

CSE-250 Fall 2022 - Section B

Sept 30, 2022

Textbook: Ch. 15, Ch. 7

QuickSort

  1. Pick a value at random as a $pivot$.
  2. Swap values until the array is subdivided into...
    • $low$: array elements that are $\leq pivot$
    • $pivot$
    • $high$: array elements that are $> pivot$
  3. Recursively sort $low$
  4. Recursively sort $high$

What's the worst-case runtime?

$$T_{quicksort}(n) \in O(n^2)$$

... but this isn't really representative of typical behavior

Let's talk probabilities

If $X$ represents a random (numerical) outcome, the average over all possibilities is the expectation of $X$, or $E[X]$

$$E[X] = \sum_{i} P_i \cdot X_i$$

  1. $E[X + Y] = E[X] + E[Y]$ (always)
  2. $E[X \cdot Y] = E[X] \cdot E[Y]$ (If $X$ and $Y$ are independent)
$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ T(0) + T(n-1) + \Theta(n) & \textbf{if } n > 1 \wedge X = 1\\ T(1) + T(n-2) + \Theta(n) & \textbf{if } n > 1 \wedge X = 2\\ T(2) + T(n-3) + \Theta(n) & \textbf{if } n > 1 \wedge X = 3\\ ..\\ T(n-2) + T(1) + \Theta(n) & \textbf{if } n > 1 \wedge X = n-1\\ T(n-1) + T(0) + \Theta(n) & \textbf{if } n > 1 \wedge X = n\\ \end{cases}$$

We pick the $X$th largest element as a pivot

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ E[T(X-1) + T(n-X)] + \Theta(n) & \textbf{otherwise} \end{cases}$$

We pick the $X$th largest element as a pivot

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ E[T(X-1)] + E[T(n-X)] + \Theta(n) & \textbf{otherwise} \end{cases}$$

$$E[T(X-1)]$$

$= \sum_{i=1}^{n} P_i \cdot T(X_i-1)$

$= \sum_{i=1}^{n} \frac{1}{n} \cdot T(i-1)$ ($T(0)$ up to $T(n-1)$)

$= \sum_{i=1}^{n} \frac{1}{n} \cdot T(n-i)$ ($T(n-1)$ down to $T(0)$)

$= E[T(n-X)]$

We pick the $X$th largest element as a pivot

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ 2 E[T(X-1)] + \Theta(n) & \textbf{otherwise} \end{cases}$$

Each $T(X-1)$ is independent.

 

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ 2 \left(\sum_{i=1}^{n} \frac{1}{n} E[T(i-1)] \right) + \Theta(n) & \textbf{otherwise} \end{cases}$$

 

$$E[T(n)] = \begin{cases} \Theta(1) & \textbf{if } n \leq 1\\ \frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + \Theta(n) & \textbf{otherwise} \end{cases}$$

Back to induction...

Hypothesis: $E[T(n)] \in O(n \log(n))$

Base Case: $E[T(1)] \leq c(1 \log(1))$

$$E[T(1)] \leq c \cdot (1 \cdot 0)$$

$$E[T(1)] \not \leq 0$$

Base Case (take Two): $E[T(2)] \leq c(2 \log(2))$

$$2\cdot E_i[T(i-1)] + 2c_1 \leq 2c$$

$$2\cdot \left(\frac{1}{2}T(0) + \frac{1}{2}T(1)\right) + 2c_1 \leq 2c$$

$$T(0) + T(1) + 2c_1 \leq 2c$$

$$2c_0 + 2c_1 \leq 2c$$

True for any $c \geq c_0 + c_1$

Assume: $E[T(n')] \leq c(n' \log(n'))$ for all $n' < n$

Show: $E[T(n)] \leq c(n \log(n))$

$$\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + c_1 \leq c n \log(n)$$

$$\frac{2}{n}\left(\sum_{i=0}^{n-1} c i \log(i) \right) + c_1 \leq c n \log(n)$$

$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$

$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$

$$c\frac{2 \log(n)}{n}\left(\sum_{i=0}^{n-1} i \right) + c_1 \leq c n \log(n)$$

$$c\frac{2 \log(n)}{n}\left( \frac{(n-1)(n-1+1)}{2}\right) + c_1 \leq c n \log(n)$$

$$c\frac{\log(n)}{n}\left(n^2 - n\right) + c_1 \leq c n \log(n)$$

$$cn\log(n) - c\log(n) + c_1 \leq c n \log(n)$$

$$c_1 \leq c\log(n)$$

$E[T_{quicksort}(n)] = O(n\log(n))$

So is Quicksort $O(n\log(n))$? No!

What guarantees do you get?

$f(n)$ is a Tight Bound
The algorithm always runs in exactly $cf(n)$ steps
$f(n)$ is a Worst-Case Bound
The algorithm always runs at-most $cf(n)$ steps
$f(n)$ is an Amortized Worst-Case Bound
$n$ invocations of the algorithm always run in $cnf(n)$ steps
$f(n)$ is an Average Bound
🤷

mutable.Seq

mutable.IndexedSeq (e.g., Array)
Efficient apply(), update()
mutable.Buffer (e.g., ArrayBuffer, ListBuffer)
Efficient apply(), update(), append()
mutable.Stack
Efficient push(), pop(), top()
mutable.Queue
Efficient enqueue(), dequeue(), head()

Stacks

A stack of objects on top of one another.

Push
Put a new object on top of the stack
Pop
Remove the object on top of the stack
Top
Peek at what's on top of the stack

Stacks in Practice

  • Storing function variables in a "call stack"
  • Certain types of parsers ("context free")
  • Backtracking search
  • Reversing sequences

Stacks


    trait Stack[A] {
      def push(element: A): Unit
      def top: A
      def pop: A
    }
  

ListStack


    class ListStack[A] extends Stack[A] {
      val _store = new SinglyLinkedList()

      def push(element: A): Unit =
        _store.prepend(element)

      def top: A =
        _store.head

      def pop: A =
        _store.remove(0)
    }
  

What's the runtime?

ArrayBufferStack


    class ArrayBufferStack[A] extends Stack[A] {
      val _store = new ArrayBuffer()

      def push(element: A): Unit =
        _store.append(element)

      def top: A =
        _store.last 

      def pop: A =
        _store.remove(store.length-1)
    }
  

What's the runtime?

Stack

Scala's Stack implementation is based on ArrayBuffer (ArrayDequeue); Keeping memory together is worth the overhead of amortized $O(1)$.

Queue

Outside of the US, "queueing" is lining up.

Enqueue
Put a new object at the end of the queue
Dequeue
Remove the next object in the queue
Head
Peek at the next object in the queue

Queues vs Stacks

Queue
First in, First out (FIFO)
Stack
Last in, First Out (LIFO / FILO)

Queues in Practice

  • Delivering network packets, emails, twokstagrams
  • Scheduling CPU Cycles
  • Deferring long-running tasks

Queues


    trait Queue[A] {
      def enqueue(element: A): Unit
      def dequeue: A
      def head: A
    }
  

ListQueue


    class ListQueue[A] extends Queue[A] {
      val _store = new DoublyLinkedList()

      def enqueue(element: A): Unit =
        _store.append(element)

      def head: A =
        _store.head

      def dequeue: A =
        _store.remove(0)
    }
  

What's the runtime?

Thought question: How could you use an array to build a queue?