Average Runtime, Stacks, Queues

CSE-250 Fall 2022 - Section B

Sept 30, 2022

Textbook: Ch. 15, Ch. 7

QuickSort

1. Pick a value at random as a $pivot$.
2. Swap values until the array is subdivided into...
   • $low$: array elements that are $\leq pivot$
   • $pivot$
   • $high$: array elements that are $> pivot$
3. Recursively sort $low$
4. Recursively sort $high$

What's the worst-case runtime?

$$T_{quicksort}(n) \in O(n^2)$$

... but this isn't really representative of typical behavior

Let's talk probabilities

If $X$ represents a random (numerical) outcome, the average over all possibilities is the expectation of $X$, or $E[X]$

$$E[X] = \sum_{i} P_i \cdot X_i$$

1. $E[X + Y] = E[X] + E[Y]$ (always)
2. $E[X \cdot Y] = E[X] \cdot E[Y]$ (If $X$ and $Y$ are independent)

We pick the $X$th largest element as a pivot

We pick the $X$th largest element as a pivot

$$E[T(X-1)]$$

$= \sum_{i=1}^{n} P_i \cdot T(X_i-1)$

$= \sum_{i=1}^{n} \frac{1}{n} \cdot T(i-1)$ ($T(0)$ up to $T(n-1)$)

$= \sum_{i=1}^{n} \frac{1}{n} \cdot T(n-i)$ ($T(n-1)$ down to $T(0)$)

$= E[T(n-X)]$

We pick the $X$th largest element as a pivot

Each $T(X-1)$ is independent.

Back to induction...

Hypothesis: $E[T(n)] \in O(n \log(n))$

Base Case: $E[T(1)] \leq c(1 \log(1))$

$$E[T(1)] \leq c \cdot (1 \cdot 0)$$

$$E[T(1)] \not \leq 0$$

Base Case (take Two): $E[T(2)] \leq c(2 \log(2))$

$$2\cdot E_i[T(i-1)] + 2c_1 \leq 2c$$

$$2\cdot \left(\frac{1}{2}T(0) + \frac{1}{2}T(1)\right) + 2c_1 \leq 2c$$

$$T(0) + T(1) + 2c_1 \leq 2c$$

$$2c_0 + 2c_1 \leq 2c$$

True for any $c \geq c_0 + c_1$

Assume: $E[T(n')] \leq c(n' \log(n'))$ for all $n' < n$

Show: $E[T(n)] \leq c(n \log(n))$

$$\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + c_1 \leq c n \log(n)$$

$$\frac{2}{n}\left(\sum_{i=0}^{n-1} c i \log(i) \right) + c_1 \leq c n \log(n)$$

$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$

$E[T_{quicksort}(n)] = O(n\log(n))$

So is Quicksort $O(n\log(n))$? No!

What guarantees do you get?

$f(n)$ is a Tight Bound

The algorithm always runs in exactly $cf(n)$ steps

$f(n)$ is a Worst-Case Bound

The algorithm always runs at-most $cf(n)$ steps

$f(n)$ is an Amortized Worst-Case Bound

$n$ invocations of the algorithm always run in $cnf(n)$ steps

$f(n)$ is an Average Bound

🤷

mutable.Seq

mutable.IndexedSeq (e.g., Array)

Efficient apply(), update()

mutable.Buffer (e.g., ArrayBuffer, ListBuffer)

Efficient apply(), update(), append()

mutable.Stack

Efficient push(), pop(), top()

mutable.Queue

Efficient enqueue(), dequeue(), head()

Stacks

A stack of objects on top of one another.

Push

Put a new object on top of the stack

Pop

Remove the object on top of the stack

Top

Peek at what's on top of the stack

Stacks in Practice

• Storing function variables in a "call stack"
• Certain types of parsers ("context free")
• Backtracking search
• Reversing sequences

Stacks

    trait Stack[A] {
      def push(element: A): Unit
      def top: A
      def pop: A
    }
  

ListStack

    class ListStack[A] extends Stack[A] {
      val _store = new SinglyLinkedList()

      def push(element: A): Unit =
        _store.prepend(element)

      def top: A =
        _store.head

      def pop: A =
        _store.remove(0)
    }
  

What's the runtime?

ArrayBufferStack

    class ArrayBufferStack[A] extends Stack[A] {
      val _store = new ArrayBuffer()

      def push(element: A): Unit =
        _store.append(element)

      def top: A =
        _store.last 

      def pop: A =
        _store.remove(store.length-1)
    }
  

What's the runtime?

Stack

Scala's Stack implementation is based on ArrayBuffer (ArrayDequeue); Keeping memory together is worth the overhead of amortized $O(1)$.

Queue

Outside of the US, "queueing" is lining up.

Enqueue

Put a new object at the end of the queue

Dequeue

Remove the next object in the queue

Head

Peek at the next object in the queue

Queues vs Stacks

Queue

First in, First out (FIFO)

Stack

Last in, First Out (LIFO / FILO)

Queues in Practice

• Delivering network packets, emails, twokstagrams
• Scheduling CPU Cycles
• Deferring long-running tasks

Queues

    trait Queue[A] {
      def enqueue(element: A): Unit
      def dequeue: A
      def head: A
    }
  

ListQueue

    class ListQueue[A] extends Queue[A] {
      val _store = new DoublyLinkedList()

      def enqueue(element: A): Unit =
        _store.append(element)

      def head: A =
        _store.head

      def dequeue: A =
        _store.remove(0)
    }
  

What's the runtime?

Thought question: How could you use an array to build a queue?