CMSI 2120

Lecture 12-1 - Heaping on the Data Structures

16 min read

Per usual, with a new data structure, let’s think about some motivations for new problems that our current tools don’t adequately address…

Motivation

How about a nice motivating example to get us started?

Example

You are managing a massive, worldwide delivery service, and need to assess some Priorities of deliveries so that the highest Priority items are delivered before the others.

Let’s think about some challenges in this scenario:

Question

What might be some challenges associated with maintaining this collection of orders as a List or a BinarySearchTree?

Remark

As such, it seems that all of our existing techniques lack some desire in this example, and we should make it a priority to find some solutions to the following:

Our objective(s) will be to find a data structure that:

  • Returns the highest “need / priority” item in some collection as quickly as possible.
  • Can efficiently handle insertions / removals of items while still guaranteeing this first property.
  • Can even be used to find the n-greatest need items from a larger collection.

Basics

Definition

A Priority Queue is a generalized version of a queue in which items can be retrieved according to the ranking / ordering of some arbitrary criteria.

This might mean, e.g.:

  • Provide the next patient with the “highest-need” triage score.
  • Provide the next closest missile with the “highest-risk” threat score.
  • Attempt the next best move for an artificial agent solving a problem.

Priority Queues are useful for pretty much anything to which you can assign some rankable-score, and in which it’s important that we receive the next “highest-priority” immediately while being able to store them effectively, and retrieve the next highest priority quickly as well.

Question

By what definition of “priority” would a standard FIFO Queue be considered a special case of a Priority Queue?

Heaps

To implement a Priority Queue, let’s talk about a new tree-based data structure. The heap!

Definition

A heap (generally referring to a max heap) is a special type of tree where all subtrees of a given Node have values less than or equal to that Node’s.

Debug

WARNING: The heap data structure is on no way related to the heap where objects are stored!

Such unfortunate naming schemas in computer science… it’s not like humans have infinite generative language capacities or anything…

Toolkit

Heaps are typically used when we need immediate access (O(1)) to the largest / smallest “priority” element in a collection, and can quickly reorder the collection to maintain this property at all times.

Here’s a fun comic that illustrates how they work:

Question

Examining this amusing XKCD, we do notice something interesting about the heap of presents: where is the largest present located, and why is this interesting / useful?

In the above comic, we see that the heap of presents at the base has the largest at the top with children decreasing in size with greater depth.

Our discussion, however, will be restricted to binary heaps, which have some additional restrictions.

Definition

A binary heap is a complete binary tree where all subtrees of a given Node have values less than or equal to that Node’s.

So we’re restricting the number of children any Node can have to 2, AND requiring the binary tree be complete.

Definition

A complete binary tree is a binary tree with all full levels, except for possibly the last level, in which case it is required that the Nodes be filled from left to right without any spaces.

Example

Is the following binary tree complete? Is it a heap?

Example

Is the following binary tree complete? Is it a heap?

Example

Is the following binary tree complete? Is it a heap?

Example

Is the following binary tree complete? Is it a heap?

Question

Is a single-node tree a heap?

Question

How many unique structural configurations (ignoring node values) are there for a complete binary tree with N nodes?

Heap Properties

Heaps have some nice properties, for example:

Definition

The largest value of a (max)heap is always at the root, and therefore accessible with constant time, O(1)!

Definition

Because heaps are complete trees, we can represent them as arrays to enable random access, with indexing starting at the root and numbering nodes level-by-level from left-to-right.

Question

What performance benefit does representing a complete tree as an array have over a Node-with-pointers implementation?

“Nice colors in that graph, Andrew, do you also design children’s play equipment?”

Shhh! It was illustrative!

Definition

We know that in the array representation of a heap, the root must always be at index 0.

But how do we determine the children and parent of any given index in an array?

Well, since it’s a complete tree, we have a simple computational way of finding these indexes…

Question

Given the index n of a “Node” in our conceptual tree structure, how can I get the index of the parent?

So, we can define a simple functional mapping, pretending that we have a Heap class (to be done later):

private int getParent (int index) {
    return (index - 1) / 2;
}

Getting the left and right children is a similar mechanical exercise:

// Child is either 'L' or 'R'
private int getChild (int index, char child) {
    int result = (index * 2) + 1;
    if (child == 'R') {
        result++;
    }
    return result;
}

Example

Draw the binary tree representation of the following heap:

Now that we have the basic tools down, we’re ready to go over some heap algorithms!

Heap Operations

The key with any heap operation is that it leaves the resulting data in heap format.

So, whenever we modify the contents of a heap, we need to make sure that we “reheapify” its elements afterwards.

This maintains the constant time access of the largest element at the root.

Definition

After every operation on a heap, we have some general steps: perform the operation and then “reheapify” the binary tree.

Insertion

For insertion, that looks like this:

  1. Create a complete binary tree with the newly inserted node in its proper location as the left-most vacant leaf position on the last level of the tree (i.e., the only place we could insert a leaf and still have the tree be complete).
  2. Starting at that newly inserted node, bubble upwards ensuring that each node is less than its parent.
  3. As soon as a parent is greater than the node we’re bubbling, stop—your array is now reheapified! (assuming it was a heap before the insertion).

Question

Why is it sufficient to conclude that in step 3, when the parent is greater than the node we’re bubbling, that we may stop? Why must we not also continue checking farther up the heap to verify its heapiness?

Example

Using the above steps, insert the node with value 50 into the heap below:

Got that?

Let’s make a simple BinaryMaxHeap class with this algorithm!

I’ll start you off…

Example

Complete the insert function and its helper, bubbleUp, in our BinaryMaxHeap below:

package tree.heap;
 
import java.util.*;
 
/**
 * Binary Max Heap storing Integer values representing
 * priorities of any arbitrary complex Node we could store
 */
public class BinaryMaxHeap {
 
    // Fields
    // -------------------------------------------------------------------------
    private ArrayList<Integer> heap;
 
    // Constructor
    // -------------------------------------------------------------------------
    public BinaryMaxHeap () {
        this.heap = new ArrayList<>();
    }
 
    // Methods
    // -------------------------------------------------------------------------
 
    /**
     * Adds the given int (representing a priority) to the
     * proper spot in the heap
     * @param toAdd Priority we desire to add
     */
    public void add (Integer toAdd) {
        // Step 1: Maintain a complete binary tree by adding the
        // new value at the bottom-most, right-most leaf spot, or
        // an append operation into the array
        this.heap.add(toAdd); // Appends toAdd to end
 
        // Step 2: Reheapify: bubble-up the value from its current
        // index into one where it is <= its parent's
        this.bubbleUp(this.heap.size()-1);
    }
 
    // Helper Methods
    // -------------------------------------------------------------------------
 
    /**
     * Recursively bubbles up the value at the given index until
     * its priority is less than or equal to its parent's
     * @param index Index at which to check for bubbling
     */
    private void bubbleUp (int index) {
        // Base case:
        if (index == 0) { return; } // At root, no where else to bubble!
 
        int parentIndex = getParentIndex(index),
            parentPri   = heap.get(parentIndex),
            currentPri  = heap.get(index);
 
        // Check if parent's priority is less than the current, if so
        // bubble up!
        if (parentPri < currentPri) {
            // bubble-up == swap operation with parent
            // Visit behavior:
            heap.set(index, parentPri);
            heap.set(parentIndex, currentPri);
 
            // Recursive case:
            bubbleUp(parentIndex);
        }
    }
 
    /**
     * Return the index of the parent of the given child in the
     * array representing this heap
     * @param childIndex Index of child from which to get the parent's
     * @return Parent's index of the given child
     */
    private int getParentIndex (int childIndex) {
        return (childIndex - 1) / 2; // [!] Integer division
    }
 
    /**
     * Returns the index of the given 'L' or 'R' child given the parent
     * @param parentIndex Index of the parent in the array
     * @param child 'L' or 'R' for desired child
     * @return Index corresponding to the parent's 'L' or 'R' child
     */
    private int getChildIndex (int parentIndex, char child) {
        int result = (parentIndex * 2) + 1; // Index of Left child
        if (child == 'R') { result++; }
        return result;
    }
 
    @Override
    public String toString () {
        return this.heap.toString();
    }
 
    // Empirical Tests
    // -------------------------------------------------------------------------
 
    public static void main(String[] args) {
        BinaryMaxHeap b = new BinaryMaxHeap();
        b.add(25);
        b.add(10);
        b.add(20);
        b.add(8);
 
        System.out.println("BEFORE adding 50:");
        System.out.println(b);
 
        b.add(50);
        System.out.println("AFTER adding 50:");
        System.out.println(b);
    }
 
}

Great! On to deletion.

Removal

Nothing particularly surprising about deletion; it’s essentially the same algorithm as insertion, in reverse!

Toolkit

Just like we might pop an item off the top of a Stack, we also typically remove the root of a heap to get the next largest / smallest elemetn.

That said, the algorithm for removing items from a heap is generalized to removing an item at any index (thus the beauty of recursive data structures).

  1. Remove the target node from the heap
  2. Promote the bottom-right-most leaf of the last level to the position of the removed node
  3. Trickle down the promoted item such that if either of its two children are greater than it, then it trades places with the greatest of its children.
  4. Continue trickling down until no child is greater than the trickling node.

Let’s try removing the 50 node at the root from the heap below:

Example

I shall leave it as an exercise to implement the deletion function into our BinaryMaxHeap class!

Question

What is the time complexity of insertion? Of deletion?

Question

To find an element in a heap, can we perform the same binary search algorithm on heaps that we can on binary search trees?

Priority Queues

Definition

Priority Queues are a data type that are implemented using the heap data structure to manage collections whose contents specify a definition of priority.

Toolkit

The PriorityQueue class is the JCF implementation which is implemented using a min-heap. It has essentially the same interface as a regular FIFO Queue (add to add items and poll to remove the highest priority), but is instead ordered by each item’s priority.

Example

Here’s a small example of a Priority Queue of Integer values; for Integers, their priority is the same as their value.

PriorityQueue<Integer> pq = new PriorityQueue<>();
pq.add(20);
pq.add(10);
pq.add(30);
// [?] What gets printed below?
System.out.println(pq.poll());
System.out.println(pq.poll());
System.out.println(pq.poll());

Comparable Interface

Toolkit

Note that the Java PriorityQueue is another Java Generic that can store any reference-type.

Question

Suppose we wished to store Forneymon in a PriorityQueue. What would we have to specify?

Toolkit

There are couple of different ways that you can specify how objects of custom types are prioritized:

  1. Have the class implement the Comparable interface, and define a public int compareTo(T other) method.
  2. Construct a PriorityQueue with a Comparator object.

Remark

The first method is appropriate when all instances of a class will be prioritized the same way, the second when you might prioritize them differently under different circumstances.

Let’s take a look at using the Comparable interface in this lecture.

Example

Consider a ForneymonTriage class that’s meant to prioritize treating the most wounded Forneymon (i.e., lowest health) before others. Implement the Comparable interface in the Forneymon class to enable this.

Toolkit

Step 1: have the class implement Comparable and stub the method public int compareTo(T other) where T specifies what classes the implementing one should be comparable to.

// [!] Note: Comparable is a generic as well, so we must specify what Forneymon are
// comparable to -- in this case, we want it to be comparable to other Forneymon
abstract public class Forneymon implements Comparable<Forneymon> {
    // ... Other Forneymon contents elided
 
    public int compareTo (Forneymon other) {
        throw new UnsupportedOperationException();
    }
 
    // ... Other Forneymon contents elided
}

Toolkit

Step 2: implement the compareTo method, which returns an integer that is:

  • 0 if this Forneymon and the other Forneymon have equal priority.
  • < 0 if this Forneymon has higher priority than the other (will be nearer to the top of the minheap).
  • > 0 if this Forneymon has lower priority than the other (will be nearer to the bottom of the minheap).
public int compareTo (Forneymon other) {
    // [!] Will be < 0 (higher priority) when this Forneymon has less health
    // than the other!
    return this.health - other.health;
}

Finally, testing it, we see that our most wounded Forneymon will be polled first!

PriorityQueue<Forneymon> fmTriage = new PriorityQueue<>();
Burnymon b1 = new Burnymon("b1"),
         b2 = new Burnymon("b2"),
         b3 = new Burnymon("b3");
 
b2.takeDamage(10, "Burny");
b3.takeDamage(5, "Burny");
fmTriage.add(b1);
fmTriage.add(b2);
fmTriage.add(b3);
 
// [?] In what order will the Burnymon be polled below?
while (!fmTriage.isEmpty()) {
    System.out.println(fmTriage.poll());
}

Remark

That’s it! Internal to the PriorityQueue implementation, its contents this.compareTo(parent) method is called during the bubble-up process and this.compareTo(child) during bubble-down to decide if a swap is necessary or not!

Hidden on the original page

The section below was commented out of the Fall 2021 course notes, so students never saw it. It is preserved here because the material is complete and usable.

HeapSort

By observing the operations above, we can start to glean some insight on a potential application:

  • Getting the largest item in a MaxHeap can be done in O(1).
  • Removing an item from the heap gives us constant time access to the next largest item after the one just removed.
  • Reheapifying after a removal takes only O(log(n)) time.

Question

How might we be able to make use of the above two facts?

Definition

HeapSort is a reduce-and-conquer algorithm for obtaining a sorted list of items by successive removal of the greatest item in an iteratively shrinking MaxHeap.

Its algorithm goes something like this:

count = size of heap
while count > 1
    swap the root with the deepest, right-most leaf
      (i.e., swap index 0 with index (count-1))
    count--
    re-heapify everything to left of count

Toolkit

Note how this is a decrease-and-conquer algorithm, since at the end of each iteration (similar to InsertionSort), the next-largest item is locked in its proper place at the end of its sublist.

Since we are representing heaps in their array format, what we are left with after heapsort is a sorted array of that heap’s elements.

Let’s do some heap sorting!

Question

What is the time complexity of heapsort?

How exciting! That looks a lot better, on average, than InsertionSort, even though Insertion can win in the best case.

We see that these two sorting algorithms are similar insofar as they maintain a sorted sub-list that grows incrementally until we are left with a sorted version of the original list.

Here’s a cool animation of that shamelessly stolen from Wikipedia:

Image source from good ole Wikipedia: HeapSort Stuff

Graph View

Backlinks