CMSI 2120

Lecture 23 - Hash Out or Crash Out

16 min read

Lecture 23: Hash Out or Crash Out

Alright, I think we’ve spent enough time talking about Trees.

It’s time to revisit a favorite operation of ours: membership search.

Motivation from the Market

Now for the tantalizing question... can we do better, and if so, what is the cost that we pay?

Andrew’s World Famous Supermarket Analogy

This part of the lecture must begin, by tradition, with Andrew’s Famous Supermarket Analogy.

Not only will this give us some intuition for what follows, but also provides a roadmap for what we can endeavor to implement in what has sometimes been heralded as the data structure to end all data structures!

Story time! Let’s consider that we’re in a supermarket looking for our favorite breakfast food: hash browns.

Now, we haven’t been to this particular supermarket before, so the layout may not be intuitive, and in particular, we may not know where to find that crunchy, delicious treat.

This is generally the approach I like to take, since I’ll usually find other snacks along the way that I want to add to my cart.

Sorry for the callout - I’m in the same boat.

Wow! Constant time lookup, that’s impressive!

AND constant time insertion?! Can we get supermarket employees to manage our data structures for us?!

More on that in a moment…

Maps & Dictionaries

Before considering the implementation, we should note that the Supermarket example has presented us with a new data type since we have Products that correspond to their store Location.

These are generally referred to as maps / dictionaries:

First, let’s think about a real-world example:

Take Webster’s Dictionary:

Let’s say I recently watched a lecture about the famous “Pigeonhole Principle” of Combinatorics, and I walked away wondering “…what’s a pigeonhole? Do pigeons live in holes?” How would I find out?

Well, I can open up Merriam-Webster and search it up! Consider this my key.

Once I hit Enter, I get my value - the definition of the word pigeonhole!

Maps / Dictionaries are ADTs that represent collections of elements consisting of some value that corresponds (i.e., maps) to a unique key.

Generally, keys are simple, unique identifiers used to quickly map to more complex values.

You’ve probably already used these data types in JavaScript objects or Python dict()s, but today we’ll take a look under the hood at some implementation details.

So, with that intuition in mind, let’s think about how to implement a Map, inspired by our Supermarket analogy.

Hash Tables

A hash table is a data structure useful for implementing the Map and Set ADTs by using an array of buckets to store its values where each bucket index corresponds to a functional mapping from each value's unique key.

Because some keys are not numeric (and therefore lack a direct translation to an index), we define hash functions that produce a semi-unique index pertaining to each object’s fields / properties / characteristics. If you’ve been wondering why you keep seeing a hashCode() method implemented in some of the classes I’ve provided you, you’re about to find out!

A hash function, f provides the index of a value's bucket corresponding to its key such that f(key(val)) = index.

Just how the buckets are arranged or arrange the data they contain depends on what's called the hashing scehma.

In a separate chaining schema, buckets are other data structures that hold the table's values (e.g., LinkedLists), while in an open addressing schema, the buckets hold the values themselves.

We’ll look at the differences between these two schemas shortly… in the meantime, let’s make sure the analogy is completely concrete…

Complete the following analogies based on our supermarket example with employees being "oracles" that can be consulted to find items in aisles of the market.

  • Supermarket = ?
  • Aisle = ?
  • Product = ?
  • Employee = ?

See how masterful that example was?

We’ll look more at hash functions later, for now, let’s consider some simple examples to make the above more concrete.

Operations

Let’s talk about the basic operations of a hash table:

Insertion in a hash table follows a basic recipe:

  1. Step 1: acquire the index of the bucket corresponding to the added value’s key using the key’s hash function.
  2. Step 2: determine if the key has already been stored at the bucket at the index found in Step 1.
  3. Step 3: if the key is new, insert* the key-value pair at that bucket; if it’s not, simply update the value corresponding to that key.

* Just how this search is done depends on if we have a separate chaining vs. open addressing hash table, to be discussed shortly.

Example

 Suppose we wish to store Products that have a unique name (the key) and map to some price values. Let’s draw the HashTable with the following properties:

  • Keys/Values: Keys are product names (Strings), values will be the product prices (doubles).
  • Buckets [separate-chaining]: buckets are an array of LinkedLists, and begin with a cacity of 8 buckets (Let ).
  • Hash Function: For Product name x (a String), we have: f(x) = x.length() % b (Note: we need to mod by , the number of buckets, so that we will always have a valid index from among those in our buckets).

Some notes on the above:

  • Note how both “corn” and “eggs” have the same number of characters, and so hash to the same bucket (4) — this is known as a collision and is generally something we want to avoid when possible, but deal with as it will inevitably come up.
  • In this Separate Chaining Hashing Schema, the buckets are LinkedLists that store the individual items. As such, when any collision happens, we simply append the new item to the existing List at that bucket.
  • Note how “buttermilk” contained more characters than there were buckets, so we took its hashed value and modded by the number of buckets to wrap around back to the 2 index.

Membership Search / Value Retrieval follow the same steps 1 + 2 as insertion, except instead of inserting / updating a value in Step 3, we return the value associated with the key if it exists.

Example

Try querying the map in the example above for the keys "apple", "pear", and "corn".

Entries

Let’s think a bit about the implementation of the picture we’ve drawn above, starting with the objects containing our key and value.

Just like LinkedLists / Trees had Nodes in order to store and organize their data, so too do HashTables!

An Entry object is a basic node storing both key and value with the following behaviors:

The equals() behavior must be overridden to compare equivalence of keys.

The hashCode() behavior must be overridden to produce a semi-unique index corresponding to the key's hash code.

Finally, we see that hashCode method I’d promised to explain many moons ago! You might remember an additional property I reminded us all of:

This is a bit harder to interpret with simple String keys, but is necessary for more sophisticated data types used as keys.

Note: if an Entry's key and value are the same thing, this is how we can use a HashTable to also implement a Set!

Performance Analysis

Now that we’ve seen a couple of examples, let’s consider the performance cases for Hash Tables:

Example

Sketch a picture of what this best case would look like!

Example

Sketch a picture of what this worst case would look like!

It seems, then, that a lot of how these best and worst cases are characterized relies on the chosen Hash Function -- let's take a look at how to design these next!

Hash Functions

Now that we’ve seen hash tables in their basic format, let’s talk about how hash functions work.

Recall that hash functions return an integer index corresponding to some characteristics of a given type.

Hash Function Basics

Example

Suppose we are designing the String class’ hashCode() method. What are some ways you might construct such a function?

Let’s consider a simple hash function that simply returns the number of letters in the given string to an int; it might look something like this.

public class String {
    @Override
    public int hashCode () {
        return this.length();
    }
}

This would be like having 1 row in our whole supermarket and then whenever we ask where something is, the employee tells us “row 1 bro.” Useless!

As such, one of the chief traits we want to try to equip our hash codes with is uniformity: the idea that different keys get very different values.

Desirable Trait 1 - Uniform Distribution: Collisions are undesirable, therefore, a good hash function should uniformly distribute hashed keys among buckets.

Avoid collisions you say! Alrighty, how about the following hash function:

@Override
public int hashCode () {
    return this.length() * Math.random();
}

Desirable Trait 2 - Consistency: A hash function for which f(key) = i should always hash the same key to the same index i.

This makes sense, intuitively; if you asked our supermarket employee which row the cookies were in twice, you wouldn’t expect them to change their answer!

And lastly, although it might go without saying…

Desirable Trait 3 - Speed: A hash function should quickly and efficiently compute its hash value, because (almost) every operation a hash table performs always starts with the hash function.

Note: generally, since the number N of items stored in a hash table will be huge compared to the size of a single item, k, it's fine if our hash function has some tolerable cost that is some function of k.

In other words, don’t make your hash function compute to the billionth digit every time it’s called, and then use that result to avoid collisions.

Hashing any key will take some time, but that time is a function of the key (small), not the number of items in the collection (huge).

Example

Consider then: what would be a good way to combine all of the above properties for a hash function?

@Override
public int hashCode () {
    int code = 0;
    for (int i = 0; i < this.chars.length; i++) {
        code = 31 * code + this.chars[i];
    }
    return code;
}

Some notes on the above:

  • The above exploits the uniqueness of a String based on the integer character codes of its composing chars: char values in Java are stored as numerical values corresponding to the character’s Unicode value.
  • What’s with the multiplication by 31? Fans of Baskin Robbins? No, turns out that in cryptography and its accompanying statistics, multiplying a code by a prime number will increase its chances of being unique.
  • Won’t the above eventually experience integer overflow? Yes! Turns out though that negative hashCodes are just fine — simply take the absolute value when you need to map them to a bucket index.
  • Wouldn’t this needlessly compute the same code many times over? Yes, so in a better implementation, we would compute the hashCode once on demand, and then store its result in a field.

A wide swath of research has investigated the best hashing methods to avoid collisions, which are outside of the scope for this introduction, so we won't detail more traits herein.

Example

Still, you can try out Java’s hashCodes for any object — see what you get for the following!

System.out.println("a".hashCode());
System.out.println("b".hashCode());
System.out.println("hash brown".hashCode());
System.out.println("hash brown".hashCode());

Analyzing Hash Tables

Before we conclude our discussion on hash tables, we have two analytic points to cover:

Ensuring Constant Time Insertion / Lookup

One question you might ask: if our hash table is aiming for O(1) insertion and lookup, how can we guarantee this performance if we have a lot of collisions?

As such, reducing collisions is really a 2-part process:

  1. Design good hash functions that we gain some theoretical guarantees of evenly distributing the values.
  2. Maintain a healthy load factor.

A hash table's load factor is the ratio of items to buckets , expressed as:

Empirically, a good load factor is kept below , which (if maintained and a good hash function is designed) leads to the following picture:

Generally speaking, a low load factor may not indicate a good hash table; for instance, we might have 100 items stored in 1 of 200 buckets, but the load factor will still be below .

Ideally, the best hash functions maintain buckets with fewer than 3 elements.

However, assuming a good hash function, a high load factor generally means that we’re taxing the high performance guarantees of our hash table (there are too few buckets to accommodate so many values).

Hash Table Use Cases

So what are the pros and cons of hash tables?

With the guarantees above, HashTables boast the fastest O(1) insertion + lookup behavior for Sets and Maps!

Memory intensive to maintain a good load factor, and we lose any guarantee of sorted ordering that a BinarySearchTree implementation could maintain.

Graph View

Backlinks