CMSI 2120

Lecture 13 - Big-O No - More Runtime Analysis

13 min read

Lecture 13: Big-O No! More Runtime Analysis!

Last time we learned about time complexity and runtime analysis - how to measure the running time of an algorithm, and how to simplify that running time into a function that described it’s maximum growth rate given it’s inputs.

Let’s revisit and review some rules of thumb for Big-O, then we’ll get into some practice.

Big-O Rules of Thumb

Disclaimer: these are simply rules of thumb to start generating intuition for finding asymptotic performance guarantees. They are not exhaustively applicable, but we will be considering exceptions explicitly later in the course.

For many algorithms, determining which run time growth class they belong to can be quite difficult and requires formal mathematical proofs.

For some basic ones, however, there are a few rules of thumb to find from a given algorithm’s .

If is a constant that does not depend on the input size, .

For example, like in the example above.

This makes intuitive sense because an algorithm whose number of steps does not change with the input size will always take the same number of steps.

If contains factors that depend on the input size, then we may ignore the constant coefficients on terms with n.

In the above example #2, we had .

Using Observation 2 from the previous section, we see that the term will dominate the number of steps this algorithm takes.

However, as n trends to infinity, the coefficient 5 plays only a tiny role, and so can be dropped entirely.

The +6 steps are even more trivial compared to a term where n is growing towards infinity, and so can be dropped as well.

So, dropping the +6 and the 5 coefficient from simply gives us .

If contains factors that depend on the input size in different degrees, we may ignore all but the highest-order term.

For example, the because it has a 2 for an exponent. The . The because it is technically .

So, we say that the highest degree terms that rely on the size of the input dominate the other.

In the example #3 above, we had .

Because is the highest order term, we ignore the others, and using our rule of thumb above, can ignore the 3 coefficient, giving us .

Example

 So, returning to our previous example, determine which algorithm is more asymptotically efficient:  

Check

; .  Therefore algorithm b is asymptotically more efficient.

Alright! Now that we’ve got these rules of thumb in mind, let’s do some practice…

Big-O in Practice

Before we do, consider one small fly in the ointment:

So, how do we qualify our analysis if we can’t concisely determine the algorithm performance across input and state variants?

For most algorithms, run time analysis is performed for three separate input assumptions: (1) the worst case, (2) the average case, and (3) the best case.

The best and worst cases are generally easy to analyze: assume either the best or worst type of inputs and then determine which statements are executed more or less in response to those types of inputs.

Average case analyses are more tricky and can often involve complex proofs to support their claims. We will not be detailing most of these proofs, but will still gain some intuition for them.

Example

We’ve written many functions thus far for Array Lists and Linked Lists! Let’s analyze them!

Find the and (best and worst case) of the following methods:

Array List Methods

private void checkAndGrow() {
    if (this.size < this.items.length) {             // c_1
        return;                                      // c_2
    }
 
    int[] newItems = new int[this.items.length * 2]; // c_3
 
    for (int i=0; i<this.items.length; i++) {        // c_4 * n
        newItems[i] = this.items[i];                 // c_5 * n
    }
 
    this.items = newItems;                           // c_6
}
public void append (int toAdd) {
    checkAndGrow();                  // c_1 * n OR c_1
    this.items[this.size] = toAdd;   // c_2
    this.size++;                     // c_3
}
public int getAt (int index) {
    if (index < 0 || index >= this.size) {      // c_1
        throw new IllegalArgumentException("Requested index out of range");//c_2
    }
    return this.items[index];                   // c_3
}

What is the asymptotic runtime for getAt() in all cases?

O(1) - random access allows us to get any element in the list instananeously!

private void shiftLeft (int index) {
    for (int i = index; i < this.size-1; i++){      // c_1 * n
        this.items[i] = this.items[i+1];            // c_2 * n
    }
}
public void removeAt (int index) {
    if (index < 0 || index >= this.size) {      // c_1
        throw new IllegalArgumentException("Requested index out of range");//c_2
    }
    shiftLeft(index);                           // c_3 * n OR c_3
    size--;                                     // c_4
}
private void shiftRight(int index) {
    for (int i = size; i > index; i--) {      // c_1 * n
        items[i] = items[i - 1];              // c_2 * n
    }
}
public void insertAt (int toAdd, int index) {
    if (index < 0 || index >= this.size) {     // c_1
        throw new IllegalArgumentException("Requested index out of range");//c_2
    }
    checkAndGrow();                            // c_3 * n OR c_3
    shiftRight(index);                         // c_4 * n OR c_4
    this.items[index] = toAdd;                 // c_5
    this.size++;                               // c_6
}

Linked List Methods

public int getAt (int index) {
    if (index < 0 || index >= this.size) {    // c_1
        throw new IllegalArgumentException("Requested index out of range");//c_2
    }
 
    Node current = this.head;                 // c_3
    while (index > 0) {                       // c_4 * n
        current = current.next;               // c_5 * n
        index--;                              // c_6 * n
    }
    return current.data;                      // c_7
}
public void append(int toAdd) {
    Node toAppend = new Node(toAdd);      // c_1
    this.size++;                          // c_2
 
    if (this.head == null) {              // c_3
        this.head = toAppend;             // c_4
        return;                           // c_5
    }
 
    Node current = head;                  // c_6
    while (current.next != null) {        // c_7 * n
        current = current.next;           // c_8 * n
    }
    current.next = toAppend;              // c_9
}

Note: Remember that this is only expensive because we haven’t implemented a tail reference in the version we wrote in class. With a tail reference append would be O(1) in all cases!

public void prepend (int toAdd) {
    Node toPrepend = new Node(toAdd);     // c_1
    toPrepend.next = this.head;           // c_2
    this.head = toPrepend;                // c_3
    this.size++;                          // c_4
}

What is the asymptotic runtime for prepend() in all cases?

O(1) - we always have access to the Nodes required to insert at the beginning!

public void removeAt (int index) {
    if (index >= this.size || index < 0) {       // c_1
        throw new IllegalArgumentException("index out of legal range"); // c_2
    }
 
    Node current = this.head;                    // c_3
    Node prev = null;                            // c_4
 
    while (current != null && index > 0) {       // c_5 * n
        prev = current;                          // c_6 * n
        current = current.next;                  // c_7 * n
        index--;                                 // c_8 * n
    }
 
    if (current == this.head) {                  // c_9
        this.head = current.next;                // c_10
    }
 
    if (prev != null) {                          // c_11
        prev.next = current.next;                // c_12
    }
    size--;                                      // c_13
}

Iterator vs getAt()

Take the following code:

IntLinkedList llcoolj = new IntLinkedList();
llcoolj.append(1);
llcoolj.append(2);
llcoolj.append(3);
llcoolj.append(4);
llcoolj.append(5);
llcoolj.append(6);
llcoolj.append(7);
 
for (int i=0; i<llcoolj.size(); i++) {       // c_1 * n
    System.out.println(llcoolj.getAt(i));    // c_2 * n^2 
}
 
                                             // Separate runtime from above
Iterator it = llcoolj.getIterator();         // c_1
 
while(it.hasNext()) {                        // c_2 * n
    System.out.println(it.getCurrentInt());  // c_3 * n
    it.next();                               // c_4 * n
}

Stack Methods

public class ConstructedIntStack implements TrueIntStack {
    private final IntLinkedList list;
 
    public ConstructedIntStack() {
        this.list = new IntLinkedList();      // O(1)
    }
 
    public void push(int toPush) {
        // A LinkedList prepend is always constant time!
        list.prepend(toPush);                 // O(1)
    }
 
    public void pop() {
        // While removeAt is usually a linear time operation,
        // it's constant if we're removing the first element!
        list.removeAt(0);                     // O(1)
    }
 
    public int peek() {
        // While getAt is usually a linear time operation,
        // it's constant if we're accessing the first element!
        return list.getAt(0);                 // O(1)
    }
}

Amortized Time

Note a small asterisk we'll attach to ArrayList insertion operations to account for the check_and_grow procedure... but this doesn't happen every time we insert a new element!

Example

Suppose we are inserting 𝑛 elements into an ArrayList that begins (for the sake of maths) with a capacity of 1.

Observation 1: Assuming that our underlying array doubled in size every time it was at capacity and needed to grow, we would incur the cost of copying all items from the old array to the new every time along the journey of adding 𝑛 items.

Observation 2: If we started with a capacity of 1, we would thus incur the following sequence of copy costs each time the array grew:

Observation 3: Observe that this is a series from which we can factor out 𝑛:

Observation 4: Note that the factor is a geometric series (get those calculus notebooks out) that converges to 2 in the limits of infinity.

Observation 5: The above cost of inserting 𝑛 items into an ArrayList thus boils down to:

Observation 6: Not forgetting that there were 𝑛 items appended with constant time apart from the grow behavior, we still have overall.

Why is the above proof cool?

Note that we were able to insert 𝑛 items with total cost of . This means that, on average, each insertion took time, i.e., time.

This analysis is known as amortized analysis, a common way to compute some average-case algorithmic performance which refers to the cost of an algorithm averaged over some sequence of repeated executions.

This is also why we conclude that appending to an ArrayList is a 𝑂(1) operation: due to its amortized analysis above!

Graph View

Backlinks