Revision 3, Sept. 8, 2026
Author: Anthony Jacaban
Revision 2, Dec. 30, 2022
Author: Seung Min Lee
Revision 1, Nov. 5, 2020
Author: Patrick Majurski

---

Requirements

A test suite is concerned primarily with providing automated checks of your code and documenting the expected behavior. Automated tests ensure that future changes to the code don't break previous behavior and give confidence to outside users (the graders) that your code is functional. As such, a well-written test file will have both comprehensive test coverage and documentation.

Test Coverage

• Normal case: Cases where the operation is successful. These are the usual cases that should occur the majority of the time.
• Edge cases: Cases where the operation is successful, but certain member variables or data structure objects are values which lie on an extreme.
• Error cases: Cases where the operation is not successful, where it will either simply not do the requested operation, return a value that indicates an error, or will throw an exception.
  These can happen due to invalid function arguments, if the data structure isn't initialized for the operation, or many other reasons like trying to pull data from an empty structure.

Note: You need to perform all testings on the GL server. Your code will be ran and graded on the GL server.

Documentation

An important aspect of testing is to document the details of each test. Documentation can come in many forms, including test names, variable names, inline or 1-line comments, and cout statements. For this course, one should communicate what is being tested (i.e., function being tested), how it is being tested (test case being covered), and why it is being tested (i.e., expected result) through documentation.

---

Creating Test Cases from Project Requirements

You are only required to create the test cases listed in the Testing section of the project website, however testing beyond that is encouraged. This area depicts what the required normal or edge cases may be for projects. Keep in mind that the line between normal/edge/error can be blurry. They don't need to be labelled perfectly in your code, as long as you cover all required cases, your tests should be accepted.

Example 1

bool LinkedList::is_empty()

This method returns true if the linked list has no nodes.

• Normal case
  • Linked list has nodes, expected to return true
  • Linked list has no nodes, expected to return false

Example 2

bool LinkedList::append(int num)

This method adds a new dynamically allocated node to the tail of the linked list. Duplicates values are not permitted and that case should return false and not insert. The success of the append operation is returned.

• Normal case
  • Append to a non-empty linked list, expected to return true and the node should be at the tail of the linked list
• Edge case
  • Append to a empty linked list, expected to succeed
• Error case
  • Inserting a duplicate value, should return false and not succeed

Example 3

int LinkedList::at(int index)

This method returns the value of the node at index. Accessing an out of bounds index shall throw an out_of_range exception.

• Normal case
  • Accessing a node at an index within bounds, node values should match the expected values based on the logic from the append() method (tail insertion)
• Edge case
  • Accessing the first and last valid nodes (at(0), at(size - 1))
• Error cases
  • Accessing a negative index throws an out_of_range exception
  • Accessing an index for an empty linked list throws out_of_range
  • Accessing an index >= size throws out_of_range

Example 4

bool operator==(const LinkedList& rhs) const;

This operator is equivalent to using: "linkedList1 == linkedList2;" All data and the order of the internal nodes must be equal for this to return true.

• Normal cases
  • Linked lists with the same data applied in the same order should be equal.
  • Linked lists created using a copy constructor/overloaded assignment operator should be considered equal
  • Linked lists with different data or different amounts of nodes should not be equal
• Edge case
  
  • LinkedLists with the same data, but different ordering of nodes should be considered unequal

Example 5

const LinkedList& operator=(const LinkedList& rhs);

This operator is equivalent to calling: "linkedList1 = linkedList2;"

• Normal case
  • For the operation: "LHS = RHS", LHS and RHS are both objects with contain nodes.
    LHS is expected to clear without memory leaks, then both objects should have the same data, member variables, and same ordered nodes.
• Edge cases
  • LHS and RHS are empty objects. LHS and RHS are expected to be empty after calling.
  • LHS is filled, but RHS isn't. This results in LHS becoming an empty object.
  • RHS is filled with nodes, but LHS isn't. This results in LHS being filled with nodes.

---

Basic Test Cases

Here is an example of an invalid test case:

bool testInsert(){
    List structure;

    bool allSuccessful = true;
    for(int i=0; i<5; i++){
        bool success = structure.insertNode(4, "cool data!");
        allSuccessful = allSuccessful && success; // Set to false if success is false, otherwise maintain value
    }

    if(allSuccessful == true){
        return true;
    }else{
        return false;
    }

}

This case should test whether the insert function of a linked list is successful, however, it only relies on the return value of the function.
If the implementation does not correctly insert the nodes, this test case fails to catch it. A simple function like this will pass this test case:

bool List::insertNode(int id, string data){
    return true;
}

Test cases need to verify that the function worked internally. Tester is a friend class of your data structure, which allows you to directly set and inspect the member variables.
After adding new checks, an incomplete function will no longer pass the test case:

bool testInsert(){
    List structure;

    bool allSuccessful = true;
    for(int i=0; i<5; i++){
        bool success = structure.insertNode(4, "cool data!");
        allSuccessful = allSuccessful && success; // Set to false if success is false, otherwise maintain value
    }

    // Check internal variables, it's internal pointer should have something and the size should be correct
    if(structure.m_head == nullptr){
        allSuccessful = false; // Could also do an early return false
    }
    if(structure.m_size != 5){
        allSuccessful = false;
    }

    // All IDs should be accessible
    for(int i=0; i<5; i++){
        string data = strucutre.getNode(i);
        if(data == "") allSuccessful = false;
    }

    // Return if all tests pass
    return allSuccessful;

}

---

Suggestions

Here are some suggestions about how to improve the readability and writability of your test file.

Do’s

• Isolate and organize your tests. Test suites can contain dozens of tests, which will inevitably lead to name overlap and confusion. Each individual test case should each be contained within it’s own labelled boolean function. These functions will return whether they pass or fail and will all be called in main.
  Having good names for your test functions can remove the need for large amounts of comments. A test named “testInsertInvalidID” is self describing and easy for graders to understand.
• Extract common testing code into their own functions. An important thing to remember is that test code is code, so you can make abstractions just like you would for your project implementations. Code that is repeated with only a few differences in the values of variables and objects are good candidates for abstraction. Here are some general categories of test code that you can extract into functions or const variables.
  • Standard pass or fail messages
  • Set-up code (i.e., setting up the data structures with the correct values or properties)
  • Checking code (i.e., checking that relevant member variables, properties, and invariants are as expected)
  • Clean-up code
• Implement a standard structure for your tests, so that emphasis is placed on the logic and intent of each test, instead of common background code (e.g., set-up, clean-up). Generally, it has the following structure:
  1. Set-up code that instantiates a data structure or data structures with specific data
  2. Execute the relevant operations such that a specific test case is covered
  3. Check that relevant properties and member variables are as expected
  4. (Optional) Perform clean-up if you dynamically allocated data structures or used system resources.
• Use STL data structures. Importing libraries is allowed inside mytest.cpp and can make initializing data for test cases much easier.

Dont’s

• Create interactive tests that require manual user input. Your test file is run by scripts, which cannot enter input. Points will be deducted.
• Use dynamic memory. Dynamic memory adds complexity to your cases and add another place where you can possibly have memory leaks. In almost all cases, you can statically allocate a large amount of memory instead.
• Catch exceptions inside your data structure methods. Exceptions inform the user that their usage of the data structure and its methods are resulting in errors. Test scripts will expect the exceptions to be thrown.
• Test exceptions without catching them in your tests. Your test file must run to completion.
• Test functions that return boolean values only by checking the boolean return value itself.
• Rely solely on visual inspection of inputs. Visual tests are a good starting point, but it is easy to overlook subtle bugs and edge cases, especially as the size of the input increases. At a certain point, visual inspection will be virtually impossible as the output will be too large to manually parse.
• Insert "cout" or other output statements in loops. As the amount of test data increases, the looping print statements will take up more visual space and resources (e.g., time, computations), obscuring important messages and running noticeably slower due to performing many more IO (input/output) operations. To be clear, it is fine to insert print statements into loops while you are debugging, but the final test file that you submit should only have essential print statements, such as which test case is being covered and the result of the test.
• Break the project coding guidelines. While you are encouraged to experiment and make the testing suite your own, but still please don't use banned coding styles.

---

Full Test Example

// Sample tester file
// This test suite will test the functionality of a custom vector class

// Vector will have this definition:
/*
class Vector{
    public:
    friend class Tester;
    Vector(int* array, int arraySize);

    void append(int element);
    bool setElement(int i, int element);
    int  readElement(int i, int element);

    int m_vectorSize; // The size the user sees, this is how many elements they've inputted

    private:
    int* m_array; // Only storing ints in this vector
    int  m_capacity; // The actual size of the internal array
    
};
*/
#include "customVector.h"

class Tester{
    public:
    // Our test cases may need alot of data, it may also be good to create helper functions to speed up the creation process    
    // This one is simple, but in cases where we need large amounts of randomized data, doing this can be a life saver
    void AppendVector(Vector& setup){
        for(int i=0; i<20; i++){
            setup.append(i);
        }
    }

    // We will define every test case as an individual bool function
    // This accomplishes the isolation and organization goals we have, as long as the functions have descriptive names,
    // the goal of the test case should be self-evident, even without comments

    // Normal case
    // When given a size, it should initialize capacity and size to that and zero out elements
    bool testConstructor(){
        int arr[10] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}; 
        Vector structure(arr, 10);

        // Check member variables
        if(structure.m_vectorSize != 10) return false;
        if(structure.m_array == arr || structure.m_array == nullptr) return false; // should be new memory
        if(structure.m_capacity != 10) return false;

        // Check internal array was initialized, we could check all or check a certain element
        if(structure.m_array[2] != 3) return false;

        return true;
    }

    // Edge case
    // A size of 0 is an acceptable size
    bool testZeroConstructor(){
        Vector structure(nullptr, 0);

        // Check member variables again
        if(structure.m_vectorSize != 0) return false;
        // What m_array could be is dependent on implementation, but we'll assume it'll be nullptr here
        if(structure.m_array != nullptr) return false;
        if(structure.m_capacity != 0) return false;

        return true;
    }

    // Error case
    // If we pass a negative number, it should default to a size of 0
    bool testInvalidConstructor(){
        Vector structure(nullptr, -5);

        if(structure.m_vectorSize != 0) return false;
        if(structure.m_capacity != 0) return false;

        return true;
    }

    // Normal case for append
    bool testAppendNormal(){
        Vector structure(nullptr, 0);
        AppendVector(structure); // This function appends 20 times

        // Another style for handling the success value
        bool successful = true;
        successful = successful && structure.m_vectorSize == 20;

        // All elements should be in order of 0, 1, 2 etc
        for(int i=0; i<20; i++){
            successful = successful && structure.readElement(i) == i;
        }

        return successful;
    }

    // Another normal case, append has no error/edge cases
    // Actually an error case is running out of computer memory but we don't want to crash the GL server
    bool testAppendOverCapacity(){
        int arr[10] = {}; 
        Vector structure(arr, 10);

        // Capacity should double if more size is needed for the array
        // This improves the average runtime, since reallocating memory and copying all elements is O(n)
        structure.append(0);

        bool successful = true;
        successful = successful && structure.m_vectorSize == 11;
        successful = successful && structure.m_capacity == 20;

        // Append another 20 times so it can double again
        AppendVector(structure);
        successful = successful && structure.m_vectorSize == 31;
        successful = successful && structure.m_capacity == 40;

        return successful;
    }

    // Normal case
    bool testSetNormal(){
        int arr[10] = {}; 
        Vector structure(arr, 10);

        bool successful = true;
        for(int i=0; i<10; i++){
            successful = successful && structure.readElement(i) == 0;
        }

        // Make sure it returns true!
        // You'd be surprised how many people forget to return true in their implementations
        successful = successful && structure.setElement(4, 10);

        successful = successful && structure.readElement(4) == 10;

        return successful;
    }

    // Error case
    // setElement returns false in this error case, so we can simply catch the return false
    // There is nothing else for us to test in this case, since the program not seg faulting is what we want
    bool testSetOutOfBounds(){
        int arr[10] = {}; 
        Vector structure(arr, 10);

        if(structure.setElement(-5, 20) == true) return false;
        if(structure.setElement(15, 20) == true) return false;

        return true;
    }

    // The normal case of read is covered in the other cases, so it can be omitted

    // Error case
    // readElement has an int return value, so we can't indicate errors with the return value
    // Instead, it will throw an out of range error
    // This test case will ensure that those errors are correctly thrown
    bool testReadOutOfBounds(){
        Vector structure();
        AppendVector(structure);

        bool caughtErrors = true;

        // Negative element
        try{
            int output = structure.readElement(-5);

            // If this part is reached, then the function didn't error out and we have undefined behavior
            caughtErrors = false;
        } catch (const out_of_range& error){
            // Out of range caught!
        }

        // Over the arraySize (but likely still within capacity)
        try{
            int output = structure.readElement(21);
            caughtErrors = false;
        } catch (const out_of_range& error){
            // Out of range caught!
        }

        return caughtErrors;
    }
    
};

// Test print function
// Just contains the formatting for how we'll print the tests

void printTest(bool testResult, const char* testName){
    if( testResult == true ){
    	cout << testName << " passed." << endl;
    }else if( testResult == false ){
    	cout << testName << " failed.*****" << endl; 
        // asterisks added for visibility
    }
}

// Test function macro
// Macros are simple text replacers, which will make testing your projects simpler
// In the majority of cases using a function is more clear and trackable, but for certain situations they cut down on the amount you write

#define TESTER_NAME tester
#define STRINGIFY(s) #s
#define TEST(testName) (printTest(TESTER_NAME.testName(), STRINGIFY(testName)))


// In main, we want to call all of our test cases
int main(){

    Tester tester;
    
    // Using an if statement can work:
    if( tester.testConstructor() == true ){
    	cout << "testConstructor passed." << endl;
    }else{
    	cout << "testConstructor failed.*****" << endl;
    }
    
    // We could also do it using the print function:
    printTest(tester.testZeroConstructor(), "testZeroConstructor");
    
    // Lastly, we can also use the macro above to write the print function
    // Doing: TEST(testInvalidConstructor)
    // Will expand to: printTest(tester.testInvalidConstructor(), "testInvalidConstructor")
    TEST(testInvalidConstructor);

    // It doesn't matter what syntax you use, as long as your test functions are called.
    // We only want to see the output of all test cases after running mytest.cpp
    // There should be no other prints, dumps, or couts.

    TEST(testAppendNormal);
    TEST(testAppendOverCapacity);

    TEST(testSetNormal);
    TEST(testSetOutOfBounds);
    TEST(testReadOutOfBounds);
    
    return 0;
}

---

Testing with Random Data

Some data structures have many different cases, like the many rotations of a binary tree or different number of possible collisions in hash maps. In order to avoid making individual test cases for every single case, we can use random data to hit every case we want.
This page will not cover specifics but it is important to note:

• Using the Random class provided in the project materials is required if random data is needed. Using other methods will result in a large point deduction.
• Large amounts of data are required to test every case, some cases (like deleting the root of a large tree) may be missed without enough data.
• Using large amounts of data can expose slow run times in your program.
  If your test file starts taking a significant time to run with hundreds of data points, it’s likely your implementation does not have the right asymptotic runtime.