Computer Science 2 Automated Grading

While regrettably not open-source due to privacy and academic honesty concerns, I am still insanely proud of these series of Python scripts.

These scripts were a fantastic and unique challenge. More so, as these assignments were quite varied the challenges of grading each assignment was also equally as varied. For most of these assignments, specific implementation details were not specified. This led to students also submitting difficult to test code programmatically. So for these assignments, I usually resolved to observing behaviors of the students' code, testing the very breaking points of the code and usually compared to how the professor's reference program reacted. All other things that I couldn't programmatically test in this way, I would have to read the student's submission.

However, for some of the later assignments, as the specific implementation and function details were more defined in the assignment instructions, I wanted to see if I could test the implementation details of a students code using Python, as opposed to reading and parsing the student's submission manually.

This led me to create such cursed beautiful tests similar to the one below.

def test_bound_harcoded(self):
    try:
        for name in ('MAX_VOLUME', 'max_volume'):
            if getattr(sut, name, None) is not None: setattr(sut, name, 5)
        tv = sut()
        tv.power()
        assert_state(tv, True, 0, 0)
        tv.volume_up()
        assert_state(tv, True, 0, 1)
        tv.volume_up()
        assert_state(tv, True, 0, 2)
        tv.volume_up()
        assert_state(tv, True, 0, 3)
        tv.volume_up()
        assert_state(tv, True, 0, 4)
        tv.volume_up()
        assert_state(tv, True, 0, 5)
        tv.volume_up()
        assert_state(tv, True, 0, 5)
    except Exception as e:
        raise e
    finally:
        for name in ('MAX_VOLUME', 'max_volume'):
            if getattr(sut, name, None) is not None: setattr(sut, name, 2)

Plus, because of the weird awesome quirks of Python. We can get the docstrings and type annotations of functions and objects programmatically at runtime:

assert typing.get_type_hints(sut.mute).get('return', None) is None
assert sut.__annotations__.get('MIN_CHANNEL', None) is None

And we can even check for visibility (even for private variables) using Python's variable name mangling.

assert any(getattr(tv, name, None) is not None for name in ('_Television__status', '_status'))
assert all(getattr(tv, name, None) is None for name in ('status'))

I would say all of the above snippets of code are pretty bad practice in real world uses. Test suites generally should not check how something gets done, but that it does. However, because of this specific use case of needing to verify how the code works, like checking for hard-coded values, annotations, docstrings, and visibility, I am not sure of any other way of solving this.