Unit Testing: from theory to real case

Thank you for your reply, Don. They are valuable words that I read and hear many times. However I'm in trouble to translate them into real testing.

When you write, test for this, test for that, what happens if the client uses the module in a wrong way, what happens when the system clock changes a little or a big, and when the task missed the exact timestamp of an event?

I was trying to write tests for *all* of those situations, but it seemed to me a very, VERY, *VERY* big job. The implementation of the calendar module took me a couple of days, tests seem an infinite job.

I have four types of events, for each test I should check the correct behaviour for each type.

What happen if the timestamp of an event was already expired when it is added to the system? I should write 4 tests, one for each type.

AddOneShotEventWithExpiredTimestamp_NoActions AddWeeklyEventWithExpiredTimestamp_NoActions AddMonthlyEventWithExpiredTimestamp_NoActions AddYearlyEventWithExpiredTimestamp_NoActions

What does it mean "expired timestamp"? Suppose the event timestamp is "01/01/2024 10:00:00". This timestamp could be expired for a few seconds, a few minutes or one day or months or years. Maybe the module performs well when the system time has a different date, but bad if the timestamp expired in the same day, for example "01/01/2024 11:00:00" or "01/01/2024 10:00:01". Should I add:

AddOneShotEventWithExpiredTimestamp1s_NoActions AddOneShotEventWithExpiredTimestamp1m_NoActions AddOneShotEventWithExpiredTimestamp1h_NoActions AddOneShotEventWithExpiredTimestamp1d_NoActions AddWeeklyEventWithExpiredTimestamp1s_NoActions AddWeeklyEventWithExpiredTimestamp1m_NoActions AddWeeklyEventWithExpiredTimestamp1h_NoActions AddWeeklyEventWithExpiredTimestamp1d_NoActions AddMonthlyEventWithExpiredTimestamp1s_NoActions AddMonthlyEventWithExpiredTimestamp1m_NoActions AddMonthlyEventWithExpiredTimestamp1h_NoActions AddMonthlyEventWithExpiredTimestamp1d_NoActions AddYearlyEventWithExpiredTimestamp1s_NoActions AddYearlyEventWithExpiredTimestamp1m_NoActions AddYearlyEventWithExpiredTimestamp1h_NoActions AddYearlyEventWithExpiredTimestamp1d_NoActions

They are 16 tests for just a single stupid scenario. If I continue this way, I will thousands of tests. I don't think this is the way to make testing, do I?

Il 30/08/2024 10:18, pozz ha scritto:

That is true if the repeated events (weekly, monthly, yearly) have a limited number of repetitions *and* the system time is over the last repetition when they are added to the system. Otherwise, even if the system time is over the event timestamp (first occurrence), other repetitions could happen in the future, so they should be added to the list of active events.

So there isn't just one test for the WeeklyEvent, but there are more:

WeeklyEventWith10Repetions_SystemTimeIsOverAllOfThem_IgnoreEvent WeeklyEventWith10Repetions_SystemTimeIsOverOnlyTwoOfThem_AddEvent WeeklyEventWithInfiniteRepetions_SystemTimeIsOverSomeOfThem_AddEvent

This for weekly, but also for monthly and yearly.

And what if the addition of events (the call to calendar_init) is made with multiple events and not only one? Maybe the module behaves correctly when just one event is added (expired or not), but behaves bad when two or more events are added.

Maybe it behaves well when all the events are completely expired, but bad when one is partially expired and on the totally expired.

The combinations that we could imagine are infinite.

So you're confirming it's a very tedious and long job.

This is a simpler case. Just test for a couple of different result and you're almost sure it works for normal case (positive integers).

I read that tests shouldn't be written for the specific implementation, but should be generic enough to work well even if the implementation changes.

Now I really know there are only two paths in the current implementation, but I'm not sure this will stay the same in the future. What happens if a developer changes the code in:

if (event_type == SINGLE) { if (event_timestamp < current_time) { // Event expired, ignore it } else { add_event_to_active_events(); } } else if (event_type == MONTHLY) { if (event_timestamp <= current_time) { ... } else { ... } } else ...

The paths could multiply.

Yes, and the test cases proliferate like ants... sigh :-(

Yes, yes, yes, please add tests, tests, tests.

My trouble isn't inventing new tests for normal and edge and corner cases, but limiting them to the real necessity, otherwise the software development stucks on tests.

Ok, but if you create tests knowing how you will implement functionalities (execution paths), it's possible they will not be sufficient when the implementation change at version 305.

I agree with you. Indeed the beautiful theory that tests shouldn't take into account real implementation is imho false.

I know a test case should verify a functionality that is indipendent of implementation, but the number and type of test cases could change

Just as a limit case, suppose you ahave a function that calculate the square of a number in the range 0-15, so the result can be returned in a unsigned char. The return value is undefined if the parameter is greater than 15.

unsigned char square(unsigned char num);

Before implementing the function I can imagine the following test cases:

assert(square(0) == 0) assert(square(1) == 1) assert(square(2) == 4) assert(square(15) == 225)

Now the developer writes the function this way:

unsigned char square(unsigned char num) { if (num == 0) return 0; if (num == 1) return 1; if (num == 2) return 4; if (num == 3) return 9; if (num == 4) return 16; if (num == 5) return 35; if (num == 6) return 36; if (num == 7) return 49; ... if (num == 15) return 225; }

My tests pass, but the implementation is wrong. To avoid this I, writing tests, should add so many test cases that I get a headache.

Am 27.08.2024 um 12:52 schrieb pozz:

Not investing at the start means: pay it later. In longer debugging sessions, or more bugs. Or more effort when you start testing.

That's part of the reason why testing is perceived as a burden, not as a help: you have this big pile of code and no idea where to start. It has not been designed for testing, so you sit there: how the f*ck am I going to test this beast, simulate time, simulate file access, etc.?

If I were to design this interface from scratch, with testing in mind, it would look like:

void calendar_task(struct Calender* p, time_t t);

One: the calendar should be an object, not global state. This means I can create as many calendars as I want for testing. Making this possible usually immediately improves the flexibility of your code, because you can of course also create multiple calendars for production. New feature: product now can support multiple users' calendars!

Two: do not have the function ask the system for the time. Instead, pass in the time. So, no more need to fake the time as global state.

At least for me, it makes testing feel quite natural: each of the tests I build is a tiny system setup. Not some Frankenstein monster with replaced system calls and linker tricks.

(One more change I would most likely make to the interface: have it tell me the time to next event, so I do not have to wake up every second; only, when an event is due.)

Depends on your requirements (black-box), and your implementation (white-box).

A calendar with zero events.

A calendar with one event, per type.

A calendar with multiple events of same or different type.

Some border cases that you happen to find in your implementation (e.g. a one-shot event that has long passed, or a repeated event with interval "every 0 seconds").

Does this really make a difference with your implementation? Isn't it just a for loop, with the cases "zero" and "one or more"? The insertion logic probably distinguishes between "one" and "two or more".

Are you measuring coverage (e.g. lcov)? That should tell you your blind spots.

The simple every-day tests make sure you do not accidentally break something simple.

The boundary case tests make sure that bringing your software to its boundaries does not break it (e.g. "nobody can hack it").

I wouldn't say one is more important than the other. But both help me sleep better.

Stefan

On 8/27/2024 6:52 AM, pozz wrote: > ...I'm a newbie in this aspect of software development. >