June 26, 2024 at 12:00 am
Comments posted to this topic are about the item Where to Test Your Code
June 26, 2024 at 10:16 am
I do a lot of Python development for cloud environments. When designing and writing software I ask myself "How can this code be tested robustly in a mechanical way"? Often at least as much thinking goes into the test approach, especially for cloud integration tests. With it being the cloud and inherently a distributed system it can be difficult to test even with careful thought.
With AWS there is a library called Moto (the main AWS client library is called boto). This does let us create testing mocks so we can test much locally. I haven't come across anything similar for Azure or GCP so if anyone has experience with this please shout up.
I find unit testing is great for spotting embarrassing errors in individual functions. The way I think of unit testing and integration testing is
I don't mock the DB as such. As it is possible to create Docker images for many DB platforms, that is what I do. For things like Snowflake and BigQuery the Docker image approach isn't possible.
I would caution against rejecting DB testing on the grounds that you can't simulate production loads or volumes. Those things are obviously important but being able to test that something mechanically works, albeit at lower volumes/load, is a big step forward. I have seen integration tests pick up surprising bugs in the way the app uses the DB. I must reiterate that there is a lot of careful thought that needs to go into designing and building your tests and everything that goes into them.
Some aspects of DB testing are not those that have widely available solutions.
June 26, 2024 at 12:16 pm
Typo: bit
June 26, 2024 at 2:33 pm
Learning how to do unit tests is what got me my current job. I never did unit testing in my previous job, so I picked up unit testing and am glad I did.
However, when I entered my current position 9 years ago, they hadn't done any unit testing, either. So, we learned together, which was a growing experience. The first thing we tried to do was 100% code coverage. But that turned out to be crazy, as it was silly to test to see if assigning a value to a property of an object, would return the same value. (No, threading involved, so it really wouldn't change, between assignment and retrieval of the value.) So, after spending many weeks to months just verifying that
x = y;
always returned the same value for x, we gave up on the idea of 100% code coverage.
The one thing I didn't agree with was when my former boss insisted that all we should do is write integration tests, and he'd call them unit tests. Ultimately, the only thing that mattered to him was that the value that came out of the database was what was expected. But when the "unit test" failed, then it was always an issue of trying to figure out if it failed in the unit test, the code the unit test was testing, the network, or the database. I prefer writing both unit tests and integration tests, but that's not what we were supposed to do. Oh well.
Kindest Regards, Rod Connect with me on LinkedIn.
June 26, 2024 at 3:00 pm
Learning how to do unit tests is what got me my current job. I never did unit testing in my previous job, so I picked up unit testing and am glad I did.
....
Any testing is good, though it's some level of debt to support, so these need to be trimmed and managed. I think the trick of testing early is to understand you'll do silly things like check x=y, or that insert into x (y) value (z) means that z is in the table. In general, we shouldn't test things that are handled by the platform.
Part of testing maturity is learning what tests to write and not to write. I think that testing should often focus on bugs, or things your team does wrong (or repeats because of copy paste).
Also, I think 100% coverage is silly.
June 26, 2024 at 5:08 pm
Test coverage as a metric is a mine field. It's like page life expectancy or buffer cache hit ratio. It's all in the context in which they are used and in what combination.
A few years back I had to upgrade a number of Python applications where there were no tests to help evaluate whether changes would cause breakages or not.
The code was huge and convoluted and when I added a test for a bit I felt I understood I was shocked to get a very high test coverage percentage. The metric was based on the number of lines touched by tests. As the code was a sprawling mass of interdependencies a lot of lines were touched by a simple function call.
Test coverage as a metric needs to be considered when combined with the metric of cyclomatic code complexity.
These metrics only give value when they are used with honest intent and knowledge of how they should be used. They should be a product of behaviour rather than a goal in their own right
June 27, 2024 at 12:56 am
Test coverage as a metric is a mine field. It's like page life expectancy or buffer cache hit ratio. It's all in the context in which they are used and in what combination.
A few years back I had to upgrade a number of Python applications where there were no tests to help evaluate whether changes would cause breakages or not.
The code was huge and convoluted and when I added a test for a bit I felt I understood I was shocked to get a very high test coverage percentage. The metric was based on the number of lines touched by tests. As the code was a sprawling mass of interdependencies a lot of lines were touched by a simple function call.
Test coverage as a metric needs to be considered when combined with the metric of cyclomatic code complexity.
These metrics only give value when they are used with honest intent and knowledge of how they should be used. They should be a product of behaviour rather than a goal in their own right
Very good points, David. I look back at those times with the realization that we were only beginners at writing unit tests. Trying to do 100% code coverage seemed the right thing to do, but we'd never done any unit testing before. Baby steps. Well, we learned better.
Kindest Regards, Rod Connect with me on LinkedIn.
Viewing 9 posts - 1 through 8 (of 8 total)
You must be logged in to reply to this topic. Login to reply