Automated Testing Will Make You Happier
This morning I realized that we’re half way to achieving our test automation goals. Excellent! I believe that success begets success, so I find it motivating to have achieved this milestone. I am encouraged to complete the second half and achieve our goal.
We didn’t launch Cheezburger with complete test automation. It’s a long story as to why, but I’ll make it short: as a startup, sometimes corners get cut in the name of progress. Unfortunately, cutting test automation is probably one of the most expensive corners to cut. Simply put, writing code without test automation adds cost in the future.
For one, it’s really time consuming and expensive to go back and write tests for code that has already been written. If it adds X more effort to a project to write the tests first, then in my experience it adds 3X to write them days, weeks or months later.
In addition, manually tested code is never tested as well as code that is tested by machines. This is because as the feature set grows, each feature gets less and less human attention on each test pass. Machine testing gives equal attention on every test pass. The lower quality testing inevitably leads to more bugs which result in a lower quality product (and takes time to fix).
Many developers complain that writing tests takes longer and they have a lot of pressure to ship code faster. WRONG! There are plenty of studies and examples that show greater effectiveness and lower total cost with test automation. A Microsoft Research study on Test-Driven Development (TDD) shows that initial development took 15-35% longer initially, but reduced bugs 40-90%. I can guarantee you dollars-to-donuts that a project with a half less bugs ships faster.
I tell my team that writing tests is simply part of the project. When a developer takes on a project writing the code is only a portion of the job–writing the tests is the other portion. The developers and teams that understand that will be more successful, I am certain.
Also, I like to tell my team that automated testing will make them happier. When you ship a project you want to celebrate the achievement, not be dragged down by issues and problems immediately thereafter. Automated testing–done well–reduces these issues leading to more successful launches. In my experience, people are happier when they have success.
Bottom line: I’m proud of my team for having reached this milestone, and I’m eager and motivated to continue on and achieve our goal.
Funny that you mention Microsoft research. They certainly take their time releasing a product (Vista), but I wouldn’t call their code bug free
Louie
24 Sep 09 at 10:49 pm
[...] Speaking Automated Testing Will Make You Happier [...]
An Example Showing How Test Automation Saves Time and Improves Quality « Scott Porad
25 Sep 09 at 9:04 am
With automated web testing, how do you manage the steep time requirements of test execution vs coverage?
What are some practices that you find to do automation “well” in your particular domain?
As the world of the web seems particularly brittle and you have some particularly unique challenges with multiple components.
Franklin Webber
26 Sep 09 at 2:56 pm
The answer to your first question is that developers should be writing the tests, not testers. It will take the person who wrote the code 1/10th of the time to write the tests over another person later on.
We had a history of development where there were developers, and like all people, developers made mistakes. So, we hired testers to check their work. As test automation as come along, now we get special “software developers in test” to write the tests.
That’s a step in the right direction, but I still think it’s not right. We should get rid of testers and SDETs and have developers write the tests. Instead of having 1x dev and 1x tester have 2x dev/test. The output of features will be the same, but the quality will be better.
At Cheezburger, devs write all the tests. We have one “user tester” who does all his testing through the browser. His job is to simply act like a user and ensure the site is working right. We also integrate him very early on in the design process, so that he can eliminate “bugs” before any code is written by simply saying, “guys, to a user, that won’t make any sense”.
In regard to your 2nd and 3rd questions:
scottporad
27 Sep 09 at 10:28 am
[...] one comment After yesterday’s post on reaching a milestone on the way to our test automation goals, I thought I would share with you an example on why test automation makes so much [...]
An Example Showing How Test Automation Saves Time and Improves Quality at Scott Porad
21 Apr 10 at 1:48 pm