|
Written by Arjan Kranenburg
|
|
Saturday, 20 March 2010 14:39 |
|
| |
Lately, it's popular to have Tests driven by something. Since long we have Requirements based and Risk based testing, which undoubtedly would have been called Requirements-driven and Risk-driven Testing if they were thought of in the past decade. Then there are Use Case-driven, Design-driven, Data-driven, Keyword-driven, Model-driven, and Business-driven testing. And I've probably forgotten a few, but Common Sense-driven testing never seems to be an option.
What's missing in the discussion is that it is in most cases best to combine approaches and techniques in order to get a diversified test approach. If you consider the types of bugs that can exist in your application and consider that every type of bug has it's own best way to be detected, it is common sense that different approaches and techniques should be applied. There are approaches that will allow you to find a majority of the bugs, but that may not be the best way to find some types of bugs.
Approaches should be taken broad. Some bugs are easily detected by reviews, white board sessions, or other static testing methods. What is the 'best way' depends on your product, your organization, and the circumstances and is certainly easier said than determined. But I don't believe that there are situations where only one approach is best. And trying to find the bug in a later phase may be best as well.
If you rely on one approach it is getting harder and harder to find the next bug. And there is a substantial risk that not all bugs will be found. To minimize that risk, diversifying the test techniques and approaches is the logical thing to do. |
|
Written by Arjan Kranenburg
|
|
Saturday, 04 April 2009 21:31 |
|
| |
A test ideology that I totally agree with is Context-Driven Testing. But I will never describe my activities like that. In short Context-Driven Testing says that the best way of testing depends on the context. There are so many external factors of influence to your test activities, there is no one best practice that will work for all.
So why won't I use the term Context-Driven Testing?
Because it is a typical engineer's answer to the question how should be tested: "it depends". This is true, but the answer will not get you any further.
Of course it depends. Every project is different. Every team is different. The customer, the budget, the timing and the time are all different. So it makes sense that testing is different as well. Therefor every test project should start with a good thinking session about how the test activities should be done. (Actually, the first question is if testing needs to be done at all.)
And best practices, as well as experience, can help you with choosing a strategy, techniques, tools, etc. Like a list of tips & tricks that can be used. Pick the ones you think are suitable or invent your own. Whether or not right for the job is you're responsibility, not that of the author. No one ever claims that best practices are universally applicable.
Besides, best practices already have a context in them. Testing for traditional engineering is far different than testing software. That's why best practices are often presented like: "Best practices in Software Testing" or "Best practices for Model Based Testing in an Embedded PLC Solution". But unfortunately that context is not always copied and the Best Practice ends up on a list of general Test Best Practices.
Nonetheless people talk about testing like it's all the same. Often you hear a presentation about a strategy on testing a web application and people ask questions with testing of an embedded application in mind. When the context is forgotten, miscommunication is the result. |
|
Written by Arjan Kranenburg
|
|
Wednesday, 11 March 2009 22:27 |
|
| |
Model Based Testing (MBT) is a term frequently used these days. It sounds well and just from the words seems like a good thing to do. But is it really worth the hype? The strict definition of Model Based Testing is when you base your testing on one or more models. This still sounds good, but not so much if you consider the following: - A model is a simplification of reality. So if your tests are solely based on one or more models, you're bound to leave some paths untested.
- If code is also generated from the same model, you're testing the generation process in stead of the product or model. Even if the code was not generated automatically, but the developer also based his code on the model, it is not likely that many faults are found.
- Models must be created or provided.
- Models can contain faults as well.
But lately the term is also used for tests described by models. The model still describes the system, because every test case does, but the primary purpose of the model is to describe the test case. Describe what path is followed, what actions are to be taken, and what is to be expected. Although I don't call this Model Based Testing, Test Case Modeling is perhaps a better term, I do see a great benefit of it. In earlier, pre-Agile times long descriptions used to be made for every test case followed by another description of the next test case that was only different by a few (essential) words. Later, Excel was used to describe test cases by key- and action-words. Test Case Modeling can be the next step in the evolution of describing test cases and to visualize them with the best possible model-type. |
|
|
|
|
