When creating test cases, we classify test case into Priority. However, how many percent for each priority is good? Do we have any "golden rate" for this?
Test case classification
© 2024 AskTester
Theme by Anders Noren — Up ↑
© 2024 AskTester
Theme by Anders Noren — Up ↑
@lanhuynh,
Actually, there’s no one-size-fit-all answer or a “golden rate” for this kind of question.
Basically, most of your test cases should be 1st Priority because those are very important test cases and those are where you put your effort on that. It also depends on priorities of features/components of your systems.
You can research more on MoSCoW priority method to see if it helps you.
I agree with you that there is no one-size-fit-all or a “golden rate” when classify test case. However, the purpose when classifying test case is easy-to-pick test case for testing purpose: Smoke Test, Light Regression, ect. Therefore, we should have a good (best) rate for Test case classification. For example, a test suite with 20% P1 is better than 50% P1; or a test suite has all of Priority 1/2/3/4 is better than all cases is P1 or P2.
So, what is your suggested rate?
@lanhuynh,
I see your need when classifying test case for testing purpose. However, I don’t limit myself to a specific number or percentage of the test based on the *priority* of the current tests I’m having. More often, when selecting the tests, I based on the goal and how much time I’ve got.
Re: “For example, a test suite with 20% P1 is better than 50% P1; or a test suite has all of Priority 1/2/3/4 is better than all cases is P1 or P2.”, can you explain more why you think 20% P1 is better than 50% P1?
Hi aThanh,
Could you please explain why we should not limit a specific number or percentage of the test based on the *priority*?
I know that there are a lot of test case’s properties like Category or Test Area. In Sprint or Functional test, we can select test case base on these properties.
However, the Priority property is a good way to select test case to execute when we don’t have much time or we want to find the high-priority bugs. In general, when manager want to run Smoke test, we can select P1 test cases only, or in Regression Test, we prioritize P1 test cases first, and following by P2, P3, P4. P1 test case is using to find P1 bug 🙂
About my example, it’s just my idea. I don’t think 50% P1 is good because a system has many major functions with few minor features is not a good system.
@lanhuynh,
Look like you forget to note that Priority #1, #2, #3 for test cases.. are just representatives. Priority just tells how important a test case is and we as testers can change these priorities if we wish or when there are change requests. Let me give you an example.
For example in the Build version 1.0 you have a test suite of 100 test cases with the ratio of:
P1-10 TCs
P2-20 TCs
P3-30 TCs
P4-40 TCs
So your smoke test will contain only 10 TCs of P1 (I assume your smoke test just needs P1 TCs)
Now in the Build version 2.0, you have more P1 feature added to the build, so you need to add more P1 test cases to your test suites:
P1-20 TCs
P2-20 TCs
P3-30 TCs
P4-40 TCs
Now, what would you do with your smoke test? Will you update your smoke test to 20 TCs of P1 of just keep 10TCs of P1 like Build version 1.0
Now, let’s assume your products go Production and you’re in Maintenance phase and you don’t have to do any Regression test, so you have to delete un-used test cases of P2, P3 and P4. Now your test suites only consist of P1 test cases.
What I’m trying to say is that things will change, code will change, business need will change, context will change. Trying to *fix* a ratio will do more harm than good. The similar question is the standard ratio between Developers and Testers in a project. Some says 1:1, some says 3:1, some says 1: 30, some says 1:0…so, you get the idea right?
P.s: Thanks for your great questions anyway. Hope it helps you with your issue.
@aThanh:
In most of situations in CI testing, I think we should use MoSCoW priority as you mentioned in previous questions.
When the team can define what’re the features should be implemented as MUST , SHOULD and COULD, …the test suite will be implemented accordingly. I think if we do that, the test coverage will be increased for each release. Then when all the team’re on the same page, the “Manager” team can pick up P1 or anything priority they want when do Smoke test, UAT. Does it make sense?
And when we do regression testing, if we don’t have enough time to run all the P1 case, I suggest we should pick up the major features, OR the features that often have many bugs (cause impacted by code changed) for run.
Any comment is welcomed.
Cheers.
@Phuoc,
Yes, your strategy totally makes sense to me :-). It’s true that we don’t have enough time to do testing. I wish we could have more time spending on testing….
@Thanh
I’m not really understand your comment. Follow your idea, the version 1 has P1-10 TCs/P2-20 TCs/P3-30 TCs/P4-40 TCs.
After that, the version 2 has P1-20 TCs/P2-20 TCs/P3-30 TCs/P4-40 TCs
I don’t know why in the next version, only P1 has more test cases. In general, if there are new features in AUT, all Ps will increase its test cases.
Moreover, for your question about Smoke Test, we still select the P1 only, it mean 20 test cases will be performed.
I agree that the Priority for test cases can be changed in SDLC, but it doesn’t mean the percent of each priority can be changed too much. I mean, it can be change from 10% to 13%, but 10% to 20% or 25% is too much.
I understand that there is no one-size-fit-all number, but we can discuss together and see which rate is better. And, what is you idea?
@lanhuynh,
Re: “I don’t know why in the next version, only P1 has more test cases. In general, if there are new features in AUT, all Ps will increase its test cases.”
> That’s just an example. The purpose of test case is to verify a feature. If I see 1 TC is enough to verify a feature, I just write 1 TC. Why I have to write TCs just to match the current priorities.
Maybe your project is running following different process and I respect that but it doesn’t make sense to me when writing TC that way.
Not sure if someone here can give a ratio for your reference 🙂
Goodluck!