Tips & Tricks – Reports by Label and Passing Parameters by Hooks
 |
In this session we will show you how you can neatly organize your test reports within a test suite. You will also learn how to define the “before test suite” and “after test suite” hooks, and consume parameters from it.
Artem Golubev: Hello, everyone. Let me jump right in and share my screen. Okay, so as you can see here, this is our UI. And in the UI, you can add additional labels to test cases. You can click on this plus label here and add the label with the title MyTestLabel, and then press ENTER. If you reload the page, you will be able to see the label added to your test case.
That is one thing. And the next is not only that, but we also will allow you to see reports by the label. So if you go to the Reports section on the left, as you can see, it says Report. Here, click on it. You will see just our standard reports, where you will see the percentages overall. The Report breakdown by labeling the number of tests overall and different results by different browsers. And this is what I was talking about. As you can see here, you can see a breakdown by labeling what’s going on. You can hover over how many tests passed in that particular label and what’s going on in general, and as you can see, this is our MyTestLabel.
And we have one failed test related to that because we just added that red label to that test suites. It’s pretty dynamic. So you can break your application into sections and see the Report in various sections using the labels. All right, this is one, and the second thing we will discuss today is passing parameters. Specifically, passing parameters by hooks. Let’s go back and here.
If you expanded it here, there are tests with hooks. You can now add scripts that will execute before the whole test suite starts and after it is complete. This goal is to help you set up test data and clean up test data before you run all of your tests because, as you remember, all the tests run in parallel. So if you need to do something across the board for the vast majority of your test, it may be convenient just to run it before the test suite. Keep in mind that it might run on a different server compared to all other tests because, as I mentioned yet again, everything is executed dynamically on the servers. However, when you create the test with a hook before the test suit, you can also create test data. And you can use that as the parameters from your tests. We can do it right here. We don’t have to make a new test suit.
For the Twitter test suite and let’s do it here. This test suite shows how to sign up for different sign-up tests here. And testing of the forms. This particular use case was that they were passing with a name, day, and month into the test, and this data came from test data sets as we discussed the last time. During one of the previous videos, however, they can pass those parameters to create something passwords parameters before the test suite. So let us do that. Let us expand it. Let’s add before the test suite, and I want to look up how I can save value on the menu. However, you probably would do this, grabbing it from the screen somewhere from a screen or doing something like this, setting up for tests.
So we can go ahead and save the value for the name Peter, day as nine and month as July. We have created before test suit here, so the good thing is how we can… Let’s copy-paste this test case which uses the first parameters. We can create a test case that will use that data. So let’s add that now, as you can see here, this test is peculiar. Because what is happening is it uses the parameters which we will not provide anywhere. We didn’t specify them in a test if you didn’t identify them in any test data or test data set. We didn’t specify them in the rule and called it to set them up. So we didn’t do anything. However, various parameters will come from our test suite hook when you do a full retest, specifically in this case.
It failed because the name wasn’t set up. That is because precisely in [unintelligible]. This is expected behavior. After all, that would come from before the test suite and before the test suite is executed. Then you rerun the whole test suite. To do it, you go to test suite details, or you can trigger it using API in the CI-CD section. If you have integrations with your Jenkins source, but in this test suite with details where there’s a button saying rerun, this is where you can change the lending URL if, for example, you want to redo those retests on a different environment temporarily. If you rerun it to restart all the tests, but yes, before running all the tests, it will execute before the test suite.
So let’s see, let’s expand that. So here, you will see that before the test suit will execute. It will set up the parameters that we need, and by the way, this example probably is not a good one because you can have if you need static data like this, your best bet is to use test data. But sometimes, you need to go through specific steps to, let’s say, create a new user and then pass that user ID and password into all other tests. That would be an example of why you would need a script instead of just constant custom in all of your tests. And then right now, the test suit hook is completed, and every single test which will be executed later on, we’ll get the data that these had produced before test suit hook like all of them, a couple of them and a curator—starting or running. So the Vega test data from this as well. So basically, it’s a combination. First, you’ll get data from test data and test data sets like this. For example, here, it will override whatever you have provided, then you trigger this in your CI call if you trigger it from Jenkins in your CI-CD.
Like this, before the test suit will execute. It might override it yet again. And inside your specific code, you will write it yet again. So everything is as you would expect. But in our particular case, we care about whatever they have been set here. I hope I provided the names with the variables correctly, you will be passed into entering test cases, and you’ll be able to use them as your exit test cases. This didn’t work because the UI was different. So this is what I’m talking about here.
As you can see, it’s used Peter is the name, even though nowhere in the test have we specified name in the test or any test data, which has the name should be Peter is not there. And these data are coming from specifically the data generated before tests suit, and once again, this is mainly for the setup use cases. I guess this one succeeded, but the form failed because it showed a different UI. The point here is that all data had proceeded. You can see it here. So July 9 is what we’re passing in here. The use case for this is any setup.
Sometimes you need to create some users. Sometimes you need to fill up specific tables, which your test suite will use, and in some cases, you do want to do that because otherwise, you wouldn’t be able to run your tests because you might be getting in a situation where once you run your tests, you cannot reuse the same data. So you will need to generate new data. It might be steps in the UI. It might be some API call to the backend to generate and set up all the data you need for your tests. But the point is you’ll be able to set it up and get the data of it. You need to set it up into variables and then use those variables in any of the tests to be able to refer to things you have just created. That is it. Just very straightforward. And I’m open to questions.
Connie Lund: Do you have any questions about using labels to organize or create reports? For us, the second topic was passing parameters by hooks before the test suite. So anybody have any questions? We can also open your microphone. It’s just a few people, only a few people here. Here’s a question. Are there any plans to make a folder structure for the tests case Organization, so maybe a folder for the labels.
Artem Golubev: So labels are a more versatile and convenient way to manage the test. The problem with folder structure is whether a test simultaneously belongs to multiple folders. It just makes it impossible to find it. For example, if one person puts the test in one folder, but you are from your perspective, it should be another you trying to find it in another folder, and you could not find it. So we decided to go for labels at this time, but we are considering releasing a folder structure that probably will come at some point in addition to the labels. Are we just trying to understand the best way to deal with it?
