Tips & Tricks – How to Autonomously Generate Tests
 |
There are many ways to create tests in testRigor. In this session we will talk about two ways to generate tests autonomously. The first way is based on our AI creating tests while analyzing the functionality of your application. The second way is to automatically generate tests matching the most frequently used functionality of your software product.
Here’s the transcript for your convenience:
Artem Golubev: Hello and welcome to testRigor tips and tricks session. Today we will talk about a couple of different ways of how to do autonomous testing and test generation on testRigor.com. Probably the best way to show this is on a mobile application. Then I will show you additional ways to do it. So first of all, for testRigor, there are two ways which testRigor provides supports on how to generate tests for you. The first one based on an intelligent crawler which will use AI to go through your application and figure out what tests should be created. And number two, you can deploy our analytics library in your production and testRigor will get analytics about how your users are using your application and use that data to generate functional end-to-end test automatically matching the most frequently used end-to-end scenarios. We will start with a simple example of a mobile application for generating tests, simple mobile test generation, and we’ll do native mobile and generate this test. Let’s use Galaxy; we’ll get an application.
All right. So then, as you noticed when we were creating the test suite, I left an option to automatically generate up to 10 functional end-to-end tests. The actual number might be lower than that depending on different factors. When the system will decide to go for specific areas or not. And how many tests it will generate where of course, there are a lot of different settings to function that, and you can generate a vast number of different tests. In general, testRigor will try to prioritize the new paths for it not go through yet. To maximize coverage as much as it can. All right, so what’s going on right now is the sensitives is a mobile application. TestRigor starts a new instance with a native mobile emulator. In this case, I think we have selected Samsung and then it will start the emulator and upload APK into the emulator start the emulator, restart the application, and then we’ll start to go for the application, and as soon as tests are ready, they are added automatically as you have seen, it was one edit, and when after some time we will expand the second one, and they’ll show you what it looks like. So basically, here. It’s a very, very simple application, quite realistic, as you can see, and we’re just two buttons and where I’m ready and about, and the first test just clicks on the first button, on the second screen and second button, and the third screen it will stop automatically because there are no more options available on the screen. And the second one is once it clicks on a second button, it’s getting a situation where no actions are available. And of course, if you have different inputs and different ways to input information, it feels like important information before clicking on the button on the screen. So this particular approach works best on mobile applications why because mobile applications have a limited interface, and therefore, you can generate a lot of meaningful tests. It is the Order of magnitude far more complex, visit the web and the interfaces containing 10s and sometimes hundreds of different actions on every single page. So the crawler might not be able to pick the right action on specific steps and therefore might not prioritize the flows, which you might consider most important all though; it tries his best. So now let me show you another example, and I’ll talk about the Generation the capturing errors specifically in this mode.
So if you’re creating your application and call it automatic tests, we’ll use this simple URL, and let’s generate some tests. In this case, it will work a little bit faster. Because this is the web, there’s no need to download and upload APK, and the application is trivial enough for us to extract. This should get the first task generated for us, so even though it is a mobile application of the UI, this particular application is a trivial one. It has just one input and one button, so testRigor generates the test and enters the data into input click buttons. And then it is as you can see, the text above changes. So now, let’s open up this application, and this is how it works. Right. If you enter something here, click on the Update button; it updates here. So let’s imagine that he released a new version of the application and as the New button here. However, if you imported data here, click Update, and nothing happens. It doesn’t update the test anymore. So let’s see if our system can catch up with that automatically. As you can see, there are no explicit validations here. I can go here, and we use a change URL instead of doing something else, but in your particular case, you probably would have deployed a new production version as the run. So basically, it runs the same tests but uses a different base URL in our case. And it was pretty fast in here, and as you can see, it’s the same actions, and when it detected that the text is no longer on the screen highlighted in red, this is one.
And the second exciting part about automatic test generation is. If you have ten tests, specify ten, and the number of tests is lower than 10. Right? Lower than you requested it to generate when it generates additional tests as soon as it sees additional functionality. And you can see the results in a tree view. And how it goes through your application for different buttons. And as you can see, click different messages. You can see the screenshot and these three views on how it goes through your application. Now, if this is cool and all, how can you control that? For this particular test, or test generation technique, it’s all in the settings then. I believe in retesting mode. You can control how you generate errors. We would only check the last page to compare text inputs and buttons, and you particular validate every page, and you can control what is being validated by default. Everything. It both texts, all kinds of inputs and all kinds of links and buttons, and all kinds of lists. And I believe we have some way of specifying a maximum number of tests in the settings.
Okay yeah. This particular tab here is exactly hotel generation. And as you can see where to toggle switch I was talking about; specifically, we’re dealing with here, they have a crawler-based test generation. And this is where you can control several tests, a maximum number of tests it will generate for the maximum depth it will go into, and so forth; several different options are available here in those books. Anyway, so this is one way of doing that. As you can see, it works well on simple and all mobile applications; however, on complex web UIs, it might not do a good job. Big just because the number of different elements on screen is gigantic and might not necessarily guess correctly which one to click in each case. However, we have another way of generating tests, as I mentioned. There is a way to automatically generate tests based on how your users use your application. And for that, you can copy-paste the script that we provide into your page. Usually, it would be in the footer of every single page. As soon as it’s done on every page, the system can continue and process it. So basically, it will not capture what your users are inputting in your inputs. However, it will video gather information about what flows your resources are taking. In a similar format as you can see in the Tree View, although it will not be represented to you in a Tree view format as just will be completed internally, and then once it is done, you will start guessing tests generated. If enough users score for a specific flow, it will be prioritized, and then a test will be generated in Order of the most frequently used flows. It’s a long process. It’s a multi-step process, and it requires enough data to be able to even kick in, but if you have enough traffic on your website, you can leverage that and get yourself tests generated. Okay, so let me switch to answering questions. They have scope-based trial versions. Is it so based on storing this best trial version? Okay, so Karen asked; Is there a desktop-based trial version? There is no such thing as test rigor.
TestRigor is a SAAS. You can get that online. You can deploy it on-premise, and several customers switch to that. However, it will be deployed in your network as in internal SAAS. So it will be some kind of internal website in your internal VPN network where you can access your internal version of testRigor. It is not a local desktop application. The next question is. Very recently tried automating the car pushers. I was able to automate users as the pitch was divided into two screens static and as dynamic changing. How do we approach this situation? It’s a generic question. Nothing to do with test generation. But let’s do that. On testRigor, the instructions that specify the testRigor do not rely on implementation details. So whatever you’re instructed to do, it does not matter. If the section is dynamic or static, it will just work. So let’s go to tesla.com. We will not generate automatic tests for this particular example to speed things up.
And I guess to be able to get to dynamic forms. You need to purchase something. Let’s purchase model three; yes, it’s clicking on model three. Order now. I want to see the purchase price; I want to select performance. What else? Let’s add full-self driving, click on continue to payment, click on Order with a card, and enter some data here. I’m not going to continue to finish this form by just entering some personal settings.
Placeholders are not going to work, of course. All right, so let’s put Tesla here and purchase model three. Anyway it was simplified a little bit. Click model three. Create. Order now. Your purchase price, click on continue to pay, or there is the card insert text without the URL changed, or they don’t care about that. We need to click on the first page, okay. So as you can see. So then, when the recording stops, it also detects the URL changes. We don’t care about this. Just specifying what needs to be done and when they click on what is this Order now. Place Order okay, all right. This is our test with clear steps. So testRigor is an executable specification engine. It is just the correct specification and English of how things should work, and it will execute it for you emulating how a manual tester will do that. Click on Order now. The purchase price as they wanted to and with the performance and Blue. Everything is blue in here. We need to specify Blue Metallic, so let me so when you’re specifying different things like this scored deeper Blue, I guess. When you specify and click certain elements, you should try to rely on something visible to the end-user. In this case, if you hover over the visible tooltip and it says the blue metallic, this is what we can rely on. Order Visa card is not available after we click to continue to payment. Okay, so this button, I guess, was not available. So before continuing to pay, we need to scroll down and scroll down instead of blue. Let’s do it sincerely, Blue. So as you can see, it’s just a more or less straightforward process. You just look using screenshots. What didn’t work and adjust accordingly? You do not usually need to look into any details of implementation. Moreover, you should note if you want to keep your tests as stable as possible. Your best bet is to let go of any implementation details and rely purely on specifications of how it should work or look from the end user’s perspective.
Al right, even the deep Blue lasts the whole significant element. I guess we need to scroll down before the deep blue. Yes, another thing which we didn’t do. We need to scroll down, and another way to force it to click on a specific element if it tries to select a larger one is sometimes more minor elements are part of a more prominent button, and the whole thing is a button. You can force it to click specifically on whether or see the most profound element. Anyway, so we have a question here. How do you manage to time to wait for the next page? If it fails to find that element in default time, wait before failing [Unintelligible] The documentation contains all the commands and areas to undo the progress, so we are creating more documentation. But let me talk about timings. So if you go to Settings and click on speed optimizations, there are many different things you can adjust here, as you can see. So by far, the most important one is maximum wait timeout. What is, by default, 40 seconds? What does mean is the system will wait up to 40 seconds; it will wait for 40 seconds every time, but after 40 seconds, the sent element is visible and available for interruption on a page, and then it will interact with that element. So this is one. The second on top of it. By default, you can turn off this behavior to speed things up. It’ll wait until the page stops loading. You can wait until all the scripts finish executing, and there are tons of other options you can meddle with for performance optimizations. Okay, let’s see what…While we are here. Okay, sorry, it didn’t scroll because you don’t. Here, you can scroll on this part.
And let’s try to record it. No, it did not record. So on scroll container, I guess the same thing could also be in here. And we can scroll down until the page contains paint and, in this case, until the page contains payments. It’s not just scrolling on the page because, by default, it will scroll the whole page, which is not scrollable. We need to scroll on the side element. And you can scroll until we get to a specific element. Yes, the question was, can you specify if I can scroll up to some element? This is precisely what we did? You can scroll until you get to a specific element.
Connie Lund: All right, should we wrap up the session? We are a little bit over time. The next two-week session will be on testing iframes and shadow Doms. So we hope you guys will come back, and I will be sending out the recording for this. Thank you so much for joining us. There is a question their item about apart from the documentation. What does the testRigor team support if you run into issues?
Artem Golubev: All our paying customers have a direction of communication with us. The support on directly on Slack and on the MS team, and you can create Zendesk tickets. So please request a demo or email us to set up a trial so we can set up your communication channels for you, and we will be able to answer your questions quickly.
Connie Lund: All right. Thank you.
Artem Golubev: Thank you very much.
