Bugs and test steps in UI
In this video, let’s cover some very cool concepts like bugs comments, collaborative regions, and so on and so forth.
So, take a look at all of my test cases here that I previously ran as part of a batch. One thing that I quickly want to tell you first is, check out these names either the test names.
Remember I told you to keep them short. Then, they’ll appear in the badge view so that you can actually see them.
Well, that’s why you should keep your test name short because if you want to use your, you know? Test method name as your test case name, then you’ll be able to see it here fully like this.
Otherwise, if it’s really long then you can’t 100% see it and then you get these, you know? Ellipses here.
But, if they’re short you can see all of the test names and it’s super easy to know what you did. Of course, you could always, you know? Hover over, you can hover over and see some more information here.
Like you can see the environment, you can see the test, you can see the step, the match level, and so on.
But this just provides you with a very quick view. Then, obviously, you can also filter on the test results using these test names anyway.
Let’s go and take a look at this page actually!
So, now the page is going to go ahead and load up. What you can see is actually I was able to previously capture a baseline. That’s what you see over here on the left. On the right-hand side is the new version of our webpage.
The very first thing I want you to notice is, look at how I captured a full-screen page. If we actually go to ultimateQA.com, you’ll notice that this is the view right now.
It’s not a full screen, it’s, this is the page that would be captured if it’s opened in a browser. However, using Applitools, I was able to capture this entire page in full-screen mode.
So pay attention to that, I’m going to teach you how to do that, don’t worry about it!
However, what I wanted to show you here are the discrepancies that are actually visible. So, what I’m going to do is place this one on top of another.
So, I’m going to go ahead and hit “S”, so that they’re one on top of another. I’m actually, remember this I’m going to zoom in on the differences. So, right now, it’s highlighting to me the first difference.
If I want to tap through this difference, you can see the problem. Actually, let me turn off the differences using the “D” key to turn them off. Now, I’m going to tap through this, so you can see that the baseline previously had overlapping text.
Now the checkpoint does not have overlapping text in a 720p resolution. So, actually, I have fixed a bug here that was previously a bug in the baseline which is fantastic.
Let’s go ahead and see the other differences. That was actually it.
So, in this case, what do you think I should do? Should I keep the baseline? Or should I accept the checkpoint as the new baseline?
Well, since the checkpoint now resolved this bug that I had previously had, what I’m going to do here is go ahead and give a thumbs up and I’m going to save.
Now every test case that runs on 720p with the parameters that I executed on Windows 10, Chrome, 720p resolution is going to be checked against the new baseline.
Now, there are other failures here, even though, I gave this a thumbs up, this is actually a failure here.
Let’s actually go ahead and take a look at this one, which I gave a thumbs up for demonstration purposes but it’s actually a failure. We need to take a look at it.
So, I’m going to do the same thing here. And, once it loads up I’m going to put it into this view which I really like. I’m going to diff it.
I’m going to hit the period to zoom in on the first differences. And, now, I can tap through it to see what happened.
You can see that the text has actually shifted down. Let me zoom out a little bit more this is super close.
So, you can see first that the text has actually shifted down, so, if we diff that, the text has shifted down. Keep in mind that I actually captured this in layout mode.
So, the text has shifted down between the baseline and the checkpoint. Also in the baseline, you can see that previously I had this overlapping text.
So, what we could have done if this was the case, we could have used a bug region. Remember I told you that we could have bug regions and now is the perfect time to use it.
So, what I could have done is add a bug here that said: “hey this is a bug”, right? And, I could give this title “Overlapping text”. Then I can hit create and then I can, you know? Say something.
But, I think this says enough. So, now, what you can do is actually continue to fail the test until this gets fixed which is exactly what I want.
I’m going to continue to fail the test until this gets fixed. Now, I’m going to save that. So, this one has a bug but and then, let’s continue to look at what else there is here to analyze.
Let’s take a look here, there are also some differences down here that Applitools is pointing us to. Again, it seems like just the shifting of everything down as a result of the larger navigation bar. So, let’s take a look at the navigation bar.
So, you can see that between the baseline and the checkpoint the navigation bar is actually smaller on the checkpoint. That has shifted everything over.
So, overall the checkpoint is actually good now, we’re happy with it. That’s why we gave it a thumbs-up, so now everything will be used from this point forward as the checkpoint.
Finally, I wanted to show you one more interesting thing here. These are actually test steps, so, what you’re going to be able to see here is now we have this scroll bar that’s enabled here.
And, you can actually look at images in sequence. So, look at this image and now look at that image. See that? look we have an image here and an image there. This one doesn’t have a baseline, that’s not a problem.
But, you can see that it was actually able to capture two images. One here and one here, And, they’re both in a single test.
These are called test steps and they are exactly as it sounds. If you would like to validate some kind of a flow, where in my case I went from the home page to the blog page. Now, I want to validate the blog page.
Well, I can do that using test steps and I can put all of them inside of a single test, right? Two actions in the test, two visual validations.
If I wanted to continue the flow, I can continue to do different kinds of visual validations. Now I could have an entire automated test with visual validations for an entire automated flow.
Don’t worry! I’ll show you how to do this using code as well. With all that said, let’s go and take a look at the code so that you can better understand everything that’s going on.
Test steps code
In this video, I want to show you how to do test steps to have multiple visual validation checkpoints in a single test.
Then, also, I want to show you a super cool technique where you can actually apply ignore regions, or layout regions, or floating regions to different kinds of elements in different parts of the page using code at runtime.
It’s super cool!
So, anyways, let’s talk about the test steps. The test steps you’ve seen before, that’s this test case right here.
It’s actually super simple, all it does is it basically allows you to have test steps that you might have in a work flow, right?
I don’t really have too many workflows on my website, but you can imagine maybe you’re navigating through a standard user workflow where I’m going to the home page, and then to the blog, and then maybe you’re going to another site.
Or you might have a workflow where you’re opening something like amazon.com, adding an item to a cart, and then going in the cart and capturing that screenshot.
Or you might even want to break this down to be at, like, you know, page level elements of the page level.
So, for example, if you add an element to a cart, you might now zoom in to the cart area and capture a screenshot of that to see how it looks. Make sure that it looks visually perfect, as well.
Basically, any point whenever you want visual validations to occur in multiple steps of your test case. This is the perfect scenario.
It’s really easy to make that happen. I simply opened up a new Applitools test and I gave it a name called “GroupTestSteps”.
Then, of course, that creates a unique baseline, then checks it for one exists and if it does and then it creates a unique one. Then, I navigated to this URL.
And, I used the check window method to check the entire window.
This is not the full page. This is only what’s visible in the viewport. That’s the difference between a check window and a check.
A check is a much more dynamic while a check window, you can see, well, actually check window has a match timeout parameter that you can pass in. Or, just a tag. A tag was a string that comes in, it’s basically like the test name that appears in the test manager.
That’s what I did, I named one homepage and one blog page. In between them, I simply navigated to a different URL and then I did another check window.
And, we continue to do this, until we are satisfied. As I said, of course, you could also use the check method and then be able to apply different checks on two elements or certain portions of a page.
I’ll show you how to do that in a couple moments.
So, if I run this, what you get is a result that you’ve seen before. I’m just going to flip to the browser and show you that here. Again, I’ll show you the filtering options. Well, actually, it’s not possible because I’m not just running that single test.
But we can take a look at one where all of the five were executed. Here’s the five that are unresolved. Then we can if we want we can filter it down based on the test steps. You can see that over here. Here are the test steps.
So, this is my baseline. Oh! By the way, this is the baseline that was captured without scrolling. You can see how these elements didn’t become live because we never scrolled.
Here’s the check window, you can see it just captures the window that is present. It was in 1920 by 1080 viewport size. So, this is all that was visible in that viewport size. And, so, that’s all that check window captured.
Then, here’s the next step that you see for the blog. I don’t have a baseline for this one. I could set a baseline for this one, but the problem is this always changes.
So, we need to figure out a way of how to handle it, of how to make this dynamic so that it doesn’t depend on the text or the image that’s there.
But instead, it just worries about maybe something like the layout, right? is the layout good? and so then that will tell us if something is broken or not.
That’s how you do steps, really simple!
Are you having a hard time with bug comments? Did this post help you? Let me know by leaving a comment below.
Nikolay Advolodkin is a self-driven SDET on a lifelong mission to create profound change in the IT world and ultimately leave a legacy for his loved ones, community, and the world at large. Today, he serves as the CEO and Test Automation Instructor at UltimateQA.com and contributes informative articles to leading test automaton websites like SimpleProgrammer.com and TechBeacon.com