Tag Archives: Automation

Initial Review of RedwoodHQ

I recently saw a post in the Test Automation community on LinkedIn suggesting that we should stop writing custom frameworks. The reasoning behind the suggestion was that it was time consuming to reinvent the wheel when there was an Open Source project, RedwoodHQ, that already had everything you need. Like most of the commenters, my first thought was that I was just reading a sales brochure for another “silver bullet” automation solution. The author, however, insisted that it was a sales pitch since his framework was free and he just wanted to raise awareness of a new tool that is available to the community.

I have long been a proponent of developing one’s own framework to build understanding of the underlying operations. In some ways it’s like Jedi training, you might start out using a lightsaber you were given but you will not become a master until you have constructed your own. In my experience, using pre-existing frameworks can limit an automator’s growth by obscuring the inner workings. Some frameworks will also limit a user’s ability to perform tasks because the developer didn’t consider that use case.

With that in mind, I decided to look at RedwoodHQ and see what it actually is. It wouldn’t be fair of me to write off someone else’s work without at least taking it for a spin. After reading a bit more about it, I downloaded the application and set it up on a VM. The installation process was smooth and I was connected to the web interface and poking around in the sample test in minutes.

For someone with very little knowledge of Selenium WebDriver, they could develop a test with RedwoodHQ right out of the box. They may even be able to begin learning by looking through the action example code. I wouldn’t, however, recommend keeping the original actions in the long term. If for no other reason, they need to be replaced because the current Selenium code uses hard sleeps and possibly relies on implicit waits.

With that said, don’t write off this framework yet. You’ll note that I recommend replacing the actions. This is extremely easy to do using the built-in web IDE and you can drop in any jars you like to use with your tests. This means that if you have built your own framework, you could compile it and reference your own classes and functions within RedwoodHQ. I am currently considering doing just this.

The efforts I have put into my personal framework have been focused on building functions around Selenium WebDriver to make developing page/component objects easier and by extension making the tests easier to write. In this way, I was not limited to using a specific test framework (although I tend to use TestNG most often). I have now realized that unlike other automation tools and solutions, RedwoodHQ operates at the same level as a test and reporting framework and will not interfere with working at the lower levels.

Since this is only an initial review, I can’t speak to how it integrates with a CI environment but I am willing to give this tool a chance. Once I have adapted my Selenium framework to RedwoodHQ, I am considering submitting it as a replacement for the existing actions. I believe the tool would benefit from having actions that utilize explicit waits and provide more than just basic functionality.

The take home from this is that I learned that RedwoodHQ is not another “silver bullet” automation nightmare. It is a test development, management, and reporting framework that won’t limit you to working within their methods. My intial opinion is “Well done!”

Advertisements

A Brief Review of “The Phoenix Project”

Over the past couple of weeks, I have been listening to The Phoenix Project during my commute to and from work. Having finished it today, I decided to jot down some of my thoughts and share them with you.

The first thing I will warn you about is that if you have already read The Goal and understand how it can be applied to IT then you are not going to find any new, eye-opening concepts in this book. The plot follows a middle manager that finds himself being promoted to the head of a struggling department. Through determination and coaching from a elusive and wise mentor, he learns to identify and control work along with how to align it with the companies needs to become successful and rise to the top of their market. For the record, that is the plot for both books. I identified the similarities between the books within the first couple of chapters, but was amused when the main character’s mentor began to quote and reference The Goal while explaining that work is work and IT is no different from a manufacturing plant.

With my main criticism out of the way, I found the story to be a fairly realistic look at IT functions within an company. Misunderstandings and unreasonable demands result in repeated disasters and a generally oppressed and depressed atmosphere. I dare say the stage was set so well that I could not only draw parallels from my experience but was starting to have actual sympathy for the characters because I could see what was coming.

Overall, I think this is a good book for IT staff, managers, and all executives to read and gain an understanding of the processes that help work flow. For the IT staff in the trenches, the purpose of reading this book should be gain an understanding of why processes may need to change and to help acknowledge that their managers need help not resistance. The executive’s take away should be to identify things that were done poorly by the executives and the board in the book so that they can recognize and correct similar issues within their organization. For the last group I recommend this book to, I suspect the take away would be a number of ideas regarding how to implement changes and structure work for their teams and that there is some hope if you can get those above you and below you to listen to reason.

I would recommend this book over The Goal for use in IT organizations simply because it makes it easier to see how the concepts apply to IT while making a point of addressing the common responses from IT personnel. I believe this was the motivation of the authors and, if I am correct, they succeeded.

Building a Selenium Grid

When testing with Selenium WebDriver, it usually becomes necessary to need access to multiple drivers simultaneously. This may be for running tests in parallel or performing cross-browser testing. One possible solution to this is the use of a Selenium grid. The following video from my course on Udemy explain what a grid is and how to configure one for use with your tests.

Automation Strategies vs. Captcha Implementation

I was contacted recently by someone looking for assistance in automating a page that uses ReCaptcha. Since there are only two reasons for doing such a thing, application testing and SPAM, I offered the following information.

First off, the purpose of a captcha is to prevent bots/automation from being used to access an application. This means that if you can successfully automate the captcha then that would be a failed test and the captcha application should be replaced with one that hasn’t been beaten.

With that said, the question of how  sites that implement captcha can be tested with automation remains. This cynical answer is simply, “you can’t”, but this is neither true nor acceptable. The correct answer is to plan ahead and work with developers to implement a test mode that can be toggled securely. This will allow the automation test environment to bypass the captcha and test the meat of the application, while providing confidence in the security of the production site after manually confirming the integrity of the captcha.

After some thought, the manual testing could be skipped (not recommended) if the test mode was built into the captcha module such that when implemented, it would always provide the same challenge. This would allow for verifying the captcha functionality is working and be able to successfully test the remainder of the application as well. The issue with this would be making sure it was not possible to put the captcha into test mode through injections or other trickery in production.

I would be remiss if I didn’t warn my fellow testers that there are methods available for beating captchas such as OCR implementations and some specialized API services. These should be considered when performing your tests and assessing the security of an application. I am not going to promote any of the sites here since I don’t want to make it easier for script kiddies to get their hands on them. They were easy enough to find without my posting links last time I checked.

Automated Testing Using Selenium WebDriver Course Launched on Udemy!

After just over a month of planning, recording, and editing, my first Udemy course, Automated Testing Using Selenium WebDriver, has been launched. This class covers the essential skills needed to begin developing tests and other automation using Selenium WebDriver.

The topics include:

  • Analyzing a web page to prepare for automation
  • Identifying element locators
  • Implementing the Page Object Model
  • Deploying and connecting to a Selenium Grid
  • Developing a universal framework to expedite any automation project using WebDriver.

To celebrate the launch of the course, we are offering a 50% discount to the first 200 students using this link

https://www.udemy.com/automated-testing-using-selenium-webdriver/?couponCode=CourseDebut

 

Automate All the Things!

“Automate all the things” has become a common battle cry heard when discussing software testing. Those who follow that banner will often support it with claims of improved ROI, faster deployment times, and lower personnel costs. In an idealized situation, this would all be true, but reality rarely deals in Utopian terms.

As a developer turned tester, I recognize the importance of good testing practices and thorough test coverage for producing a quality product. As applications become more complex, the possibly of introducing bugs into a seemingly unrelated module increases. Granted that this becomes less likely when using well-planned and executed architecture, but I have found few developers that have had the fortune to never deal with legacy systems (usually some of the most intricate and fragile balls of mud known to man). Since it is always better to err on the side of caution, someone needs to makes sure that new features are working as expected and that the rest of the system is also functioning as it should. Enter the manual testers.

Management teams often consider their developers to be too precious of a resource to have them spend time testing applications and instead bring in the “QA Team”.  This team of typically less technical employees is usually charged with reviewing and verifying all of the functionality in the application prior to release. While this sounds reasonable at first glance, QA typically gets the application two days before release because development ran into snags or was given a short deadline  that would look better at the quarterly meeting. This means that if the company intends to actually get the testing done, the number of testers required becomes a payroll issue. The alternative to this is to automate all of the tests and thus reduce many salaries to a smaller team of developers (wait, aren’t they a precious resource?).  The problem with this strategy is that it is usually implemented with the concept of a “one-time cost”, which is rarely the case. In fact, the only successful automation efforts are those that take into account that tests will require regular maintenance to remain current with the applications.

Another issue with the “Automate Everything” concept is that there are things that can be tested manually more efficiently. Computers may work faster than human employees, but they are either slower to adjust to changes or require a significant amount of time and expense to develop. With this in mind, it becomes obvious that the best strategy for ensuring quality is to develop a team with the skills and authority to implement a blended solution.

In my experience, a team that will produce the most successful results is cross-functional and aligned towards a common goal. Bear with me if that sounded like a case of jargon dropping, it wasn’t. In order to keep the focus narrow I will focus on the developers and QA. This team doesn’t deal in hand-offs between members. Instead, they share the the whole cycle. Many testers may not understand the code that resides within the bowels of the program, but they do know what it is expected to do. It is their job to provide this insight to the developers prior to development to reduce the chances of the finished product diverging from the expected one. Armed with the knowledge of what is expected, the developers should implement appropriate unit tests while writing the new features to make sure that each portion of the application performs as expected. On the other end of the development cycle, the testers are charged with confirming the functionality prior to release.

Ok, you are probably wondering what happened to the automation portion or you think that it was dumped solely onto the developers. You would be wrong. Unit and integration testing are important and should be done as part of development, but they may not cover the full spectrum of tests and requirements. Burdening the development team with writing regression suites would bring them to a crawl in regards to producing new features. Likewise, manual testers will rarely be able to complete such tests within deadlines and would probably be bored to tears performing such tedious tasks repeatedly. This is the proper place for automation.

A good automation strategy identifies the tests that provide the most benefit to the organization that can also be reasonably automated. Scripting a test that accesses the UI and clicks on every button on the screen is a waste of resources. The only thing this test catches is a lazy developer that didn’t even check to see if his code worked. While it can be argued that there is value there, I think such an issue would show itself in other ways. Better automation candidates would be the main business process flows through the application because they are usually time consuming to be done manually and will cover the majority of the applications uses. Having these paths automated will not only save time, but will free the manual testers to explore fringe cases and usability considerations, which machines either don’t do well or would offer little ROI from.

Using Page Objects

The quality of anything you build is going to be a reflection of the underlying infrastructure. This fact applies to temples and test code alike. You wouldn’t build a cathedral on a shoddy foundation, so why would you be willing to build a test suite using fragile coding practices?

Looking down a spiral staircase
Photo By Karl-Ludwig Poggemann

Good coding concepts and practices are not just for use in final applications. Those same ideas should be used when developing tests that will be used to confirm the application’s functionality. I recommend skipping the unit tests of the tests though, you do not want to go down that rabbit hole.

When developing test suites, it is a good idea to utilize encapsulation and abstraction. This will simplify the building of tests while simultaneously increasing code reuse and reducing the fragility of them. One method to achieve this is through the use of page objects. Continue reading Using Page Objects

An Overview of the Selenium Suite

First Things First

For those unfamiliar with Selenium, it is a suite of open-source browser automation tools consisting of three main products: Selenium IDE, Selenium Server, and Selenium WebDriver. Selenium IDE is a Firefox plugin that can be used to record and playback tests. Selenium Server is a java application that is used to control browsers on remote machines and/or to create what is called a Selenium Grid. The final component of the suite is Selenium WebDriver, which is a browser automation API designed to create tests or perform other required tasks. Each of the suite components serves a specific purpose in a web tester’s toolbox although all of them may not be needed within an organization.

Continue reading An Overview of the Selenium Suite

What is Test Automation?

Since this is the first post for my blog, it seems appropriate to start by defining test automation and then adding some scope to rein it in. In the broadest of termstest automation is the process of consistently obtaining and verifying results with minimum (or no) human interactionThis means that the tests could be physical or virtual. An example of a physical test for a manufacturing environment would be a sorting machine that checks for over-sized or misshapen parts. In the virtual realm, automated tests come in numerous forms and functions such as unit, integration, regression, and performance. My personal experience lies predominantly with software so my posts will most frequently deal with that aspect of automation.

Continue reading What is Test Automation?