Tag Archives: Testing

Initial Review of RedwoodHQ

I recently saw a post in the Test Automation community on LinkedIn suggesting that we should stop writing custom frameworks. The reasoning behind the suggestion was that it was time consuming to reinvent the wheel when there was an Open Source project, RedwoodHQ, that already had everything you need. Like most of the commenters, my first thought was that I was just reading a sales brochure for another “silver bullet” automation solution. The author, however, insisted that it was a sales pitch since his framework was free and he just wanted to raise awareness of a new tool that is available to the community.

I have long been a proponent of developing one’s own framework to build understanding of the underlying operations. In some ways it’s like Jedi training, you might start out using a lightsaber you were given but you will not become a master until you have constructed your own. In my experience, using pre-existing frameworks can limit an automator’s growth by obscuring the inner workings. Some frameworks will also limit a user’s ability to perform tasks because the developer didn’t consider that use case.

With that in mind, I decided to look at RedwoodHQ and see what it actually is. It wouldn’t be fair of me to write off someone else’s work without at least taking it for a spin. After reading a bit more about it, I downloaded the application and set it up on a VM. The installation process was smooth and I was connected to the web interface and poking around in the sample test in minutes.

For someone with very little knowledge of Selenium WebDriver, they could develop a test with RedwoodHQ right out of the box. They may even be able to begin learning by looking through the action example code. I wouldn’t, however, recommend keeping the original actions in the long term. If for no other reason, they need to be replaced because the current Selenium code uses hard sleeps and possibly relies on implicit waits.

With that said, don’t write off this framework yet. You’ll note that I recommend replacing the actions. This is extremely easy to do using the built-in web IDE and you can drop in any jars you like to use with your tests. This means that if you have built your own framework, you could compile it and reference your own classes and functions within RedwoodHQ. I am currently considering doing just this.

The efforts I have put into my personal framework have been focused on building functions around Selenium WebDriver to make developing page/component objects easier and by extension making the tests easier to write. In this way, I was not limited to using a specific test framework (although I tend to use TestNG most often). I have now realized that unlike other automation tools and solutions, RedwoodHQ operates at the same level as a test and reporting framework and will not interfere with working at the lower levels.

Since this is only an initial review, I can’t speak to how it integrates with a CI environment but I am willing to give this tool a chance. Once I have adapted my Selenium framework to RedwoodHQ, I am considering submitting it as a replacement for the existing actions. I believe the tool would benefit from having actions that utilize explicit waits and provide more than just basic functionality.

The take home from this is that I learned that RedwoodHQ is not another “silver bullet” automation nightmare. It is a test development, management, and reporting framework that won’t limit you to working within their methods. My intial opinion is “Well done!”

Advertisements

Testing Is Overrated

As I sit here on the morning of this sunny April 1st, I find myself thinking about the actual importance of testing. We’ve all heard the speeches from QA about how important their job is to the company and the product. How they keep the developers honest and reduce the risk of bugs reaching the customers. Seriously? All they do is hold up release while complaining that some obscure path through the product results in a 5 second delay every 50th time it is performed. Who needs that? The project team certainly has better things to do than listen to that kind of nit-picking.

Now I suggest you think about how much time we can save on projects if we can eliminate the testing phases. In some companies, testing can take up half the project because some module changed and now we have to go through regression testing. As if the developers didn’t already make sure it worked. Of course, the saving for some companies won’t be as significant since they are already practicing the superior process of using hard dates that result in QA only being able to look at the product for a day or less before going live. In today’s fast moving environment, every second counts when getting a new offering or feature out the door.

In the grand scheme, does anyone really expect software to be bug free or even fully functional on launch day? All customers know that the initial release is more of a prototype than a polished product and unless the want to get a feel for application they really should wait until the first or second patch is sent out. With that in mind, who is going care about the testing efforts done pre-release. Heck, we can even save money by letting the early adopters report any real issues found. They are going to be better suited to the task anyway since they are actually using the product rather than making up hypothetical uses. They are also more likely to follow the instructions for using the product and stay on the “happy path” rather than performing “Exploratory Testing”.

All in all, testing is completely overrated and unnecessary beyond the spot checks performed by the developers when they write the code. If anyone tells you otherwise, they are probably just whiny testers trying to feel like they matter, just ignore them.

 


The post above was written for April Fool’s Day and does not represent the actual opinion of the author or Wolvesbane Academy. If this article sounds like an opinion expressed by teammates or managers in your organization, I am so, so sorry for you.

Preventing Issues or Helping to Mitigate Risk

I recently read an article by Cassandra H. Leung claiming that promoting the idea that testers prevent issues is harming the “tester brand” by establishing unrealistic expectations for testers. My initial reaction was to think that she was crazy because testers do prevent issues if we are allowed to join the project early and influence development. After reading a bit farther, I realized that the latter part of that thought was what she was talking about. Testers can’t prevent issues if they are not allowed to be part of planning or their warnings go unheeded. Unfortunately, both of these scenarios are more common than teams would like to admit.

This revelation leads us to an identity crisis as we are forced to ask, “What should we being doing as testers?”.

Over the years, testers have been charged with many formidable tasks including: assuring quality, catching bugs, policing issues, preventing issues, enforcing acceptance criteria, etc. As a whole, we’ve done a fair job of achieving success despite having the odds stacked against us by these vague definitions. The reason we cannot be completely successful is that we rarely have the level of control needed to perform these tasks.

As thought-leaders continue to move quality considerations “left” and raise awareness of the need to have testers involved early in the process, the need for testers gets questioned. If developers are testing and product owners are writing tests in the form of acceptance criteria, testers need to properly define their role on the team. Since Quality is a team responsibility, we cannot claim it as our goal. My recommendation is to define the tester role as mitigating risk though observation and review of project quality considerations and conversations with the team.

This means that testers are not gatekeepers, defenders of quality, or even responsible for catching all of the bugs. It is our job to bring our unique perspective and abilities to the team and use them to minimize the risk of issues within an implementation.

I admit that this definition is still a bit vague but it is something that we can accomplish. Testers cannot control the quality developed into the product. Testers cannot prevent decisions that could result in issues down the road. Testers can raise concerns to the team and make sure that quality conversations occur so that everyone involved has the best information available to them before making a risky decision.

Selenium Basics for Java is Coming Soon!

In 2016, I published my first course—Automated Testing Using Selenium WebDriver—on Udemy. With that course, my intent was to provide an intermediate level introduction to Selenium WebDriver. I decided to further differentiate it from other courses by working in both C# and Java. While this succeeded in showing that it is easy to apply the concepts taught in multiple languages, I fear that it could be confusing for students that are not experienced in development. This is where my new course will come in.

Students will be required to have a basic understanding of development concepts and Java, but I will be covering the additional technologies and concepts used in greater detail. From installing an IDE and using Git through running tests using a Selenium Grid, my new course will provide you with the foundation you need to become successful in your automation endeavors.

Topics to be covered include:

  • Installing an IDE
  • Installing browser drivers
  • Creating your first Selenium WebDriver test
  • Locating web elements on a page
  • Waiting for web elements
  • An overview of the Page Object Model
  • Running tests against a Selenium Grid
    • Local Dockerized
    • BrowserStack
    • Sauce Labs
    • Testing Bot
  • Other tips, tricks, and concepts for writing great automation

The following is a sample video showing how to setup a Selenium Grid using Docker and Docker-Compose on an Ubuntu virtual machine. This lesson will be part of the Selenium Grid section of the course.

Establishing a Selenium Grid Using Docker and Docker-Compose from Shawn Conlin on Vimeo.

Adventures in LoadRunner

As we came into the new year, I was in process of learning how to use LoadRunner for a new project. I’ve been meaning to do this for quite a while so I’ve been rather excited. I started by going through some legacy tests and then I signed up for a course on Udemy, Load Testing Using HP LoadRunner 12. While the audio and video quality of the course was lacking, the content of the course was very informative. I still have a couple of other courses I picked up, but I was able to get my confidence up enough that I jumped in with both feet and started working on tests.

As is normal, the first test was a bit sketchy. I hadn’t gotten a real feel for the application nor I had I realized that the scripts are “C like” and not “written in C”. This means that you may not have all of the functionality you expect from C readily available to you unless you manually install additional libraries. I also found that the normal method of creating header files and class files doesn’t seem to apply either. Rather than creating separate header and class files, the developer simply includes the classes and methods in the same file. I’m not sure if this considered a “best practice” for LoadRunner, but it does seem to be widely practiced.

As I continued working through the tests, one utilizing system level commands and others operating a terminal, I continued to improve them as I got more familiar with the application. After the initial runs were working, I had to start refactoring for use with the controller so that the tests and data were accessed such that multiple users could be generated and run concurrently. Those changes went surprisingly well and I am now working on the environment the tests will be run on.

I know that there is much more to LoadRunner than I have seen thus far and I am looking forward to mastering this new tool.

 

Be An Eternal Student

No, I’m not talking about spending your entire life jumping from one major to another so you don’t have to graduate. When I say, “Be an eternal student.”, I am advising you to keep learning throughout your life both actively and passively. Don’t let opportunities pass you by. Just because you don’t need to know something right now doesn’t mean that it won’t come in handy later.

One question I get from many people is how I know all of the things I do. I tell them that I just pick it up as I go. Growing up, I was surrounded by mechanics, electricians, carpenters, and other tradesmen. If they needed help, or were helping me, I paid attention and picked up some tricks from them that help me around the house today. With my career, it’s been a bit different. Until recently, I didn’t have a lot of computer people around to talk to so I had spend a lot of time tinkering on my own and doing research. I learned a lot of ways not to do things; I learned some bad habits; and I gained some excellent insight into how our magical toys work from the ground up.

Aside from the hands-on/trial and error approach, I have also invested a significant amount of time reading, watching videos, and taking courses on various topics. Usually I try to focus on things I am either actively working on or expect to in the near future, but sometimes I throw in something brand new or just plain fun to keep me excited about learning. Recently I did this by taking all of the courses in the Docker Path on Pluralsight.com. While I started the courses for fun, I quickly found that what I was learning could be implemented in my current projects, which was an added bonus since I get to practice what I learned and improve my working environment.

It has been experiences like that and some unpleasant bills, replacing things I didn’t know how to fix, that helped me realize the importance of not growing stale or letting my aptitude for learning atrophy because I already know how to do my job. I was also lucky to have grown up around other perpetual students who gave me a solid understanding of how to acquire knowledge. I have found that the keys to learning are very simple:

  1. Find something you are curious about or need to learn.
  2. Gather resources that cover the topic.
    1. Talk to people who already do or know what you need.
    2. Read books and articles about the topic.
    3. Watch videos about it.
    4. Look for someone teaching a course that you can sign up for.
    5. Experiment on your own.
  3. Do something with what you have learned.
    1. Complete a project using what you learned.
    2. Share what you learned with someone else.
  4. Appreciate yourself for learning something.
  5. Repeat.

If that sounds easy, that’s because it is most of the time. I use this approach in my daily life for everything from plumbing to performance testing applications. Granted I will never be a master plumber (it just isn’t my calling) but I also don’t need to call one when I need to unclog a drain or replace a faucet. When it comes to computers and software, there is always something new to learn regardless of your level of mastery. This is part of the reason it is important to be an eternal student. If being armed with new tools and ideas isn’t enough to fuel your desire to learn, remember that once you stop growing you begin to become stale and obsolete. Don’t let your potential sit idle. Take the time and spend the effort to find out exactly what you are capable of. You might even find out that you can do anything you set your mind to.


I have never let my schooling interfere with my education – Mark Twain

There Is No Right/Wrong Way: Writing Tests

Over the years, I have encountered and practiced many different ways of creating, documenting, and executing tests. I have also worked with a number of testers, including myself at times, who harbored strong opinions about how tests should be handled. As a result, I’ve found that the “right way” to handle tests depends upon your environment and your team.

If we look at the methodology and maturity of any team’s development life cycle, we will see that certain approaches to testing are better suited than others. For example, a waterfall team that has a high turn over rate will get more value out of high detail, scripted tests because there will be a smaller product knowledge base among testers. Alternately, an Agile team dedicated to a project/product will likely benefit from more generalized tests and exploratory testing. A company’s culture, regulatory bodies, and procedures also play into how testing is performed and documented.

With this in mind, it is important for testers to keep an open mind and consider their processes carefully. Small changes can result in major savings in time and costs. A good example of this would be changing the test team’s charter from exhaustive to risk-based testing. This change will quickly switch their focus from reviewing every possible combination of data paired with a complete review of the system’s functionality to examining the changes and identifying the sections of the software with the greatest need for testing, which typically results in a significant reduction of test time. As a counter-point to that example, there are situations in which the team may need to perform more in-depth testing, such as when there is a major refactoring and the project is disturbingly light on unit tests.

The point of my ramblings here is that as professionals, we should focus on learning and improving our skills and finding solutions to our pain points rather than arguing over who’s methodology is better. The reason there are so many ways to approach testing is because each situation is different and cannot be handled by a one-size-fits-all solution. Keep learning and remember there is no right or wrong way, it’s just a different style.

 

Limbering Up QA: Adapting to an Agile Process

In the days of Waterfall, testers were seen as the guardians of quality and the protectors of user experience. They were the last line of defense to prevent a flawed product from being released. This meant that QA needed to have all of the requirements provided to them so they could prepare their tests for when the product was finally complete and they could begin their process. Sure, the deadlines were usually tight because the release couldn’t be moved and development ran long, but that was just part of thrill. QA were sometimes viewed as an obstacle rather than an aid, but they remained strong and provided their sworn services for client and company.

All of this changed with the dawning of Agile.

The once-powerful QA is now faced with shorter deadlines, stories instead of specifications, and seemingly incomplete features being submitted for test. Worse yet, the developers are encouraged to develop unit tests that automate a chunk of what the tester once handled.  How can we possibly work like this? It’s utter madness.

It may seem like madness and chaos, but there is also method and rhythm to be found in the new processes. The first step is to stop fighting the current and dive off the waterfall instead. After taking the plunge, testers can learn to navigate the flow of sprints and iterations. Granted, this is often easier said than done, but any habit can be kicked and new ones formed. It just requires time, effort, and a willingness to change.

Unfortunately, two out of three of those things are often not in the testers’ inventory, time and willingness to change. I’m sure that a number of those reading this might be offended by that statement, but bear with me. I know that no-one will argue with the time part, but everyone trips on the willingness to change. It’s natural for people to find a comfort zone and settle in. It’s also natural to be startled and scared by change. In my experience, testers are often leery of Agile because it seems to value speed over quality, but this couldn’t be farther from the truth. Agile places emphasis on quality but it is done by building it in rather than straining out errors after development.

This change in both thought and method is the key to producing software quickly without compromising quality. The catch is that it requires developers and testers to step out of their silos and work closely together. They should also include product owners and other stakeholders in their discussions to keep everyone aligned.

The idea is to inject the knowledge of QA from the beginning rather than waiting until everything is done. While this may sound like a pitch for TDD or BDD, that is only a piece of the picture. Sometimes, it is already too late for feature to maintain quality once it gets to development. This case is most often found with legacy modifications or a stakeholder pet project that seems “simple enough” but hasn’t been fully evaluated for ripples or pitfalls. This isn’t because someone missed  something. It is a result of their standard thought processes.

  • Developers tend to think in terms of “how can I make this work?”. They are focused on solving the puzzles.
  • Product owners focus on the value of the feature to clients. Most POs leave the technical concepts to the dev team.
  • Stakeholders are usually focused on how the feature will improve their positions or increase the company’s profits

When testers are left out of the planning stages, teams sacrifice an opportunity to reduce bugs and head off troubled projects before they are sent to development. A tester’s mindset generally includes looking at contingencies, interactions, and risks within a system. Even if the person doesn’t know the  system, asking the right question during planning can shine a light on a major issue that may have been glossed over, such as “Have we considered how the new shipping system handles an order including both physical and downloadable content?”.

After planning, QA should be helping to translate business process knowledge from the POs into paths through the software and tests for the new feature. A common method of doing this is by writing Given-When-Then (or similar concept) scenarios. These scenarios will then form a basis for both automated and manual tests used to confirm the functionality during development and prior to release.

In short, testers should make an effort to be involved throughout the project cycle rather than sweeping up at the end. By doing this they can help set the stage for successful projects and avoid the stress of being the roadblock or bearer of bad news just before release. While this is a major change, it is one worth embracing.

An Overview of the Selenium Suite

First Things First

For those unfamiliar with Selenium, it is a suite of open-source browser automation tools consisting of three main products: Selenium IDE, Selenium Server, and Selenium WebDriver. Selenium IDE is a Firefox plugin that can be used to record and playback tests. Selenium Server is a java application that is used to control browsers on remote machines and/or to create what is called a Selenium Grid. The final component of the suite is Selenium WebDriver, which is a browser automation API designed to create tests or perform other required tasks. Each of the suite components serves a specific purpose in a web tester’s toolbox although all of them may not be needed within an organization.

Continue reading An Overview of the Selenium Suite

What is Test Automation?

Since this is the first post for my blog, it seems appropriate to start by defining test automation and then adding some scope to rein it in. In the broadest of termstest automation is the process of consistently obtaining and verifying results with minimum (or no) human interactionThis means that the tests could be physical or virtual. An example of a physical test for a manufacturing environment would be a sorting machine that checks for over-sized or misshapen parts. In the virtual realm, automated tests come in numerous forms and functions such as unit, integration, regression, and performance. My personal experience lies predominantly with software so my posts will most frequently deal with that aspect of automation.

Continue reading What is Test Automation?