Given my experience with creating automated regression tests, one would think that I’d remember the dangers of improperly planned data. Unfortunately, I am susceptible to getting wrapped up in how the function works and getting to an initial success and sometimes forget to see if the feature works differently if the data varies. While I take comfort in not being alone in making this mistake, I still regret not seeing it coming sooner.
With any automation, developers should always prepare a plan for providing reliable and predictable data. Without that data, the automation will become “flaky” and produce undependable results. As an example, I was recently running a performance test that we’ve been working with for a few weeks and has been running smoothly. We decided it was time to start ramping up the load and expanded the product data available. Unfortunately, we did realize that the new products had a setting that caused them to trigger a different process flow. The result was that our tests began failing and because the data was being selected at random, we couldn’t determine if we had hit a product limitation or an issue in the test. We lost several hours to debugging this situation.
An important lesson I have learned through this and similar experiences is that automation requires a data strategy that takes into account how the tests will be run and what their purpose is. In the case mentioned above, the solution is to generate a controlled set of products that will result in a predictable path through the software. We will also need to regulate any changes to the data load or develop an automated system to produce the desired entries in the required state before the test runs. It may sound obvious but it really is more complicated than you might think.
At least now I know what one of my future articles will be about, developing a data strategy.
Measuring Tape or yard stick
Materials (per wall):
3 – ½” PVC pipes (10’ each)
6 – ½” PVC elbow connectors
4 – ½” PVC Tee connectors
2 – Spring-loaded Clamps (Optional)
3 – clamp lights (Optional)
1 – twin size blanket
- Cut the 10’ PVC pipes into three 3 foot sections and one 1’ section each
- Cut one of the 3’ sections into 1’ pieces.
- Assemble two 6’ upright poles by connecting two 3’ foot pipes with a tee connector for each one.
- Put elbows on either end of a 3’ pipe and set it aside. This will be the top bar.
- Lay the side poles down with the Tees facing each other. Insert a 3’ pole into the tee connectors to serve as a brace. You assembly should now look like an H.
- Now take the top bar and fit it on the top of the poles.
- Place two elbows on each of the remaining 3’ pipes and set them aside
- Make two 2’ pipes using 4 1’ sections and 2 tee connectors
- Assemble a rectangle using the two 2’ pipes and the 3’ pipes with the elbows.
- The final step is place the upright poles into the tee connectors in the center of the 2’ sides of the base.
- Use clamps or lights to attach the blanket to the frame
I hope these instructions are useful and make sense. I am including a picture of one of my frames. This one is a little different than the instructions because I decided to add an extension to the top rather than having a full rectangle for the base.
Be creative and build what you need. I added another pole and a 3-way connector to support the corner and connected the two walls so I could completely enclose the area rather than just having separate screens.
The following text has been created to serve as an introduction to mead and mead making. The material contained within is not specific to the Medieval and Renaissance periods as that time frame is just a small portion of the history of mead. Instead, I have chosen a broader scope which begins in Antiquity and ends in your kitchen. I believe the result is a well rounded overview which will not only whet your appetite for knowledge but teach you how to brew something to slake your thirst while delving deeper into your own studies.
A Brief History of Mead
Mead has been enjoyed by man through the ages from pre-history to the present day.
A Not So Brief History of Mead
Most people are familiar with the argument of whether the chicken or the egg came first, but not as many realize that there is a similar argument surrounding beer, wine, and mead. All three of these beverages can be traced back into antiquity through writings, art, and archaeological findings which makes finding an absolute answer to the question extremely complex. Rather than trying to unravel the threads and divine an altruistic answer, this work will present the facts and theories surrounding the origins and history of mead while making no apologies for perceived bias in the writing.
It is important to have a basic definition to work from when interpreting historical references. The definition must be neither too broad nor too narrow to be effective. In its most basic form, mead is the product of combining honey and water. Yeast would be the third ingredient in the list if we wanted to restrict the definition to alcoholic beverages, but this would also make it harder to locate references since most historical texts do not mention yeast. For the purposes of this article, mead will be defined as a beverage made from a solution of honey, water, and optionally other ingredients for flavoring.
One of the most intriguing facts about mead is that it occurs naturally when honey reaches a sufficient moisture level. With this in mind, it is not unreasonable to theorize that man did not invent mead but rather discovered it while seeking food in a hollow tree. An alternate theory for the discovery of mead works on the same basis but includes man actively gathering and storing honey and some of the containers became contaminated with water which resulted in the wondrous beverage being created. Using either of these concepts, we can speculate that once man began using honey as a normal food source the possibility of discovering mead rose exponentially. The earliest indications found relating to honey gathering are cave paintings and rock drawings dating back to around 15,000 BC (Schramm, 5). Given the nature of mead, there is no way to actually determine when the first batch was created but it should be very safe to say that man was enjoying mead long before he ended his days of cave-dwelling.
The skeptical reader is no doubt wondering why mead would have been forgotten until the Middle Ages if it was discovered so long in the past. Mead wasn’t forgotten; it just existed under different names such as soma, pyment, hydromel, and nectar. References to mead can be found throughout the ancient world from mythology and religious texts to accounts by Greek and Roman historians.
The Many Names of Mead
Let me start by saying that this section is not going to be about what the word “mead” translates to in other languages. It will, however, cover the various types of meads which have had names coined for them. In my studies, I have found that there is some contention over the appropriateness of some of these terms. The main issue is whether or not the distinctions are modern inventions. While a number of the names do date to the Medieval period or earlier, the actual adherence to the terms and definitions is not strict before modern times (Krupp, 7). My general opinion is that they are all meads and the names are more guidelines for discussing them in a modern context.
This designation is used to describe meads that have been flavored with fruits. The term derived from the Roman name for a beverage made from honey and fruit juices, melomeli (Papazian/Gayre, 113).
The exact nature of pyment is a question of perspective. Some sources state that pyment is wine that has been sweetened with honey . Under the broader definition of what mead is, a fermented beverage made with honey, pyment would classify as a mead made with grapes. While Papazian continually refers to pyment as a wine, he gives proof that it is more a mead while quoting Chaucer (Papazian/Gayre, 116).
A form of hard cider which uses honey rather than mead as an additive. It can also be made as a mead with apples, cider, or apple juice added for flavor.
Mulberries are used to provide a deep color and rich flavor to this form of mead.
Pears or pear juice adds its delicate flavor to this form of melomel to produce a crisp refreshing beverage.
A braggot is often described as “a mead with a beer in it”. This description is fairly accurate since the additional flavors come from barley or hops which is the main ingredient for making beers and ales.
A Greek and Roman mead flavored with rose petals. (Schramm, 19-20) (Papazian/Gayre, 114).
Some references use the term hydromel and mead interchangeably but it has been my experience that this term refers to a mead that has been cut with water to reduce its potency.
A sack mead is made by increasing the ratio of honey to water such that the finished product is much sweeter than a typical mead.
Brewing a Basic Mead
Now I am hoping that you have prepared yourself to unravel the complicated and intricate methods and practices involved in the making of mead. If you are truly ready to learn the arcane knowledge that will start you on your way to producing the drink of the gods, you are about to be very surprised by the simplicity of it all.
The most basic of meads is also known as traditional mead. The ingredients list consists entirely of two things honey and water. Of course, yeast will help if you intend to make an alcoholic beverage, but it isn’t required to be on the shopping list. When working from many early recipe sources, yeast is not listed as an additive. This is due to the fact that brewers of the time didn’t know that yeast is what caused the fermentation. Most often the must of these ancient brewers would be impregnated by yeast in the environment. This could be wild, airborne yeasts, bread yeast if the mead was made in a kitchen where baking was being done, or those that were residuals in the vessels used to make previous batches.
Once you have the ingredients, the next step is to gather the tools and equipment needed. Many people think that this means spending large amounts of money on gadgets and gizmos, but this really isn’t the case. The essential list of things you need is: a stock pot (preferably steel so it doesn’t impart a metallic taste to your mead), a long spoon, a funnel, and something to put your must in to ferment. There are some additional items that will make life easier and help you to produce “better” mead, such as airlocks, hydrometers, carboys of various sizes, hoses, bottles, corks, corkers, filters, and the list goes on long enough to make your head spin and your checkbook cry for mercy. For our purposes, I will be covering how you can start brewing with items you probably already have around the house or can obtain for $20 or less.
Referring back the basic equipment list, the first item we need to find is a large enough stock pot. The recipe to be used here makes 1 gallon of traditional mead. The stock pot you will need should hold 6-12 quarts. The next items on the list are also fairly common in any kitchen, a spoon long enough to reach the bottom of the pot without putting your hand in the boiling liquid and a funnel (any size will do but a wide mouth funnel will make pouring easier). The last item in our essentials list is something to put the must in to ferment. Most brewers will recommend glass bottles or carboys for this because they are easy to clean and do not absorb smells the way plastic can. For someone just starting out, I recommend 1 gallon plastic bottles. I used plastic apple juice bottles since my kids made them readily available in abundance. If you would prefer glass, I have found that 1 gallon wine bottles work wonderfully, in fact, they are the same as the 1 gallon carboys you can purchase at brewing supply shops only you need to empty them first. Once you have all of these items, you are ready to begin.
As a general rule, the ratio of honey to water is 2.5-3lbs honey per gallon of water (Papazian/Gayre, 169). As we are making a 1 gallon batch, we will need 3lbs of honey (less measuring that way) and 1 gallon of water. Start by bringing the water to a boil and then stir in the honey. Continue stirring until the honey is completely dissolved and then remove your must from the heat. Allow your must to cool to room temperature and then pour it into your bottle (this is where the funnel comes in handy). At this point we need to get some yeast into the must and to do this you have 2 options, adding some yourself or sitting hopefully waiting for some stray yeast to come along. I recommend using some store bought yeast.
Since I didn’t go into this earlier, you can get brewer’s yeast from any number of brewing supply shops either locally or on-line. If for some reason you can’t find brewer’s yeast, you can use regular bread yeast from the grocery store. Using bread yeast will change the taste of your resulting mead. I have done this and received compliments, so it really boils down to your personal tastes.
Before adding the yeast to your must, called pitching, you will need to activate it. There are directions on the yeast packet regarding how to do this. Basically, you dissolve the yeast in a small amount of warm water or juice. A better method is to take a small amount of your must to use as a starter. Once your yeast has been activated, it will usually start to foam after a few minutes, pour this into your must and mix it in. I use the handle of my long spoon for this.
You are now ready to sit back and let the yeast do its magic. Since we are working with very basic equipment found in the kitchen, you will need to make a decision about how to seal the bottle while the yeast is working. You can use the lid for the bottle but you will need to be very diligent if you do. While the yeast is producing alcohol, it is also making CO2. This means that if your bottle is air tight it will build up pressure and could explode. During a vigorous fermentation, even putting the cap on loosely could be too tight, so you will want to carefully open the cover from time to time to release the pressure. Another option is to use a cloth held over the opening with a rubber band. This works fairly well. If you are planning to continue brewing, I strongly recommend investing in airlocks. They only cost around $1 and they will protect your mead and your kitchen (I’ve seen carbouys explode from pressure and it isn’t pretty!!)
After about a week, you will notice that the bubbles in your brew are decreasing or have stopped. You will also notice a build up of sludge (actually called lees) on the bottom of your bottle. Most brewers recommend racking, transferring your mead to a new bottle and leaving the sludge behind, at this point. Racking is much easier to do if you have a length of food grade tubing, which can be found in most hardware stores, but can be done by carefully pouring your mead into a new bottle.
If you have made it this far, then you are now the proud brewer of a batch of mead. The question that you are undoubtedly asking is, “When can I drink it?” This is a question that will bring about debate from brewers everywhere. Technically, your mead can be served when the fermentation stops, or even before if you really wanted to. Most mead makers today prefer to age their meads to improve the quality. Aging can take anywhere from 1 month up to 2 years or more depending on the mead. For the beginning brewer, I recommend tasting your mead to see if you like it. If you like the taste, then you are ready to serve, if not then close it up and put it in a cool dark place for a month or so.
The following recipes were gathered from various sources including my own recipe book.
Wulfric’s Traditional Mead (AKA BOOM!)
This wondrous mead may never truly be duplicated, but we are extremely hopeful. The first time I used this recipe, the result was a lightly sweet mead with an alcohol content of 22%, hence the name “BOOM!” This recipe is for a 5 gallon batch. If you do not have a pot large enough to heat all of the water and honey at once, you can hold 3 gallons aside and add it to the carboy later. This method is good for cooling the must quicker if you use cold water.
25 lbs Raw Dark Honey
5 Gallons Water
1 tsp Yeast Nutrient
1 pkg Lalvin D47 Active Dry Yeast
Heat water and honey to just below boiling, stirring to until the honey is completely dissolved. Skim the solution as needed to keep the surface relatively clear. Remove from heat and let stand until cool. While the must is cooling, you can start your yeast. When the must is below 90°F pour it into a carboy, pitch yeast and seal with a airlock.
When the bubbles slow to one or two per minute, you should rack the mead into a clean carboy. You will want to have an airlock on the new carboy in case a secondary fermentation starts. Allow the mead to sit in the carboy in a cool place until it clears, or clears enough for you (I don’t mind cloudy meads, they have more character), and then bottle.
“A Most Excellent Metheglin” (Digbie, 68-69)
This recipe is a nearly period recipe for metheglin from Digbie. I will list both the original text as well as the redaction.
“Take one part of honey, to eight parts of Rain or River-water; let it boil gently together, in a fit vessel, till a third part be wafted, skimming it very well. The sign of being boiled enough is, when a New-laid-egg swims upon it. Cleanse it afterwards by letting it run through a clean Linnen-cloth, and put it into a wooden Runlet, where there hath been wine in, and hang in it a bag with Mustard-seeds by the bung, that so you may take it out when you please. This being done, put your Runlet into the hot Sun, especially during the Dog-days, (which is the onely time to prepare it) and your Metheglin will boil like Must; after which boiling take out your Mustard-seeds, and put your vessel well stopped into a Cellar. If you will have it the taste of wine, put to thirty measures of Hydromel, one measure of the juice of the hops, and it will begin to boil without any heat. Then fill up your vessel, and presently after this ebullition you will have a very strong Metheglin.” (Digbie, 68-69)
For my interpretation of this recipe, I will plan to produce a 5 gallon batch. I have chosen this number because of the specification to pour your must into a “Runlet”. A runlet (rundlet) is a cask or barrel that holds between 3 and 20 gallons (14.5 is most common). Since most brewers don’t have barrels, I’m opting to plan for a 5 gallon carboy.
5 pints Honey (approximatly 5 lbs)
5 gallons Water
1 tbl Mustard Seed
1 pint Juice of Hops (optional)
1 pkt Yeast (if using a clean carbouy)
Boil the water and honey until the starting volume is reduced by one third. Alternately, you can check if it has boiled enough by placing an egg in the must. The egg should be fairly round and as fresh as possible. If the egg floats then you have boiled it down far enough. In this situation, the egg is serving as a hydrometer.
After the boiling, filter the must by pouring it through a clean linen cloth into your carbouy (or cask if you have one). The original recipe specifies that it should be a container that had wine in it before. While this may seem odd, the reason behind it is that there would be residual yeast in the container and that is why the recipe doesn’t call for the addition of any. Since sanitation is a major concern for most modern brewers, I have included a pkt of yeast in the recipe. The yeast should be added once the must has cooled a bit so as not to kill the yeast. You should then place the mustard seed in a linen bag and hang it in the must so that it can be easily removed later. A string tied around the bag that runs out of the stopper works well for this. When you have everything in place, put your carbouy in a warm place to work.
When the initial fermentation is complete, remove the mustard seeds, securely seal the carbouy and move it to a cellar (or other cool, dark place). At this time, you can optionally add 1 pt of “Juice of Hops”, which I suspect is the result of boiling hops in water, to 30 pints of your Metheglin. Since 5 gallons is the equivalent of 40 pints, you can either remove a portion of the metheglin, increase the amount of hops to 1 1/3 pints, or call it close enough. Your metheglin will begin a secondary fermentation at this point. When that completes, you should have “a most excellent metheglin”.
“Damn Shame”: A non-alcoholic mead
There have been a number of occasions in which I wanted to serve mead but could not distribute alcoholic beverages. I rose to this challenge by creating the following recipes. While the steps are very similar to making the “fully leaded” version, you do not add yeast. The omission of yeast means that there is no waiting period outside of allowing the must to cool before bottling.
2 gallons water
4 cups honey
2 cinnamon sticks
2 gallons water
4 cups honey
1 tsp whole allspice (crushed coarsly in a mortar & pestle)
Heat the water in a large pot. Once the water is hot, add the honey and stir until all of the honey dissolved. Continue heating the mixture but do not boil it. If you notice a scum floating on top, skim this off and discard it. I typically keep the mixture in low heat for about 20 minutes or so after mixing. Remove from heat and add the spices. Allow your brew to stand until it reaches room temperature and then strain and bottle it. Alternately, you could simply add all of the ingredients in at once and then heat it, but I prefer to do it in stages.
Digbie, Sir Kenelme, The Closet of the Eminently Learned Sir Kenelme Digbie Opened… s.l., London: E.C. for H. Brome, 1669. (Reproduction by Mallinckrodt Chemical Works 1967)
Griffith, Ralph T. H., Sacred Writings Volume 5 – Hinduism: The Rig Veda, New York, NY: Quality Paperback Book Club, 1992.
Krupp, Christina M and Bill Gillen, “Making Medieval Mead or Mead Before Digby”, The Compleat Anachronist 120 (July 2003) : .
Papazian, Charlie and G. Robert Gayre, Brewing Mead : Wassail! In Mazers of Mead, Boulder, CO: Brewers Publications, 1986.
Schramm, Ken, The Compleat Meadmaker, Boulder, CO: Brewers Publications, 2003.
Shapiro, Mark, “Alcoholic Drinks of the Middle Ages”, The Compleat Anachronist 60 (March 1992) : .
Sibley, Jane, “The CA Guide to Brewing”, The Compleat Anachronist 5 (March 1983) : .
Over the past couple of weeks, I have been listening to The Phoenix Project during my commute to and from work. Having finished it today, I decided to jot down some of my thoughts and share them with you.
The first thing I will warn you about is that if you have already read The Goal and understand how it can be applied to IT then you are not going to find any new, eye-opening concepts in this book. The plot follows a middle manager that finds himself being promoted to the head of a struggling department. Through determination and coaching from a elusive and wise mentor, he learns to identify and control work along with how to align it with the companies needs to become successful and rise to the top of their market. For the record, that is the plot for both books. I identified the similarities between the books within the first couple of chapters, but was amused when the main character’s mentor began to quote and reference The Goal while explaining that work is work and IT is no different from a manufacturing plant.
With my main criticism out of the way, I found the story to be a fairly realistic look at IT functions within an company. Misunderstandings and unreasonable demands result in repeated disasters and a generally oppressed and depressed atmosphere. I dare say the stage was set so well that I could not only draw parallels from my experience but was starting to have actual sympathy for the characters because I could see what was coming.
Overall, I think this is a good book for IT staff, managers, and all executives to read and gain an understanding of the processes that help work flow. For the IT staff in the trenches, the purpose of reading this book should be gain an understanding of why processes may need to change and to help acknowledge that their managers need help not resistance. The executive’s take away should be to identify things that were done poorly by the executives and the board in the book so that they can recognize and correct similar issues within their organization. For the last group I recommend this book to, I suspect the take away would be a number of ideas regarding how to implement changes and structure work for their teams and that there is some hope if you can get those above you and below you to listen to reason.
I would recommend this book over The Goal for use in IT organizations simply because it makes it easier to see how the concepts apply to IT while making a point of addressing the common responses from IT personnel. I believe this was the motivation of the authors and, if I am correct, they succeeded.
One of the fastest growing segments of the modern internet is the use of social networking sites and software. With users numbering in the millions, applications such as Facebook, Twitter, LinkedIn, and Google+ are helping to keep people connected despite distance between them. While all of these websites are recent creations and are considered to be new concepts, the fundamental ideas behind this type of computer use began to surface in the 1970s with the introduction of the bulletin board system (BBS).
The Humble Beginnings
A BBS is a “[c]omputerized system used to exchange public messages or files” (Encyclopædia Britannica, 2011). Bulletin board systems were originally restricted to companies and educational institutions until early 1978 when Ward Christensen launched CBBS in Chicago, IL (BBS Corner, 2009). From this point, BBSs began to spread as hobbyists and computer enthusiasts created their own systems. Most bulletin boards served small areas because they required users to connect via telephone lines and few people wishes to incur long distance charges.
As the boards became more sophisticated, their services expanded from simple file sharing and public message forums to include games, private messaging, and chat rooms. For the computer-savvy, these services became the equivalent of the barbershops, ice cream parlors, and coffee shops of previous years. The influence, acceptance, and use of bulletin board systems continued to grow until the 1990s when the internet became readily available to the public and World Wide Web blossomed.
Evolution and Revolution
Beginning in the 90s, the nature of the internet began to change drastically. Local bulletin board systems were being replaced by national services (CompuServe and AOL) and a new type of company, the internet service provider. These new companies hosted large modem banks to allow their client to dial in and connect to the internet. They typically provided email access as well.
As ISPs became dominant and the internet more accessible, BBSs began to fade away. In their place, email, internet forums, and internet relay chat (IRC) gained popularity. Some websites included all of these methods of communication. While users now only needed to connect to a single access point, they were still visiting multiple virtual locations to carry on their business.
The next evolution of public internet was the introduction of high-speed, broadband internet connections. Users could now be online 24/7 and accessing data at ever-increasing rates. The increase in speed and connections led to the more innovations in software offering once they were combined with the increase in processing power of the average PC. Forums, Email, and Instant Messaging—a child of IRC—continue to be staples for communicating, but in the middle of the first decade of the 21st century a new moniker was coined for websites that performed these functions. The term that was now used to describe these services was Social Networking.
Where Do We Go From Here?
Just as the current incarnation of social networking has come to overshadow the bulletin board systems of the 1980s, future software will enhance the functionality and speed of communication between individuals. Some of the technological advances are already beginning to show themselves in the form of video chats and simultaneous discussions between groups of people—all of whom are in different locations. Since most methods of communication have already been introduced via the internet, it is likely that the future will hold mostly improvements on existing technology rather than pioneering completely new methods of communicating. An exception to this could be the introduction of holographic projections to replace the current video feed. Regardless of the kinds of changes seen in the future, it is clear that the computers, electronics, and internet will continue to make the world a smaller place to live.
BBS Corner. (2009, November 29). The BBS Corner – A Brief History of BBS Systems! Retrieved August 30, 2011, from The BBS Corner: http://www.bbscorner.com/usersinfo/bbshistory.htm
Encyclopædia Britannica. (2011). bulletin-board system (BBS). Retrieved August 30, 2011, from Encyclopædia Britannica: http://www.britannica.com/EBchecked/topic/183745/bulletin-board-system-BBS
I recently read an article by Cassandra H. Leung claiming that promoting the idea that testers prevent issues is harming the “tester brand” by establishing unrealistic expectations for testers. My initial reaction was to think that she was crazy because testers do prevent issues if we are allowed to join the project early and influence development. After reading a bit farther, I realized that the latter part of that thought was what she was talking about. Testers can’t prevent issues if they are not allowed to be part of planning or their warnings go unheeded. Unfortunately, both of these scenarios are more common than teams would like to admit.
This revelation leads us to an identity crisis as we are forced to ask, “What should we being doing as testers?”.
Over the years, testers have been charged with many formidable tasks including: assuring quality, catching bugs, policing issues, preventing issues, enforcing acceptance criteria, etc. As a whole, we’ve done a fair job of achieving success despite having the odds stacked against us by these vague definitions. The reason we cannot be completely successful is that we rarely have the level of control needed to perform these tasks.
As thought-leaders continue to move quality considerations “left” and raise awareness of the need to have testers involved early in the process, the need for testers gets questioned. If developers are testing and product owners are writing tests in the form of acceptance criteria, testers need to properly define their role on the team. Since Quality is a team responsibility, we cannot claim it as our goal. My recommendation is to define the tester role as mitigating risk though observation and review of project quality considerations and conversations with the team.
This means that testers are not gatekeepers, defenders of quality, or even responsible for catching all of the bugs. It is our job to bring our unique perspective and abilities to the team and use them to minimize the risk of issues within an implementation.
I admit that this definition is still a bit vague but it is something that we can accomplish. Testers cannot control the quality developed into the product. Testers cannot prevent decisions that could result in issues down the road. Testers can raise concerns to the team and make sure that quality conversations occur so that everyone involved has the best information available to them before making a risky decision.
When I started thinking about this post, I realized that it had the potential to just be my obligatory test automation pyramid post that tells everyone to focus on unit tests and use UI tests sparingly. So I decided to take manual testing into account and the idea turned into a software testing cupcake post.While both of those concepts are useful for setting goals, they rarely survive implementation in a real world process. This is because they are based on an ideal scenario rather than being modeled against actual projects, teams, and applications. This has led me to focusing on the process flow rather than where to focus testing efforts.
Most processes can fail when faced with the actual day-to-day requirements of a project. The key is provide enough wiggle room in the implementation for the plan to survive minor variations and be acceptable to the whole team. It should also take into account the roles to be played be the entire team and not just those of the testers.
Suggested Software Testing Process
I have found that the placement of test planning in the development cycle can greatly effect how testing is done throughout the process. I have most frequently seen this activity placed after requirements planning and concurrently with development. While this seems like an effective use of time, the result is a separation of responsibility and usually a breakdown of communication. To avoid this common pitfall, test planning should coincide with requirements planning and include the entire team—or at least representatives from product planning, development, and testing.
One method to accomplish this is to use “the three amigos” approach when planning the project. This allows testers and developers to address concerns and offer guidance while establishing ways to handle the business requirements. The final product of this session will then become an alignment and plan for each stage of the cycle.
As part of the planning, testers can assist in advocating quality throughout the development cycle by identifying checks and tests to be performed and the team can decide where best each of the tests can be implemented. As an example, data validation can more effectively be handled within unit tests implemented by the developers. This will reduce the functional testing requirements for the testers allowing them to focus on business scenarios to be handled in manual or UI automation later in the process.
While writing the application or feature, developers should verifying their work with unit tests. These small bits of code are designed to confirm the functions are performing tasks as expected and respond gracefully should an error occur. This is often an area that is neglected either because of time constraints or a lack of testing expertise. This where the importance of test planning prior to development comes into play.
By providing insight earlier in the process, the testers will have provided the developers with added insight for creating a more comprehensive unit test suite. With the required tests already provided, the developers will be able to save a significant amount of time while writing them in much the same way as having good requirements documentation makes for faster application development in general.
Automated Smoke Testing
Each time a build is performed, a concise set of previously automated tests should be run to assure that there are no glaring problems with pre-existing functionality. This set of tests is commonly referred to as smoke tests. The suite in use will likely consist of tests from unit, integration, API, and UI automation efforts but should run in a minimal amount of time while covering a maximum amount of business paths (or at least the main “happy paths”).
The next stage of testing after confirming each of the individual pieces is working and to check that the modules are working together. This is the realm of the integration test. These higher level checks are closely related to unit tests and should be created by the developers as they complete individual modules of functionality. I recommend integration tests be handled by developers because they need to occur at a point in code where there may not be an easy way for a non-coding tester to have access. As with unit tests, however, the tester will have had input and helped design the tests that would be required so as not place an additional burden on development.
Some people consider API testing as a subset of integration testing. I choose to have it stand alone for two reason. The first is that an API may consist of multiple modules that should have integration tests performed prior to testing the combined interface. The second reason is that the task of testing an API can be performed by either a developer, who may write a programmatic test similar to an integration test, or a tester using a tool such as Postman, SoapUI, or one of the many others available. I find the latter to be the most influencing reason to keep them separate.
API tests represent a turning point in the project timeline. Prior to an API being available, testers can offer input but are limited in the direct assistance they can provide to the developers. Once prepared, the team can decide on who will be handling the testing based on workload rather than skill set (although the skills available may still be a factor).
Automated Regression Testing
Normally viewed as the bane of a testers existence, regression testing is a tedious process through which we verify pre-existing functionality continues to work. While it is important, performing this task manually is extremely time consuming. For this reason, I recommend regression testing be relegated to automation that has been created over time.
As with the smoke tests, a regression test suite will consist of unit, integration, API, and UI automation tests created during previous projects. The difference is that a regression test employs more than the basic functionality. Regression suites should include edge cases and less common paths through the product but should not necessarily contain every test ever written for the product. Depending on the teams merging process, this suite may or may not contain the tests from the current development cycle. In the interest of time, I would recommend leaving those tests out of the regression suite until the project has been completed and the tests reviewed.
Scripted Manual Testing
In recent years, there has been a growing movement away from scripted testing within the testing community. This has been largely been as a result of Agile adoption and the need to perform more testing in less time. While I agree that testers should be focusing more effort on exploratory testing, there are practical reasons to continue providing this type of testing.
The most pressing reasons I have found for taking the time to write and execute scripted tests are to feed the next automation stage and to prevent the automation stage from delaying a release. Once written, the scripted tests provide an easy to follow guide for the automation engineer which will make the process of automating them much faster. By performing the tests manually, testers confirm both the validity of the test steps for automation and the functionality of the application under test so risk is reduced if the product needs to go out before the automation is completed. There is also the possibility that some of the tests may be found to be redundant or unnecessary which will further reduce the load for automation efforts.
New UI Automation
As the end of the project approaches, the application should become stable enough to reduce the risk of immediate refactoring of UI tests while they are written. As mentioned previously, new UI automation tests should be based on the scripted tests that have been written. You may ask what is the purpose of adding this automation as part of the process when they have already been passed manually. It’s a good question. By automating these tests prior to release, the team is able to focus on functionality that is still fresh in their minds. Completing before the team moves on to the next project also means that these tests will be available use in Smoke and Regression suites in the next project which will reduce manual testing requirements.
Exploratory testing is a form of manual testing that focuses on business processes and paths through the software. Rather than being tied to a script, the tester is guided by a charter which gives them greater flexibility in their testing. The goal is to examine certain portions of the application as a user would do but with a bit more emphasis on what they might do than what they are supposed to do. My inclusion of this step at the end will likely be met with resistance because the benefits of this technique can be felt at any point in the development process. My thoughts of keeping it towards the end are based solely on experience and acceptance of it in workplaces.
Many companies are skeptical of performing exploratory testing because it is misunderstood. many believe that it will increase the risk of missing vital functionality because it is unstructured. My choice of performing exploratory testing at the end of a cycle will generally result in it being allowed and providing a foothold for expanding it’s use in the organization.
Final Thoughts on the Testing Process
I’m sure you noticed that a number of the testing activities in the diagram were paired. This was done to show which ones I believe can be performed concurrently. In the following sections I’ll explain my reasoning.
Integration and API Testing
Considering I stated that API testing often relies on integration tests, it may seem odd that I would list these as concurrent stages. I did this as a consideration for close coupling of the two types of tests and possibility of splitting the duty between developers and testers. If developer is creating both integration and API tests, it is very likely that they would be written at the same time as necessary pieces are being completed. If the duty is being separated, the tester can begin working on the API while the developer is working on integration tests for segments that do not effect it. By running these tests concurrently, the team is able to be more efficient with the time available.
Automated Regression and Scripted Manual Testing
These two stages make an obvious pairing in terms of utilization. An automated regression suite can take hours or even days to complete. Since the expectation for these tests is a clean run, there is no reason why testers shouldn’t begin their manual tests against the new functionality.
New UI Automation and Exploratory Testing
The final set of testing activities can be seen as cleanup or polishing for the application. Assuming that the same person is not responsible for performing both tasks, automating the scripted tests and performing exploratory tests can provide productive activities for the team as they are preparing for release.
The Release Stage
Release is generally considered as the final step of a process where everyone gets to celebrate and take a breath. I’m sorry to say that I disagree. When the product is released, the team now has the opportunity to reflect on the project and do some additional administrative steps before starting in on their next project. This is the stage in which they will review the tests and processes for relevance going forward, select the tests that are to be persisted in the smoke and regression suites, and perform any tasks left over (such as completing new UI automation) so their next project starts with a clean slate.
It’s Not Perfect
The process I propose is entirely based on my observations and experience working with multiple teams using various methodologies. While I have found that this can work in most environments, I am not naive enough to think it will work for every project or team. It is offered here as a suggestion for those seeking direction. If it works as is then I am glad to have been of assistance. If your team dynamic prevents the stages to work in this order, maybe you can use it as a starting point and then tweak the process to better fit your team. Even if you found my process to be completely unhelpful, I thank you for taking the time to read it through.
Just in case the original diagram wasn’t cool enough to impress anyone, I made an alternate one. Seriously, could we get any more awesome than the Testing Cobra?
Over my 20+ years in software development and testing, I have witnessed (and suffered from) a common phenomenon regarding tool use and technique adoption. When a tester or company invests in an application or process, it has a tendency to become a hammer and every test looks like a nail. This can effect both manual and automated testing with the most common consequences being lost time, confidence, effectiveness, and ultimately money.
My experience has shown me that we, as teams, need to take the time to identify and sort the nails (unit tests), screws (integration test), and other fasteners (UI automation, exploratory and scripted testing, etc.) so we can select the best tool for implementing them. For example, Selenium is one of my favorite Web UI automation tools, but you shouldn’t use it to test the functionality of an API; other programs such as SoapUI, Swagger, and Postman are much better suited to that task. The same is true regarding methodologies. As I discussed in a previous post, Automate All the things, manual testing and automation should be used to augment each other rather than as opposing or mutually exclusive solutions. The key is to provide the most value for the effort.
While that is a simple statement to make, it is a difficult task to complete because of the number of variable involved. The value of a unit test for field validation may be significant for a new site in development, but that test may no longer matter on a legacy page. Similarly, test that requires only five minutes for a manual tester wouldn’t warrant an automated test, unless that test needs to be performed repeatedly over an extended period of time at which point it would be worth spending hours to develop it. The point I am attempting to make here is that decisions need to be based on each team’s needs and current situation and there is no “magic bullet”.
It may seem like I am swinging a little wide of the topic and you may be asking, “what does this have to with choosing the right tool?”. Keeping an open mind and reviewing the complete situation is the most important part of tool selection. A clever automation engineer can find a way to use her favorite tool to to perform a specified task, but is there a better option? Will the test be needed in the future? If it was a one-time feature test, then it should have been handled manually. Was there a tool available that would have allowed the test to written faster and more stable rather than forcing the process to conform to the tool?
I seem to presenting more questions than answers in this post and I hope that I haven’t just rambled aimlessly. There is an answer hidden within all the examples. That solution is think objectively about process and tools if you want to find the best solution available. Sometimes, you will need to compromise on the tools you use. That doesn’t mean you can’t keep looking for a better way. You also shouldn’t be afraid of changing tools. Over time, great tools become mediocre as new tools are developed and technology changes. It may just prove to be more effective to phase out the old in favor of an improved solution rather than limping the original along because “we’ve already invested so much in it”. If you want to provide value to your organization, you need to be willing to look for it.
At some point in their lives, most people will find themselves battling “analysis paralysis” and being completely unable to get something done (or even started). Sometimes this is caused by a lack of confidence or an underlying desire to procrastinate. It can also be the result of trying to tackle something that is simply too large and becoming overwhelmed. No matter what the cause is, the result—getting nothing done—is the same. Fortunately, there are solutions to these impediments and they all start with a decision.
The choice will vary according to the exact situation but ultimately it can be reduced to selecting a task to accomplish and committing to getting it done. This probably seems like an oversimplification and a rather obvious step, but there is a lot more to it than you first see. For example, you have to not care if the decision you just made is right or wrong so long as you have begun to move. Once in motion, you can always change the direction if you need to. This is the most difficult part for those struggling with a lack of confidence.
For those of you who are overwhelmed by a large project, you need to identify something small and obvious that you can get done quickly and then do it. The first task will lead to another small piece that can then be accomplished in the same way. Of course, you might find that how you solved the first piece doesn’t fit with what you need to complete the next and now you have to change your approach. I can already hear some of you saying that if you had just spent a bit more time analyzing the project that wouldn’t have happened. That’s true. It’s also true that if you had continued to analyze the project rather than starting to develop it then you would still be at square one and possible still not been aware of the problem at all.
In both of the situations above, you may have noticed some common elements: doing something rather than thinking about it and not being afraid to make a mistake. The latter of these is probably the most important and hardest to deal with. No one wants to spend their time being wrong. Many of us grow up being told that we can only get somewhere in life is if we “do it right the first time” otherwise we’ll lose our spot to someone that can. This logic runs counter to any form of innovation. How do you do something that has been done before correctly the first time? How do you even know it is correct? You won’t until you’ve tried it.
Just to be clear, I am not suggesting that you should run off half-cocked and start projects without any planning. That is as sure of a way to fail as never getting started at all. The correct time to take the kind of step I’m suggesting is when you have a basic understanding of what you want to accomplish but you are delaying because you aren’t sure of what should be done first or if what is required can be done with your current tool set. Those are times that you need top take a deep breath and jump in. It really is ok to make mistakes so long as you learn from them.
I offer you a few quotes to back up my philosophy:
- “A journey of a thousand miles begins with a single step” – Lao Tzu
- “I have not failed. I’ve just found 10000 ways that won’t work.” – Thomas A. Edison
- “You have brains in your head. You have feet in your shoes. You can steer yourself in any direction you choose. You’re on your own, and you know what you know. And you are the guy who’ll decide where to go.” – Dr. Seuss
For those of you reading this who fall into the group of procrastinators like me, I also have some words of wisdom. Set a goal and hard dates for accomplishing your tasks. This will provide you with some additional motivation. If you are concerned about possible failure, make your tasks a little more flexible. Instead of promising finished deliverables, you can specify that you will complete a proof of concept or perform a simple experiment. Ultimately, my advice is that doing something is better than doing nothing.