Reading:
Serenity Automation Framework Basics
Share:

Serenity Automation Framework Basics

Learn the basics of the automation framework Serenity in this comprehensive article from Ioan Solderea

As a software tester you likely have come into contact or at least heard of test automation. There are debates that automation will eliminate all manual jobs and that in the future all will be automated, but that is for another time, another article.

Today let’s begin with the following premise: You are a tester in an organization and your project manager or CTO tells you that a client is coming today and that they would like you to present a test automation framework. Have no fear, that is why you are here, together we are going to set up a test automation framework quickly and easily.

What Is an Automation Framework?

Before we set up the framework, I would like to explain what a framework contains.

From my experience, for a framework to be truly useful, it needs to have:

  • Structure - the code needs to be well organized and all the packages linked together in an easy to understand way

  • Reporting - test run reporting is essential so that an overview of the current system state can be easily provided to the team or the client

  • Test data and screenshots - being able to read and write files while running your tests helps you cover a lot of scenarios faster, and making a screenshot when a test fails will help you debug what went wrong

  • Integration with bug trackers - pushing direct updates to tools like JIRA after a test has run provides quick feedback to the team

  • Integration with other libraries - no matter how well you can code or how good your framework, it is always ideal to be able to integrate helpful third party libraries if needed

  • Continuous integration - it is good that the tests run on your machine but it is even better if they run on a virtual machine on a schedule

Considering the above list, you can either create a framework from scratch or use an existing framework. However, your time constraint in this scenario tips the scale in favor of using an existing framework.

Prerequisites and setup

The framework which we will cover is called Serenity. It has all the features we covered in The Premise and more.

Before starting setup we need to check that your system has Java and Maven installed. In case you know for sure that you do not have Java or Maven, then please follow the steps in the article below before continuing the setup.

https://mkyong.com/maven/how-to-install-maven-in-windows/

If you do have Java and Maven installed, check the versions as noted below.

Verify Java Installation

Open your command prompt and type java -version. You should see information similar to the one below. Make sure that the installed version meets current Serenity requirements.

java -version
java version "15.0.1" 2020-10-20
Java(TM) SE Runtime Environment (build 15.0.1+9-18)
Java HotSpot(TM) 64-Bit Server VM (build 15.0.1+9-18, mixed mode, sharing)

Verify Maven Installation

Open your command prompt and type maven -version. You should see information similar to the one below . Make sure that the installed version meets current Serenity requirements.

    
mvn -version
Apache Maven 3.8.2 (ea98e05a04480131370aa0c110b8c54cf726c06f)
Maven home: C:\Program Files\apache-maven-3.8.2
Java version: 15.0.1, vendor: Oracle Corporation, runtime: C:\Program Files\Java\jdk-15.0.1
Default locale: de_DE, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
    

Install And Set Up Serenity

Now that all the prerequisites are met it is time to move on to setting up Serenity.

  1. Open a command prompt.

  2. Navigate to a new folder where you want to make the installation.

  3. In your new folder, type mvn archetype:generate -Dfilter=serenity and then press enter. What this will do is show you all the maven archetypes that exist for Serenity. You can think of an archetype as a customizable template. Do not get scared, it may take a while for all the dependencies to be downloaded to your device.

  4. In the next step we will need to tell maven what archetype we want to install. In the section where you need to add a number type 5 f(Serenity automated acceptance testing project using Selenium 2 and JUnit) and then press enter.

  5. For the next step you can go with the default and hit enter. This will install the latest version of that archetype.

  6. We are almost done with the setup. Now even if the archetype contains all the relevant information for our setup we still need to add standard information for a maven project: groupid, artifactd, version, package.

  7. For the groupid I added net.article.mot, but you can add your own groupid ( see reformat description) and press enter.

    Define value for property 'groupId': net.article.mot

    • Now you just need to enter an artifactId ( I added framework) and press enter.

    Define value for property 'artifactId': framework

    • Enter 1 for version and press enter.

    Define value for property 'version' 1.0-SNAPSHOT: : 1

    • For package just press enter and maven will take the value of the groupid.

    Define value for property 'package' net.article.mot: :

    • You will be prompted for final confirmation of the standard information. Press enter to begin the final installation step.

    Confirm properties configuration:

                
                    groupId: net.article.mot
                    artifactId: framework
                    version: 1
                    package: net.article.mot
                    Y: : y
                
            
  8. If all goes well you should see Build Success. However in case your build was not successful please repeat all the steps up to this point.

                
    [INFO] ----------------------------------------------------------------------------
    [INFO] Using following parameters for creating project from Archetype: serenity-junit-archetype:2.3.2
    [INFO] ----------------------------------------------------------------------------
    [INFO] Parameter: groupId, Value: net.article.mot
    [INFO] Parameter: artifactId, Value: framework
    [INFO] Parameter: version, Value: 1
    [INFO] Parameter: package, Value: net.article.mot
    [INFO] Parameter: packageInPathFormat, Value: net/article/mot
    [INFO] Parameter: package, Value: net.article.mot
    [INFO] Parameter: groupId, Value: net.article.mot
    [INFO] Parameter: artifactId, Value: framework
    [INFO] Parameter: version, Value: 1
    [INFO] Project created from Archetype in dir: D:\artile2\framework
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 12:50 min
    [INFO] Finished at: 2022-04-15T15:09:32+02:00
    [INFO] ------------------------------------------------------------------------
                
            

And with that final step we have completed the setup. Now all that remains is to open the folder in an IDE (I will use Intellig, but Eclipse or any other IDE for java should also work fine) and to explore and understand the framework.

Exploring Your Serenity Installation

Folders and Files

After all that time spent on installation and configuration I am pleased to tell you that the setup is done and that you can actually run a test.

However, before running a test, we will inspect and explore the structure of our project in terms of the folders and files that were created.

There are three main folders that define the Serenity framework structure:

  1. Features - features.search in the side image

  2. Pages - pages in the side image

  3. Steps - steps.serenity in the side image

Do not worry if it is not clear at the moment what these represent for the framework and how they all fit together. We will explain each part in the next sections.

Folders are good for structure, you might think but where are the tests, the elements, the assertions, the webdriver?

We are going to link the entities discussed above to test-specific elements in the following sections.

Tests

All tests have to be in the features folder as per the default configuration. In the project structure if we open the feature.search folder we will see the test that was created for us.

If we open the test class, SearchByKeywordStory, and check its structure, the following sections can be seen:

  • The Serenity Test Runner sets up the test and also records the results for us

  • The Webdriver will allow us to run our tests against a real browser

  • Steps, which provide essential implementation detail

  • The test itself contains the business logic that we want to check

    
@RunWith(SerenityRunner.class)
public class SearchByKeywordStory {
    
    @Managed(uniqueSession = true)
    public WebDriver webdriver;

 

@Steps public EndUserSteps anna; @Issue("#WIKI-1") @Test public void searching_by_keyword_apple_should_display_the_corresponding_article() { anna.is_the_home_page(); anna.looks_for("apple"); anna.should_see_definition("A common, round fruit produced by the tree Malus domestica, cultivated in temperate climates."); } }

Steps

In the project structure, if we open the steps.serenity folder, we will see the steps class that was created for us.

But what are these steps, you might ask, what do they contain, and what are they good for? In simple terms, steps contain different types of actions that you can perform on a page, ranging from entering text, clicking element, or performing checks ( assertions), and more.

To better understand steps, let us open and explore the EndUserSteps class structure:

  • Pages - the pages that you want to use in your steps have to be imported and instantiated in the steps

                
    public class EndUserSteps {
        
        DictionaryPage dictionaryPage;
                
            
  • Steps - the steps define interactions made on the page. This can be anything from clicking on a button to performing a food order. Any method that we want can become a step once it has the @Step annotation. As you can see below, you can use steps for different types of interactions.

  • Steps for a simple action

                
    @Step
    public void enters(String keyword) {
        dictionaryPage.enter_keywords(keyword);
    }
                
            
  • Steps for multiple actions

                
    @Step
    public void looks_for(String term) {
        enters(term);
        starts_search();
    }
    
                
            
  • Steps to perform a check

                
    @Step
    public void should_see_definition(String definition) {
        assertThat(dictionaryPage.getDefinitions(), hasItem(containsString(definition)));
    }
                
            

Pages

All pages have to be under the pages folder as per the default configuration. In the project structure if we open the pages folder we will see the page class that was created for us.

Pages encapsulate how a test interacts with a specific web page. They hide the WebDriver implementation details of how elements on a page are accessed and manipulated behind more enterprise-friendly methods. Like steps, pages are reusable components that make tests easier to understand and maintain.

To better understand pages let us open and explore the DictionaryPage class structure:

  • DefaultUrl - it is recommended that you add a default URL to your page. This URL will be used when the page is called in your steps.

            
    @DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary")
    public class DictionaryPage extends PageObject {
            
        
  • Extend PageObject - all pages in Serenity need to extend the PageObject

  • Element locators - to interact with the elements on a page we will need to uniquely identify those elements. In Serenity this is done using the @FindBy annotation and the locator type. Serenity supports all the known existing selectors in selenium. For a full list see https://www.selenium.dev/documentation/webdriver/elements/locators/ .

        
    @FindBy(name="search")
    private WebElementFacade searchTerms;
        
    
  • Element interactions - once the elements are identified, you can define methods to map different actions performed with the elements. All those methods can be used by the Steps that instantiate the page.

            
        public void enter_keywords(String keyword) {
            searchTerms.type(keyword);
        }
            
        

Serenity Configuration File

Apart from the folders and files described before, there is one more section that we should explore before running the tests, and that is the serenity.properties file.

This file controls the setup for running our test and provides different customizable options, for example:

  • Browser used

  • Running test headless

  • Browser size

We will change a few of these settings in a future section to see how they affect the test run, but for now it is important just to know where the file is located.

    
# Define the default driver
#webdriver.driver=phantomjs

# Appears at the top of the reports
serenity.project.name = Demo Project using Serenity and JUnit
webdriver.autodownload = true
webdriver.driver = chrome
headless.mode = true

# Root package for any JUnit acceptance tests
serenity.test.root=net.article.mot.features

# Customise your riequirements hierarchy
#serenity.requirement.types=feature, story

# Run the tests without calling webdriver - useful to check your JUnit wiring
#serenity.dry.run=true

# Customise browser siz
#serenity.browser.height = 1200
#serenity.browser.width = 1200
    

Running The Sample Tests

Now that you know what each section is for, let's run some tests. A few of them were already generated for us so we just need the right command.

The test set that was generated for us will open the Wikipedia dictionary page and search for a keyword, then check if its definition can be found.

To run the test set follow one of the options below:

  • Open a terminal, navigate to the folder where you have installed the framework (make sure it is the folder with a pom.xml inside) and run maven clean verify

  • From your IDE open your terminal and run mvn clean verify

                
    Regardless of the option you selected you should see the output below:
    [INFO] Scanning for projects...
    [INFO]
    [INFO] ---------------------< net.article.mot:framework >----------------------
    [INFO] Building Serenity project with JUnit and WebDriver 1
    [INFO] --------------------------------[ jar ]---------------------------------
    Downloading from central: https://jcenter.bintray.com/io/cucumber/messages/maven-metadata.xml
    Downloaded from central: https://jcenter.bintray.com/io/cucumber/messages/maven-metadata.xml (2.1 kB at 2.4 kB/s)
    [INFO]
    [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ framework ---        
                
            

    At the end of the run you will notice that a summary is generated and that some of the tests failed. That’s as expected: we will find out in the next section why they failed and will even fix the failures by the end of this article.

        
    ] -----------------------------------------
    [INFO] SERENITY TESTS : FAILURE
    [INFO] -----------------------------------------
    [INFO] | Tests executed     | 3
    [INFO] | Tests passed      | 1
    [INFO] | Tests failed      | 1
    [INFO] | Tests with errors   | 0
    [INFO] | Tests compromised   | 0
    [INFO] | Tests pending     | 1
    [INFO] | Tests ignored/skipped | 0
    [INFO] ------------------------ | --------------
    [INFO] | Total Duration     | 16s 249ms
    [INFO] | Fastest test took   | 338ms
    [INFO] | Slowest test took   | 11s 206ms
    [INFO] -----------------------------------------
        
    

View Test Results In A Report

Since I said at the outset that a good framework should provide for test reporting, you might ask where the report is. The good news is we have a report, and we also know where Serenity stored it for us. Just have a look under the summary where it states Full Report.

    
[INFO] | Total Duration     | 16s 249ms
[INFO] | Fastest test took   | 338ms
[INFO] | Slowest test took   | 11s 206ms
[INFO] -----------------------------------------
[INFO]
[INFO] SERENITY REPORTS
[INFO]  - Full Report: file:///D:/artile/framework/target/site/serenity/index.html
    

Open index.html in a browser, and you will see an overview of the tests run and their status.

If you navigate to the failed test it shows exactly where the test failed, what the error was, and even a screenshot of the browser as the test proceeded through the steps. In our case the text that the test was expecting did not match the text on the page.

Doing A Root Cause Analysis of Failed Tests

Now that you have seen how the setup is done and how the tests are run, we will expand on that and have some more fun.

When we ran the test there were a few things that were unexpected:

  • We saw no browser being opened

  • We had tests that failed

  • We had tests that did not run at all: one test was marked as Pending

And this is just to name a few. We will fix all the issues one at a time in the sections that follow.

Watch The Test Run In A Browser

To see tests run in a browser, we need to change the default behavior of Serenity, which is to run tests in headless mode. This is easily done by changing the value for headless mode from true to false in the serenity.properties file.

    
# Appears at the top of the reports
serenity.project.name = Demo Project using Serenity and JUnit
webdriver.autodownload = true
webdriver.driver = chrome
headless.mode = false —-> this needs to be changed from true to false
    

Once that is done, save your changes and run the tests again (see Running The Sample Tests).

Now a browser window will open and you can see all the actions defined in our test being performed in the browser.

Fix The Failing Test By Changing The Assertion

We can now watch the tests run, but we don’t yet understand why one test is failing. To fix the failing test we first need to understand what the steps do and what they verify.

According to the test we need to navigate to the home page for http://en.wiktionary.org/wiki/Wiktionary, then look for the word “pear,” and check if a specific definition for “pear” is present on the page.

    
@Test
public void searching_by_keyword_banana_should_display_the_corresponding_article() {
    anna.is_the_home_page();
    anna.looks_for("pear");
    anna.should_see_definition("An edible fruit produced by the pear tree, similar to an apple but elongated towards the stem.");
}
    

 

If we do the steps manually in a browser we will come to the conclusion that there is no definition for “pear'' that matches what we want. But the closest to what we want is the first entry in the list.

So to make the test pass we need to replace the searched text with the first entry.

    
@Test
public void searching_by_keyword_banana_should_display_the_corresponding_article() {
    anna.is_the_home_page();
    anna.looks_for("pear");
    anna.should_see_definition("An edible fruit produced by the pear tree, similar to an apple but typically elongated towards the stem.");
}        
    

Before you run the test again, you should change its name, since we are not really searching for “banana” but for “pear”.

    
@Test
public void searching_by_keyword_pear_should_display_the_corresponding_article() {
    anna.is_the_home_page();
    anna.looks_for("pear");
    anna.should_see_definition("An edible fruit produced by the pear tree, similar to an apple but typically elongated towards the stem.");
}
    

Rerun the tests: this time they should all run and pass. See the output below.


[INFO] SERENITY TESTS : PENDING
[INFO] -----------------------------------------
[INFO] | Tests executed     | 3
[INFO] | Tests passed      | 2
[INFO] | Tests failed      | 0
[INFO] | Tests with errors   | 0
[INFO] | Tests compromised   | 0
[INFO] | Tests pending     | 1
[INFO] | Tests ignored/skipped | 0

Create A New Test

Before you can say that you are comfortable using the framework, you should try to add your own test. Well, Serenity did the starting work for us here since we have one test that did not run in all our test runs.

That test should, according to its name, search for an ambiguous keyword (instead of pear or apple) and return the disambiguation page.

    
@Pending @Test
public void searching_by_ambiguious_keyword_should_display_the_disambiguation_page() {

}
    

Make The Pending Test Run

Before adding content to the test you first need to make sure that the test will actually run when you start the test. In our case we need to remove the @Pending annotation in order to achieve this.

    
@Test
public void searching_by_ambiguious_keyword_should_display_the_disambiguation_page() {

}
    

If we run the tests again we will see that now all three tests passed. But since our new test is not checking anything we cannot really consider it a test.

    
[INFO] SERENITY TESTS : SUCCESS
[INFO] -----------------------------------------
[INFO] | Tests executed     | 3
[INFO] | Tests passed      | 3
[INFO] | Tests failed      | 0
[INFO] | Tests with errors   | 0
[INFO] | Tests compromised   | 0
[INFO] | Tests pending     | 0
[INFO] | Tests ignored/skipped | 0
    

Verify The Scenario Before Adding Test Steps

Now that we know that the test will run, we need to make sure that we have all steps for our tests defined and clear. And one of the best ways of achieving this is to run them manually and document the process.

The scenario for our test has the following steps:

  1. The user goes to the DictionaryPage home url ( https://en.wiktionary.org/wiki/Wiktionary )

  2. The user enters some ambiguous keyword or phrase, for example - This is the best MOT article, and presses enter

  3. Since no page was found for our ambiguous keyword, a Special page opens. This is our disambiguation_page

Add A New Page

We now know our test scenario, so we can do the preparations needed to make our test. First, we need to take is to create a new page for our disambiguation_page, since we do not yet have the Page in our framework.

Add a new class under the pages folder. I named mine DisambiguationPage ( you can use any name that you want) and also made sure that it extends PageObjects.


package net.article.mot.pages;

import net.thucydides.core.pages.PageObject;

public class DisambiguationPage extends PageObject {

@FindBy(id="searchMenuNecLink") // locator
private WebElementFacade createPage; // element


}    

In the new Page, to check that we are on the correct page, we will add at least one element and one function that checks the text for that element.

The element that we want to add is the create new Page link, which is easily identifiable since it has a unique id.

To add the element for “create a new page” to the Page object, we need to add two imports:

  • the findby annotation - allows you to define the locator for your element

            
    import net.serenitybdd.core.annotations.findby.FindBy;
            
        
  • the WebElementFacade - WebElementFacades are largely interchangeable with WebElements: you just declare a variable of type WebElementFacade instead of type WebElement

            
    import net.serenitybdd.core.pages.WebElementFacade;
            
        

    Once you do the imports, you can define your element by first using the @FindBy annotation and underneath providing a name for your Element as seens below.

        
    import net.serenitybdd.core.annotations.findby.FindBy;
    import net.serenitybdd.core.pages.WebElementFacade;
    import net.thucydides.core.pages.PageObject;
    
    public class DisambiguationPage extends PageObject {
    
        @FindBy(id="searchMenuNecLink") // locator
        private WebElementFacade createPage; // element
    
    
    }
        
    

    Apart from the element for the “create new page” link and its locator, we will need to add a function that reads the text from the element. This can be achieved by calling the gettext() method on the Element. You can name your function any name you desire; however, I would recommend you use a proper name for its purpose and also make sure that it has a return type of type string, since we want to return some text. Once you have defined your function your Page class should look similar to the one below.

        
    import net.serenitybdd.core.annotations.findby.FindBy;
    import net.serenitybdd.core.pages.WebElementFacade;
    import net.thucydides.core.pages.PageObject;
    
    public class DisambiguationPage extends PageObject {
    
        @FindBy(id="searchMenuNecLink")
        private WebElementFacade createPage;
    
        public String getLinkText() {
            return createPage.getText();
        }
    
    }
        
    

Add Some Steps

Now that we’ve added an object representation for the page and its elements to Serenity, we can proceed to add some new entries in the Steps. We do not need to add a new class here; we can simply update the existing EndUserSteps.

But before we can use the new page and its functions we need to perform two steps:

  • import the page

            
        import net.article.mot.pages.DisambiguationPage;
            
        

    If you named your page differently than we did, you will need to use the name you chose.

  • instantiate the page in the steps.

                
    public class EndUserSteps {
        DictionaryPage dictionaryPage;    
        DisambiguationPage disambiguationPage;
                
            

If you named your page differently than we did, you will need to use the name you chose.

Once the import and instantiation operations are completed, we will need to add a new function to check that we see the newPage link. To do this you need to:

  • Add a @Step annotation (without it, the function is not considered a step)

            
    @Step
            
        
  • Define a function that takes a string argument (you can use any name you desire, we chose should_see_newPage)

            
    public void should_see_newPage(String link) {
    
    }
            
        
  • Define the check (assertion) . We want to check that the page contains a link with some specific text.

                
    assertThat(disambiguationPage.getLinkText(),containsString(link));
                
            

    If all steps have been performed correctly the new Step should look like this:

    
    @Step
    public void should_see_newPage(String link) {
        assertThat(disambiguationPage.getLinkText(),containsString(link));
    }
    
    

Update The Test

Now that we have done the updates on the Pages and Steps sections, all that remains to be done is to put all of this together in our test.

To make sure we cover all aspects of our test scenario, we can add the code corresponding to each of our manual steps:

        
anna.is_the_home_page();
        
    
  1. The user goes to the DictionaryPage home url (https://en.wiktionary.org/wiki/Wiktionary).

  2. The user enters some ambiguous keyword or phrase, for example - This is the best MOT article, and presses enter.

                
                    anna.looks_for("This is the best MOT Article");                
                
            
  3. Since no page was found for our ambiguous keyword, a Special page opens. We check this by making sure that the page that we see has a create new page link.

                
    anna.should_see_newPage("create");
                
            

The final structure of the test should look as indicated below.

    
@Test
public void searching_by_ambiguious_keyword_should_display_the_disambiguation_page() {
    anna.is_the_home_page();
    anna.looks_for("This is the best MOT Article");
    anna.should_see_newPage("create");
}
    

Review The Result

With the new test update, all that remains is to run mvn clean verify and then check the report. If we did everything correctly, you should see that all tests have passed. In the report we have our new test with all the steps and screenshots.

Conclusion

As promised at the beginning of this article, you now have the setup of a full automation framework. With what you have learned here you can create a functional proof of concept for situations where you need to show a structured approach to either your manager or a potential client.

Also with some more time on your hands you can explore and experiment with the other available setups of Serenity, like Cucumber integration or the Screenplay pattern. Go ahead and try them out and share your thoughts in the MOT Club!

For More Information…

Introduction to Java - Mike Talks | MoT

Selenium WebDriver API Introduction in Java - Richard Bradshaw

Essentials - Introduction to Software Development and Testing | MoT

Author Bio

A passionate QA with focus on quality and test automation, creator of https://funquote.herokuapp.com/quotes and part time YouTuber.

I am one of those people who want to know all about all, but will also be happy knowing a lot about a lot. Because of my curious nature and desire to learn, I choose to be a tester.

After my day job is done I usually write code or post videos online.

If you want to connect just ping me on LinkedIn or search for me in the MOT Club as ResterTest.

Ioan Solderea's profile

Ioan Solderea

Lead QA

I am one of those people who want to know all about all but will also be happy knowing a lot about a lot. Because of this I choose to be a tester since you get to learn always new technologies, you get to test in the most diverse areas and it is always fun to tell people you found a bug.


Share this article: