Libertad ("Freedom"), a character on Quino's Mafalda describing her father's fear on elections:

• Mafalda: And your dad, Libertad, who is he going to vote for in the next elections?
• Libertad: Oh my... he has been with such a depressed face!
• Mafalda: ah, he hasn't decided for any candidate yet?
• Libertad: Yes, he has, and he has such a depressed face!
• Mafalda: Why? he thinks his candidate is going to loose?
• Libertad: No, he thinks he is going to win, and he has such a depressed face!
• Mafalda: I don't understand your dad: he knows who he is going to vote for, and he thinks his candidate is going to win, and he is not happy?
• Libertad: No, and he has such a depressed face!
• Mafalda: but... why? Does he thinks they are not going to let his candidate govern?
• Libertad: Sometimes that's what he thinks and then he has such a depressed face!... but sometimes he thinks they are going to let him govern, and then again he has such a depressed face!
• Mafalda: BUT DAMN IT! IF HIS IS SO WORRIED ABOUT THAT CANDIDATE, WHY NOT JUST VOTE FOR ANY OTHER ONE?
• Libertad: He thought of that... and he was with such a depressed face!

I have been working on the Approvals Workflow Plugin (AWP) since 2007 and just recently released version 2.

Because of the nature of the application, it appeals to enterprises, and they're not that keen on upgrading that often, therefore I have to support several versions of Confluence, and that can be challenging.

For the new version of the plugin, I started using web integration tests to run tests which I used to do manually.

The supported versions of Confluence are from 2.6 to 2.10, so although I have managed to have only one binary, I still have to test on all the supported versions on every release.

In addition to the 80+ unit tests I already have, there are the manual tests that require creating pages, approving documents, adding labels, changing permissions, etc.

But now that I created the Web Tests and included them into my Bamboo plans, every time I commit changes, the tests run in just a few minutes on 2.8.2, 2.9.1 and 2.10.1, which I used to do only when I was about to release.

Although I am not able to run all the tests against all the versions I need, it reduces the amount of manual testing required.

There is not too much documentation on the subject, so I started by looking at the chart-plugin test and still took me a while to get it going, so for the community, here is what I have learnt.

UPDATE: Customware has put together this neat documentation

POM Changes

The dependency is already included in the confluence-plugin-base so as long as you created your plugin using the plugin archetype you are all set.

To start, you only have to make a couple of changes in your xml.pom (here's mines, which you could use a reference).

First you need to define some properties:

<properties>
    <confluence.version>2.10.1</confluence.version>
    <atlassian.product.test-lib.version>1.4.2</atlassian.product.test-lib.version>
    <atlassian.product.data.version>2.10</atlassian.product.data.version>
    <confluence.plugin.bundled>true</confluence.plugin.bundled>
    <confluence.jars.to.remove.regex.list>${basedir}/test/resources/jar-files-to-remove-from-bundle.txt</confluence.jars.to.remove.regex.list>
</properties>

Property	Description	Notes
confluence.version	The version of Confluence on which you are going to be compiling and running the tests.	I was only able to run against 2.8.x, 2.9.x and 2.10.x. There are missing libraries on earlier versions. In theory you also have to define atlassian.product.version but I've found that confluence.version is enough
atlassian.product.test-lib.version	test lib version	I am currently using version 1.4.2 of the confluence-functestrpc-plugin. The latest stable version (2.1) uses a newer version of JWebUnit and changed all the packages. I tried upgrading but had several problems running it.
atlassian.product.data.version	data set version	You can omit defining this one if you are testing against 2.8 or 2.9, but you must set it to 2.10 if running on 2.10
confluence.jars.to.remove.regex.list	the name says it all The file has to be included as part of the test resorces, see below. Mines contains the rexep of my plugin: ^approvalsworkflow-.*.jar$	This important when rerunning the tests.

Then you need to define your test resources, where you need to store that jar-files-to-remove-from-bundle.txt file and any other files you need for your testing (more on this below).

<build>
    ...
    <testSourceDirectory>test</testSourceDirectory>
    <testResources>
        <testResource>
            <directory>test/resources</directory>
            <includes>
                <include>**/*.*</include>
            </includes>
        </testResource>
    </testResources>                          
    ...
</build>

You can look at my pom.xml.

The abstract web test case

You can start by looking at ChartMacroTestCase.java for a set of simple tests. Something to be noted is that the Web tests' package names must start with 'it.'.

My tests were more complex because:

• Some of tests deal with content permissions
• Some require the Metadata Plugin to be installed
• The plugin has some macros which are disabled by default.
• The tests require workflows to be loaded. (workflows are a connection of macros, which contain approvals and triggers)

I had to create an abstract web test class, to provide some common services. Some of them can be useful for other applications and also can give you some ideas.

public class AbstractAwpWebTestCase extends AbstractConfluencePluginWebTestCase {
    protected void setUp() throws Exception {
        super.setUp();
        SpaceHelper spaceHelper;
        spaceHelper = getSpaceHelper();
        spaceHelper.setKey(SPACE_KEY);
        spaceHelper.setName("AWP Test Space");
        spaceHelper.setDescription("AWP Test Space");
        assertTrue(spaceHelper.create());

        configureSpacePermissions();

        enableMacro( "workflow");
        enableMacro("include-segment");
        enableMacro("send-email");
        configurePlugin();
        configureUsers();
        loadRequiredPlugins();
    }
    ...
}

Setting space permissions

The first part of setUp() only creates test space. However, because some of my testing deals with permissions, I had to configure space permissions.

What is done through the UI like this:

It is done in the web test like this:

private void configureSpacePermissions() throws Exception {
    gotoPage("/spaces/editspacepermissions.action?key=" + SPACE_KEY);
    setFormElement("groupsToAdd",GROUP_CONFLUENCEUSERS);
    submit("groupsToAddButton");

    gotoPage("/spaces/editspacepermissions.action?key=" + SPACE_KEY);
    checkCheckbox("confluence_checkbox_viewspace_group_confluence-users");
    checkCheckbox("confluence_checkbox_editspace_group_confluence-users");
    checkCheckbox("confluence_checkbox_exportpage_group_confluence-users");

    checkCheckbox("confluence_checkbox_removepage_group_confluence-users");
    checkCheckbox("confluence_checkbox_editblog_group_confluence-users");
    checkCheckbox("confluence_checkbox_comment_group_confluence-users");
    checkCheckbox("confluence_checkbox_createattachment_group_confluence-users");
    submit();
}

You can use firebug or look at the HTML source to identify the form element names.

Enabling plugin's modules

Then I need to enable some of the plugin's modules that are disabled by default upon installation.

So again, what is done through the UI like this:

It is done in the web test like this:

protected void enableMacro(String name) throws Exception {
    gotoPage("/admin/plugins.action?mode=enable&moduleKey=com.comalatech.workflow%3A" + name);
    assertEquals(200,getDialog().getResponse().getResponseCode());
}

Adding users

The Confluence web test library provide some helpers which use the SOAP API for some operations.

Because of my test cases, I need to create some users:

(view code)

private void configureUsers() throws Exception {
    createGroup(GROUP_REVIEWERS);
    createGroup(GROUP_CONTENTPROVIDERS);
    createUser(AUCTOR,"Auctor Incertus",new String[] {GROUP_CONFLUENCEUSERS, GROUP_CONTENTPROVIDERS});
    createUser(EMENDATOR,"Emendator Asperrimus",new String[] {GROUP_CONFLUENCEUSERS, GROUP_CONTENTPROVIDERS, GROUP_REVIEWERS});
    createUser(PRIMUS,"Primus Emendator",new String[] {GROUP_CONFLUENCEUSERS, GROUP_CONTENTPROVIDERS});
    createUser(READONLY,"Read only user",new String[] {GROUP_CONFLUENCEUSERS, GROUP_CONTENTPROVIDERS});
}

private void createGroup(String name) throws Exception {
    GroupHelper groupHelper = getGroupHelper();
    groupHelper.setName(name);
    assertTrue(groupHelper.create());
}

private void createUser(String name, String fullName, String [] groups) throws Exception {
    UserHelper userHelper = getUserHelper();
    userHelper.setName(name);
    userHelper.setFullName(fullName);
    userHelper.setEmailAddress(name + "@localhost.localdomain");
    userHelper.setPassword(PASSWORD);
    userHelper.setGroups(Arrays.asList(groups));
    assertTrue(userHelper.create());
}

Loading other plugins

My tests require the Metadata Plugin so I copied the jar to the test resources directory and this is how it gets loaded:

private void loadRequiredPlugins() throws Exception {
    ConfluenceWebTester confluenceWebTester = getConfluenceWebTester();
    File plugin = new File(SystemUtils.JAVA_IO_TMPDIR,"metadata-2.1.0.jar");
    InputStream in = null;
    OutputStream out = null;
    try {
        in = new BufferedInputStream(getClass().getClassLoader().getResourceAsStream("metadata-2.1.0.jar"));
        out = new BufferedOutputStream(new FileOutputStream(plugin));
        IOUtils.copy(in, out);
    } finally {
        IOUtils.closeQuietly(out);
        IOUtils.closeQuietly(in);
    }
    confluenceWebTester.setConfluencePluginJar(plugin);
    confluenceWebTester.setConfluencePluginName("Metadata Plugin");
    installPlugin();
}

Loading test pages

I need some predefined workflows for my tests. The workflows are plain wiki markup. The workflows are stored also in the test resources directory, and loading them is just a matter of creating a page using the helper classes:

(view code)

public long loadWorkflow(String workflowName) throws Exception {
    String workflow = getWorkflow(workflowName);
    long pageId = createPage(workflowName,workflow);
    assertTextNotInElement("content","error");
    return pageId;
}

public long createPage(long parentId, String title, String content) throws Exception {
    final PageHelper pageHelper;
    pageHelper = getPageHelper();
    pageHelper.setParentId(parentId);
    pageHelper.setSpaceKey(getSpaceHelper(SPACE_KEY).getKey());
    pageHelper.setTitle(title);
    pageHelper.setContent(content);
    assertTrue(pageHelper.create());
    gotoPage("/pages/viewpage.action?pageId=" + pageHelper.getId());
    return pageHelper.getId();
}

public String getWorkflow(String workflowName) throws Exception {
    InputStream path = AbstractAwpWebTestCase.class.getClassLoader().getResourceAsStream(workflowName);
    if (path == null) {
        throw new Exception("could not get " + workflowName);
    }
    BufferedReader br = new BufferedReader(new InputStreamReader(path));
    StringBuffer sb = new StringBuffer();
    String line;
    while (null != (line = br.readLine())) {
        sb.append(line).append("\n");
    }
    return sb.toString();
}

There are other methods I created, which include workflow-specific operations (i.e. approving pages) and assertion (i.e. asserting approvals availability and status) .

The test cases

Developing the first tests was very time consuming, as it takes a while to get confluence running, but it was worth it, as the new ones were really easy.

Here's an example of a test that simulates what this demo does:

(view workflow)

{workflow:adminusers=admin}
    {approval:Author|weight=10|permission=edit}
    {approval:Reviewer|hasapproval=Author|weight=20|permission:edit|usergroup=reviewers}
    {approval:Editor in Chief|hasapproval=Reviewer|weight=30|user=primus|final=true}
    {trigger:pagecreated}
        {set-label:label_created}
        {set-message:style=note}AAA{set-message}
    {trigger}

    {trigger:pageapproved|approval=Author}
        {remove-label:label_created}
        {set-label:label_author}
        {set-message:style=info}BBB{set-message}
    {trigger}

    {trigger:pageapproved|approval=Author|hadapproval=Author}
        {set-label:reapproved}        
    {trigger}

    {trigger:pageapproved|approval=Reviewer}
        {set-message:style=info}Page reviewed by [~@user@] _@approvalcomment@_ {set-message}
    {trigger}

    {trigger:pagerejected|approval=Reviewer}
        {set-message}Under review{set-message}
    {trigger}

    {trigger:pageapproved|approval=Editor in Chief}
        {set-message:style=tip}Approved page!{set-message}
        {set-label:released}
        {remove-label:label_author}
    {trigger}
    
    {trigger:labelremoved|label=released|user=!primus}
        {set-message:user|style=warning}NOTALLOWED{set-message}
        {set-label:released}
    {trigger}

    {trigger:pageupdated|hasapproval=Editor in Chief}
        {set-message:style=warning}MMMNNNOOO{set-message}
    {trigger}
{workflow}

(view code)

package it.com.comalatech.confluence.workflow;

import org.apache.log4j.Logger;

/**
 * Created Jan 19, 2009 @ 9:15:20 AM
 * Copyright (C) 2008 Comala Technology Solutions, Inc.
 * confluence-dev@comalatech.com
 * All rights reserved.
 */
public class LoremIpsumWebTestCase extends AbstractAwpWebTestCase {

    protected static final String AUTHOR_APPROVAL = "Author";
    protected static final String REVIEWER_APPROVAL = "Reviewer";
    protected static final String EDITORINCHIEF_APPROVAL = "Editor in Chief";

    public void testLoremIpsum() throws Exception {
        loadWorkflow("loremipsum.workflow");
        changeUser(AUCTOR);
        long pageId = getPageHelper(createPage("lorem ipsum","")).getId();
        // let's get auctor to approve first:
        viewPage(pageId);
        assertState("Draft");
        assertApproval(AUTHOR_APPROVAL, true, true, true);
        assertApproval(REVIEWER_APPROVAL, false, false, false);
        assertApproval(EDITORINCHIEF_APPROVAL, false, false, false);
        assertMessage("AAA");
        assertLabel("label_created");
        approvePage(pageId,AUTHOR_APPROVAL, "comment");
        viewPage(pageId);
        assertApproval(AUTHOR_APPROVAL, true, false, true);
        assertNotLabel("label_created");
        assertLabel("label_author");
        assertNotLabel("reapproved");
        assertMessage("BBB");

        changeUser(EMENDATOR);
        viewPage(pageId);
        assertState("Draft");
        assertApproval(REVIEWER_APPROVAL, true, false, true);
        rejectPage(pageId,REVIEWER_APPROVAL,"rejected");


        // now let's update
        appendToPage(pageId,"123456");
        viewPage(pageId);
        assertState("Draft");
        assertApproval(REVIEWER_APPROVAL, false, true, true);
        assertNotMessage("MMMNNNOOO");

        // let's reapprove
        changeUser(AUCTOR);
        viewPage(pageId);
        assertState("Draft");
        assertApproval(AUTHOR_APPROVAL, true, true, true);
        approvePage(pageId,AUTHOR_APPROVAL, "comment");
        viewPage(pageId);
        assertLabel("reapproved");

        // emendator to approve
        changeUser(EMENDATOR);
        viewPage(pageId);
        assertState("Draft");
        assertApproval(REVIEWER_APPROVAL, true, true, true);
        approvePage(pageId,REVIEWER_APPROVAL, "XXXYYYZZZ");
        viewPage(pageId);
        // let's check the user name and the comment in the message
        assertMessage("XXXYYYZZZ");
        assertMessage(getCurrentUserFullName());

        changeUser(PRIMUS);
        viewPage(pageId);
        assertState("Draft");
        assertApproval(EDITORINCHIEF_APPROVAL, true, true, true);
        approvePage(pageId,EDITORINCHIEF_APPROVAL, "comment");
        viewPage(pageId);
        assertState("Published");
        assertLabel("released");

        appendToPage(pageId,"blah blah blah");
        viewPage(pageId);
        assertState("Draft");
        assertMessage("MMMNNNOOO");
        releaseViewPage(pageId);
        assertState("Published");
    }
}

Running the tests and adding to Bamboo

Running the tests is just a matter of:

mvn clean integration-test

I also added the tests to the Trunk Plan which compiles and tests on 2.8.2.

Running the tests take a few minutes because some of them depend on queued tasks on Confluence.

I also created two other plans, one for 2.9.2 and another for 2.10.1.

(Note that for 2.10 I had to define atlassian.product.data.version).

The tests on 2.9.2 and 2.10.1 are fired when the Trunk build is completed:

Conclusion

I found difficult getting the integration testing going. Besides having to dig into code because of lack of documentation, running the tests can take a while.

However, as everything invested in repeatable testing, it was worth the effort. The benefit is not only for checking changes but also for verifying new versions of confluence. i.e. running the tests on 3.0-m3 was just a matter of:

mvn  -Dconfluence.version=3.0-m3 -Dmaven.test.unit.skip=true clean integration-test;

References

• http://confluence.atlassian.com/display/DEVNET/Testing+your+plugin
• http://svn.atlassian.com/fisheye/browse/public/contrib/confluence/confluence-plugin-func-test
• http://www.customware.net/repository/display/GREENHOUSE/2009/03/18/Integration+Testing+of+Confluence+Plugins