java - TestNG - cannot run same test twice in a suite

Question

I have a TestNG test suite that is working fine except for one thing: I cannot run the same test twice. The reason I want to run a test twice is that I am testing web services that log a user in/out, and I want to verify that the login works, then that the logout work, and then log them in again so that the subsequent tests, which require the user to be logged in, can proceed.

Here is the relevant section of my testng.xml:

<suite name="WebServiceTestSuite">
    <test name="webServiceTest" verbose="10" parallel="none">
        <classes>
            <class name="com.abcco.webservice.DivisionTest">
                <methods>
                    <include name="registrationCreateTest"></include>
                    <include name="registrationActivateTest"></include>
                    <include name="appUserLoginTest"></include>
                    <include name="appUserLogoutTest"></include>
                    <include name="appUserLoginTest"></include>
                    <include name="sessionValidateTest"></include>
                    <include name="sessionExtendTest"></include>
                </methods>
            </class>
        </classes>
    </test>
</suite>

As you can see, the "appUserLoginTest" is called twice. However, when I debug this I can clearly see that it is only actually run the first time it is called. After that the execution proceeds as if the second line calling it didn't exist.

Answer 1

This is actually by design in TestNG. I see two potential options:

First, look at using a DataProvider with your tests so they can be executed repeatedly with different parameters.

Or, alter the XML layout so that each method is part of its own tag. In this configuration, the XML will be longer, but you'll be able to invoke a test method as many times as you like as part of a suite.

EDIT: More details now.

In response to your comment, here's a sample XML layout that worked for me when I was up against this same problem. Try this:

<suite name="WebServiceTestSuite">
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="registrationCreateTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="registrationActivateTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="appUserLoginTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="appUserLogoutTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="appUserLoginTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="sessionValidateTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="sessionExtendTest"></include>
            </methods>
        </class>
    </classes>
</test>
</suite>

I removed the verbose="10" parallel="none" from the <test> tags, as the layout that worked for me didn't have this... You may be able to add the verbose="10" to the <suite> tag to get any specific functionality there that you need. Have a go with this and post your results and I'll be happy to help further. Without seeing your runner configuration or knowing too much about the rest of your setup, there may be some other things to consider.

You'll find that this is a much more verbose XML layout, but it solved this same problem for me at the time. You could also consider re-arranging your code a bit and making an @Test method that calls all the methods on this XML as a single test, allowing you to reduce the length of your XML file.