Discussion:
Running a Test Conditionally -- Including/Excluding Tests with Profiles
Dan Kigelman
2007-11-11 22:46:47 UTC
Permalink
Hello!

I am trying to execute a test if and only if a certain condition is set
(such as a profile is activated). The test would fail if it is run on some
systems, so I am trying to disable it by default, and run it only when run
on the relevant system.

I made a sample project to simplify the problem. Below is the relevant part
my pom:

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>

<profiles>
<profile>
<id>condition-profile</id>
<activation>
<property>
<name>some-condition</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rare-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

When I execute "mvn test" it works as expected:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.085 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------


However, when I execute "mvn test -Dsome-condition" the
ExecuteSometimesTest.java is not executed:

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] [surefire:test {execution: rare-test}]
[INFO] Surefire report directory:
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports

-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------

Am I doing something wrong or is this a known problem?

Help would be much appreciated!!

-- Dan
Wayne Fay
2007-11-11 23:41:08 UTC
Permalink
Try renaming the profile to some_condition, and then using
-Dsome_condition=true. I've never tried to use a property named a-b-c
and I'm not entirely sure how different operating systems pass that to
Java and how it is interpreted prior to handing it to Maven (eg -D, -h
etc). So I would just avoid -'s in profile names, personally.

Wayne
Post by Dan Kigelman
Hello!
I am trying to execute a test if and only if a certain condition is set
(such as a profile is activated). The test would fail if it is run on some
systems, so I am trying to disable it by default, and run it only when run
on the relevant system.
I made a sample project to simplify the problem. Below is the relevant part
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>condition-profile</id>
<activation>
<property>
<name>some-condition</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rare-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.085 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
However, when I execute "mvn test -Dsome-condition" the
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rare-test}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Am I doing something wrong or is this a known problem?
Help would be much appreciated!!
-- Dan
Dan Kigelman
2007-11-12 00:18:19 UTC
Permalink
Thank you Wayne,

I will be careful about that. It seems it was working on my linux laptop,
but I'm sure you've averted some of my future headaches for executing this
on all kinds of shells. I renamed the profile and activation to the
following:

<profile>
<id>conditionProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>true</value>
</property>
</activation>
<build>

I execute now with "mvn test -Dsome_condition=true" but still with the same
result:

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] [surefire:test {execution: rareTest}]
[INFO] Surefire report directory:
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports

-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------

Somehow the second surefire execution includes no tests...

-- Dan
Post by Wayne Fay
Try renaming the profile to some_condition, and then using
-Dsome_condition=true. I've never tried to use a property named a-b-c
and I'm not entirely sure how different operating systems pass that to
Java and how it is interpreted prior to handing it to Maven (eg -D, -h
etc). So I would just avoid -'s in profile names, personally.
Wayne
Post by Dan Kigelman
Hello!
I am trying to execute a test if and only if a certain condition is set
(such as a profile is activated). The test would fail if it is run on
some
Post by Dan Kigelman
systems, so I am trying to disable it by default, and run it only when
run
Post by Dan Kigelman
on the relevant system.
I made a sample project to simplify the problem. Below is the relevant
part
Post by Dan Kigelman
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>condition-profile</id>
<activation>
<property>
<name>some-condition</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rare-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.085sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
However, when I execute "mvn test -Dsome-condition" the
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rare-test}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Am I doing something wrong or is this a known problem?
Help would be much appreciated!!
-- Dan
---------------------------------------------------------------------
Wayne Fay
2007-11-12 17:07:47 UTC
Permalink
Without looking at the code I can't be sure, but I would assume that
you can't include -and- exclude the same file. Probably the exclude
list is processed "last' in the code which means it won't run.

Wayne
Post by Dan Kigelman
Thank you Wayne,
I will be careful about that. It seems it was working on my linux laptop,
but I'm sure you've averted some of my future headaches for executing this
on all kinds of shells. I renamed the profile and activation to the
<profile>
<id>conditionProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>true</value>
</property>
</activation>
<build>
I execute now with "mvn test -Dsome_condition=true" but still with the same
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rareTest}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Somehow the second surefire execution includes no tests...
-- Dan
Post by Wayne Fay
Try renaming the profile to some_condition, and then using
-Dsome_condition=true. I've never tried to use a property named a-b-c
and I'm not entirely sure how different operating systems pass that to
Java and how it is interpreted prior to handing it to Maven (eg -D, -h
etc). So I would just avoid -'s in profile names, personally.
Wayne
Post by Dan Kigelman
Hello!
I am trying to execute a test if and only if a certain condition is set
(such as a profile is activated). The test would fail if it is run on
some
Post by Dan Kigelman
systems, so I am trying to disable it by default, and run it only when
run
Post by Dan Kigelman
on the relevant system.
I made a sample project to simplify the problem. Below is the relevant
part
Post by Dan Kigelman
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>condition-profile</id>
<activation>
<property>
<name>some-condition</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rare-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.085sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
However, when I execute "mvn test -Dsome-condition" the
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rare-test}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
Post by Wayne Fay
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Am I doing something wrong or is this a known problem?
Help would be much appreciated!!
-- Dan
---------------------------------------------------------------------
Dan Kigelman
2007-11-12 22:02:59 UTC
Permalink
Mm.. Is there any way to do what I'm trying though? Execute one or
more of the tests if and only if? Maybe by running on different
executions of surefire?

I tried to attach a .zip of my test code, but the message was rejected
as spam. Here is the sample program as an in-text attachment. It was
generated with archetype:create and modified to show the problem.

Thank you again for your help,

-- Dan
.
|-- pom.xml
`-- src
|-- main
| `-- java
| `-- sample
| `-- test
| `-- App.java
`-- test
`-- java
`-- sample
`-- test
|-- ExecuteAlwaysTest.java
`-- ExecuteSometimesTest.java

pom.xml:
-------------
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>sample.test</groupId>
<artifactId>execute-test-conditionally</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>execute-test-conditionally</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>

<profiles>
<profile>
<id>conditionProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rareTest</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

App.java: (ignore me)
------------------------------
package sample.test;

/**
* Hello world!
*
*/
public class App
{
public static void main( String[] args )
{
System.out.println( "Hello World!" );
}
}


ExecuteAlwaysTest.java:
------------------------------------

package sample.test;

import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;

/**
* Unit test for simple App.
*/
public class ExecuteAlwaysTest
extends TestCase
{
/**
* Create the test case
*
* @param testName name of the test case
*/
public ExecuteAlwaysTest( String testName )
{
super( testName );
}

/**
* Rigourous Test :-)
*/
public void testApp()
{
assertTrue( true );
}
}

ExecuteSometimesTest.java:
------------------------------------------

package sample.test;

import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;

/**
* Unit test for simple App.
*/
public class ExecuteSometimesTest
extends TestCase
{
/**
* Create the test case
*
* @param testName name of the test case
*/
public ExecuteSometimesTest( String testName )
{
super( testName );
}

/**
* Rigourous Test :-)
*/
public void testApp()
{
fail();
}
}
Post by Wayne Fay
Without looking at the code I can't be sure, but I would assume that
you can't include -and- exclude the same file. Probably the exclude
list is processed "last' in the code which means it won't run.
Wayne
Post by Dan Kigelman
Thank you Wayne,
I will be careful about that. It seems it was working on my linux laptop,
but I'm sure you've averted some of my future headaches for executing this
on all kinds of shells. I renamed the profile and activation to the
<profile>
<id>conditionProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>true</value>
</property>
</activation>
<build>
I execute now with "mvn test -Dsome_condition=true" but still with the same
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rareTest}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Somehow the second surefire execution includes no tests...
-- Dan
Post by Wayne Fay
Try renaming the profile to some_condition, and then using
-Dsome_condition=true. I've never tried to use a property named a-b-c
and I'm not entirely sure how different operating systems pass that to
Java and how it is interpreted prior to handing it to Maven (eg -D, -h
etc). So I would just avoid -'s in profile names, personally.
Wayne
Post by Dan Kigelman
Hello!
I am trying to execute a test if and only if a certain condition is set
(such as a profile is activated). The test would fail if it is run on
some
Post by Dan Kigelman
systems, so I am trying to disable it by default, and run it only when
run
Post by Dan Kigelman
on the relevant system.
I made a sample project to simplify the problem. Below is the relevant
part
Post by Dan Kigelman
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>condition-profile</id>
<activation>
<property>
<name>some-condition</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rare-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.085sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
However, when I execute "mvn test -Dsome-condition" the
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rare-test}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
Post by Wayne Fay
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Am I doing something wrong or is this a known problem?
Help would be much appreciated!!
-- Dan
---------------------------------------------------------------------
---------------------------------------------------------------------
Wayne Fay
2007-11-12 22:12:11 UTC
Permalink
Assuming what I said earlier about excludes overriding includes, the
most obvious way I can see you handling this would be with 2 profiles
-- one that is active by default and excludes the test, and another
that requires -P etc that includes the test. Then you wouldn't have
any surefire declaration in the main pom, but instead 2 profiles that
would each declare it independently with different configuration
options.

Wayne
Post by Dan Kigelman
Mm.. Is there any way to do what I'm trying though? Execute one or
more of the tests if and only if? Maybe by running on different
executions of surefire?
I tried to attach a .zip of my test code, but the message was rejected
as spam. Here is the sample program as an in-text attachment. It was
generated with archetype:create and modified to show the problem.
Thank you again for your help,
-- Dan
.
|-- pom.xml
`-- src
|-- main
| `-- java
| `-- sample
| `-- test
| `-- App.java
`-- test
`-- java
`-- sample
`-- test
|-- ExecuteAlwaysTest.java
`-- ExecuteSometimesTest.java
-------------
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>sample.test</groupId>
<artifactId>execute-test-conditionally</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>execute-test-conditionally</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>conditionProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rareTest</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
App.java: (ignore me)
------------------------------
package sample.test;
/**
* Hello world!
*
*/
public class App
{
public static void main( String[] args )
{
System.out.println( "Hello World!" );
}
}
------------------------------------
package sample.test;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
/**
* Unit test for simple App.
*/
public class ExecuteAlwaysTest
extends TestCase
{
/**
* Create the test case
*
*/
public ExecuteAlwaysTest( String testName )
{
super( testName );
}
/**
* Rigourous Test :-)
*/
public void testApp()
{
assertTrue( true );
}
}
------------------------------------------
package sample.test;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
/**
* Unit test for simple App.
*/
public class ExecuteSometimesTest
extends TestCase
{
/**
* Create the test case
*
*/
public ExecuteSometimesTest( String testName )
{
super( testName );
}
/**
* Rigourous Test :-)
*/
public void testApp()
{
fail();
}
}
Post by Wayne Fay
Without looking at the code I can't be sure, but I would assume that
you can't include -and- exclude the same file. Probably the exclude
list is processed "last' in the code which means it won't run.
Wayne
Post by Dan Kigelman
Thank you Wayne,
I will be careful about that. It seems it was working on my linux
laptop,
Post by Wayne Fay
Post by Dan Kigelman
but I'm sure you've averted some of my future headaches for executing
this
Post by Wayne Fay
Post by Dan Kigelman
on all kinds of shells. I renamed the profile and activation to the
<profile>
<id>conditionProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>true</value>
</property>
</activation>
<build>
I execute now with "mvn test -Dsome_condition=true" but still with the
same
Post by Wayne Fay
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rareTest}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
Post by Wayne Fay
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Somehow the second surefire execution includes no tests...
-- Dan
Post by Wayne Fay
Try renaming the profile to some_condition, and then using
-Dsome_condition=true. I've never tried to use a property named a-b-c
and I'm not entirely sure how different operating systems pass that to
Java and how it is interpreted prior to handing it to Maven (eg -D, -h
etc). So I would just avoid -'s in profile names, personally.
Wayne
Post by Dan Kigelman
Hello!
I am trying to execute a test if and only if a certain condition is
set
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
(such as a profile is activated). The test would fail if it is run
on
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
some
Post by Dan Kigelman
systems, so I am trying to disable it by default, and run it only
when
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
run
Post by Dan Kigelman
on the relevant system.
I made a sample project to simplify the problem. Below is the
relevant
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
part
Post by Dan Kigelman
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>condition-profile</id>
<activation>
<property>
<name>some-condition</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rare-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
0.085sec
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
However, when I execute "mvn test -Dsome-condition" the
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04
sec
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rare-test}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
Am I doing something wrong or is this a known problem?
Help would be much appreciated!!
-- Dan
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
Dan Kigelman
2007-11-13 03:58:05 UTC
Permalink
Thank you!! It works great for just two profiles, and it works ok for
n profiles. Maintenance would be a bit difficult with numerous such
profiles (excludes would need to be updated in several places when
adding a new test), and it wouldn't be possible to activate more than
one profile at a time.

Executing "mvn test -P2ndProfile,3rdProfile" would have the 3rd
profile's excludes override the 2nd.

I don't quite understand how the execution in
<profile><executions>...</executions></profile> works, but if it were
to work with surefire, one could exclude a pattern of tests in the
main build, and include only certain patterns for separate unique
executions of surefire.

However, this works for me for now... Thanks again!

-- Dan

Here is my pom:

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>sample.test</groupId>
<artifactId>execute-test-conditionally</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>execute-test-conditionally</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>

<profiles>
<profile>
<id>defaultProfile</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/SecondEnvironmentTest.java</exclude>
<exclude>**/ThirdEnvironmentTest.java</exclude>
<exclude>**/FourthEnvironmentTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>

<profile>
<id>2ndProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>2</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ThirdEnvironmentTest.java</exclude>
<exclude>**/FourthEnvironmentTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>

<profile>
<id>3rdProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>3</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/SecondEnvironmentTest.java</exclude>
<exclude>**/FourthEnvironmentTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>

<profile>
<id>4thProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>4</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/SecondEnvironmentTest.java</exclude>
<exclude>**/ThirdEnvironmentTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>

</profiles>
</project>
Post by Wayne Fay
Assuming what I said earlier about excludes overriding includes, the
most obvious way I can see you handling this would be with 2 profiles
-- one that is active by default and excludes the test, and another
that requires -P etc that includes the test. Then you wouldn't have
any surefire declaration in the main pom, but instead 2 profiles that
would each declare it independently with different configuration
options.
Wayne
Post by Dan Kigelman
Mm.. Is there any way to do what I'm trying though? Execute one or
more of the tests if and only if? Maybe by running on different
executions of surefire?
I tried to attach a .zip of my test code, but the message was rejected
as spam. Here is the sample program as an in-text attachment. It was
generated with archetype:create and modified to show the problem.
Thank you again for your help,
-- Dan
.
|-- pom.xml
`-- src
|-- main
| `-- java
| `-- sample
| `-- test
| `-- App.java
`-- test
`-- java
`-- sample
`-- test
|-- ExecuteAlwaysTest.java
`-- ExecuteSometimesTest.java
-------------
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>sample.test</groupId>
<artifactId>execute-test-conditionally</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>execute-test-conditionally</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>conditionProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rareTest</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
App.java: (ignore me)
------------------------------
package sample.test;
/**
* Hello world!
*
*/
public class App
{
public static void main( String[] args )
{
System.out.println( "Hello World!" );
}
}
------------------------------------
package sample.test;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
/**
* Unit test for simple App.
*/
public class ExecuteAlwaysTest
extends TestCase
{
/**
* Create the test case
*
*/
public ExecuteAlwaysTest( String testName )
{
super( testName );
}
/**
* Rigourous Test :-)
*/
public void testApp()
{
assertTrue( true );
}
}
------------------------------------------
package sample.test;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
/**
* Unit test for simple App.
*/
public class ExecuteSometimesTest
extends TestCase
{
/**
* Create the test case
*
*/
public ExecuteSometimesTest( String testName )
{
super( testName );
}
/**
* Rigourous Test :-)
*/
public void testApp()
{
fail();
}
}
Post by Wayne Fay
Without looking at the code I can't be sure, but I would assume that
you can't include -and- exclude the same file. Probably the exclude
list is processed "last' in the code which means it won't run.
Wayne
Post by Dan Kigelman
Thank you Wayne,
I will be careful about that. It seems it was working on my linux
laptop,
Post by Wayne Fay
Post by Dan Kigelman
but I'm sure you've averted some of my future headaches for executing
this
Post by Wayne Fay
Post by Dan Kigelman
on all kinds of shells. I renamed the profile and activation to the
<profile>
<id>conditionProfile</id>
<activation>
<property>
<name>some_condition</name>
<value>true</value>
</property>
</activation>
<build>
I execute now with "mvn test -Dsome_condition=true" but still with the
same
Post by Wayne Fay
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rareTest}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
Post by Wayne Fay
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Somehow the second surefire execution includes no tests...
-- Dan
Post by Wayne Fay
Try renaming the profile to some_condition, and then using
-Dsome_condition=true. I've never tried to use a property named a-b-c
and I'm not entirely sure how different operating systems pass that to
Java and how it is interpreted prior to handing it to Maven (eg -D, -h
etc). So I would just avoid -'s in profile names, personally.
Wayne
Post by Dan Kigelman
Hello!
I am trying to execute a test if and only if a certain condition is
set
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
(such as a profile is activated). The test would fail if it is run
on
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
some
Post by Dan Kigelman
systems, so I am trying to disable it by default, and run it only
when
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
run
Post by Dan Kigelman
on the relevant system.
I made a sample project to simplify the problem. Below is the
relevant
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
part
Post by Dan Kigelman
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/ExecuteSometimesTest.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>condition-profile</id>
<activation>
<property>
<name>some-condition</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<id>rare-test</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
<configuration>
<includes>
<include>**/ExecuteSometimesTest.java</include>
</includes>
<excludes />
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
0.085sec
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
However, when I execute "mvn test -Dsome-condition" the
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running sample.test.ExecuteAlwaysTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04
sec
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] [surefire:test {execution: rare-test}]
/home/danik/workspace/test2/execute-test-conditionally/target/surefire-reports
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
-------------------------------------------------------
T E S T S
-------------------------------------------------------
There are no tests to run.
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
Post by Wayne Fay
Post by Dan Kigelman
Post by Wayne Fay
Post by Dan Kigelman
Am I doing something wrong or is this a known problem?
Help would be much appreciated!!
-- Dan
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
Loading...