testing - Run additional tests by using a feature flag to "cargo test"

Question

I have some tests that I would like to ignore when using cargo test and only run when explicitly passed a feature flag. I know this can be done by using #[ignore] and cargo test -- --ignored, but I'd like to have multiple sets of ignored tests for other reasons.

I have tried this:

#[test]
#[cfg_attr(not(feature = "online_tests"), ignore)]
fn get_github_sample() {}

This is ignored when I run cargo test as desired, but I can't get it to run.

I have tried multiple ways of running Cargo but the tests continue to be ignored:

cargo test --features "online_tests"

cargo test --all-features

I then added the feature definition into my Cargo.toml as per this page, but they continue to be ignored.

I am using workspaces in Cargo. I tried adding the feature definition in both Cargo.toml files with no difference.

Answer 1

Without a workspace

Cargo.toml

[package]
name = "feature-tests"
version = "0.1.0"
authors = ["An Devloper <[email protected]>"]

[features]
network = []
filesystem = []

[dependencies]

src/lib.rs

#[test]
#[cfg_attr(not(feature = "network"), ignore)]
fn network() {
    panic!("Touched the network");
}

#[test]
#[cfg_attr(not(feature = "filesystem"), ignore)]
fn filesystem() {
    panic!("Touched the filesystem");
}

Output

$ cargo test

running 2 tests
test filesystem ... ignored
test network ... ignored

$ cargo test --features network

running 2 tests
test filesystem ... ignored
test network ... FAILED

$ cargo test --features filesystem

running 2 tests
test network ... ignored
test filesystem ... FAILED

(some output removed to better show effects)

With a workspace

Layout

.
├── Cargo.toml
├── feature-tests
│?? ├── Cargo.toml
│?? ├── src
│?? │?? └── lib.rs
├── src
│?? └── lib.rs

feature-tests contains the files from the first section above.

Cargo.toml

[package]
name = "workspace"
version = "0.1.0"
authors = ["An Devloper <[email protected]>"]

[features]
filesystem = ["feature-tests/filesystem"]
network = ["feature-tests/network"]

[workspace]

[dependencies]
feature-tests = { path = "feature-tests" }

Output

$ cargo test --all

running 2 tests
test filesystem ... ignored
test network ... ignored

$ cargo test --all --features=network

running 2 tests
test filesystem ... ignored
test network ... FAILED

(some output removed to better show effects)

With a workspace with a virtual manifest

Virtual manifests do not support specifying features (Cargo issue #4942). You will need to run the tests from within the sub project or specify the path to the appropriate Cargo.toml

Layout

.
├── Cargo.toml
└── feature-tests
    ├── Cargo.toml
    └── src
        └── lib.rs

feature-tests contains the files from the first section above.

Cargo.toml

[workspace]
members = ["feature-tests"]

Output

$ cargo test --all --manifest-path feature-tests/Cargo.toml --features=network 

running 2 tests
test filesystem ... ignored
test network ... FAILED

$ cargo test --all --manifest-path feature-tests/Cargo.toml

running 2 tests
test filesystem ... ignored
test network ... ignored

(some output removed to better show effects)