multithreading - Sharing String between threads in Rust

Question

I'm trying to request multiple URLs with multiple std::thread. This is how my code looks so far:

fn fetch(urls: Vec<&str>) {
    let (tx, rx) = mpsc::channel();

    for url in urls {
        let tx = tx.clone();

        thread::spawn(|| {
            let ssl = NativeTlsClient::new().unwrap();
            let connector = HttpsConnector::new(ssl);
            let client = Client::with_connector(connector);
            let mut res = client.get(url).send().unwrap();
            let mut result = String::new();
            res.read_to_string(&mut result);

            tx.send(result).unwrap();  
        });
    }

    //let mut result: Vec<String> = vec![];
    for _ in urls {
        println!("{}", rx.recv().unwrap());
    }
}

But I got an error that said:

error[E0277]: the trait bound `std::sync::mpsc::Sender<std::string::String>: std::marker::Sync` is not satisfied
    --> src/lib.rs:18:9
    |
 18 |         thread::spawn(|| {
    |         ^^^^^^^^^^^^^ the trait `std::marker::Sync` is not implemented for `std::sync::mpsc::Sender<std::string::String>`
    |
    = note: `std::sync::mpsc::Sender<std::string::String>` cannot be shared between threads safely
    = note: required because of the requirements on the impl of `std::marker::Send` for `&std::sync::mpsc::Sender<std::string::String>`
    = note: required because it appears within the type `[closure@src/lib.rs:18:23: 29:10 url:&&str, tx:&std::sync::mpsc::Sender<std::string::String>]`
    = note: required by `std::thread::spawn`

When I tried to put the move in the thread::spawn:

thread::spawn(move || {
    ...

I got another error related to lifetime:

error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
  --> src/lib.rs:15:16
   |
15 |     for url in urls {
   |                ^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the block at 12:26...
  --> src/lib.rs:12:27
   |
12 | fn fetch(urls: Vec<&str>) {
   |                           ^
note: ...so that expression is assignable (expected std::vec::Vec<&str>, found std::vec::Vec<&str>)
  --> src/lib.rs:15:16
   |
15 |     for url in urls {
   |                ^^^^
   = note: but, the lifetime must be valid for the static lifetime...
note: ...so that the type `[closure@src/lib.rs:18:23: 27:10 url:&str, tx:std::sync::mpsc::Sender<std::string::String>]` will meet its required lifetime bounds
  --> src/lib.rs:18:9
   |
18 |         thread::spawn(move || {
   |         ^^^^^^^^^^^^^

So, what is the proper way to send strings from threads through channel here? And how can I solve the lifetime problem in the later error?

Thank you so much!

Answer 1

Adding the move is the correct solution to your first problem. The second error indicates a problem in your code that was there before already, but is detected only in a later compiler stage. So what does this second error mean?

Well, a spawned thread can run forever (more precisely: as long as the main thread/the whole program runs). In your case they don't, because you block the calling thread waiting for the results from the channel. But the compiler doesn't know that. Therefore, thread::spawn() requires the passed closure to be : 'static which means that it doesn't reference anything that lives shorter than the whole program.

But in your case the closure has a reference to the url, a &str. But how long does the string behind that reference actually live? Not necessarily forever! That's the problem here.

The typical solution to problems like these is to use an Arc and wrap the owned value in it. But this is not possible here, because your function does not have access to the owned value. There are a few possible solutions for you:

• Use a scoped thread API, like crossbeam offers. This API makes sure that the spawned thread doesn't outlive its parent, so you can just reference the &str inside of your closure. I think this is actually the best solution with the only downside of pulling in a new dependency.

• Change your function signature to fn fetch(urls: Vec<&'static str>). It works, but it limits the callers of your function as they have to provide static strings. I guess that the list of URLs is not just a list of string literals, but dynamically generated; so this is not really a solution for you.

• Clone the &str to move the resulting String into the closure. This is not really a nice solution, though, as useless clones should be avoided. But it could be tolerable in your situation as the HTTP request will takes a whole lot longer than cloning a rather small (url) string.