Scala Future.sequence(Iterator) throws NoSuchElementException

Question

Why is the following code throwing java.util.NoSuchElementException: next on empty iterator???

     implicit val ec = ExecutionContext.global
     val it = (0 until 10).toIterator.map{
       x =>
         Thread.sleep(500)
         println(x)
         x
     }
     val it2 = new Iterator[Future[Int]] {
       def hasNext = it.hasNext
       def next() = Future { blocking { it.next() } }
     } //This should be equivalent to it.map{x => Future{blocking{x}}}
     Await.result(Future.sequence(it2), Inf)

Subsidiary question: why is it2 not behaving as it.map{x => Future{blocking{x}}} ??

question from:https://stackoverflow.com/questions/66052171/scala-future-sequenceiterator-throws-nosuchelementexception

Answer 1

You have a concurrency error, someone calls hasNext it returns true because the underlying iterator is capable of producing new elements. Then it calls next this returns a future that when completed will hold the next value.
However, for the caller (in this case Future.sequence) it already finished so it calls hasNext again which will return true again but because the underlying iterator hasn't finished producing the first element so we produce a second that is enqueued until the first one finish (so you win nothing with the future).

And this continues creating more and more and more futures that will eventually call next and get a NoSuchElementException because the underlying iterator didn't have more elements but you already create more and more futures waiting for more and more elements.
So, in other words, you are getting a true in hasNext just because it hasn't finished producing elements and as such haven't mutated its internal state; not because it actually has more elements.

This is a good recordatory that mutability + concurrency = headaches.
Again, I recommend you to use an appropriate abstraction for thi