Out of our depth

fs2 gives us the tools to work with infinity: the fs2.Stream datatype can describe an infinite list. But here’s the problem: infinity is impossible to fathom.

In functional programming, we’re used to primitive data types like String and algebraic data types like Option. We can even describe recursive data types, such as the Scala linked List, but these are still always finite: we can imagine how a list is laid out in memory, regardless of how big it is.

What’s happening in memory in notSoManyKittens? Surely we don’t have ten million kittens in our computer at once? And what about manyKittens? What’s stopping our JVM from exploding?

Thinking of streams as infinite lists doesn’t help us answer these questions. It doesn’t give us any insight into how streams work.

The challenge of composition

This leads to more problems. If we don’t know how an infinite stream works, we can’t get a sense of how to compose one.

If you’ve done enough functional programming, you might have tried your hand at writing your own linked list datatype and your own take function: you’d be able to describe how take worked, and how it composed with other operators on lists like drop and map.

But we can’t easily describe how stream operators compose. What exactly is happening when we take and repeat?

Try and figure out the difference between the following two streams:

Stream("Mao", "Popcorn").take(3).repeat
Stream("Mao", "Popcorn").repeat.take(3)

There’s a big difference between them, but without an understanding of composition, we can’t explain why that is.

This gets even more confusing when we start working with side-effects. Here’s a sneak peak at some effectful code:

Stream("Mao", "Popcorn")
 .evalMap(eat)
 .evalMap(nap)
 .repeat
 .compile
 .last

eat and nap are both suspended side-effects, regardless of how they’re implemented, so the order in which they happen is important. In what order do Mao and Popcorn eat and nap?

To answer that question, we need a better intuition of infinite streams.