Chapter 4.3: A Lack of Values

Where We Left Off

At the end of the previous chapter, our program crashed because we tried to use a reference to access an experience object’s rating — another.rating — but what the variable another actually stored was null: a reference that leads nowhere. This occurred as we were adding a first experience to a category; the trouble started because the category didn’t yet have a favorite experience, and we had used a null reference to record that fact.

On its own, the computer can’t figure out how it should handle the scenario where no favorite experiences exists (yet). As programmers, we must somehow specify this behavior, too, in the program code. Our earlier solution attempt completely neglected the issue.

We could approach the problem in two different ways:

We could use a different data type that helps us represent the fact that a category has “one or zero favorites” rather than always having one.
Or we could continue to use the null reference to mark a nonexistent value; null is a valid value for a variable of (almost) any data type. We must then use other commands to tiptoe around the null value and make sure we never try to access the attributes of a nonexistent object.

There are reasons why the first option is generally better. (We’ll get to the reasons later in this chapter.) We’ll take that path — choose a new data type — as we fix our Category class. First, though, let’s get to know this new data type in general terms.

`lift`ing for Inspiration

Programs commonly need to manipulate values that may or may not exist. It’s easy to find an example in Scala’s standard library: a vector may have an element at index 100 (if the vector’s large enough) or it may not (if it’s small).

Sure enough, we know we can check what a vector contains at a given index as shown below, but it’s a shame that it crashes our program if the index is too high or negative:

val words = Vector("first", "second", "third", "fourth")words: Vector[String] = Vector(first, second, third, fourth)
words(2)res0: String = third
words(100)java.lang.IndexOutOfBoundsException: 100
  [...]

Couldn’t we have a method that tells us both 1) whether or not there is an element at the given index, and if there is one, 2) what it is?

Yes we could, and we do. The method is called lift.

words.lift(2)res1: Option[String] = Some(third)
words.lift(100)res2: Option[String] = None
words.lift(-1)res3: Option[String] = None

The return values are pretty easy to interpret:

When the element exists, the return value is “some string, "third" to be more precise”.

When no such element exists, the return value is “no value at all”. Note that this isn’t the same thing as null; we’ll get to that.

Even if the given index is invalid, lift doesn’t crash. The method lets us pick out an element safely.

What we get as a return value is not quite a String, though, but something else.

lift relies on a data type named Option. Let’s find out what it is and how it helps us improve Category.

The `Option` Class

Option is a class defined in package scala, which means it’s always available for you to use in any Scala program. You can think of an object of type Option as a “wrapper” that either contains a single value of a particular type or nothing.

`Some` and `None`

Let’s take another look at the above example:

val words = Vector("first", "second", "third", "fourth")words: Vector[String] = Vector(first, second, third, fourth)
words.lift(2)res4: Option[String] = Some(third)
words.lift(100)res5: Option[String] = None

A Some object is a “full wrapper”: an Option object that contains a single value. In our example, the wrapped value happens to be a string.

None is a singleton object defined as part of Scala. It represents an “empty wrapper”. It, too, counts as an object of type Option.

We can see that lift returns an Option that contains a String or nothing. That is, the method always returns either a Some object wrapped around a string or the None object.

Let’s introduce another vector, which contains integers:

val numbers = Vector(10, 0, 100, 10, 5, 123)numbers: Vector[Int] = Vector(10, 0, 100, 10, 5, 123)
numbers.lift(0)res6: Option[Int] = Some(10)
numbers.lift(-1)res7: Option[Int] = None

Notice that the type parameter is different. You can read Option[Int] as: “a wrapper containing an Int or nothing”. The type parameter in the square brackets serves exactly the same purpose as it does with vectors and buffers: it indicates the type of the value that may be contained within the “outer” object.

The concept of Option encompasses both Some and None.

To summarize, class Option is defined so that it guarantees us a number of things:

Every Some object is an Option object. Cf. “Every llama is an animal.” None is also an Option object.
There are exactly two kinds of Option object. Every Option object is either a Some object or the None object. Cf. “Every human is either a mortal or Chuck Norris.”
Every single value of type Option[X] is either None or a Some object that contains exactly one value of type X.

Variables of type `Option`

In the examples above, we received Option values from a method. We can also create Option values of our own and use Option as the type for variables in our programs.

var test: Option[Int] = Nonetest: Option[Int] = None

Here we have a variable that initially gets the value None. More specifically, the variable stores a reference to the singleton object None.

None has been defined to be type compatible with class Option, so we can assign the reference to a variable of type Option. Note that we could not assign None to, say, variables of type Experience or Int.

What we have now is a variable that we could use to store an integer. Currently, though, there’s no number stored there, as the variable has the value None instead. We can change that by assigning a “full wrapper”, a Some object, to the variable:

test = Some(5)test: Option[Int] = Some(5)

The expression Some(...) “wraps” a value. Since we’re working with Option[Int], the wrapped value must be an integer. Some(5) constructs a wrapper that contains the integer value 5.

(We could have written new Some(5), but it’s possible to omit new in this context and people usually do.)

We can also express a computation and wrap the result in an Option:

test = Some(10 + 50)test: Option[Int] = Some(60)

An Option object is immutable. Once you create an object with Some(5), there’s no way to change which object is stored inside the wrapper or to empty the wrapper. However, if we have a var that stores a reference to an Option, we may discard that reference and store another Option in the variable instead. Indeed, we just did that.

Pictures or it didn’t happen

Wrapped values vs. unwrapped values

A value of type Int is an integer. A value of Option[Int] might contain an integer. These types are distinct from each other and you can’t cross-assign them. For example, you can’t simply assign an Int to a variable of type Option[Int]:

test = 5<console>:11: error: type mismatch;
found   : Int(5)
required: Option[Int]
      test = 5
              ^

Conversely, you can’t assign a value of type Option[Int] to a variable of type Int or otherwise use that value in computations that call for an integer:

var numericalValue = 10numericalValue: Int = 10
var possibleValue: Option[Int] = Some(10)possibleValue: Int = Some(10)
numericalValue = possibleValue<console>:13: error: type mismatch;
found   : Option[Int]
required: Int
      numericalValue = possibleValue
                       ^
possibleValue - 1<console>:13: error: value - is not a member of Option[Int]
      possibleValue - 1
                    ^

Similarly, if a method takes an Int parameter, it won’t accept an Option[Int] and vice versa.

This is a very good thing. What it means is that if a part of your program requires an integer, it’s not enough that you have “an integer that might exist”. You receive, even before you run your program, an error message that reminds you to ensure that where an integer is needed, an integer must be delivered.

So, what’s the difference between `null` vs. `None`?

null is a value that means “a reference to nowhere, no object at all”. It’s technically possible to use null quite freely in a variety of places. It can be assigned to variables of type Experience or String, for instance.

By using null, one effectively adopts a style of programming where any value is, in a sense, “optional” and may be missing.

None is a singleton object that has the Option type. It is specifically built to be used in places that demand a value of type Option. A reference to None can be stored in a variable whose type is Option[something] but cannot be stored in an arbitrary sort of variable.

By using None and Option, we adopt a style of programming where the programmer explictly marks the parts of the program where a value is “optional” and may be missing. Further down on this page, we’ll discuss why this is often the better approach and you would do well to avoid null in your programs.

If you have trouble discerning the difference between the null reference and the None object, you may find it helpful to contrast the animations in this chapter with the one at the end of the previous one.

And what about `Unit`?

Unit, first introduced in Chapter 1.6, is a special value that we use to mean “no value of interest” and especially “the function produces no value of interest under any circumstances”. The value Unit is of a data type of its own, also called Unit; the value can’t be assigned to an arbitrary variable (as null can) or to a variable of type Option (as None can). In O1, we won’t use Unit for any purpose other than as the return value of some effectful functions.

Opening an `Option` with `match`

In order to use the contents of an Option, we need a way to “open up the wrapper” and see what, if anything, we find there.

To do that, we’ll use the match command. match is a relative of if in the sense that both commands make choices between alternatives.

There are other ways than match for accessing the contents of an Option but, for now, we’ll use match for this specific purpose. Moreover, there are other things that we can do with match besides handle Options but, for now, we’ll use match specifically for that purpose.

The `match` command

First, some familiar-looking code to set things up:

val numbers = Vector(10, 0, 100, 10, 5, 123)numbers: Vector[Int] = Vector(10, 0, 100, 10, 5, 123)
val possibleElement = numbers.lift(5)possibleElement: Option[Int] = Some(123)
val thousandthIfAny = numbers.lift(999)thousandthIfAny: Option[Int] = None

Let’s now make the computer select what it should do by considering two cases and seeing which one possibleElement’s value matches. The cases are: 1) a full Option wrapper, or 2) an empty one.

possibleElement match {
  case Some(wrappedNumber) => "there is some number in the wrapper"
  case None => "there is no number in the wrapper"
}res8: String = there is some number in the wrapper

We want to evaluate the expression possibleElement and examine its value. We follow the expression with the keyword match and an obligatory set of curly brackets.

Here we have two distinct cases to choose from. We specify them by writing the keyword case and an “arrow” that consists of an equals sign and a greater-than sign.

In between, we define each case. When we use match to process an Option, like here, we’ll often use one case to cover the possibility that the Option is a Some and another case to cover None.

Since it so happens that our example expression evaluates to a Some, the first case matches.

The expression that follows the arrow produces the result of the entire match command. In our example, we produce a string.

wrappedNumber is a programmer-chosen name for a variable that we didn’t yet use for any purpose.

Of course, it could be that it’s the other case that matches the expression:

thousandthIfAny match {
  case Some(wrappedNumber) => "there is some number in the wrapper"
  case None => "there is no number in the wrapper"
}res9: String = there is no number in the wrapper

The value of thousandthIfAny matches the latter case, so the computer selects that branch and uses the code therein as the match expression’s value.

In the preceding examples, we cared only about whether or not the Option object had content, not about what that content might be. The following example attends to that, too.

possibleElement match {
  case Some(wrappedNumber) => "the wrapper contains: " + wrappedNumber
  case None => "there is no number in the wrapper"
}res10: String = the wrapper contains: 123

In the Some case, we include a variable name as shown. When the expression matches that case, the Option object’s content is copied into a new local variable of that name before the expression on the right is evaluated. Therefore...

... we can use that variable to refer to the value that was “found” inside the Option. In this example, that value is an Int.

Here’s an additional example that illustrates some of match’s other features:

val numbers = Vector(10, 0, 100, 10, 5, 123)numbers: Vector[Int] = Vector(10, 0, 100, 10, 5, 123)
val result = numbers.lift(4) match {
  case None => 0
  case Some(number) => number * 1000
}result: Int = 5000

A match expression is an expression like any other. We can use it as part of other commands. We can, for instance, assign its value to a variable, as here.

A match expression begins with a (sub)expression whose value is matched on. In our earlier examples, that expression was a variable’s name, but other expressions work just as well. Here, we call lift and immediately match on the value that it returns (skipping the intermediate step of assigning the return value to a variable).

You aren’t obliged to order the cases in a specific way; it’s perfectly fine to write the None case before the Some case. The computer goes through the cases in order until it finds a match.

In our earlier examples, the match expression evaluated to a string, but other data types are fine, too. Here we have two cases that produce integers, so the result type is Int.

It’s customary to indent the cases. Like indentation in general, this is not strictly required and doesn’t actually impact on the behavior of a Scala program.

You have the option of watching the following animation.

The pseudocode below summarizes, in general terms, how we’ve used match to work on Options:

expression to examine match {
  case Some(variableForContent) => an expression to evaluate if the “wrapper is full” and may use the variable
  case None => an expression to evaluate if the “wrapper is empty”
}

You can also call effectful functions such as println within a case:

numbers.lift(7) match {
  case None => 
    println("There was no number at index seven.")
    println("No can do.")
  case Some(number) => 
    println("At index seven, I found " + number + ".")
}There was no number at index seven.
No can do.

If you include effectful commands, it’s customary to use line breaks and indent deeper, as shown.

You may include multiple sequential commands in a case.

Tracking the Favorite Experience (A New Implementation)

An instance variable of type `Option`

Now to our Category class. We can use Option to define its instance variable fave. Like so:

class Category(val name: String, val unit: String) {

  private val experiences = Buffer[Experience]()
  private var fave: Option[Experience] = None

  def favorite = this.fave

  def addExperience(newExperience: Experience) = {
    // We'll need to re-implement this.
  }

}

We enter Option[Experience] as the type of fave. This means the variable may hold a single experience or none. This definition exactly describes what we’re trying to achieve with the variable.

The value of our variable can be None or Some(...), where the “wrapper” contains a reference to an Experience object. Initially, there is no favorite and the value is None.

Re-implementing `addExperience`

Let’s rewrite the pseudocode that outlines addExperience:

  def addExperience(newExperience: Experience) = {
    this.experiences += newExperience
    How the favorite is updated depends on what we find inside the “fave wrapper”:
    1) In case we find nothing, we record that the favorite is a Some* that contains the newly added (first) experience.*
    2) In case we find an old favorite, we choose the better of the new experience and the old favorite and wrap it.
  }

We have two cases: either this.fave previously holds just an empty wrapper or some old favorite wrapped in an Option.

We can’t simply compare the ratings of Option[Experience] objects. We can compare the ratings of Experience objects. To do that, we need to extract the old favorite from its Option wrapper.

Here is the same algorithm implemented as Scala:

def addExperience(newExperience: Experience) = {
  this.experiences += newExperience
  this.fave match {
    case None =>
      this.fave = Some(newExperience)
    case Some(oldFave) =>
      val newFave = newExperience.chooseBetter(oldFave)
      this.fave = Some(newFave)
  }
}

Once an experience has been added, the category will definitely be associated with a favorite experience. We wrap a reference to that experience in a Some before storing it in the instance variable (since our new fave variable demands an Option[Experience] rather than Experience).

You can define local variables within a match command. Here, we’ve used a temporary variable newFave to emphasize the algorithm’s two steps. We could have also done everything on one line: this.fave = Some(newExperience.chooseBetter(oldFave))

A variation of the above

This alternative implementation works just as well.

def addExperience(newExperience: Experience) = {
  this.experiences += newExperience
  val newFave = this.fave match {
    case Some(oldFave) => newExperience.chooseBetter(oldFave)
    case None          => newExperience
  }
  this.fave = Some(newFave)
}

A change in the `Category` interface

The version of Category that we just produced (and that comes with the GoodStuff project) needs to be used a bit differently than we originally envisioned in Chapter 4.2.

This is because the return type of favorite is no longer Experience but Option[Experience], and the method returns either None or a Some that holds an experience. When we use the class, we must attend to the return type, as in this example:

val wineCategory = new Category("Wine", "bottle")
// ... (We may or may not add experience objects to the category here.)

wineCategory.favorite match {
  case Some(bestWine) =>
    println("The favorite is: " + bestWine.name)
  case None =>
    println("No favorite yet.")
}

An expression such as wineCategory.favorite.name isn’t valid, as indeed it shouldn’t be. If you try to use that expression with this Category class, you immediately receive a complaint from the Scala toolkit: the Option returned by wineCategory.favorite doesn’t have a name variable (even though any experience that it may contain has one).

An Alternative Implementation without `Option`

We didn’t strictly need Option in order to make addExperience work. At the beginning of this chapter, we suggested that there is another way to solve our original problem with null. Here’s how it works: we keep using a null value in fave but are super-careful that we never access the attributes of a non-existent object. Consider this implementation:

class Category(val name: String, val unit: String) {

  private val experiences = Buffer[Experience]()
  private var fave: Experience = null

  def favorite = this.fave

  def addExperience(newExperience: Experience) = {
    this.experiences += newExperience
    this.fave =
      if (this.fave == null)
        newExperience
      else
        newExperience.chooseBetter(this.fave)
  }

}

Since we first compare to see if fave is null and place our chooseBetter call in the else branch, our method never attempts to access the rating of a nonexistent object. The method works as originally intended.

Come on, you could just have told us that to begin with!?

Doesn’t that null-based version work just as well as the Option-based one? Didn’t Option just add complexity to our class for not much gain? Was it even worth learning about?

The Option-based implementation has some highly desirable features that the null-based one lacks. We’ll now review some of them.

The Billion-Dollar Mistake

I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object-oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe. — — But I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years.

—C. A. R. Hoare

The quotation above is from a 2009 speech where Sir Charles Antony Richard Hoare apologizes for inventing the null reference, originally for the ALGOL programming language. ALGOL contributed a great number of (mostly good) ideas that have been adopted into other programming languages, including present-day ones. null also lives on in many programming languages, whether by that name or a different one. Unfortunately, a great many runtime defects similar to the one you saw in the previous chapter occur every day because someone used a null reference. Often, those defects show up in programs that are more complex than our example and are more challenging to spot.

Many of the most notorious error messages (such as segmentation fault and null pointer exception) arise from programmers’ failure to treat null appropriately.

Why `Option`?

As you have seen, Scala doesn’t stop us from using null. However, the language has been designed to encourage alternative approaches, Option being one of them. null is rarely used in (good) Scala programs.

When an expression has the type Option[X] rather than just X, we can’t simply use it in a context that calls for a value of type X. An attempt to do so produces an error notification before we even run our program. What this implies is that we programmers have to explicitly extract the value from the Option wrapper. As we do so, we are reminded and even forced to attend to the fact that the Option might be None and the value of type X might not exist.

For example, the fact that a Category’s favorite has a return type of Option[Experience] is a clear signal to any user of the class: the favorite may or may not exist and you must consider both alternatives. What’s more, any oversight will be brought to your attention immediately rather than left as a defect in the software, perhaps to be discovered by an unfortunate end user much later.

Yes, you may be able to implement a class such as Category with null. Yes, you may ensure that its methods carefully check that no internal problems occur. Even so, the problem fails to vanish if the class’s public methods (such as favorite) return null values. Any user of the class, which could be you or someone else, will still need to be on their toes.

If you are a beginner programmer or an experienced programmer resigned to using null, you might not realize just how much bother you can save yourself by using Option where a value may be missing.

On error messages

We just suggested that a major factor in why Option is a good idea is that you get error messages. This may sound odd.

To a beginner programmer, the whole notion of error messages has a distinctly negative vibe. But an error message is your friend. An error message at compile-time is a friend indeed! It means that the computer has been able to detect a problem automatically. It’s great that you get a message about that — even if it’s a complaint — as immediate feedback. Runtime bugs are commonly much harder to squash, and it’s by no means assured that they’ll even be detected to begin with.

A good chunk of Tony Hoare’s billion would have been saved by earlier error messages.

Summary of Key Points

You can use class Option to represent a value that “maybe exists”.
- An Option has a type parameter. Option[Int], for example, stands for “zero or one integer values”.
- A value of type Option is either a reference to the singleton object None (“an empty wrapper”) or a Some object that “wraps” a single value within itself.
- It’s almost always better to use an Option than to rely on null references.
Scala’s match command works well together with the Option type. You can use match to select what to do based on whether a given Option contains a value or not. Later on, you’ll see other uses for match, too.
The Option class provides a number of convenient methods, which include getOrElse, isDefined, and isEmpty. Once you get to know more of these methods (in Chapter 8.2), you’ll find Option increasingly easy to work with.
A programming language and its standard libraries may be designed to eliminate software defects and to support the rapid detection of programmer errors. The Option class is an example of this.
Links to the glossary: Option, null; compile-time error, runtime error.

Feedback

Please note that this section must be completed individually. Even if you worked on this chapter with a pair, each of you should submit the form separately.

Credits

Thousands of students have given feedback that has contributed to this ebook’s design. Thank you!

Weeks 1 to 13 of the ebook, including the assignments and weekly bulletins, have been written in Finnish and translated into English by Juha Sorva.

Weeks 14 to 20 are by Otto Seppälä. That part of the ebook isn’t available during the fall term, but we’ll publish it when it’s time.

The appendices (glossary, Scala reference, FAQ, etc.) are by Juha Sorva unless otherwise specified on the page.

The automatic assessment of the assignments has been developed by: (in alphabetical order) Riku Autio, Nikolas Drosdek, Joonatan Honkamaa, Jaakko Kantojärvi, Niklas Kröger, Teemu Lehtinen, Strasdosky Otewa, Timi Seppälä, Teemu Sirkiä, and Aleksi Vartiainen.

The illustrations at the top of each chapter, and the similar drawings elsewhere in the ebook, are the work of Christina Lassheikki.

The animations that detail the execution Scala programs have been designed by Juha Sorva and Teemu Sirkiä. Teemu Sirkiä and Riku Autio have done the technical implementation, relying on Teemu’s Jsvee and Kelmu toolkits.

The other diagrams and interactive presentations in the ebook are by Juha Sorva.

The O1Library software has been developed by Aleksi Lukkarinen and Juha Sorva. Several of its key components are built upon Aleksi’s SMCL library.

The pedagogy behind O1Library’s tools for simple graphical programming (such as Pic) is inspired by the textbooks How to Design Programs by Flatt, Felleisen, Findler, and Krishnamurthi and Picturing Programs by Stephen Bloch.

The course platform A+ has been created by Aalto’s LeTech research group and is largely developed by students. The current lead developer is Jaakko Kantojärvi; many other students of computer science and information networks are also active on the project.

For O1’s current teaching staff, please see Chapter 1.1.

Additional credits appear at the ends of some chapters.

Where We Left Off

`lift`ing for Inspiration

The `Option` Class

`Some` and `None`

Variables of type `Option`

Pictures or it didn’t happen

Wrapped values vs. unwrapped values

Practice on `Option`s

So, what’s the difference between `null` vs. `None`?

And what about `Unit`?

Opening an `Option` with `match`

The `match` command

Practice on `match` + `Option`

Tracking the Favorite Experience (A New Implementation)

An instance variable of type `Option`

Re-implementing `addExperience`

A change in the `Category` interface

An Alternative Implementation without `Option`

Come on, you could just have told us that to begin with!?

The Billion-Dollar Mistake

Why `Option`?

On error messages

Summary of Key Points

Feedback

Credits

Chapter 4.3: A Lack of Values

Where We Left Off

lifting for Inspiration

The Option Class

Some and None

Variables of type Option

Pictures or it didn’t happen

Wrapped values vs. unwrapped values

Practice on Options

So, what’s the difference between null vs. None?

And what about Unit?

Opening an Option with match

The match command

Practice on match + Option

Tracking the Favorite Experience (A New Implementation)

An instance variable of type Option

Re-implementing addExperience

A change in the Category interface

An Alternative Implementation without Option

Come on, you could just have told us that to begin with!?

The Billion-Dollar Mistake

Why Option?

On error messages

Further Reading: Option and Related Topics

Summary of Key Points

Feedback

Credits

`lift`ing for Inspiration

The `Option` Class

`Some` and `None`

Variables of type `Option`

Practice on `Option`s

So, what’s the difference between `null` vs. `None`?

And what about `Unit`?

Opening an `Option` with `match`

The `match` command

Practice on `match` + `Option`

An instance variable of type `Option`

Re-implementing `addExperience`

A change in the `Category` interface

An Alternative Implementation without `Option`

Why `Option`?

Further Reading: `Option` and Related Topics