This course has already ended.

The latest instance of the course can be found at: O1: 2024

Luet oppimateriaalin englanninkielistä versiota. Mainitsit kuitenkin taustakyselyssä osaavasi suomea. Siksi suosittelemme, että käytät suomenkielistä versiota, joka on testatumpi ja hieman laajempi ja muutenkin mukava.

Suomenkielinen materiaali kyllä esittelee englanninkielisetkin termit.

Kieli vaihtuu A+:n sivujen yläreunan painikkeesta. Tai tästä: Vaihda suomeksi.


Chapter 4.3: A Lack of Values

About This Page

Questions Answered: How do I express that some information may or may not exist? What can I use instead of null?

Topics: The trouble with null. Basic use of the Option data type. Using the match command to select between alternatives.

What Will I Do? Read. Work on small assignments. (The next chapter contains larger practice tasks on the same topics.)

Rough Estimate of Workload:? An hour or more. Depends on how quickly you take to the new concepts. Many students initially find this chapter’s main topic — class Option — challenging.

Points Available: A15.

Related Modules: GoodStuff.

../_images/person08.png

Where We Left Off

At the end of the previous chapter, our program crashed because we tried to use a reference to access an experience object’s rating — another.rating — but what the variable another actually stored was null: a reference that leads nowhere. This occurred as we were adding a first experience to a category; the trouble started because the category didn’t yet have a favorite experience, and we had used a null reference to record that fact.

On its own, the computer can’t figure out how it should handle the scenario where no favorite experiences exists (yet). As programmers, we must somehow specify this behavior, too, in the program code. Our earlier solution attempt completely neglected the issue.

We could approach the problem in two different ways:

  1. We could use a different data type that helps us represent the fact that a category has “one or zero favorites” rather than always having one.

  2. Or we could continue to use the null reference to mark a nonexistent value; null is a valid value for a variable of (almost) any data type. We must then use other commands to tiptoe around the null value and make sure we never try to access the attributes of a nonexistent object.

There are reasons why the first option is generally better. (We’ll get to the reasons later in this chapter.) We’ll take that path — choose a new data type — as we fix our Category class. But first, let’s get to know this new data type in general terms.

lifting for Inspiration

Programs commonly need to manipulate values that may or may not exist. It’s easy to find an example in Scala’s standard library: a vector may have an element at index 100 (if the vector’s large enough) or it may not (if it’s small).

Sure enough, we know we can check what a vector contains at a given index as shown below, but it’s a shame that it crashes our program if the index is too high or negative:

val words = Vector("first", "second", "third", "fourth")words: Vector[String] = Vector(first, second, third, fourth)
words(2)res0: String = third
words(100)java.lang.IndexOutOfBoundsException: 100 is out of bounds (min 0, max 3)
  [...]

Couldn’t we have a method that tells us both 1) whether or not there is an element at the given index, and if there is one, 2) what is that element?

Yes we could, and we do. The method is called lift.

words.lift(2)res1: Option[String] = Some(third)
words.lift(100)res2: Option[String] = None
words.lift(-1)res3: Option[String] = None

The return values are pretty easy to interpret:

When the element exists, the return value is “some string, "third" to be more precise”.

When no such element exists, the return value is “no value at all”. This isn’t the same thing as null; we’ll get to that.

Even if the given index is invalid, lift doesn’t crash. The method lets us pick out an element safely.

What we get as a return value is not quite a String, though, but something else.

lift relies on a data type named Option. Let’s find out what it is and how it helps us improve Category.

The Option Class

Option is a class defined in package scala, which means it’s always available for you to use in any Scala program. You can think of an object of type Option as a “wrapper” that either contains a single value of a particular type or nothing.

Some and None

Let’s take another look at the above example:

val words = Vector("first", "second", "third", "fourth")words: Vector[String] = Vector(first, second, third, fourth)
words.lift(2)res4: Option[String] = Some(third)
words.lift(100)res5: Option[String] = None

A Some object is a “full wrapper”: an Option object that contains a single value. In our example, the wrapped value happens to be a string.

None is a singleton object defined as part of Scala. It represents an “empty wrapper”. It, too, counts as an object of type Option.

We can see that lift returns an Option that contains a String or nothing. That is, the method always returns either a Some object wrapped around a string or the None object.

Let’s introduce another vector, which contains integers:

val numbers = Vector(10, 0, 100, 10, 5, 123)numbers: Vector[Int] = Vector(10, 0, 100, 10, 5, 123)
numbers.lift(0)res6: Option[Int] = Some(10)
numbers.lift(-1)res7: Option[Int] = None

The type parameter is different here. You can read Option[Int] as “a wrapper containing an Int or nothing”. The type parameter in the square brackets serves exactly the same purpose as it does with vectors and buffers: it indicates the type of the value that may be contained within the “outer” object.

../_images/inheritance_option.png

The concept of Option encompasses both Some and None.

To summarize, class Option is defined so that it guarantees us a number of things:

  • Every Some object is an Option object. Cf. “Every llama is an animal.” None is also an Option object.

  • There are exactly two kinds of Option object. Every Option object is either a Some object or the None object.

  • Every single value of type Option[X] is either None or a Some object that contains exactly one value of type X.

Variables of type Option

In the examples above, we received Option values from a method. We can also create Option values of our own and use Option as the type for variables in our programs.

var test: Option[Int] = Nonetest: Option[Int] = None

Here we have a variable that initially gets the value None. More specifically, the variable stores a reference to the singleton object None.

None has been defined to be type compatible with class Option, so we can assign the reference to a variable of type Option. Note that we can not assign None to, say, variables of type Experience or Int.

What we have now is a variable that we could use to store an integer. But currently, there’s no number stored there, as the variable has the value None instead. We can change that by assigning a “full wrapper”, a Some object, to the variable:

test = Some(5)test: Option[Int] = Some(5)

The expression Some(...) “wraps” a value. Since we’re working with Option[Int], the wrapped value must be an integer. Some(5) constructs a wrapper that contains the integer value 5.

We can also express a computation and wrap the result in an Option:

test = Some(10 + 50)test: Option[Int] = Some(60)

An Option object is immutable. Once you create an object with Some(5), there’s no way to change which object is stored inside the wrapper or to empty the wrapper. However, if we have a var that stores a reference to an Option, we may discard that reference and store another Option in the variable instead. Indeed, we just did that.

Pictures or it didn’t happen

Wrapped values vs. unwrapped values

A value of type Int is an integer. A value of Option[Int] might contain an integer. These types are distinct from each other and you can’t cross-assign them. For example, you can’t simply assign an Int to a variable of type Option[Int]:

test = 5-- Type Mismatch Error:
  |test = 5
  |       ^
  |       Found:    (5 : Int)
  |       Required: Option[Int]

Conversely, you can’t assign a value of type Option[Int] to a variable of type Int or otherwise use that value in computations that call for an integer:

var numericalValue = 10numericalValue: Int = 10
var possibleValue: Option[Int] = Some(10)possibleValue: Int = Some(10)
numericalValue = possibleValue-- Type Mismatch Error:
  |numericalValue = possibleValue
  |                 ^^^^^^^^^^^^^
  |                 Found:    (possibleValue : Option[Int])
  |                 Required: Int
possibleValue - 1-- Not Found Error:
  |possibleValue - 1
  |^^^^^^^^^^^^^^^
  |value - is not a member of Option[Int]

Similarly, if a method takes an Int parameter, it won’t accept an Option[Int] and vice versa.

This is a very good thing. What it means is that if a part of your program requires an integer, it’s not enough that you have “an integer that might exist”. You receive, even before you run your program, an error message that reminds you to ensure that where an integer is needed, an integer must be delivered.

Practice on Options

Study the expressions below. Which of them have a value that can be assigned to a variable of type Option[String]?

Suppose the variable exp stores a reference to an Experience object. Further assume that the following line of code has been executed:

val realGoodExperience = if exp.rating >= 10 then Some(exp.name) else None

What is the type of the variable realGoodExperience?

So, what’s the difference between null vs. None?

null is a value that means “a reference to nowhere, no object at all”. It’s technically possible to use null quite freely in a variety of places. It can be assigned to variables of type Experience or String, for instance.

By using null, one effectively adopts a style of programming where any value is, in a sense, “optional” and may be missing.

None is a singleton object that has the Option type. It is specifically built to be used in places that demand a value of type Option. A reference to None can be stored in a variable whose type is Option[something] but cannot be stored in an arbitrary sort of variable.

By using None and Option, we adopt a style of programming where the programmer explictly marks the parts of the program where a value is “optional” and may be missing. Further down on this page, we’ll discuss why this is often the better approach and you would do well to avoid null in your programs.

If you have trouble discerning the difference between the null reference and the None object, you may find it helpful to contrast the animations in this chapter with the one at the end of the previous one.

And what about Unit?

Unit, first introduced in Chapter 1.6, is a special value that we use to mean “no value of interest” and especially “the function produces no value of interest under any circumstances”. The value Unit is of a data type of its own, also called Unit; the value can’t be assigned to an arbitrary variable (as null can) or to a variable of type Option (as None can). In O1, we won’t use Unit for any purpose other than as the return value of some effectful functions.

Opening an Option with match

In order to use the contents of an Option, we need a way to “open up the wrapper” and see what, if anything, we find there.

To do that, we’ll use the match command. match is a relative of if in the sense that both commands make choices between alternatives.

There are other ways than match for accessing the contents of an Option but, for now, we’ll use match for this specific purpose. Moreover, there are other things that we can do with match besides handle Options but, for now, we’ll use match specifically for that purpose.

The match command

First, some familiar-looking code to set things up:

val numbers = Vector(10, 0, 100, 10, 5, 123)numbers: Vector[Int] = Vector(10, 0, 100, 10, 5, 123)
val possibleElement = numbers.lift(5)possibleElement: Option[Int] = Some(123)
val thousandthIfAny = numbers.lift(999)thousandthIfAny: Option[Int] = None

Let’s now make the computer select what it should do by considering two cases and seeing which one possibleElement’s value matches. The cases are: 1) a full Option wrapper, or 2) an empty one.

possibleElement match
  case Some(wrappedNumber) => "there is some number in the wrapper"
  case None => "there is no number in the wrapper"res8: String = there is some number in the wrapper

We want to evaluate the expression possibleElement and examine its value. We follow the expression with the keyword match and break the line.

Here we have two distinct cases to choose from. We specify them by writing the keyword case and an “arrow” that consists of an equals sign and a greater-than sign.

In between, we define each case. When we use match to process an Option, like here, we’ll often use one case to cover the possibility that the Option is a Some and another case to cover None.

Since it so happens that our example expression evaluates to a Some, the first case matches.

The expression that follows the arrow produces the result of the entire match command. In our example, we produce a string.

wrappedNumber is a programmer-chosen name for a variable that we didn’t yet use for any purpose.

Of course, it could be that it’s the other case that matches the expression:

thousandthIfAny match
  case Some(wrappedNumber) => "there is some number in the wrapper"
  case None => "there is no number in the wrapper"res9: String = there is no number in the wrapper

The value of thousandthIfAny matches the latter case, so the computer selects that branch and uses the code therein as the match expression’s value.

In the preceding examples, we cared only about whether or not the Option object had content, not about what that content might be. The following example attends to that, too.

possibleElement match
  case Some(wrappedNumber) => "the wrapper contains: " + wrappedNumber
  case None => "there is no number in the wrapper"res10: String = the wrapper contains: 123

In the Some case, we include a variable name as shown. When the expression matches that case, the Option object’s content is copied into a new local variable of that name before the expression on the right is evaluated. Therefore...

... we can use that variable to refer to the value that was “found” inside the Option. In this example, that value is an Int.

Here’s an additional example that illustrates some of match’s other features:

val numbers = Vector(10, 0, 100, 10, 5, 123)numbers: Vector[Int] = Vector(10, 0, 100, 10, 5, 123)
val result = numbers.lift(4) match
  case None => 0
  case Some(number) => number * 1000result: Int = 5000

A match expression is an expression like any other. We can use it as part of other commands. We can, for instance, assign its value to a variable, as here.

A match expression begins with a (sub)expression whose value is matched on. In our earlier examples, that expression was a variable’s name, but other expressions work just as well. Here, we call lift and immediately match on the value that it returns (skipping the intermediate step of assigning the return value to a variable).

You aren’t obliged to order the cases in a specific way; it’s perfectly fine to write the None case before the Some case. The computer goes through the cases in order until it finds a match.

In our earlier examples, the match expression evaluated to a string, but other data types are fine, too. Here we have two cases that produce integers, so the result type is Int.

Indent the cases.

You have the option of watching the following animation.

The pseudocode below summarizes, in general terms, how we’ve used match to work on Options:

expression to examine match
  case Some(variableForContent) => an expression to evaluate if the “wrapper is full”; may use the variable
  case None => an expression to evaluate if the “wrapper is empty”

You can also call effectful functions such as println within a case:

numbers.lift(7) match
  case None => 
    println("There was no number at index seven.")
    println("No can do.")
  case Some(number) => 
    println("At index seven, I found " + number + ".")There was no number at index seven.
No can do.

If you include effectful commands write line breaks and indent deeper, as shown.

A case may include multiple commands in sequence.

Practice on match + Option

Before we get to work on class Category, answer the following practice questions. Use the REPL as needed to help you find the answers. Since Option is about to become an important tool for us, it makes sense to go through a few basic drills.

Assume the following commands have been executed.

var first: Option[Int] = None
var second: Option[Int] = Some(10)

What is the value of the following match expression?

first match
  case Some(value) => value
  case None        => -1

What about this expression?

second match
  case Some(value) => value
  case None        => -1

And this?

second match
  case Some(value) => value + 100

One more.

Vector(15, 18, 50, 12).lift(3) match
  case Some(value) => if value >= 18 then "adult" else "child"
  case None        => "age not available"

What does the match command below do to the value of experiment?

var experiment = 100
val possibleResult = if experiment > 50 then Some(2 * experiment) else None
possibleResult match
  case Some(result) =>
    experiment += result
  case None =>
    experiment = 0

Option objects have methods, one of which is named getOrElse. A Some object responds to a getOrElse method call by returning the value wrapped inside itself. A None object, in contrast, evaluates whichever expression you pass in as a parameter to getOrElse and returns the value of that expression.

Try calling getOrElse on Some objects and None. You can try the following, for example. A bit further down on this page, you can also find an animation of these commands.

val words = Vector("first", "second", "third", "fourth")
println(words.lift(100).getOrElse("no word"))
println(words.lift(2).getOrElse("no word"))

Which of the following claims are correct? Select all that apply.

Option objects also have the parameterless methods isDefined and isEmpty. Experiment with these methods in the REPL.

Then examine the following match expression:

x match
  case Some(thing) => true
  case None        => false

Which of the following claims are correct? Select all that apply.

Tracking the Favorite Experience (A New Implementation)

An instance variable of type Option

Now to our Category class. We can use Option to define its instance variable fave. Like so:

class Category(val name: String, val unit: String):

  private val experiences = Buffer[Experience]()
  private var fave: Option[Experience] = None

  def favorite = this.fave

  def addExperience(newExperience: Experience) =
    // We'll need to re-implement this.

end Category

We enter Option[Experience] as the type of fave. This means the variable may hold a single experience or none. This definition exactly describes what we’re trying to achieve with the variable.

The value of our variable can be None or Some(...), where the “wrapper” contains a reference to an Experience object. Initially, there is no favorite and the value is None.

Re-implementing addExperience

Let’s rewrite the pseudocode that outlines addExperience:

  def addExperience(newExperience: Experience) =
    this.experiences += newExperience
    How the favorite is updated depends on what we find inside the “fave wrapper”:
    1) In case we find nothing, we record that the favorite is a Some* that contains the newly added (first) experience.*
    2) In case we find an old favorite, we choose the better of the new experience and the old favorite and wrap it.

We have two cases: either this.fave previously holds just an empty wrapper or some old favorite wrapped in an Option.

We can’t simply compare the ratings of Option[Experience] objects. We can compare the ratings of Experience objects. To do that, we need to extract the old favorite from its Option wrapper.

Here is the same algorithm implemented as Scala:

def addExperience(newExperience: Experience) =
  this.experiences += newExperience
  this.fave match
    case None =>
      this.fave = Some(newExperience)
    case Some(oldFave) =>
      val newFave = newExperience.chooseBetter(oldFave)
      this.fave = Some(newFave)

Once an experience has been added, the category will definitely be associated with a favorite experience. We wrap a reference to that experience in a Some before storing it in the instance variable (since our new fave variable demands an Option[Experience] rather than Experience).

You can define local variables within a match command. Here, we’ve used a temporary variable newFave to emphasize the algorithm’s two steps. We could have also done everything on one line: this.fave = Some(newExperience.chooseBetter(oldFave))

A variation of the above

This alternative implementation works just as well.

def addExperience(newExperience: Experience) =
  this.experiences += newExperience
  val newFave = this.fave match
    case Some(oldFave) => newExperience.chooseBetter(oldFave)
    case None          => newExperience
  this.fave = Some(newFave)

A change in the Category interface

The version of Category that we just produced (and that comes with the GoodStuff module) needs to be used a bit differently than we originally envisioned in Chapter 4.2.

This is because the return type of favorite is no longer Experience but Option[Experience], and the method returns either None or a Some that holds an experience. When we use the class, we must attend to the return type, as in this example:

val wineCategory = Category("Wine", "bottle")
// ... (We may or may not add experience objects to the category here.)

wineCategory.favorite match
  case Some(bestWine) =>
    println("The favorite is: " + bestWine.name)
  case None =>
    println("No favorite yet.")

An expression such as wineCategory.favorite.name isn’t valid, as indeed it shouldn’t be. If you try to use that expression with this Category class, you immediately receive a complaint from the Scala toolkit: the Option returned by wineCategory.favorite doesn’t have a name variable (even though any experience that it may contain has one).

An Alternative Implementation without Option

We didn’t strictly need Option in order to make addExperience work. At the beginning of this chapter, we suggested that there is another way to solve our original problem with null. Here’s how it works: we keep using a null value in fave but are super-careful that we never access the attributes of a non-existent object. Consider this implementation:

class Category(val name: String, val unit: String):

  private val experiences = Buffer[Experience]()
  private var fave: Experience = null

  def favorite = this.fave

  def addExperience(newExperience: Experience) =
    this.experiences += newExperience
    this.fave =
      if this.fave == null then
        newExperience
      else
        newExperience.chooseBetter(this.fave)

end Category

Since we first compare to see if fave is null and place our chooseBetter call in the else branch, our method never attempts to access the rating of a nonexistent object. The method works as originally intended.

Come on, you could just have told us that to begin with!?

Doesn’t that null-based version work just as well as the Option-based one? Didn’t Option just add complexity to our class for not much gain? Was it even worth learning about?

The Option-based implementation has some highly desirable features that the null-based one lacks. We’ll now review some of them.

The Billion-Dollar Mistake

I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object-oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe. — — But I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years.

—C. A. R. Hoare

The quotation above is from a 2009 speech where Sir Charles Antony Richard Hoare apologizes for inventing the null reference, originally for the ALGOL programming language. ALGOL contributed a great number of (mostly good) ideas that have been adopted into other programming languages, including present-day ones. null also lives on in many programming languages, whether by that name or a different one. Unfortunately, a great many runtime defects similar to the one you saw in the previous chapter occur every day because someone used a null reference. Often, those defects show up in programs that are more complex than our example and are more challenging to spot.

Many of the most notorious error messages (such as segmentation fault and null pointer exception) arise from programmers’ failure to treat null appropriately.

Why Option?

As you have seen, Scala doesn’t stop us from using null. However, the language has been designed to encourage alternative approaches, Option being one of them. null is rarely used in (good) Scala programs.

When an expression has the type Option[X] rather than just X, we can’t simply use it in a context that calls for a value of type X. An attempt to do so produces an error notification before we even run our program. What this implies is that we programmers have to explicitly extract the value from the Option wrapper. As we do so, we are reminded and even forced to attend to the fact that the Option might be None and the value of type X might not exist.

For example, the fact that a Category’s favorite has a return type of Option[Experience] is a clear signal to any user of the class: the favorite may or may not exist and you must consider both alternatives. What’s more, any oversight will be brought to your attention immediately rather than left as a defect in the software, perhaps to be discovered by an unfortunate end user much later.

Yes, you may be able to implement a class such as Category with null. Yes, you may ensure that its methods carefully check that no internal problems occur. Even so, the problem fails to vanish if the class’s public methods (such as favorite) return null values. Any user of the class, which could be you or someone else, will still need to be on their toes.

If you are a beginner programmer or an experienced programmer resigned to using null, you might not realize just how much bother you can save yourself by using Option where a value may be missing.

On error messages

We just suggested that a major factor in why Option is a good idea is that you get error messages. This may sound odd.

To a beginner programmer, the whole notion of error messages has a distinctly negative vibe. But an error message is your friend. An error message at compile-time is a friend indeed! It means that the computer has been able to detect a problem automatically. It’s great that you get a message about that — even if it’s a complaint — as immediate feedback. Runtime bugs are commonly much harder to squash, and it’s by no means assured that they’ll even be detected to begin with.

A good chunk of Tony Hoare’s billion would have been saved by earlier error messages.

Summary of Key Points

  • You can use class Option to represent a value that “maybe exists”.

    • An Option has a type parameter. Option[Int], for example, stands for “zero or one integer values”.

    • A value of type Option is either a reference to the singleton object None (“an empty wrapper”) or a Some object that “wraps” a single value within itself.

    • It’s almost always better to use an Option than to rely on null references.

  • Scala’s match command works well together with the Option type. You can use match to select what to do based on whether a given Option contains a value or not. Later on, you’ll see other uses for match, too.

  • The Option class provides a number of convenient methods, which include getOrElse, isDefined, and isEmpty. Once you get to know more of these methods (in Chapter 8.4), you’ll find Option increasingly easy to work with.

  • A programming language and its standard libraries may be designed to eliminate software defects and to support the rapid detection of programmer errors. The Option class is an example of this.

  • Links to the glossary: Option, null; compile-time error, runtime error.

Feedback

Please note that this section must be completed individually. Even if you worked on this chapter with a pair, each of you should submit the form separately.

Credits

Thousands of students have given feedback and so contributed to this ebook’s design. Thank you!

The ebook’s chapters, programming assignments, and weekly bulletins have been written in Finnish and translated into English by Juha Sorva.

The appendices (glossary, Scala reference, FAQ, etc.) are by Juha Sorva unless otherwise specified on the page.

The automatic assessment of the assignments has been developed by: (in alphabetical order) Riku Autio, Nikolas Drosdek, Joonatan Honkamaa, Antti Immonen, Jaakko Kantojärvi, Niklas Kröger, Kalle Laitinen, Teemu Lehtinen, Jaakko Nakaza, Strasdosky Otewa, Timi Seppälä, Teemu Sirkiä, Anna Valldeoriola Cardó, and Aleksi Vartiainen.

The illustrations at the top of each chapter, and the similar drawings elsewhere in the ebook, are the work of Christina Lassheikki.

The animations that detail the execution Scala programs have been designed by Juha Sorva and Teemu Sirkiä. Teemu Sirkiä and Riku Autio did the technical implementation, relying on Teemu’s Jsvee and Kelmu toolkits.

The other diagrams and interactive presentations in the ebook are by Juha Sorva.

The O1Library software has been developed by Aleksi Lukkarinen and Juha Sorva. Several of its key components are built upon Aleksi’s SMCL library.

The pedagogy of using O1Library for simple graphical programming (such as Pic) is inspired by the textbooks How to Design Programs by Flatt, Felleisen, Findler, and Krishnamurthi and Picturing Programs by Stephen Bloch.

The course platform A+ was originally created at Aalto’s LeTech research group as a student project. The open-source project is now shepherded by the Computer Science department’s edu-tech team and hosted by the department’s IT services. Markku Riekkinen is the current lead developer; dozens of Aalto students and others have also contributed.

The A+ Courses plugin, which supports A+ and O1 in IntelliJ IDEA, is another open-source project. It has been designed and implemented by various students in collaboration with O1’s teachers.

For O1’s current teaching staff, please see Chapter 1.1.

Additional credits appear at the ends of some chapters.

a drop of ink
Posting submission...