Lambdas: Myths and Mistakes – by Richard Warburton



Lambdas: Myths and Mistakes – by Richard Warburton

1 0


Lambdas--Myths-and-Mistakes

A slide deck on what the LJC has learnt trying out lambda expressions in Java 8.

On Github RichardWarburton / Lambdas--Myths-and-Mistakes

Lambdas: Myths and Mistakes

by Richard Warburton

the critical design tool for software development is a mind well educated in design principles. It is not ... technology.

Craig Larman

Before we go into details ... Explain quote What does he really mean: idioms and technique are what is important.

Talk Structure

  • Why am I talking about this?
  • Intro to Lambda Expressions
  • Beyond the Myths
  • Functional Thinking
intro-ing in case there are people in the audience who don't know lambda expressions.

Why am I talking about this?

How did I get here?

Lambda Expressions are coming in Java 8!

Also massively improved collections library Biggest change since generics Language changes relatively small, library changes are the important thing Language change enables library change

lots of discussion/debate

Everyone has an opinion That's healthy Amount of irrelevant debate that goes on unhealthy, people shouting opinions problems are hard, need to balance specialist implementor knowledge with mass-market usecases lambda-dev was ineffective tool for discussion

How can we help?

Adopt-a-JSR

Community involvement and community driven standards. Poll if people have heard of it?

Adopt-a-JSR?

  • More community driven standards
  • Hackdays
  • Serve on Expert Groups
Community involvement and community driven standards Opinions of one person not so relevant entire community is important - tough because people differ report feedback to EG trialled by James Gough and I on JSR-310.

Some discussion unimportant

Concrete Examples focus discussion

A lot of the mistakes people made were similar similar to the mistakes I made, know several java champs similar mistake Conclusion: other people not at hackdays "We're not so different you and I"

Intro to Lambda Expressions

Poll people about to ask about whether they have tried out binaries and been following changes

Overview of Lambdas

  • Goal: Better Libraries
  • Example: Collections with Data Parallelism
  • Approach: Allow Code as Data
Specific example threading model very tied in hard to abstract the what from the how

Action Listener

button.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent event) {
        System.out.println("button clicked");
    }
});
                    
existing code as data solution aim: register a callback for when someone clicks a button familiar with anonymous inner classes bulky and verbose

Code as Data

button.addActionListener(
    ?
);
                    
outer method call is fine issue is what goes in the middle, something that represents an action.

Need a parameter

button.addActionListener(event

);
                    
well we know the that we're passing in something that gets given an event as a parameter

Lambda Example

button.addActionListener(event ->
    System.out.println("button clicked")
);
                    
previously required the type of the event can be inferred from context already in java 7: diamond operator

No parameters

Runnable helloWorld =
    () -> System.out.println("Hello World");
                    
also possible to infer the type from an assignment no arguments variant of a lambda

Variable Capture

String name = getUserName();
button.addActionListener(event ->
    System.out.println("hi " + name)
);
                    
improvement over final variable capturing non-final variables that are 'effectively final'.

Functional Interfaces

  • Everything in Java has a type
  • Problem: Need types to represent Functions
  • Solution: Use interfaces
6 is an int new HashMap is a HashMap;

Functional Interfaces

public interface ActionListener extends EventListener {
    public void actionPerformed(ActionEvent event);
}
                    
here's the example method from earlier single abstract method

Streams

  • Support automated data parallelism
  • Build computation pipelines
  • Iterator with inversion of control
aim to seperate out the threading model provide high level operations which are composable and reusable not just beneficial for parallelism also makes code easier to read and reuse

External Iteration

int count = 0;
for (Artist artist : artists) {
    if (artist.isFrom("London")) {
        count++;
    }
}
                    
familiar concept the control flow is driven by the code performing the operation can't do parallel iteration

Internal Iteration

artists.stream()
       .filter(artist -> artist.isFrom("London"))
       .count();
                    
same code rewritten to use internal iteraion there's no visible iteration! iteration is delegated to the stream this is why its an inversion of control: stream knows how to iterate effectively. you may call into streams, but they call you back, which lets them control the threading model.

map

briefly talk about a few different types of stream operation map: transform

map

List<String> collected = Stream.of("a", "b", "hello")
                               .map(string -> string.toUpperCase())
                               .collect(toList());

assertEquals(asList("A", "B", "HELLO"), collected);
                    
in code Explain Stream.of explain general form point out the toUpperCase()

reduce

reduce

int sum = Stream.of(1, 2, 3, 4)
                .reduce(0, (acc, x) -> acc + x);

assertEquals(10, sum);
                    
remember to mention the sum()

filter

filter

List<String> beginningWithNumbers =   
    Stream.of("a", "1abc", "abc1")
          .filter(value -> isDigit(value.charAt(0)))
          .collect(toList());

assertEquals(asList("1abc"), beginningWithNumbers);
                    
retain only strings whose first character is a digit.

Putting it all together

for a given an album, find the nationality of every band playing on that album

go through a worked example of how to put together a pipeline of stream operations.

Putting it all together (2)

transform an album into its artists figure out which artists are bands find the nationalities of each band give concrete example, eg 60s music compilation, beatles and monkeys

Putting it all together (3)

List<String> origins =
  album.getMusicians()
       .filter(artist -> artist.getName().startsWith("The"))
       .map(artist -> artist.getNationality())
       .collect(toList());
                    
make joke about bands starting with the

Method References

str -> str.length
String::length

x -> foo.bar(x)
foo::bar

str -> new Name(str)
Name::new
                    
one more note about introductory stuff

Beyond the Myths

Claim: Syntax is the most important thing about Lambda Expressions

Yeah, I liked the # syntax proposal better, too. One less character to type! :) Referring to a preference over ::
Have you considered 'default null'? It will save a keyword asking for things being too short at the expense of readability.
How about a single punctuation mark, currently unused, as syntax sugar for "()->". Extreme operator usage.
(_, _) -> _ + _ This is starting to look like risque ASCII art :) Joke Slide: absurdism

Its a Myth!

People asked about the syntax but very little explanation and only a few examples were required. Vast amounts more thought going into type inference, type system changes and overload resolution.

Claim: Syntax is irrelevant

Whilst token substitution not important, people did get tripped up on syntax.
// Originally invalid
Stream.of(1, 2, 3)
      .forEach(x -> System.out.println(x));

// Required extra ;
Stream.of(1, 2, 3)
      .forEach(x -> System.out.println(x););
                    
Tripped up loads of people Incredibly confusing compiler message Issue was that println returned void and was a statement Need a ; at the end of statements thankfully fixed after being reported

Difference between expectations

  • Many language features stolen! adapted
  • Missing Features
    • Stronger Type System
    • Tuples
    • List construction syntax
stolen vs adapted: frame of reference. Reality is there's a lot of work to bring changes into Java 8. Point is no Java dev I spoke to ever complained about the lack of higher kinded types. experienced fpers from other areas wanted features they were used to. Not a good or bad feature change, just a different perspective.

Framing Effect

Different reactions depending on whether something is presented as a loss or a gain.

Basic Psychology "Prospect theory shows that a loss is more significant than the equivalent gain" Coming from Scala: you might expect tuples and higher kinded types Coming from Java 7 you're happy with what's provided Also results in the general reaction from Scala devs that they aren't coming back

Recall our earlier example

List<String> origins =
  album.getMusicians()
       .filter(artist -> artist.getName().startsWith("The"))
       .map(artist -> artist.getNationality())
       .collect(toList());
                    
Recall from earlier

Eager vs Lazy (2)

  album.getMusicians()
       .filter(artist -> artist.getName().startsWith("The"))
       .map(artist -> artist.getNationality())

// What's happened?
       .collect(toList());
                    
does nothing Streams are really builders for collections people not at all understanding eager vs lazy was a common mistake nothing is constructed untill the last call just building up a recipe key trick: anything that returns a Stream is lazy

Very little Testing

Maybe ...

  • a reflection on popularity of TDD
  • spikes are good for learning
  • unfamiliarity with testing lambdas
might be part of a hackday environment: experimentation not reliable code some people tried to do TDD theoretically functional code easier to test specification based testing

How do I test this?

list.stream()
    .map(x -> 1.0 / Math.ceil(1 + Math.pow(x) + Math.atan2(y, x)))
    .collect(toList());
                    
Issue is that lambda expressions aren't reference-able. Say your lambda expression interacts with a database: how do you mock it? Anyone spot the subtle bug? atan2(0,0) is mathematically undefined, Java gives an answer NaN return, +ve/-ve 0, 10 infinity related special cases

Approach 1: Test surrounding method

  • Don't test the lambda
  • Test the method its surrounded by
  • Works well for simple lambdas

Approach 2: Extract Method

double complexFunction(double x) {
    return 1.0 / Math.ceil(1 + Math.pow(x) + Math.atan2(0, x));
}

list.stream()
    .map(this::complexFunction)
    .collect(toList());
                    
extract your lambda out to a function if its complex. Still get the code reuse and stream functionality through method references remember to explain what a method reference is. Method references really help testability for things like this.

Mistake: debugging

// Streams
list.stream()
    .filter(filteringFunction)
    .map(mappingFunction)
    .collect(toList());

// Ye olde for loop
List<Bar> bars = new ArrayList<>();
for (Foo element : list) {
    if (filteringFunction(element) {
        Bar result = mappingFunction(element);
        bars.add(result);
    }
}
                    
Issue is understanding intermediate values.

peek

list.stream()
    .filter(filteringFunction)
    .peek(e -> System.out.println("Filtered value: " + e));
    .map(mappingFunction)
    .map(e -> e);
    .collect(toList());
                    
could also be logging output or debugger breakpoint. similar to the unix tee program. Hands up.

Compiler Error Messages

Very important influential in the learning process. Are they idiomatic? Are they easy to understand?

Comparators

Comparator<String> comparator = comparing(String::length);

Comparator<String> comparator = comparing(str -> str.length);
                    
Comparator is a functional interface. provides some static methods for comparing by keys explanatory example

Compiler Error

java: reference to comparing is ambiguous both
method
<T>comparing(java.util.function.ToIntFunction< ? super T>)
in java.util.Comparator and method
<T,U>comparing(java.util.function.Function< ? super T,? extends U>)
in java.util.Comparator match
                    
Overload resolution failure. current state of play, but not likely to have a language fix may change libraries

What happened?

// Generic object variant
public static <T, U extends Comparable< ? super U>>
    Comparator<T>
        comparing(Function< ? super T, ? extends U> keyExtractor)

// Specialised primitive variant
public static <T>
    Comparator<T>
        comparing(ToIntFunction< ? super T> keyExtractor)
                    
overloads are an issue explain primitive specialised variant overloads + type inference can be a bad combo, quite unusual recommendation to library writers: don't overload on functional interfaces

Summary

  • Syntax important, but not in the way people think
  • New approaches for debugging and testing
  • Take care of overloads and compiler error messages

Functional Thinking

Say what I mean look at a couple of examples try and think of solutions

Functional Thinking?

Thinking in terms of the input to output relationship and not a sequence of steps

What is it? Functional means different things to different communities clojure: immutability, haskell: purity + monads, scala: a series of nouns which can be used for trolling. Biggest issue that I saw: easy to develop and understand, but takes a bit of practise. In the same way that patterns such as SOLID exist in OOP and take a while to develop.

First code that people write

List<Integer> numbers = Arrays.asList(1, 2, 3);
numbers.forEach(x -> {
    System.out.println(x);
});
                    
Pretty consistent in understanding internal vs external iteration. not initially trying higher order functions.

Non-idiomatic Proposals

Eg: capture non-final local variables

A lot of people want features which aren’t in the direction of java 8 doesn't make parallelism easier.

Example Problem

Count the number of instances of each word in a document.

Example problem from the devoxx uk hackday nominally simple problem Required understanding of the collectors abstraction. Given a reader as input.

Ideal Solution

reader.lines()
      .flatMap(s -> s.splitAsStream(" "))
      .collect(groupingBy(s -> s, 
               counting()));
                    
Explain code explain the breakdown emphasize the downstream collectors

Ideal Solution (then)

reader.lines()
      .flatMap(s -> Stream.of(s.split(" ")))
      .collect(groupingBy(s -> s,
               reducing(s -> 1, Integer::sum)));

// Map entries for "dad"
// [ "dad", "dad", "dad" ] => [1, 1, 1] => 3
                    
Counting not originally there. Subsequently added. People not only failed to get there, but also failed to understand the solution when presented

Bad Solution (Part 1)

Map<String, List<String>> initial
      = br.lines()
          .flatMap(s -> Arrays.stream(s.split(" ")))
          .collect(groupingBy(s -> s));
  
  Map<Map.Entry<String, Integer>, Integer> freq1 = initial
    .entrySet().stream()
    .map(entry -> new AbstractMap.SimpleImmutableEntry<String,
Integer>(entry.getKey(), entry.getValue().size()))
    .collect(Collectors.toMap(entry -> entry.getValue()));
                    
nearly right to begin with building the map up doesn't realise about downstream collectors

Bad Solution (Part 2)

  Supplier<HashMap<String, Integer>> supplier = () -> new
HashMap<String, Integer>();
  BiConsumer<HashMap<String, Integer>, Map.Entry<String, Integer>> accum =
    (HashMap<String, Integer> result, Map.Entry<String, Integer>
entry) -> result.put(entry.getKey(), entry.getValue());
  BiConsumer<HashMap<String, Integer>, HashMap<String, Integer>>
merger = HashMap::putAll;

  Map<String, Integer> freq2 = initial.entrySet().stream()
    .map(entry -> new AbstractMap.SimpleImmutableEntry<String,
Integer>(entry.getKey(), entry.getValue().size()))
    .collect(supplier, accum, merger);
                    
downhill from there, don't worry about understanding this code make point of weird approach if things start looking like this also, choose first steps wisely sometimes indicates a lack of library knowledge you'll write something like this, treat it as a learning experience not "FP is unreadable."

This takes thought

emphasize he’s a really good programmer, conference speaker, jodatime author, JSR lead and EG member mention annecdote with stuart marks' laptop

Summary

  • Idioms are vital
  • Not Java specific at all
  • Requires Practise
Recognising the problem is the first step

Conclusions

  • Gone through a bunch of examples of specific issues
  • ‘Functional Thinking’: Not necessary to start learning.
  • Try before you buy

Q & A

http://is.gd/lambdas

@RichardWarburto

http://insightfullogic.com