command-pattern

View Github Repository Open presentation in a new window

parsonsmatt

See all presentation from parsonsmatt

command-pattern

0 1

command-pattern

On Github parsonsmatt / command-pattern

I Command You

To Be Free!

http://www.parsonsmatt.org/command-pattern

Hi! I'm Matt Parsons, and I'd like to talk to you about some cool software practices I've been using recently to make better software.

If you want to follow along with the slides, they're available at the URL above.

Better?

  • Performance?
  • Correctness?
  • Ease of understanding?
  • Ease of reuse?
  • Easy to modify?
  • Easy to test?

Of course, better is subjective. Faster code is better, but performance isn't free -- you have to spend time implementing it. Code that gives the right answer is important, but sometimes "close enough" is good enough.

Ease of understanding is important and subjective. I personally have no issue understanding Haskell code, and may trip up with complex Java hierarchies or JavaScript scoping rules that are perfectly obvious and idiomatic for a familiar programmer.

Ease of reuse is a little easier to understand. How hard is it to repurpose this code for other related tasks? This is difficult to know without actually reusing the code, which may be too late to understand how good the code actually is.

Modifiability is important. If business rules need to change, then the code needs to change. How difficult this is determines how quickly the business react to changing requirements.

Testability -- this is a really good metric for good code!

Test Driven Design

aka

If it sucks to test,

it sucks

Test Driven Design

  • Informs API/library design
  • Immediate feedback on code reusability
  • Free regression/unit testing

Writing tests first is a great proxy for writing good code. Having "tests" to cover correctness and find bugs is just about a nice side effect. You get immediate feedback on your library design. If it sucks to write tests for your code, it probably sucks to use your code.

TDD is hard!

Writing test-first is really difficult. This is partially because writing "better" code is really difficult.

This isn't a TDD talk

I don't care about TDD. Testability is a nice proxy for "good" code.

Code that's easy to test tends to be easy to modify, update, reuse, and verify. It is like a finger pointing at the moon of good software: don't look at the tests, look at the good software! You don't have to write tests to write good code, and just because you wrote tests doesn't mean your code is nice.

Methods and Functions

the unit of

uh, doing stuff

So, when we talk about "doing stuff" in computer programming, we have a bunch of different ways of organizing it. If you're writing in OOP, you'll start by making a class, and then defining some methods on it. In functional programming, you start by defining a data type and writing functions that operate on it. In imperative languages, you define procedures that run a sequence of commands on the underlying machine.

Values and Effects

Input and Output

Let's talk about how we use methods and functions. Generally, we can talk about a function in terms of the inputs and outputs that it has. We can also talk about a function in terms of the values and effects that it deals with.

The code in this talk will be a combination of Ruby and Haskell.

Values

class Foo
  def my_func(x, y)          # value!
    z = User.all.length
    FooResult.insert(x, y, z)
    x + y + z                # value!
  end
end
  • Input arguments
  • Return value

The values in this code snippet are the simple, easy bits. We pass in the numbers x and y, which are both values. We return x + y + z, which is a simple value.

Effects

class Foo
  def my_func(x, y)
    z = User.all.length       # effect!
    FooResult.insert(x, y, z) # effect!
    x + y + z
  end
end
  • Input effects
  • Output effects

The effects in this function are reading all of the users out of the database, and inserting a new result into the database.

Haskell

module Foo where

myFunc x y = do
    z <- fmap length selectUsers
    insert (FooResult x y z)
    pure (x + y + z)

Haskell's purity makes it really easy to figure out what's an effect and what's a value. In Ruby (or any other language, really) you have to either know or read the entire call graph of the code you're talking about. Haskell tracks it in the type, which makes these refactors really easy.

Values and Effects

Values are, at a first approximation, the things we pass directly into functions or methods, and the things that are returned directly from functions. Input effects are the things that provide information to the function that we don't explicitly pass in. Output effects are the things that happen as a result of calling the method, that aren't explicitly part of the return value.

Values are explicit

Effects are implicit

So values are explicit. Testing values is easy, and testing is an OK approximation of good software. Effects are implicit. And testing effects is difficult.

Testing Values:

def add(x, y)
  x + y
end

describe "add" do
  it "should add" do
    expect(add(2, 3)).to eq 5
  end
end

So, here's the super simple add function. Testing it is stupid easy. We just pass in some input values, and make assertions about the output value. This test is pretty silly, but it's easy to come up with more advanced test cases.

describe "Add" do
  it "is commutative" do
    100.times do
      x, y = 2.times { Random::rand }
      expect(x + y).to eq(y + x)
    end
  end

  it "is associative" do
    100.times do
      x, y, z = 3.times { Random::rand }
      expect((x + y) + z).to eq(x + (y + z))
    end
  end
end

Here, we're testing that add is associative and commutative. We're taking a random sample of values, and ensuring that we can reorder our operations and group them however we want. These tests are easy to write. They're fun, almost. And they're kinda pretty! They look extremely close to the mathematical definitions of associativity and commutativity.

This sounds trivial, but many difficult concepts in distributed systems involve guaranteeing properties like commutativity. Making these properties easy to test is important.

In Haskell, too!

add x y = x + y

spec = 
  describe "add" $ do
    it "should add numbers" $ do
      add 2 3 `shouldBe` 5    
    prop "is commutative" $ \x y -> do
      add x y `shouldBe` add y x
    prop "is associative" $ \x y z -> do
      add x (add y z) `shouldBe` add (add x y) z

The tests we have for the Ruby and Haskell are just about the same! It's a little easier to write the tests in HAskell, but we've got basically the same thing going on.

Testing Effects:

So you're going to read the code on these slides and you might wince. I'm gonna be calling out some methods of testing effects that we've all done (probably?!). So if you are feeling personally offended, just know that I've done all of these too, and maybe I can help find a way out!

Testing Effects:

class Foo
  def my_func(x, y)
    z = User.all.length       # effect!
    FooResult.insert(x, y, z) # effect!
    x + y + z
  end
end

Testing effects is a lot harder. Suddenly we have to worry about what User is, and what happens when we do FooResult. I'm going to evolve testing this example.

Wrong

describe "Foo#my_func" do
  it "adds inputs" do
    expect(Foo.new.my_func(1,2)).to eq(3)
  end
end

So this attempt is flat out wrong. However, on an uninitialized database with 0 users, it'll return the right answer. This is a fragile test, even though it may pass sometimes.

Wrong Haskell

describe "Foo.myFunc" $ do
  it "adds inputs" $ do
    Foo.myFunc 1 2 `shouldReturn` 3

Haskell isn't going to protect us here. While we know that we have effects going on in Foo.myFunc, that's all we know, and as long as we acknowledge that, then GHC is satisfied. Since the correctness of this depends on something that we are not tracking in the type, the type system can' help us!

Slow

describe "Foo#my_func" do
  it "adds inputs" do
    User.insert(name: "Matt", age: 28)
    expect(User.all.length).to eq(1)
    expect(Foo.new.my_func(1,2)).to eq(4)
    x = FooResult.find_by(x: 1, y: 2, z: 1)
    expect(x).to_not be_nil
  end
end

So this isn't wrong anymore. However, the test relies on the database state, and has to do five SQL queries in order to verify the code. These tests are monstrously slow and will kill your TDD cycle, in addition to being fragile and annoying to write.

Stubs!

The next "level up" that often happens is to take advantage of stubs or mocks. Let's look at that real quick:

describe "Foo" do
  it "adds some numbers" do
    x, y, z = 3, 4, 3

    expect(User)
      .to receive(:all)
      .and_return([1,2,3])

    allow(FooResult)
      .to receive(:insert).with(x, y, z)

    expect(Foo.new.my_func(x, y))
      .to eq(x + y + z)
  end
end

This test is a lot nicer. We need to stub out the User.all method to ensure it returns a value that suits our expectation. We also need to stub out the FooResult class and verify that it receives the arguments we expect. Finally, we can do some assertions on the actual values involved.

This kinda sucks! You can imagine extending this to more complex things, but it gets even uglier, pretty quickly. Furthermore, stubs and mocks are pretty controversial in the OOP community. They're not a clear best practice.

Stubs in Haskell

So stubbing global terms like this in Haskell? It's not possible. Sorry, or not, I guess, depending on whether you find the previous code disgusting or pleasantly concise.

Dependency Injection?

Dependency injection is usually heralded as the solution or improvement to just stubbing out random global names. You define an interface (or duck type) for what your objects need and pass them in your object initializer

class Foo
  def initialize(user, foo_result)
    @user = user
    @foo_result = foo_result
  end

  def my_func(x, y)
    z = @user.all.length        # effect!
    result = x + y + z
    @foo_result.insert(x, y, z) # effect!
    result
  end
end

Dependency injection can be used to make testing like this a little easier, especially in languages that aren't as flexible as Ruby (like Haskell). Instead of overriding a global name, you make the class depend on a parameter that's local. Instead of referring to the global User class, we're referring to the instance variable user which is ostensibly the same thing. This is Good, as we've reduced the coupling in our code, but we've introduced some significant extra complexity. And the testing story isn't great, either:

describe "Foo" do
  it "adds stuff" do
    x, y, user = 2, 3, double()
    user.stub(:all) { [1,2,3] }

    foo_result = double()
    foo_result.stub(:insert)

    expect(foo_result)
      .to receive(:insert).with(x, y, 3)

    foo = Foo.new(user, foo_result)

    expect(foo.my_func(x, y)).to eq(x + y + 3)
  end
end

This is clearly worse than before. If we're going by the metric that easier to test code is better code, then this code really sucks. So dependency injection is clearly not the obvious solution to this problem.

You can write helpers and stuff to obscure the difficulty of testing this. But that doesn't make it better, it just hides the badness. Sometimes that's great! Perfect is the enemy of good etc.

How to even do that in Haskell

So how do we even do that in Haskell? Well, everything is just a function, so you just pass functions.

myFuncAbstract 
    :: IO [a]               -- select users
    -> (FooResult -> IO ()) -- insert FooResult
    -> Int -> Int -> IO Int -- The rest of the function
myFuncAbstract selectUsers insert x y = do
    z <- fmap length selectUsers
    insert (FooResult x y z)
    pure (x + y + z)

myFunc = myFuncAbstract DB.selectUsers DB.insert

Here we make selectUsers and insert into functions that we pass in, and for the real version, we provide the database functions. The tested version can provide different functions based on what you want to test.

This is about as awkward as the OOP Version! Alas.

Values vs Objects

Are objects values? Not necessarily. Objects have a notion of identity that is separate from the values of their member variables. By default, objects in Ruby and most object oriented languages compare each other based on reference equality: these two objects are equal iff they refer to the same objet in memory. Two users, each with the same name and age, are different if they are stored in different places in memory.

Objects

class User
  attr_reader :name, :age
  def initialize(name, age)
    @name = name
    @age = age
  end
end

a = User.new("Matt", 28)
b = User.new("Matt", 28)

a == b # False!

Here we've got a User class with name and age. We instantiate two users with the same values. THese are different objects, and equality checking returns false for them.

Values

class User
  # ...
  def eql?(other)
    name == other.name && age == other.age
  end

Values, on the other hand, are equal if every component of the value is equal. 5 is equal to 5, regardless of where the two fives are stored in memory. This modification to the User class converts it into a value, where two users are now equal if their name and age are the same.

Values

https://github.com/tcrayford/Values

User = Value.new(:user, :age)

I'm going to refer to the Values library. The above code creates an immutable value object with two fields, user and age. Equality is done by comparing the members for equality.

Resolving Input Effects

# Ruby
class Foo
  def my_func(x, y)         
    z = User.all.length
    FooResult.insert x, y, z
    x + y + z                
  end
end

Ok, so let's look at our example again. We want to get rid of that input effect so we can make the method easier to test, and by extension, easier to understand, modify, reuse, etc.

three magic steps

Identify input effects Take result of input effect as a parameter High five your coworkers 
# Ruby
class Foo
  def my_func(x, y, users)
    z = users.length
    FooResult.insert(x, y, z)
    x + y + z
  end
end

Fortunately, resolving input effects is super easy. You identify the input effects, and then take them as a parameter. In this example, we've stopped talking to the database, and now just take the object as a parameter. The tests just got way simpler.

-- Haskell
myFunc x y users = do
    let z = length users
    insert (FooResult x y z)
    pure (x + y + z)

And here's the Haskell. It's pretty much exactly what you'd expect.

Testing:

# Ruby
describe Foo do
  it "adds stuff" do
    x, y, arr = 1, 2, [1,2,3]

    allow(FooResult)
      .to receive(:insert)
      .with(x, y, arr.length)

    expect(Foo.new.my_func(x, y, arr)).to eq 6
  end
end

Three lines of test code. One of those is just initialize values. Nice!

-- Haskell
describe "Foo" $ do
  prop "adds stuff" $ \arr x y ->
    Foo.fooResult x y arr 
      `shouldReturn`
        x + y + length arr

The Haskell version even allows us to turn this into a QuickCheck property test now, which gives us way more confidence on the correctness. of this implementation.

Draw the input line?

Why not...?

# Ruby
class Foo
  def my_func(x, y, z)
    FooResult.insert x, y, z
    x + y + z
  end
end

describe Foo do
  it "uhh" do
    x, y, z = 1, 2, 3
    allow(FooResult)
      .to receive(:insert).with(x, y, z)
    expect(Foo.new.my_func(x, y, z)).to eq 6
  end
end

You might know about the law of Demeter, which is roughly is described as: the fewer expectations you have on your inputs, the easier your code is to deal with, and the less likely it is to be fragile. In this example, we just pass in the length of the users collection directly, removing that dependency entirely.

The code and the test fit on the same slide now! That's new.

Input effects are ezpz

# Ruby
def should_bill?(user_id)
  user = User.find(user_id)
  user.last_billing_date <= Time.now - 30.days
end

It's easy to resolve input effects. You just take the value you'd extract as a parameter. Does this function have any input effects? If so, what are they?

-- Haskell
shouldBill userId = do
  user <- Sql.get userId
  time <- getCurrentTime
  pure (
    lastBillingDate user 
      `isBefore` 
        time .-^ days 30
    )

Haskell makes it super easy to figure out whenever you have input effects. The IO type is basically "I do all the effects," and if you're binding out of IO or anything like IO, then you've got input effects. This code kinda cheats the answer for us: the user is an input effect, as is getting the current time.

Input effects are ezpz

# Ruby
def should_bill?(user, time)
  user.last_billing_date <= time - 30.days
end

-- Haskell
shouldBill user time =
  lastBillingDate user `isBefore` time .-^ days 30

We've extracted all of the input effects from the method. The method depends solely on the values we pass in as input now. The Ruby starts to look a lot like the Haskell!

Comparison of tests:

OK, so let's compare the tests of our methods, before and after extracting the input effects.

Implicit:

# Ruby
it "is true if older than 30 days ago" do
  user = double()
  user.stub(:last_billing_date) { 31.days.ago }

  expect(User)
    .to receive(:find).with(1)
    .and_return(user) 

  expect(should_bill?(1)).to be_true
end

We're overriding the global User name and making it return a stub. Since we can't even do this in Haskell I won't show the test, and because (let's be real) making our functions abstract in the functionality they do sounds boring.

Explicit:

# Ruby
it "is false if newer than 30 days" do
  user = double()
  user.stub(:last_billing_date) { 15.days.ago }

  expect(should_bill?(user, Time.now))
    .to be_false
end

These tests are pretty similar. Now we get to explicitly control the time that we're comparing against. We also don't have to worry about mocking the User class. We just create some object that responds to last billing date, and can provide our constraints.

Output Effects

# Ruby
class Foo
  def my_func(x, y)
    z = User.all.length       
    FooResult.insert(x, y, z)  # output effect!
    x + y + z
  end
end

So we've covered input effects and how we can push them out of methods. Output effects, unfortunately, aren't as easy to deal with. We'll follow a similar strategy. It's obvious how to convert an input effect into an input value. It's less obvious how to convert an output effect into an output values.

The Command Pattern

So the command pattern is a nice and convenient way to convert output effects into return values. Let's refactor the above code to use a Command.

A Command

# Ruby
class InsertFooResult
  attr_reader :x, :y, :z
  def initialize(x, y, z)
    @x = x
    @y = y
    @z = z
  end
end

First, we need a way to encapsulate the arguments to the effect that we want to have. In this case, we can create a read-only class that just carries those values around.

You want these to be values, because values are nice and easy. Making them a class tempts us to add extra functionality.

A value command

# Ruby
InsertFooResult = Value.new(:x, :y, :z)

-- Haskell
data InsertFooResult = InsertFooResult Int Int Int
  deriving (Eq, Ord, Show)

This is that values library I talked about previously. Highly recommended. It's even more concise than the Haskell!

Using:

# Ruby
class Foo
  def my_func(x, y, z)
    x + y + z, InsertFooResult.new(x, y, z)
  end
end

value, action = Foo.new.my_func(1, 2, 3)

-- Haskell
myFunc x y z = (x + y + z, InsertFooResult x y z)

Ok, so we're going to return the command value as one of the values returned from our method.

Ruby lets us return two things in a method, which is pretty cool. It implicitly returns an array of stuff, which you can then destructure when you call the method.

// Java
class Pair<A, B> {
    public final A first;
    public final B second;

    public Pair(A a, B b) {
        this.first  = a;
        this.second = b;
    }

    public static <A, B> 
    Pair<A, B> of(A a, B b) {
        return new Pair<A, B>(a, b); 
    }
}

You can do this in less flexible languages, it's just not fun or convenient. This is a Java implementation of a pair of values.

class Foo {
    public Pair<Integer, InsertFooResult> 
    myFunc(int x, int y, int z) {
        return Pair.of(
            x + y + z,
            new InsertFooResult(x, y, z)
        );
    }
}

All languages can mimic this feature by returning a single composite value, so don't fret if you're using Java or C# or PHP or whatever. Half of my dayjob is in PHP and I use these techniques there!

Now we can write a test for it:

Testing

# Ruby
describe Foo do
  it "does the thing" do
    expect(Foo.new.my_func(2, 3, 4))
      .to eq(
        [
          2 + 3 + 4,
          InsertFooResult.new(2, 3, 4)
        ]
      )
  end
end

We don't have to stub anything out. We're just comparing values to each other. This is super easy, basic, 2 + 2 level stuff. Our output effect is now a simple value.

I personally find this much nicer to look at than the previous code. We can compare how much complexity we're invoking, and how much of that is difficult to achieve in other languages.

describe "Foo" $ do
  prop "does the thing" $ \x y z ->
    myFunc x y z 
      `shouldBe` 
        (x + y + z, InsertFooResult x y z)

And here's the test in Haskell. Easy to express as a QuickCheck property, too!

Refactoring Example

Ok, now I'm going to demonstrate a quick example of refactoring some code with commands.

Business Logic

We're writing a subscription/billing system We have many products, and many plans for each product (small, large, enterprise) We only want customers to have one plan for a given product at a time

Here's a brief overview of the spec we've been given.

# Ruby
def subscribe(user, plan)
  user.subscriptions.each do |subscription|
    if subscription.product == plan.product
      subscription.cancel!
    end
  end

  user.subscribe! plan
end

Note:

This method takes a user and a plan that we want to subscribe them to. We're going to iterate over all the users current subscriptions and cancel anything on the same product. Then we finally subscribe them to that plan.

This is the super imperative algorithm that requires stubs, mocks, etc. to test. So we need to refactor it to have fewer effects, so the test story is nicer!

subcribe user plan = do
  for_ (userSubscriptions user) $ \subscription ->
    when (subProduct subscription == planProduct plan)
         (Stripe.cancel subscription)

  Stripe.subscribe user plan

Note:

So this is the same thing, but in Haskell. Haskell won't save you! You can write grossnasty code in Haskell, and it's only a little more inconvenenient than writing it in Ruby or PHP or whatever. 

# Ruby
Cancel    = Value.new(:subscription)
Subscribe = Value.new(:user, :plan)

def subscribe(user, plan)
  user.subscriptions.filter do |subscription|
    subscription.product == plan.product
  end.map do |subscription|
    Cancel.new(subscription)
  end.push(Subscribe.new(user, plan))
end

We start with the users current subscriptions. We filter that list, resulting in a list of the users subscriptions that match the new plan's product. These are the plans we want to cancel. We create a new Cancel command for each of these subscriptions. Finally, we push a new Subscribe command to the end of that list, subscribing them to the new plan.

-- Haskell
data SubscribeCommand
  = Cancel Subscription
  | Subscribe User Plan
  deriving (Eq, Show)

subscribe :: User -> Plan -> [SubscribeCommand]
subscribe user plan = cancels user ++ [Subscribe user plan]
  where
    cancels = map Cancel 
            . filter (\sub -> subProduct == product)
            . userSubscriptions
    product = planProduct plan

Here's the same thing in Haskell! It's just a pure function that maps a user and plan to a list of commands to execute. This is a pure specification of our business logic.

An interesting thing here is that we're returning an array of commands to execute. In any case, it's now really easy to set this up and write tests:

# Ruby
describe "subscribe" do
  it "should subscribe to an empty user" do
    user, plan = 2.times { double() }
    user.stub(:subscriptions) { [] }
    plan.stub(:product) { "foo" }

    commands = subscribe user, plan

    expect(commands)
      .to include(Subscribe.new(user, plan))

    expect(commands.length).to eq(1)
  end
end

So our tests are pretty nice now. We don't have to worry about the details of plan subscription. Perhaps that's handled through Stripe or some other provider. We don't have to mock that API or anything.

We are stubbing out objects here. In this sense, though, it is less that we are stubbing methods on global terms, and more that we are documenting the duck type that our method works with. Smaller duck types are easier to reuse and compose, so the less stubbing and test setup we have to do, the better.

-- Haskell
describe "Subscribe" $ do
  it "should subscribe to an empty user" $ do
    let user = fakeUser { userSubscriptions = [] }
        plan = fakePlan { planProduct = "foo" }

    subscribe user plan `shouldBe` [Subscribe user plan]

Here's that test in Haskell. It's basically the same thing, and a pretty clear declaration of our business logic. If we have arbitrary instances for our types, then we can do something similar:

-- Haskell
describe "subscribe" $ do
  prop "should subscribe an empty user" $ \user plan -> do
    let user' = user { userSubscriptions = [] }
    subscribe user' plan `shouldBe` [Subscribe user' plan]

Since our commands and inputs are just dumb data, we can easily generate arbitrary ones to get confidence that we're doing the right thing, as well as ignoring stuff that's incidental to what we're working on.

# Ruby
describe "subscribe" do
  it "cancels a related plan" do
    user, plan, old_sub = 3.times { double() }
    old_sub.stub(:product) { "foo" }
    plan.stub(:product) { "foo" }
    user.stub(:subscriptions) { [old_sub] }
    commands = subscribe user, plan

    expect(commands)
      .to include(Subscribe.new(user, plan))
    expect(commands)
      .to include(Cancel.new(old_sub))
    expect(commands.length).to eq(2)
  end
end

We can easily write tests for more complex business logic too. Here we're verifying that our subscribe method cancels old plans on the same product.

Discard Stubs,

Acquire Values

# Ruby
User         = Value.new(:subscriptions)
Subscription = Value.new(:product)
Plan         = Value.new(:product, :amount)

OK, so if you're really anti-stubbing, then you hopefully are just using dumb value classes. If you do that, then you don't need to stub anything out at all. Here's our minimal data model.

# Ruby
describe "subscribe" do
  it "cancels a related plan" do
    sub  = Subscription.new("foo")
    user = User.new([sub])
    plan = Plan.new("foo", 123)

    commands = subscribe user, plan

    expect(commands)
      .to include(Subscribe.new(user, plan))
    expect(commands)
      .to include(Cancel.new(sub))
    expect(commands.length).to eq(2)
  end
end

And here are our fancy new stub-free tests.

Property Testing

Remember adding? We wanted to verify properties about it, so we generated a bunch of random values and asserted that the property held for all of the values we generated.

We can describe properties of our business logic too. Then we can generate random values and assert that they hold.

# Ruby
describe "Add" do
  it "is commutative" do
    100.times do
      x, y = 2.times { Random::rand }
      expect(x + y).to eq(y + x)
    end
  end

  it "is associative" do
    100.times do
      x, y, z = 3.times { Random::rand }
      expect((x + y) + z).to eq(x + (y + z))
    end
  end
end

So here are the property tests for adding two numbers. We can recover a more natural language specification for these properties as:

Properties:

Subscribing a user to a plan should always issue a subscribe command for that plan Subscribing a user to a plan should always issue cancel commands for subscriptions on the same product Subscribing a user to a plans should not alter plans on other products 
# Ruby
describe "subscribe" do
  it "always subscribes to given plan" do
    forall_users_and_plans do |user, plan|
      expect(subscribe(user, plan))
        .to include(Subscribe.new(user, plan))
    end
  end
end

So this property generates a bunch of random users, a bunch of random plans, and asserts that the commands always include a subscribe command. We'll assume that the generator makes sensible choices.

# Ruby
describe "subscribe" do
  it "always cancels related plans" do
    forall_users_and_plans do |user, plan|
      cancellations = subs_on_product(
        user.subscriptions, plan.product
      ).map { |sub| Cancel.new sub }

      expect(subscribe(user, plan))
        .to include(*cancellations)
    end
  end
end

This property tests that we always cancel related plans.

# Ruby
describe "subscribe" do
  it "doesn't cancel unrelated plans" do
    forall_users_and_plans do |user, plan|
      unrelated_cancels = subs_not_on_product(
        user.subscriptions, plan.product
      ).map { |s| Cancel.new(s) }

      expect(subscribe(user, plan))
        .to not_include(*unrelated_cancels)
    end
  end
end

Finally, we can test that we don't cancel any plans that aren't related to the one we're subscribing for. This technique is crazy powerful for verifying the correctness of our business logic.

Property Testing

QuickCheck is the original property testing library, written in Haskell. There's an interesting paper on how it's used that I'd recommend reading. QuickCheck has been ported to many other langauges, though it's somewhat less pleasant because you need to specify generators explicitly and many langauges don't encourage working with easily testable values in the same way that Haskell does.

Using Commands

Now that we've got all these commands, how do we actually use them? There are a number of ways we can do this, in increasing complexity/power. As with most things in software development, it's best to ask for as little power as you need. Shooting yourself in the foot hurts a lot less if you only have a water gun.

Command Interpreter

class FooResultInterpreter
  def call(command)
    FooResult.insert(
      command.x, command.y, command.z
    )
  end
end

For each command, you implement an interpreter. Here's a basic interpreter for the FooResult class.

Single Command

value, action = Foo.new.my_func(1,2,3)

InsertFooResultInterpreter.new.call(action)
# or,
TestInsertFooResultInterpreter.new.call(action)
# or,
RedisFooResult.new.call(action)

So, if you've got a single command, this is the easiest. You take that object and you pass it to the interpreter of your choice. You can easily define many different varieties of interpreters for a given command.

class StripeSubscriber
  def call(command)
    Stripe::Subscription.create(
      command.user, command.plan
    ) 
  end
end

class InternalSubscriber
  def call(command)
    InternalBilling.create_subscription(
      command.user, command.plan
    )
  end
end

These classes share the same interface and can be used interchangeably. So we can easily pass a single command to them and have it be executed.

Running Multiple Commands

That covers the single command case, like our toy example. What about multiple commands, like our more involved refactoring example? Since we're dealing with simple values, we can use simple functions to chain commands. We'll start with a simple case: multiple of the same command.

commands = [
  Subscribe.new(x, y), 
  Subscribe.new(z, a),
  Subscribe.new(b, c)
]

# answer?
# wat do
# halp

Ok, so we've got an array full of Subscribe commands. And we want to actually execute each one. Does anyone have a suggestion on how to do this?

interpreter = StripeSubscriber.new
commands = [
  Subscribe.new(x, y), 
  Subscribe.new(z, a),
  Subscribe.new(b, c)
]

commands.map do |command| 
  interpreter.call command
end

We can simply iterate over the commands, and for each command, call it through the interpreter.

Dumb Data

Commands are just dumb data. Since they're dumb data, we can easily stuff them in data structures are do interesting things to them. While the above example just used an array, you could easily have a lazy list of commands, or a binary stree, or a hash, or whatever.

Optimizing Commands

Since we have introduced a data layer between business logic and execution, we can easily optimize command sequences.

InsertFooResult

commands = [
  InsertFooResult.new(1, 2, 3),
  InsertFooResult.new(4, 5, 6),
  InsertFooResult.new(7, 8, 9)
]

commands.map { |cmd| interpreter.call cmd }

Here's our naive logic. It works. But it's inefficient! We issue three SQL queries here, when we could save a tremendous amount of time by doing a bulk insert.

Let's optimize this.

def optimize(commands)
  inserts, rest = commands.partition do |command|
    command === InsertFooResult
  end

  rest.push(BulkInsert.new(inserts))
end

Ok, so here we're going to partition the commands into two lists; the first is one where the command's class is an InsertFooResult. The second list is all of the other commands. We return the list of non-insert commands with a BulkInsert command appended to the end.

Distinct Commands

Subscribe = Value.new(:user, :plan)
Cancel    = Value.new(:subscription)

commands = [
  Cancel.new(:foo),
  Cancel.new(:bar),
  Subscribe.new(:lol, :baz)
]

# wat do?

We can't just map over this with our StripeSubscriber, because it doesn't know how to handle Cancel commands. Fortunately, it's real easy to compose two command interpreters.

Composing Interpreters

class StripeCancel
  def call(command)
    Stripe::Subscription.cancel(
      command.subscription
    )
  end
end

class StripeSubscribe
  def call(command)
    Stripe::Subscription.create(
      command.user, command.plan
    )
  end
end

Here are our cancel and subscribe command interpreters. Super basic, no logic, easy to read, test, understand, etc. Now let's compose them:

class CancelOrSubscribe
  def call(command)
    case command
    when Subscribe
      StripeSubscribe.new.call(command)
    when Cancel
      StripeCancel.new.call(command)
    else
      raise UnkownCommandError.new(command)
    end
  end
end

Ruby lets you use case..when syntax against the classes we're comparing against. So when the command is a Subscribe (or a subclass of Subscribe), we'll use the Stripe Subscriber. When it's Cancel, we'll call the stripe canceller. Otherwise we throw an error.

interpreter = CancelOrSubscribe.new

commands = [
  Cancel.new(:foo),
  Cancel.new(:bar),
  Subscribe.new(:lol, :baz)
]

commands.map do |command|
  interpreter.call command
end

Here's how we solve the problem of multiple classes in our command array.

class CancelOrSubscribe
  def initialize(canceller, subscriber)
    @canceller  = canceller
    @subscriber = subscriber
  end

  def call(command)
    case command
    when Subscribe
      @subcriber.call(command)
    when Cancel
      @canceller.call(command)
    end
  end
end

We can make this more generic by passing the specific canceller and subscriber in.

stripe_manager = CancelOrSubscribe.new(
  StripeCanceller.new, StripeSubscriber.new
)

internal_manager = CancelOrSubscribe.new(
  InternalCanceller.new, InternalSubscriber.new
)

Now, it's pretty easy to construct interpreters for sets of commands. However, I'm still not entirely happy with this. I want to describe a generic structure that accepts a mapping of commands to their handlers.

stripe_manager = ComposeInterpreters.new(
  {
    Cancel => StripeCanceller.new,
    Subscribe => StripeSubscriber.new
  }
)

Now, this is easy! We just pass a simple hash in, and our composition is done.

class ComposeInterpreters
  attr_reader :handlers

  def initialize(handlers)
    @handlers = handlers
  end

  def call(command)
    handlers[command.class].call(command)
  end
end

The implementation is even a lot simpler. And we can easily compose existing composed commands using Ruby's hash merge methods:

stripe_handlers = { 
  Cancel    => StripeCanceller.new,
  Subscribe => StripeSubscriber.new 
}
user_handlers = {
  Delete => StripeUserDelete.new,
  Create => StripeUserCreate.new
}

stripe_manager = ComposeInterpreters.new(
  stripe_handlers.merge(user_handlers)
)

Here we have two hashes of handlers. One handles subscriptions, with a canceller and a subscriber. The other handles users, with a delete and create command. To compose them, we can just use Ruby's hash merge method.

Another kind of composition?

This covers a way to extend interpreters to handle multiple distinct commands. What if we also want to assign multiple handlers for a command? Suppose we want to additionally log every subscription and cancellation that happens.

class CommandLogger
  def call(command)
    puts command
  end
end

How can we combine our handlers? We extend our handlers to require arrays of handlers, not just a single one.

class ComposeInterpreters
  def initialize(interpreters)
    @interpreters = interpreters
  end

  def call(command)
    @interpreters[command.class].map do |interpreter|
      interpreter.call(command)
    end
  end
end

Now, when we're composing commands, we can still do hash merge, but we'll want to control how merging occurs. By default, merging hashes in Ruby considers a collision to be an update. Instead, we want it to be list append.

def compose_handlers(commands_1, commands_2)
  commands_1.merge(commands_2) do |_, c_1, c_2|
    c_1 + c_2
  end
end

Now, we can easily merge our logging command with our other commands.

stripe_handler = {
  Cancel    => [StripeCanceller.new, Logger.new],
  Subscribe => [StripeSubscriber.new, Logger.new]
}

Final Boss

Okay, so that's the easiest and most common form of the command pattern. We can use our easy to test, easy to understand, and easy to read functions to generate a list of actions to take, and then have dead simple handlers to interpret the commands into real actions.

There's something missing, though.

Changing mid course?

What if we want to change our commands based on a return value of a handler? Let's look at the following code.

def charge_user(user)
  user.subscriptions.each do |subscription|
    if user.balance? >= subscription.price
      user.charge!(subscription.price)
    else
      send_balance_notice!(user, subscription)
    end
  end
end

I've used the question mark to indicate input effects, and the exclamation mark to indicate the output effect. We iterate over the users subscriptions, and if the user's current balance is at least the subscription price, then we charge the subscription to their account.

We can't just take the balance as a parameter, because it changes during execution.

def subs_to_charge(user)
  balance = user.balance?
  result = []
  subscriptions.each do |subscription|
    if balance >= subscription.price
      result << subscription
      balance -= subscription.price
    end
  end
  result
end

We could create an array of subscriptions to charge for based on the users current balance, and then run charge! on those subscriptions.

def charge_user(user)
  charges = subs_to_charge(user)
  charges.each do |subscription|
    user.charge!(subscription.price)
  end

  # Array difference:
  subs_to_notify = user.subscriptions - charges
  subs_to_notify.each do |subscription|
    send_balance_notice!(user, subscription)
  end
end

This is about as nice as that looks. It's still not great. Let's use the command pattern.

Identify Commands

Charge =
  Struct.new(:user, :amount)

SendBalanceNotice = 
  Struct.new(:user, :subscription)

UserBalance = 
  Struct.new(:user)

Our three commands are to Charge the user, send a balance notification, and query the users balance. The query class is new. It represents a query that we want to make. It's like an input effect, but it is supposed to change throughout the execution of the plan.

After identifying commands, we need an interpreter. The interpreter is pretty easy to create, even with this new restriction.

Functionally speaking, the interpreter is mapping over the structure of our computation. They're "maps". We need a way to construct our action plans dynamically, without having access to the actual values.

Allowing input effects...

def charge_user(user)
  user.subscriptions.map do |subscription|
    if user.balance? >= subscription.price
      Charge.new(user, subscription.price)
    else
      SendBalanceNotice.new(user, subscription)
    end
  end
end

If we allow input effects, then we just build the command list as normal. This can be an OK solution if your input queries aren't too complicated or difficult to deal with. However, we really don't want to have to deal with that. So let's figure out how to build our dynamic instructions without actually running the effects.

What to do next?

[ action_1, action_2, action_3 ]

With the array mapping, we answer "What do we do next" by taking the next item in the array. If we want to have a more dynamic structure, then we can't be using an array. Let's add an :and_then parameter to each command.

Next!

Charge = 
  Struct.new(:user, :amount, :and_then)
        #    ^      ^        ^
        #    |      |        +-- output
        #    +------+----------- input

So, this is a new Charge command that has an and_then parameter to specify what happens afterwards. What is "next" going to look like? Well, Charge doesn't have a meaningful output, so the next thing in the sequence won't take any input. Since we're talking about the next action to take in our sequence, it should be a command.

And then, ...

UserBalance =
  Struct.new(:user, :and_then)
        #    ^      ^
        #    |      +-- output
        #    +--------- input

This is our new User Balance struct. It has an and_then parameter also. This parameter will get access to the balance returned from the command when it is finally executed, and will use that information to construct the next action in the sequence.

Finally!

Return = Struct.new(:result)

We need to have some way of terminating the chain of execution. This class does it. We call it Return because we can think of it like the return keyword in imperative programming: "stop executing and return this value".

A single charge

Let's look at how this is with a single charge.

def charge_or_email(user, subscription)
  UserBalance.new(user, -> (balance) do
    if balance >= subscription.price
      Charge.new(user, subscription.price, -> _ do
        Return.new nil
      end)
    else
      SendBalanceNotice.new(
        user, subscription, -> _ do
          Return.new nil
        end
      )
    end
  end)
end

So, we create our first command. We check the user balance. The first parameter is the user, and the second parameter is a function that accepts the balance and determines what to do next. If the balance is greater than or equal to the price, then we create a Charge command on the user with that price, and finally Return the user. Otherwise, we create a SendBalanceNotice command with the user and the subscription.

Interpreting

Interpreting these commands is relatively straight forward.

class UserBalanceStripe
  def run(command)
    user = command.user
    balance = user.balance?
    command.and_then.call(balance)
  end
end

For the user balance, We extract the user from the command, call Stripe to get the user's balance, and pass the balance to the command's "And then" method. Finally, we return the command that is generated from this lambda.

class UserBalanceLocal
  attr_reader :balances

  def initialize(balances)
    @balances = balances
  end

  def run(command)
    balance = balances[command.user.id]
    command.and_then(balance)
  end
end

We can also create a mock interpreter, that lives locally. Here we initialize it with a mapping from user IDs to their balances, and when we receive the command, we just check that hash map.

class ChargeStripe
  def run(command)
    command.user.charge!(command.subscription.price)
    command.and_then.call(nil)
  end
end

For charging against stripe, it's basically the same thing. We call the charge method on the command's user, and then generate the next command in the sequence.

Put it all together

Now, our functions are returning the next thing in the sequence. But we haven't put them all together yet. Whereas the older command pattern used a "map" to go over the commands, we need to "reduce" over them.

Reduce

[1, 2, 3]                    #collection
  .reduce(0,                 # starting value
  -> (accumulator, next_value) do
    accumulator + next_value # reducer
  end
)
# => 6

Let's recap reducing, since it can be difficult. Reducing a collection takes a combination function, a starting value, and a collection of things to reduce. We take the first item in the collection, combine it with the starting value using our combination function. That gives us a new reducing value, which we then combine with the second element of our collection.

Illustrated

Since reduce can be a bit tricky to get at first, here's how it evaluates.

f = -> (acc, x) { acc + x }

[1, 2, 3].reduce(0, f) 

[2, 3].reduce( f(0, 1), f)
[2, 3].reduce( 1, f)

[3].reduce(f(1, 2), f)
[3].reduce(3, f)

[].reduce(f(3, 3), f)
[].reduce(6, f)

6

We start by defining our combiner, which just adds the two numbers. We pluck the first number off the array, pass it to the combining function with the initial accumulator 0, and that becomes our new accumulator. We continue plucking the first element off and combining with the accumulator until the array is empty. Once the array is empty, the accumulator is returned.

Reducing Command Trees

class Interpeter
  attr_reader :handlers

  def initialize(handlers)
    @handlers = handlers
  end

  # ...

This can be a bit tricky to think about. Like the ComposeCommand class we made earlier, we've got our hash of handlers that we're initialized with.

  # ...

  def interpret(command)
    case command
    when Return
      return command.result
    else
      handler = handlers[command.class]
      next_command = handler.run(command)
      self.interpret(next_command)
    end
  end
end

We have two cases: If the command is a Return, then we terminate execution and return whatever the result of the command is. Otherwise, we get our handler for the given command, and run it with the command. This gives us a next command to work with. We then recurse on the function, calling interpret on the next command.

Grafting Trees

There's one last piece of the puzzle. We need a way to compose two command trees. That is, given some command sequence, we need to have a way to say "And after you do all of this stuff, run this other sequence of commands."

def charge_or_email(user, subscription)
  UserBalance.new(user, -> (balance) do
    if balance >= subscription.price
      Charge.new(user, subscription.price, -> _ do
        Return.new(user) 
      end)
    else
      SendBalanceNotice.new(
        user, subscription, -> _ do
          Return.new(user) 
        end
      )
    end
  end)
end

Here's our command structure for deciding whether to charge or email a user. Now, we need some way to compose it, so we can iterate over all of the users subscriptions and run this command tree.

We could naively generate a list of these commands and then run the interpreter over each one individually.

Or we could bind the sequences together!

Return = Terminate

We terminate execution when we use the Return command. So we should be able to bind two trees together by replacing the Returns in the original tree with the sequences we're building out.

def Interpreter
  attr_reader :binds
  def initialize
    @binds = []
  end

  def bind(next)
    @binds << next
    self
  end

  # ...

First, we create an empty list of binds, and we expose a function for binding another command to our interpreter. These binds are going to be the same thing as the and_then parameters we've been dealing with: a function that returns a command.

  def interpret(command)
    case command
    when Return
      result   = command.result
      and_then = binds.shift

      if and_then
        interpret(and_then.call(result))
      else
        result
      end
    else 
      # ...

We want to identify the Returns, or ends, in our current command structure. If we have a return, then we want to take the first "next path" out of our array, and interpret that path. Otherwise, we just return the result of this and we're done.

def charge_user(user, subscriptions)
  interpreter = Interpreter.new
  user.subscriptions.each do |subs|
    interpreter.bind -> {
      UserBalance.new user, -> (balance) {
        if balance >= subs.price
          Charge.new user, subs.price, -> {
            Return.new nil
          }
        else
          SendBalanceNotice.new user, subs, -> {
            Return.new nil
          }
        end
      }
    }
  end
end

So this is how we'd write that with our new command pattern.

The syntax is a little ugly. Let's write some helpers that clean it up.

nothing = -> _ { Return.new nil }

def charge(user, amount)
  Charge.new(user, amount, nothing)
end

def send_balance_notice(user, subscription)
  SendBalanceNotice.new(
    user, subscription, nothing
  )
end

def user_balance(user)
  UserBalance.new(user, nothing)
end

So these functions make up our "empty constructors." These trees just do one thing, and then they return nothing.

Improved Syntax!

def charge_user(user)
  user.subscriptions.reduce(Interpreter.new) 
    do |commands, subscription|
    commands.bind -> _ do
      user_balance(user)
    end.bind -> (balance) do
      if balance >= subscription.price
        charge(user, subscription.price)
      else
        send_balance_notice(user, subscription)
      end
    end
  end
end

So this syntax is a good bit nicer. It's not as clean as the plain ol' imperative syntax, but it gives us something much more powerful.

The return of this is a data structure. It's a series of instructions that we can carry out and interpreter however we like.

All we have to do is provide the right interpreters for each of the individual commands, and the code will glue and compose super nicely.

def charge_user(user)
  user
    .subscriptions
    .reduce(Interpreter.new) do |cmd, sub|
      cmd.bind -> _ do
        charge_or_email(user, sub)
      end
    end
end

Here's the same code that uses the function we defined earlier, demonstrating that it's pretty easy to compose code that uses this structure.

Intrigued?

It's a free monad

If you want to learn more about this final evolution of the command pattern, the theory behind it is called the free monad. It's super interesting and very powerful. Unfortunately, most typed languages can't express it well. Haskell and Scala both have great free monad libraris, but Java/F#/C#/etc. are unable to express these things.

Dynamic languages aren't restricted by types, but it's on you to ensure everything plugs together well.

The End

Matt Parsons

Seller Labs