Unretrofittable: Starting off right – Some abstractions are hard to (un|re)do...

View Github Repository Open presentation in a new window

cwinters

See all presentation from cwinters

Unretrofittable: Starting off right – Some abstractions are hard to (un|re)do...

0 0

unretrofittable-201406

On Github cwinters / unretrofittable-201406

Unretrofittable: Starting off right

Some abstractions are hard to (un|re)do...

Chris Winters (@cwinters) - ModCloth

Pittsburgh Techfest 2014

goo.gl/J4Tcoq

Things start out great

Greenfield projects are the best, right?

Starting a new project is one of my favorite things. Probably yours too. People start new jobs just to do new stuff. So many things to play with, so much to discover.
Gift to work on new! You don't know how lucky you are! Audience Q: How many in last year? Every? New/rewritten subsystem?

It all made sense at the time

Why awesome? No usual mistakes -- troubling bugs, difficult users, interface with terrible awful systems, compromises work org, dreadful assumptions, hidden agendas... But... many are good ideas at the time! Nobody wants crap -- just willing to trade off for it.
Thing is: I will do them all. You will too. And I'll forget all these lessons because SO EXCITING NEW SYSTEM.

Does it matter why?

Lots of reasons, that's another talk. Now: our levels of abstraction, side effects of working with right and wrong ones.

"agile" vs AGILE

We love "agile", maybe not capital A Agile. Lot of it geared toward making developer acutely aware of all joy + pain, tightening feedback between awareness and influence on them.

What is your job?

Help others be awesome

...what we do is a means, not an end

? TAB Most of us don't work on software for ourselves. (Audience Q) TAB Hard to remember because the means is fun! Argue about editors and frameworks! But like Dustin said this AM the end is what we're here for.

Change as change does

Biology definition of "stable". We create complex systems. Not quite life, but not bridges either. Our main technical lever: make it easy to change.

Empathy

Change ++ surface area for empathy. Theme at devopsdays -- with co-workers, other organization parts, but mostly with users. Last is hard because you rarely see them, hardwired for tribes + proximity. So anything that helps is awesome.

Frying pan?

But you probably didn't sign up for people feedback loops, right? We have analysts and product manager to take care of these things, right? Nope, you're FULL STACK, baby. That includes the user.

Empathy with doing, empathy with tasks

Role of our systems in user's work / life More: what users DO. Leads to assertion: Focus on tasks as organizing abstraction will make your system better at what it does. Thinking about tasks user POV not only happier users, clarity on what we should actually be focusing on.

Also: side effects

Sometimes the main benefit, sometimes a risk or something to be mitigated, and sometimes can lead to other side effects or enable other practices.
Take a drink! (~5 min)

Lever 1: Creating a usable system

One command to create a system with real(-ish) data.

How can we manifest this empathy with the user?
Seriously! Everybody ignores. Happy to limp along with data we've built up. Show system to family: data real? real life user names? products have generic photos? history -- was everything created the same day? what happens when you run a report?

Hard part: real(-ish)

"But I have fixtures!"

One command is easy. Hard part is looking real. TAB Oh, you have fixtures...?

Can your domain experts edit them?

Fixtures are bare minimum. Write once read never. Relational data without SQL is awful. In YAML files? LOL They work table by table. Because objects mapped to tables, data flow directly into them. THIS SHOWS THE PROBLEM.

Abstractions shape ideas

Storage shapes how we present and think about our domain. How many tools exist to navigate this path? How many apps have you seen that just barf relations to user? "They just make it easy" is problematic. People do what's easy.

So many forms

Example: hospital discharge nurse. Discharge is A Thing to her. To you? One or two dozen tables. How to make that real? How can you hope to rope domain experts like her into defining your data for you?

Thinking in tasks

Users don't think in normalized tables. Don't care about efficiency, or query planners. Do care about duplication, but less than you think. Do care about completeness, doing very little mapping their brain to The System. Our job to mediate between tasks and metal. But we focus on metal. Should focus on tasks. Metal is what we read, argue about, school for, care about.

Tasks will cause you pain

Fowler: "If it hurts, do it more often."

This path is non trivial. Even painful. But this pain is good! Data too complex and hard to recreate give pain early. Pointer to dissonance between user view and your view. TAB Ideal world pain will induce change, most often we callus up.

Side effects, you

Quick startup for new hires or changing teams. Which means people can change teams more often!

Head off model drift and too-generic data models

Side effects, others

Domain experts define and maintain data

Sales + marketing can create and customize demos

Potential users can create demos with their own data (woo cloud!)

Take a drink! (~13 min)

Lever 2: Audit

Seriously?! There aren't many terms that evoke enterprise software more than "auditing". TURNS OUT auditing has great first and second-order effects to make lots of people (including you) happy.

Why do you even?

Answer "How did this thing get this way?" at any time. Who created it with what data, who updated it and when? Can be invaluable for support staff, though reciting back to a user her actions may be Big Brother

Cooler than it sounds

To outsiders: auditing adds determinism. Think about how nutty our systems look to them. Auditing helps our crazy system finally MAKE SENSE. Not just a bunch of people guessing or shrugging and offering no help.

More abstractions

Most frameworks hook into persistence. Attractive: leveraging tool for orthogonal concern -- a big reason we use frameworks!

Refactoring resistant

Can work ok, but resist refactoring. How to represent *logical* record when history is tied to *physical* record and physical storage changed?

Record tasks, not results

Traditional auditing: record results. What if we record actions themselves? Similar to loading data: reuse those ideas and attach 'who' and 'when' and 'from where' metadata about these actions.

What could we do with that record?

Not only auditing, but replay. Replay for load, replay for modified business logic, replay for debugging to get back to a point in time. Maybe replay done elsewhere to run through an expensive computation (risk profile) -- see Fowler's article on LMAX.

Other side effects

Confidence in the system: things can be explained!

Exposing intent -- RecoverPassword vs ResetPassword and SendPasswordRecoveryEmail

Take a drink! (~19 min)

Lever 3: Undo

Lastly, we'll focus on an activity that's almost entirely end-user focused. Undo.
Users expect small undo, not large. Undo font change or typing: just exist. Undo order? Hospital admission? Import a school of students? MAGIC. The sort that users will LOVE YOU for. People flip out that they can undo gmail deletions. What if they accidentally import a spreadsheet of 500 students to the wrong school and that can click a button to make it better?

Again with the tasks

But they expect undo of actions, don't care about side effects in your system. Undo should mean it's like the action never happened. The form went in the shredder, the file never uploaded. Couple of tricks depending on action. Both easier to do at start.

Grace period

Wait a while to do your work, don't do it if asked

I place an order, but cancel without consequence in the half hour after. (Efficiencies in logistics can actually make this more difficult, ask Amazon. Simple on surface: do nothing until period up. I like doing nothing!

No free lunch

Actually: everything EXCEPT the final action. e.g., validation logic must fire so user doesn't get "Order invalid" email half hour later. Inventory needs to be up to date. Tricky: validation dependent on earlier persistence. Transaction fix? Maybe... None of this is impossible, just easier if you have it in mind from the beginning

Change as Commands

Represent change as commands, represent undone change as commands

Woohoo! Already user task abstraction. Great handle to UNDO task. Commands that touch many you have to associate... but you've already done! You want to treat it as A Thing and redisplay or enable editing.

Side effects

Users can experiment! Less fear! So much happiness!

Take a drink! (~25 min)

Lever n: Feature flags

Compile/build setting controlling feature accessibility

Compile/build is typical, not mandated; it speaks more to their typical scope. Who's using them?

Why?

Timing features to releases is harder than you'd think, particularly the older and larger your system gets. (Yes, this is a good argument for lots of small systems. That's another talk.)
Dependencies usually aren't this bad (crazy graph), but they can be tricky. Feature X may require an update to a database table but Feature Y also needs to update it, and the updates need to be done at the same time. BUT Feature X needs to be one two weeks before Feature Y.

Thinking differently about features

Full functionality vs not breaking

Instead of thinking of features as a discrete set of files implementing a feature that get delievered (sorry, "merged") at once, what if it were a set of changes that get delivered over time -- maybe even a long time -- but none of which break existing functionality?

Not the prettiest...

Most implementations can be kind of gross... See Kellan Elliott-McCrea's presentation on Change at Etsy for an example. (PHP, but still...)

I'll just code that up...

You don't have to! (Flip, Setler, Togglz)

May even be able to use third party runtime A/B testing services for lightweight version.

But I want clean!

Having part of a feature's code released is okay! (Koan: is a feature even partially released if it cannot be used?)

Side effects++

Release more often; A/B testing

Releasing more often is good in and of itself. But it also means you're releasing smaller changes, which are generally less risky. Lowered risk increases development experimentation. And that can bubble up the organization in great ways.

Related: Kill switch

Who's using these?

You might also know me...

Runtime flag removing functionality due to disruption, planned or not

Credit card processor went down, some backend system has collapsed and we don't want to continue throwing traffic at it, etc. Results can be reduced functionality, or may even be hidden to user -- for example, orders are put into a queue and not touched but we can catchup later.

Wouldn't read-only be nice?

Why? Enabling schema change for broken databases

This could help you work around database shortcomings (MySQL 5.5, how do substantial systems use this!?) by ensuring that you can keep your app up during blocking schema changes.

Side effects

<discuss>What actually happens when things fail?</discuss> vs

Most of the time these discussions are much better handled by other parts of the business, not you. Giving an easily understood hook into the ability to turn things off leads to discussions of what we should actually do when they're off.

Others to think about

  • Start production sequences at 1000 so have an easy 'this is special' hook (id < 1000). (REI Outlet items, Larry Wall Huffman coding)
  • Avoid 'magic' values your platform might have (woe be the user with ID 4 in a ruby system...)
  • Extract metrics via API: generic performance as well as performance specific for your app (Carol's example from devopsdays)
  • Classify and contextualize errors - enable that intent you've exposed via your user actions to people troubleshooting later on. (YOU NEVER DO THIS LATER)

Others to think about

  • Validation
  • Act-as-user for internal admins and customer support
  • Authen/Authz
  • Schema updates
  • Caching hooks everywhere

Others to think about

  • Version internal dependencies to enable change (weight documentation example)
  • URI creation
  • Serialization and marshalling
  • Timezones!
  • "It's easier to make a stream system batch than make a batch system stream." (@dehora)

Thank you!

@cwinters - goo.gl/J4Tcoq

References

Photos