Constantly Start Over

Never Rewrite

Constantly Rewrite

@mattjmorrison

Constantly Start Over

What are the Best Projects?

Green Field

Green Field == Starting Over

Some rights reserved by Robert Crum

Why?

So Many Reasons

No Technical Debt

No Legacy Code

Chance to Learn from Your Mistakes

Chance to Choose

• New Languages
• New Platforms
• New Frameworks
• Updated Versions

The Unfortunate Reality

Is That Most of Us Work In a…

Brown Field

Some rights reserved by Lauren Tucker Photography

The Trick is Turning

Your Brown Field Green

Not as Simple as Watering

What Makes a Brown Field Brown?

Many Reasons, But Mostly

Something That I Call…

The Monolith

Some rights reserved by p-a-t-r-i-c-k

What is The Monolith?

It’s One Large Code Base

It Does All the Things

Monolith Advantages

• All the Things in One Place
• Simple to Get Started
• Easy to Reuse Code
• Simple Networking & Hardware Needs
• Simple Deployment

Neutral Fact

Many Frameworks Have Monolithic Tendencies

• Rails
• Django
• «Insert Full Stack Framework Here»

Caveat - Rails Engines & Django Apps - Use These

Monolith Disadvantages

• It’s Big, Scary and Causes Nightmares
• Difficult to Fit in Your Head
• Merge Conflicts
• Slow Tests (because there are LOTS of them, right?)
• Gaps in Coverage (100k LOC @ 96% = 4k Untested LOC) *
• Difficult to Leverage Open Source Community
• Easy to Introduce Unnecessary Coupling
• You Don’t Know You’re Building One Until It’s Too Late

  * MS DOS 1.0 was 4k Lines of Assembly in 1980

What Does the Monolith Have to do With a Brown Field?

Wikipedia: Expansion or redevelopment of (brownfields) may be complicated by environmental contaminations.

The Monolith is a Contaminated Environment

coders anonymous

Hi, my name is Matt

and I’ve built Monoliths

Let’s Travel into the Past

It is Now 2009

Chesley B. Sullenberger III Saves 155 Lives by Crash Landing in the Hudson River

And We Begin Building a New Web Based System

Fast Forward to 2012

Robotic Rover Curiosity successfully lands in Gale Crater on Mars

And Our Web Based System has Grown Significantly

• Customer Management
• Customer Usage Reports
• Manage Users
• Authentication
• Authorization
• Display Customer Data
• Generate Reports for Customer Data
• Create New Business
• Work Flow Management
• Integrate with Geo Coding Services
• Integrate with Multiple Property Valuation Services
• Integrate with Multiple Reporting Agencies
• Manages Attaching Documents
• Migrating Legacy Data into Database
• Exporting Data into Multiple 3rd Party Systems
• More…

It Does Too Much

• Everyone Fears the Monolith
• Everyone Tries to Avoid Changing It
• Bugs are Explained Away
• Bugs are Shrugged Off

Time for a Rewrite

Never Rewrite

An Entire System

What value is there in a Rewrite?

• Cleaner Code
• Better Techniques
• Fewer Bugs
• Better Test Coverage
• Newer, more up-to-date Technologies
• You know everything
• Green Field, Baby!

What Business Value is there in a Rewrite?

• Long Time to Market
• Fewer Features
• Training Users on a New System
• Second System Syndrome
• Existing Production Support

So Then, What Do You Do?

Make Smaller Pieces

If You Do Not Have One Giant Code Base You Will Not Have One Giant Rewrite

And What About the Monolith?

Iterate

As opportunities arise…

• Add Tests & Refactor
• Extract Services
• Extract Libraries
• Extract Applications
• Integrate the Old with the New

Let’s Take a Look at Some Examples

Extract Services

Example

What is a Service?

A Thing in the Cloud that You Can Use

• Some Code Deployed Out on the Web Somewhere
• Exposes an API to Allow Manipulation or Retrieval of Data
• Clients Need to Know Very Little About It

New Vendor Integration

• UI To Order Reports From Vendor
• UI To Display Report Results
• Use Report Results in Existing New Business Application

Our Choices

Add to the Monolith?

Boo, Brown Field

- or -

Start Over?

Yay, Green Field

Start Over

• New Project
• New Code base
• New Database
• New Web System with an API
• Green Field, Baby!

Roadblock #1

Authentication

Easy with a Monolith

Not So Easy with Multiple Servers

Solution?

Start Over

Green Field, Baby!  

• New Project
• New Code Base
• New Database
• New Web System with an API
• Replace Authentication in Monolith with Calls to the New Authentication API
• Migrate Existing User Data to New Database

Roadblock #2

Customers

Again, Easy with a Monolith

And Again, Not So Easy with Multiple Servers

Remember that Unnecessary Coupling Thing?

Solution?

Start Over

Green Field, Baby!

Where are we now?

Where We Ended Up

Pros

• Smaller Pieces with Fewer Responsibilities
• Pieces are More Loosely Coupled
• Pieces Can be Updated in Isolation
• Continuous Reevaluation of Technology
• Fewer Technology Restrictions
• Lots More Green Field Work

Cons

• Slow to Get Started
• No Out-of-the Box Solution
• More Complex (or at least more) Deployments
• No Single Place to Find All Answers
• Still a Bit of Work in the Brown Field
• Performance - DB Join is Now an HTTP Request

A Few More Things About Starting Over

It Sounds Daunting, But the More You Do It

• You Get Better
• You Get Faster
• You Get to Apply Things that You’ve Learned

Extract Libraries

Example

What is a Library?

A Portable Bundle of Code

• Can Be Included Into Other Systems or Libraries
• Provides Reusable Bits of Code

How Do We Share Common Code?

• Monolith
• Authentication
• Customer
• Vendor Integration

Duplicate Code

Extract to Libraries

Things to Consider

• Semver
• Tag Versions in Source Control
• Internal Hosting
• Dependency Management
• Open Source

Open Source

Just By Trying to Name Your Open Source Library You May Find That It Already Exists

Open Source

django-pretty-times

Translates a Timestamp into:

• Just Now
• _ Seconds Ago / In _ Seconds
• _ Minutes Ago / In _ Minutes / A Minute Ago / In a Minute
• _ Days Ago / In _ Days / Yesterday / Tomorrow
• _ Weeks Ago / In _ Weeks / Last Week / Next Week
• _ Months Ago / In _ Months / Last Month / Next Month
• _ Years Ago / In _ Years / Last Year / Next Year

django-pretty-times

• Very Small Library
• Very Specific Function
• Open Source on Github
• Available on PyPI
• 9 Community Contributions
• 5 Community Contributors

Extract Applications

Example

What is an Application?

A Vertical Slice of a System

• Has A UI and/or API
• Has Some Logic
• Has Persistence
• Is Pluggable

Authorization

Each System Needs Permissions

And a Way to Maintain Them

The Monolith Can Do It

Time to Extract

Green Field, Baby!

Hold On…

Have I Contradicted Myself?

Extracting

≠

Rewriting

Or Is It?

When I Say “Extract”

I Do Mean “Rewrite”

But

On Such a Small Scale That it Does Not Count

Extracting is more of a Refactoring than it is a Rewriting

Also, since it is a Refactoring…

For The Love of All that is Good and Holy

Get some tests around the code AS IT IS prior to extracting anything!

Ideal Situation

* With Tests, of Course!

Bad Situation

I Have Tried This, It Does Not Work

The Rewrite to Never Do

Is the One Called The Rewrite

Integrate the Old with the New

Do Not Rewrite, Integrate

Legacy Desktop Application

The Web is the Future

We Want to Leverage What We Already Have

Legacy Desktop Application

• ~ 30 Years Old
• Windows Desktop Application
• PC COBOL & Flexus SP2
• PC COBOL does not speak HTTP

First Bit of Integration

Legacy Application Does Some Calculations

We Want to Leverage That from the Web

We Do Not Want Possible Inconsistencies

How Do We Integrate?

Second Bit of Integration

Legacy Application Has Some Data

We Want that Data to be Pushed in Real Time to the Web

How Do We Integrate?

COBOL Does Not Speak HTTP, We Start There

We Made COBOL Speak HTTP

• New Web Service to Accept Legacy Data (Green Field, Baby!)
• C# COM Component to Talk to New Web Service (Green Field, Baby!)
• Active X Control to Pass Data to COM Component
• Flexus SP2 with Active X Control Add-On
• PC COBOL Program to Pass Data to SP2
• Lots and Lots and Lots of Integration Tests

What Do We Have Now?

• Our Legacy System Can Communicate with New Systems
• Our New Systems Can Communicate with our Legacy System
• We Can Incrementally Add, Enhance and Replace Features

Important Saftey Tip

Use Adapters

What is an Adapter?

Code that Sits Between

• Your Code and Other Code
• Your Services and Other Services
• Your Old Systems and Your New Systems

Why Use an Adapter?

Because it Allows Your Code to

Do Not Allow External Variables to Compromise Your Design

Examples

• Command Line Interface to Legacy Desktop System
• Web Service that Wrapped Command Line Interface
• Cobol Program to Talk to C#
• C# Code to Talk to Web Service
• Web Service that Accepts Data from C#
• Web Service to Talk to Vendor(s)
• Libraries that Talk to Web Services

How To Get Started

Start Small

Start Simple

Extract One Small Thing at a Time

Extracting Everything at Once is The

Rewrite that Should Never Be Done

Simplest First

Tools

(In Order of Importance)

• Source Control

• Tests

• Environment Isolation

• Automated

  • Database Migrations
  • Dependency Management
  • Deployment
  • Server Configuration

Need Buy-In?

Back Yourself into a Corner

Remove the Choice and Require it

The Trick is Turning

Your Brown Field Green

Started With A Brown Field Project

Added A Whole Bunch of Small Green Field Projects

Currently at

55

Web Services, Libraries, Applications and Other Small Systems

The Monolith Has Not Changed All That Much

But It Is Starting to Look a Lot More Green

Thanks!

@mattjmorrison

http://mattjmorrison.com/icc11