a brief history

• Let's start by talking a bit about django's roots.

django's 10th birthday

• django was open sourced in 2005, 10 years ago
• it's aged remarkably well

django is not a cms

(but it was built for one)

• django was conceived as a CMS for Lawrence Journal-World
• eventually they decided to split the codebase in 2, license the CMS bits, and open source the more generic web framework underneath.
• So django has its roots deep in journalism
• contrib belies those roots (admin, content types, localflavor)

"perfectionists with deadlines"

• Django's slogan is "perfectionists with deadlines"
• This gives us an idea of the the purpose of Django, the motivation for the project that has dictated its trajectory
• most of django's strengths and weaknesses can be summed up in this one phrase

the web

2005-2015

• While Django's core architecture has remained the same, the web is a very different place than it was 10 years ago.

we've moved from this

• Remember the days when you could describe your stack with an acronym?
• 1 data store, usually relational
• request/response cycle ecapsulated all business logic
• server-side views responsible for all content rendering
• this is the web django was born out of, the web whose problems it was intended to solve
• so we've moved from this ...

to this

er, this

• A single service is potentially comprised of many processes
• Services (mostly) talk HTTP to each other
• Pubsub (usually via redis) allows efficient 1:many communication between services
• MOST IMPORTANTLY: the web client is now just another service, a consumer of public services published by the server

expectations

web "site" 🆚 "app"

• over the past decade, expectations have changed dramatically
• If your site looks like an application, users expect it to act like one
• Fallacies of distributed computing mean nothing to users.
• responsiveness is now an expectation, and sever-side mvc doesn't cut it

django's

strengths

• As I said, Django has aged well, and it still brings a lot to the table for developing modern web apps

python

• python is probably django's biggest strength
• libraries are the lifeblood of a language, and the python community has a tremendous bredth and depth of libraries
• plus, we all have a fondness for the language, otherwise we wouldn't be here

"perfectionists with deadlines"

• I mentioned that most of Django's strengths and weaknesses stem from this one phrase
• decisions made with this in mind mean django is still an awesome tool for getting stuff done

orm & migrations

• Some people bash django's orm as not complete or expressive enough, but it's still my favorite
• Now that migrations are built in, it's a tremendous battery included that sets the foundation for other productive tools

from django.contrib import admin

• yesterday morning Ola gave an entire talk about how powerful the admin is for getting stuff done
• nobody wants to spend their time writing an admin from scratch, so it's exactly the kind of feature that makes django valuable

django rest framework

• super quick to get up and running with basic CRUD actions for resources based on orm models

celery

• every non-trivial web application needs an asynchronous work queue
• celery is a best in class tool, a huge advantage to putting core business logic in python

django's

weaknesses

• but for all those strengths, django is of course not without its weaknesses

c10k

• The c10k problem has plagued django for years
• (c10k shorthand for 10,000 concurrent connections)
• most evident in long-polling and websocket apps
• python, at least python 2, wasn't built to be good at massive concurrency
• solutions do exist, but either at the expense of library support or by monkeypatching, which can have some weird and subtle edge case behavior
• Python, at least python + django, is just not the right tool for this job
• node.js and go were built with concurrency in mind

js client integration

• As we move toward richer js clients for our web apps, we've developed a suite of tools to help
• But Most of those tools also exist within the js community, js being the de facto language in the browser
• We don't get the full benefit of these tools without js on the server

example problems

• let's look at a few example problems and how we might solve them using a multi-platform web architecture

push to web clients

• Lots of applications want to be able to push data to web clients. Django still doesn't have a great solution.

django solution

• Long-polling. Ties up web workers, will DDOS busy sites.
• Some gevent monstrosity. Please, just don't.

polyglot solution

• unix pipes: websocketd
• pure go: built-in websocket package
• js: socket.io

async task results

• Let's say your client calls an API endpoint which launches an async task, but the client needs to know the result of the task before it can reasonably continue.
• These problems commonly surface in a "waiting" UI state. (Think travel search engines like Hipmunk.)
• How do you solve this problem?

django solution

• Include task ids in endpoints which launch async tasks
• Expose an API endpoint which takes a task id and responds with the task state and result
• setInterval in the client to poll for the task result

polyglot solution

• Include task ids in endpoints which launch async tasks
• Client opens a websocket to a server which acts as a bridge to redis pubsub
• Allow clients to subscribe to tasks using task id as a pubsub channel name

server side js rendering

• It's becoming increasingly common for javascript libraries to support server side rendering of initial state in order to minimize time to usable page.
• This cannot happen without js on the sever.

django solution

¯\_(ツ)_/¯

subprocess.call(['node', 'render.js'])

• Subprocess to node?
• Straw man: slow, still running node
• I think Eric Florenzano actually tried doing this at some point and gave up because it was so slow

polyglot solution

React.renderToString(BaseElement)

• Depending on how you handle setting element state, there's a bit more to it than this, but that's the gist
• Use CORS to serve your client app from a different domain than your django api.
• Alternatively, have node proxy api calls back to django

shared authentication

• As soon as you have clients accessing more than 1 service directly, you need a shared authentication mechanism
• On web clients, there are 2 stateful mechanisms: local storage and cookies

django.contrib.auth

• If you use django.contrib.auth, make sure you issue the cookie on a domain that will be sent with requests to each public service
• One benefit if you use the admin, you won't have to log into it separately

access token

• The alternative is to store a 2-legged oauth token in local storage and append it to requests to public services from the client
• It's worth noting that in both of these scenarios, django will need to serve an auth page (either to provide a session cookie or an access token)
• You can also use local storage, but then there's no way to take advantage of server side rendering

authentication service

• If you don't like the idea of delegating auth to a public service, you can create a dedicated private authentication service and delegate to it from all public services

conclusions

say no to full stack python

• I know this was a popular sentiment a few years ago. I think it shows a lack of appreciation for the bredth of tools available to us as web developers.
• Right tool for the job means there are some jobs python is just not the right tool for

http is your friend

(when it's not, pubsub is)

• All of the web services you build know how to speak and listen in http. Take advantage of that.
• When 1:1 communication between services won't do, pubsub is a great tool.

web browsers are service consumers

• When you embrace the web browser as nothing more than a consumer of your public services, then you're free to think of much more appropriate application architectures for moden web apps.