• so…
• another web application framework
• really???
• aren't there already dozens, or even hundreds of them around?
• why not just pick one of them?
• As a matter of fact all of the Kotti core developers have some common history

Yet another web framework?

Features

• full featured CMS
• lots of add ons (varying quality)
• OFS (object file system)
• security (permissions, roles, groups)
• workflows

• Full featured CMSHas lots and lots of add ons (of varying quality, but still.Uses Object File SystemA simple, yet effective way to persist objects.More or less plain Python objects are stored in an object DB.It supports a folder / file like object tree.securityHas great security features.Has the concepts of groups, users, roles and permissions which enables you to write applications with very fine grained security.also: great security track record with very few vulnerabilities compared to other systems of that kind.Workflowsare an absolute killer feature, that was unprecedented at that time.

Can be a perfect choice when it fits your needs

but

does not fit all kinds of applications.

• if you have a classic CMS use case, or need an enterprise CMS application, it's probably a perfect choice.
• BUT
• if you need to implement a highly customized application with some CMS like features, it can quickly become overkill and a major pain.

It often has too many features out of the box, that you just don't need.

and you constantly find yourself fighting the framework

• This is particulary caused by the underlying software stack
• huge: consists of some 300 packages
• uses multiple competing technologies, which
• don't conform to "The Zen of Python"
  • to have “preferably only one obvious way to do” things
• based on the monolithic Zope 2 AND the component based Zope 3 stacks
• uses adapters, multiadapters and utilities all over the place
• long story short…

• multiple competing technologies
• doesn't conform to "The Zen of Python"
• complex

• have a similar history and
• wanted to preserve the most important features, that Zope pioneered back in the days.
• This all with a modern, clean and maintainable code base.
• This was the birth of Pyramid, then known as repoze.bfg

Features

• small core
• excellent documentation
• pythonic
• low level (micro framework) It has
  • small core,
  • excellent documentation,
  Is
  • pythonic and rather
  • low level, which is why you can also sort it into the group of microframeworks
  And it's also… unopinionated, which means it
  • makes no assumptions about stuff like persistence and forms
  • and only basic assumptions about authentication and authorization, which you could call
  • the least common denominator for all kinds of web applications.
  This all makes it a great…
  • framework framework
  • a framework to build your own framework, which supports your opinions
• unopinionated
  • persistence
  • templating / forms
  • authentication / authorization sources
• “framework framework”

Conclusion

• only provides stuff we need
• doesn't come with unneeded ballast
• no need to “waste time fighting the framework's decisions”
• perfect foundation!

• Pyramid only provides what you need in every web application.
• Doesn't come with unneeded ballast.
• There's no need to waste time fighting the framework's decisions
• This makes it a perfect foundation for your own frameowrk.

Make some choices!

• persistence
• traversal or URL dispatch
• templating & forms
• authentication & authorization sources

• persistence: obviosly need some means of storage for most applications
• templating, forms and user input validation:
• authentication & authorization sources

• probably the most advanced ORM for Python
• database agnostic
• has many nice, useful features
  • hybrid properties
  • association proxies
  • ordering list
• transaction integration with pyramid

• Particularly useful: transactions can be bound to the lifecycle of a Pyramid request through the use of the pyramid_tm and zope.sqlalchemy packages.

Kotti implements a Node class in pure SQLAlchemy, that implements what we had with the Object Filesystem in Zope.

(bulletpoints)

The Node Class

• adjacency list pattern
  • parent
  • children
• single root node => node tree
• dictionary protocol

The Node Class

Dictionary Protocol

from kotti.resources import Document
from kotti.resources import get_root

root = get_root()

root['about']
<Document 2 at /about>

root['my-document'] = Document(title='My Document', description='foo',
                               body='<p>some HTML</p>')
            

• you can treat any instance of the Node class as an dictionary and
• get items by their name / key and
• also set them like you would with a dictionary.

The Node Class

Traversal

a = root['a'] = Document(title='A', description='Document A')
b = root['a']['b'] = Document(title='B', description='Document B')
c = root['a']['b']['c'] = Document(title='C', description='Document C')
            

This instantaneously gives us traversal support with Pyramid, as Pyramid "only" requires a __getitem__ method to be implemented, which is exactly the same what a dict does.

Beyond that we only need to have an ACL property on the node class to have persistent and inheritable ACLs.

Polimorphic queries

from kotti.resources import get_root
from kotti.resources import Node

root = get_root()
print root.children:
    print(type(c))

"<class 'kotti.resources.Document'>"
"<class 'kotti.resources.File'>"

print Node.query.filter(Node.title == 'My Document').one()
"<Document 5 at /my-document>"
            

• In Kotti SQLAlchemy is set up to do polymorphic queries.
• Single query can return objects of multiple types.
• For example querying for Nodes will hand you back instances of Document and File if the Nodes happen to be of that types.

Joined Table Inheritance

• class hierarchy is broken up among dependent tables
• each class represented by its own table
• the respective table only includes attributes local to that class

Joined Table Inheritance is another nice feature of SQLAlchemy that we use in Kotti.

It means that…

• class hierarchy is broken up among dependent tables.
• Each class is represented by its own table.
• The respective table only includes attributes local to that class.

Events

• before_flush
  • ObjectUpdate
  • ObjectInsert
  • ObjectDelete

• SQLAlchemy has a lot of them, but in Kotti we use only one: before_flush
• A flush is basically the moment when SQLAlchemy talks to the database.
• It is NOT the same as a commit. It rather is SQLAlchemy emitting SQL and sending it to the DB, which can happen multiple times during a transaction. ((unit of work pattern))
• Of course a commit always includes a flush.

Alembic

• DB migrations
• DDL (transactional, if supported by DBMS)
• DML
• environments

• DB Migration tool that supports
• transactional Data Definition Language operations (such as CREATE or ALTER TABLE) and
• transactional Data Manipulation Language operations.

• supports multiple environments.
• Comes in extremely handy for add ons in Kotti: each add on can have its own, independent migration steps.

Kotti provides a script to perform either specific migrations or all migrations for itself and all add ons at once. You can either upgrade to specific revisions or to the latest known revision.

Downgrades are also supported for the case that something goes south during a migration and your DB does not support transactional DDL.

• has all the components for modern UIs
• responsive
• well known
• easy to customize

Unless you have a UI designer on your team and can be sure that he or she will stay, you shouldn't even try to invent something yourself.

Instead: just use Bootstrap.

(bulletpoints)

• creation of forms as well as their validation.
• Colander and Deform are different in that aspect.
• With Colander you can (bulletpoints).
• But it doesn't know anything about forms at all.

Colander

• define data schema
• validate & deserialize
  • HTML forms
  • JSON
  • XML
• serialize Python structures to
  • strings
  • mappings
  • lists

Deform

• render HTML forms from structures serialized by Colander
• outputs Bootstrap 3 forms (Deform 2)

• …only renders forms from datastructures passed by colander.
• Deform 2 uses Bootstrap 3 by default.
• In fact Deform 2 is a merge of Deform 1 and deform_bootstrap, that we developed for earlier versions of Kotti. (it was a set of bootstrap templates for deform)
• Doesn't know anything about serialization or validation

• take user input not only from forms, but as well as from say Javascript applications or XML RPC endpoints or similar.

• A content workflow system that supports the concepts of states and transitions.
• States define a role to permission mapping
• Transitions define transitions between states, and the circumstances in which the transition may be executed

• a content workflow system
• states define
  • role / permission mapping
• transitions define
  • from_state
  • to_state
  • required permission

• storing and serving files in web applications
• multiple backends
  • local filesystem
  • S3
  • GridFS
  • roll your own
• integrates with SQLAlchemy
  • files are handled like a plain model attribute
  • transaction aware

• storing and serving files in web applications.
• it supports: (bulletpoints)
• transaction aware rollback or abort of a transaction causes the files to be deleted from the configured backend
• Don't get drawn away by its version (0.0.6 ATM)!
• Excellent docs, good codebase, complete test coverage.
• There's a dedicated talk on Depot tomorrow at 11:00 in the Barria2 room. Don't miss it!

Wiring it all together…

Let's finally come to Kotti itself.

As mentioned in the talk outline, it's a rather small package, that wires all those giants together in a sensible way.

• Started by Daniel Nouri in 2011.
• He did the first version in just 2 days and that version already contained most of the Node class i mentioned before.
• I learned about Kotti and joined the project a year later, in 2012 (Plone Konferenz)

• 1.0.0 in January 2015
• Biggest mistake in terms of marketing: we should've released a v1.0 in place of 0.6 or 0.7 already.
• Kotti was already absolutely stable and used in several production systems at that time, so calling it 1.0 would definitively have been justified.
• The Number of features in 0.10 alone would have justified 3 or 4 releases / versions.
• With 1.0.0 switched to semver
  • major version increases with backward incompatible changes
  • minor version increases with feature additions
  • and patch is reserved for bugfixes
• ~9k dl/month according to PyPI
• 5 contributors with push permission
• ~40 contibutors overall.
• That makes us a still small, but active & healthy community.
• Of course contributions are always welcome, even (or especially) if it's just a missing comma or some unclear wording in the docs.

• started by Daniel Nouri in 2011
• BSD licensed
• 1.0.0 in January 2015
• current version: 1.1.4
• 9k downloads per month
• still small, but active & healthy community
• contributions are always welcome

Code Quality

a.k.a. "everyone loves badges"

• extensive test suite based on py.test
• Heavy use of reusable fixtures.
• Fixtures are exposed a a pytest plugin, that you can easily use in your addons' tests.

• Travis CI
• every commit and pull request

• recently a number of tools for static source code analysis came into life
• you shouldn't blindly follow all of their suggestions / issues they report, some of them are non-issues
• BUT they can give VERY valueable insights on your code and help you improve

• Also try very hard to keep all of our requirements up to date and continue to succeed with that with one minor exception in our testing requirements. So technically the last badge is lying a bit.

Configuration

[app:kotti]
use = egg:kotti
sqlalchemy.url = sqlite:///%(here)s/Kotti.db
# sqlalchemy.url = postgres://user:pass@host/db
kotti.configurators =
    kotti_tinymce.kotti_configure
    kotti_youraddon.kotti_configure

[filter:fanstatic]
use = egg:fanstatic#fanstatic

[pipeline:main]
pipeline =
    fanstatic
    kotti

[server:main]
use = egg:waitress#main
host = 127.0.0.1
port = 5000

• Almost every aspect of Kotti can be configured with an option in the INI file.
• Kotti provides sensible defaults for each option, so you don't have to specify any of them.
• I'm starting to run out of time already, so I won't go into detail here.
• The important thing to take away from this is: you can override almost everthing in Kotti, but you dont need to.

Example Options

Example Options (continued)

Let me give you just 1 example of how we combine multiple "best of breed" components to a fully functional system.

w.r.t. security we use: (bulletpoints)

• ACLs are JSON serialized Pyramid ACLs

Security

• use SQLAlchemy to…
  • store pricipals (users & groups) in the DB
  • attach (inheritable) ACLs to each node
• use Pyramid for…
  • authentication
  • authorization
• use repoze.workflow to…
  • recompute ACLs on workflow state changes

• We provide a complete Kotti scaffold (based on Pyramid's pcreate command) with Kotti.
• After running that command, you have a fully functional add-on for Kotti, with a complete test suite and CI on Travis CI setup already.

• a custom content type
• a default and alternate view for that type
• completely setup i18n infrastructure
• an also completely setup alembic environment, so that you only need to write the actual migration steps (if that should become necessary in future versions of your add on)
• fanstatic resources local to this add on

• …always kept up to date, to adhere to the most recent coding conventions as suggested by the Kotti team
• completely tested within our CI setup, so that we can be sure it is working as expected at any time.

Creating an addon

$ pcreate -s kotti kotti_myaddon
Author name [Andreas Kaiser]:
Author email [disko@binary-punks.com]:
Github username [Kotti]:
[… lot of output …]
===============================================================================
Welcome to Kotti!

Documentation: http://kotti.readthedocs.org/
Development:   https://github.com/Kotti/Kotti/
Issues:        https://github.com/Kotti/Kotti/issues?state=open
IRC:           irc://irc.freenode.net/#kotti
Mailing List:  https://groups.google.com/group/kotti
===============================================================================
            

• It inherits from Kotti's Content class, which is again a sane default that you probably should inherit all your types from.
• It has an ID column, that is both the PK and a FK to the parent table's id column. This is needed for SQLAlchemy's Joined Table Inheritance.
• Then it has the attributes that are actualy added by this class in addition to the inherited ones.
• Last there's a type_info, that tells Kotti how, where and under wich conditions that Content Type should be made available through the UI.

Custom Content Type

from kotti.resources import Content
from sqlalchemy import *

class Document(Content):
    id = Column(Integer(),
                ForeignKey('contents.id'),
                primary_key=True)
    body = Column(UnicodeText())
    mime_type = Column(String(30))
    type_info = Content.type_info.copy(
        name=u'Document',
        title=_(u'Document'),
        add_view=u'add_document',
        addable_to=[u'Document'])

Next there's the Colander schema definition, responsible for serialization, deserialization, validation and form creation through Deform.

It by default also inherits from the parent's schema and only adds schema nodes provided by the custom content type.

Schema definition for validation and form creation

import colander
import deform
from kotti.views.edit.content import ContentSchema

class DocumentSchema(ContentSchema):
    body = colander.SchemaNode(
        colander.String(),
        title=_(u'Body'),
        widget=deform.widget.RichTextWidget(),
        missing=u"")

• They're, like everything in Kotti, plain Pyramid views that are configured by the usual view_config decorator.
• They're protected by some permissions, that you can – again, like everything in Kotti – adjust to fit your specific needs.
• The actual form rendering and validation is again provided by the base classes. You don't have to do anything yourself, unless you want something to be done differently.
• What you see here is a complete and working example!

Add / edit forms

from kotti.resources import Document
from kotti.views.form import AddFormView
from kotti.views.form import EditFormView
from pyramid.view import view_config

@view_config(name=Document.type_info.add_view, permission='add',
             renderer='kotti:templates/edit/node.pt')
class DocumentAddForm(AddFormView):
    schema_factory = DocumentSchema
    add = Document
    item_type = _(u"Document")

@view_config(context=Document, name='edit', permission='edit',
             renderer='kotti:templates/edit/node.pt')
class DocumentEditForm(EditFormView):
    schema_factory = DocumentSchema

• All error messages, as well as labels are fully internationalized.
• We currently ship with translations for 8 languages. As always: contributions are very welcome.

• again, it's just ordinary Pyramid views, just like everything in Kotti.
• In this first view we specify a template for rendering.
• In Kotti we use Chameleon templates (as we're used to them because of our Plone / Zope history).
• If you prefer some other templating language: just use it. You can even mix arbitrary templating languages, because Pyramid supports that of course.

• Nice thing about it: you can group multiple views that share some common code or view configuration.
• You can then add additional views with even less code or configuration.
• Of course views don't need to be rendered via templates, they can for example also provide JSON.
• Again: this all is working and complete code, not just excerpts.

View(s)

from pyramid.view import view_config
@view_config(name='view', context=Document, permission='view',
             renderer='kotti:templates/view/document.pt')
def document_view(context, request):
    return {}

from pyramid.view import view_config
from pyramid.view import view_defaults
from kotti.views import BaseView

@view_defaults(context=Document, permission='view')
class DocumentViews(BaseView):
    @view_config(name='view', renderer='kotti:templates/view/document.pt')
    def view(self):
        return {}
    @view_config(name='view2', renderer='kotti:templates/view/document2.pt')
    def view(self):
        return {'answer': 42}
    @view_config(name='json', renderer='json')
    def view(self):
        return {'title': self.context.title, 'body': self.context.body, ...}
        # return self.context
        

Finally this is the template our view uses. Nothing special again.

The templates provided by Kotti (like the tags template included in this example) can of course be individually overriden.

Template(s)

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"
      metal:use-macro="api.macro('kotti:templates/view/master.pt')">
  <article metal:fill-slot="content" class="document-view content">
    <h1>${context.title}</h1>
    <p class="lead">
      ${context.description}
    </p>
    <div tal:replace="api.render_template('kotti:templates/view/tags.pt')" />
    <div class="body" tal:content="structure context.body | None">
    </div>
  </article>
</html>