gevent-talk

WSGI?

• Plug in a Flask app, patch and go
• Multiple instances bound to a single socket/port

exempli gratia

from gevent.wsgi import WSGIServer
from gevent.socket import tcp_listener
from multiprocessing import Process, cpu_count
from serverdensity.proxy.app import app


def run(app, listener):
    http_server = WSGIServer(listener, app)
    http_server.serve_forever()

exempli gratia cont.

listener = tcp_listener(('127.0.0.1', 8000))

for i in xrange((cpu_count() - 1)):
    Process(
        target=run,
        args=(app, listener)
    ).start()

run(app, listener)

Spawning

• Greenlet class
• Greenlet.spawn(c, [...])
• greenlet.spawn(c, [...])

Flow control

• gevent.sleep(n) (yield from)
• gevent.join(g)
• gevent.joinall([...])
• greenlet.link(g)
• gevent.shutdown
• Greenlet states
• Timeouts

exempli gratia

from gevent import spawn, joinall, sleep

def g():
    print('foo')
    sleep(0)
    print('bar')

joinall([spawn(g), spawn(g)])

exempli gratia cont.

foo
foo
bar
bar

Data structures

• Events and Queues (like the standard library)
• Groups/Pools/Locks/Semaphores
• Thread locals

Monkey patching

• patch_socket()
• patch_ssl()
• patch_time()
• patch_select()
• patch_thread()
• patch_all()

NOOP-ish exempli gratia

# -> HTTP request
mongo.payloads.insert(payloads)
tasks.process_payload.apply_async(payload_id)

# -> celery task (inside a greenlet)
@task
def process_payload(payload_id):
    payload = mongo.payloads.findOne({...})

process = PayloadProcessor(payload)
...
return process_alerts.apply_async(payload_id)

cont.

# inside PayloadProcessor
# (still within a greenlet, controlled by Celery

def send_metrics(self):
    payload = copy(self.payload)

    # remove some fields we don't want to send,
    # turn some to strings

    gevent.spawn(self._handle_metrics_request, [payload])

cont.

    def _handle_metrics_request(self, payload):

        resp = requests.post(METRICS_URL,
                   data=json.dumps(payload))

        if resp.status_code == 200:
            logging.info(...)
            return True
        else:
            logging.error(...)
            return False

deus e.g.

def update_check(self, response):
    # update collections in Mongo from response
    ...

def update_metrics(self):
    # send a request to the metrics service
    ...

def update_inventory(self):
    # send a request to the inventory service
    ...

deus e.g. cont.

# run from a celery task
def health_check(self):
    # get HTTP response from actors
    ...

    update_greenlet = spawn(self.update_check, [response])

    update_greenlet.link(spawn(self.update_metrics))
    update_greenlet.link(spawn(self.update_inventory))

sys.exit(0)

• Code: git.io/sd, git.io/1v
• Slides: 1stvamp.github.com/gevent-talk

^D

• Jonathan "Joff" Oliver: about.me/joffie
• Grab me and say hi :-)