ipython-brownbag

IPython and the IPython Family

Christopher Kotfila INF Brown Bag- Spring 2015 ckotfila@albany.edu

Context

Tool talk - not going to cover research really

Hopefully this will be mostly demonstrations

Before that thought I want to make sure we're on the same page

Who am I?

Programming since I was 17.

CS undergraduate at RPI - 'raised' in a formal CS environment - O(n), balancing Red-Black trees, automata, translation look aside buffers for virtual memory all of it

started research in CogWorks lab at RPI - cognitive computation engineering, eyetracking, LISP programming on DARPA grant

Adjunct technology instructor overseas for a couple of years

Was lead developer at local marking firm, web dev, dev ops, sys admin

Huge open source enthusiast

I bring this up because in a little bit i'm going to talk like IPython is the second comming

I have some pretty strong personal biases

Dewy style instrumentalism, problem oriented instead of knowledge oriented

quantitative, predictive, computational

want to be upfront about those at the beginning

Why this presentation?

Evangelism

Opportunity

So why this presentation?

Zealous advocacy of a cause or doctrine, proselytizing zeal.

I don't currently teach - but I really think this tool belongs in the curriculum

Want to bring this tool to your attention.

things that are changing right now that make this viable in the classroom instead of just a reserach lab

Start at the End

Mei Chen - No overview slide

Lead with conclusions

Reduce the hassle

Tell stories with data

Learn by doing

(almost) Classroom ready

Never hit a ceiling

A lot of research gets done with Microsoft Excel - don't want to disparage the program - but at some point you hit a wall with Excel

Not really going to talk about research - but the tools i want to show you are things I use every day

I use IPython & the notebook to tune paramaters by evaluating hundreds of thousands of statistical models on Reserach IT's cluster

Graph generated in python of course

from matplotlib import pyplot as plt
import math
import os

pts = range(1,100)

plt.plot(pts, [n**2.3 for n in range(1,100)], label="Python & IPython")
plt.plot(pts, [400 * math.log(n) for n in pts], label="Most other tools") 

plt.legend(loc=2)
plt.ylim([0,10000])
plt.xlabel("Investment")
plt.ylabel("Capabilities")

plt.title("Investment vs. Capabilities")
plt.savefig(os.getcwd() + "/.publication/ipython-brownbag/img/plt.png")

Framing the problem

Programming is hard

FALSE!

Programming is hard [click]

False!

Want to argue that this is a myth -

we give and recieve complex directions all the time.

Engagement

When we're really engaged with a topic or a project we can coordinate and execute complex plans of action

Ask an undergrad about coordinating a raid in League of Legends

or what they need to get on their final to not fail the class

What are the roadblocks to engaged programming?

Feeling overwhelmed

where to start?

Cognitive overload

wait where am I?

lack of eary success

lack of empowerment

Programming is hard

for students it shouldn't have to start hard

Programming is hard for faculty

Programming also hard for faculty -

Doubt anyone here disagrees that programming is importaint for students

From a pedagalogical perspective there is a barrier to entry

How much time am I going to take? what is going to be the payoff of including programming?

Programming Environments

ITS provided programming environments insufficient

Stuck with putty and an amazon server - or laptops and dozens of different installs

Making sure the libraries are correctly installed

SSH and command line are scary the first time (usually scary the second time too)

Scalability

Manageable problems with a class of 25 or 30. How do I handle 1100 students and they're problems?

Faculty are faculty - they shouldn't also have to be system administrators

Getting students to "Wow…"

All these stand in the way of getting to wow as quickly as possible

getting to that engaging empowered experience

capable of solving a problem i couldn't solve before

seeing the world in a different way - fully self-actualized digital citizenship

if you want to convince someone that programming is worth their time you need to get them to wow quickly

IPython Notebook

History

Want to cover just a little bit of history first

The Python REPL

(Read Evaluate Print Loop)

The IPython REPL

Improving the REPL by ignoring your dissertation

Client/Kernel Architecture

Different Kinds of Clients

The Power of Client/Kernel Architecture

Because we've seperated the input and output from the processing, we can move the input/output off the same machine

The 'python' can be run on a remote machine (or a local machine) and the display can be handled locally

(Importaint if you need to use RIT's 256Gig RAM server but don't want to brave the -10 degree weather)

Enough History - Lets Hack!

Live Coding

(almost always a bad idea)

Lets do it anyways!

An Example Notebook Server

Do scratch notebook - hello world, basic explination of interface

Do Lotto example as something a little more interesting

Do Markdown - talk about markdown

Do Rich Display - walk through rich display notebook quickly

Back to slides

Blogging, Tutorials and Books

Many resources available -

Jake VanderPlas - Director of Research in the Physical Sciences for University of Washington's eScience institute Bugra Akyildiz - Data scientist at Axial in Brooklyn Olivier Grisel - French contributor to sklearn Matthew A. Russell - CTO Digital Reasoning ("cognitive computing")

NBViewer

An example using nbviewer

Notebooks are jus JSON - can be version controlled, can be shared via github

Then clone github repo - don't forget to clear output

Learn by doing

Widgets & Simulation

What does "Almost" classroom ready mean?

IPython's Roots

IPython notebook really scientific notebook

Designed to be used on local computer, or on remote computer but by one or a few trusted users

Not really secure - not really designed for a classroom setting

Project Jupytr

tmpnb.org

Project Jupytr is the evolution of IPython notebook (last 6 to 12 months)

Different name because its not just a Python notebook anymore - also does R, Julia and a few others

Extends the Client/Kernel architecture

Also adds system level authentication, your notebook runs inside a virtualized container as your user

No access to other user's notebooks, containment incase of accidents (better resource allocation)

Its ready - "Almost" just means this is cutting edge (Informatics history of being a pilot program)

Jupytr is happening right now

https://github.com/jupyter

Solving the problem

Direct Impact?

Student

Browser and an internet connection

Instructor

Only need to setup/install a single instance

Spheres of Responsibility

Amason, ITS and EIS

Program or be Programmed

Douglas Rushkoff - 2010

Not suggesting everyone should be a software engineer

About de-mystifying the technologies - teaching people they can have control over their digital lives

There are people out there who are making choices about the face of our digital landscapes

We can choose to participate in that conversation - or let others make choices for us.

IPython and the IPython Family – Objectives – What is Python and why should I care?

kotfic

IPython and the IPython Family – Objectives – What is Python and why should I care?

0 0 (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/platform.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();