Writing faster code

Writing faster code and not hating your job as a software developer

Writing faster Python

@SebaWitowski

Python was not made to be fast...

...but to make developers fast.

Would you like your FIRST program EVER to be like:

public class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello, world!");
    }
}
                        

or

print("Hello, world!")
                        

OPTIMIZATION

1. Don't

2. Don't... yet

• Finish your code
• Have tests
• Now

3. Profile

• cProfile
• pstats
• RunSnakeRun, SnakeViz

Levels of optimization

• Design
• Algorithms and data structures

sum = 0
for x in range(1, N + 1):
    sum += x
print sum
                        

print N * (1 + N) / 2
                            

Levels of optimization

• Design
• Algorithms and data structures
• Source code
• Build level
• Compile level
• Runtime level

Optimization is all about the speed

Writing fast Python

a.k.a source code optimization

Setup

Python 3.5.1 (IPython 1.2.1)

def ultimate_answer_to_life():
    return 42
                        

>>> %timeit ultimate_answer_to_life()
10000000 loops, best of 3: 87.1 ns per loop
                        

2.72 × 1021 times faster than in The Hitchhiker's Guide to the Galaxy ;-)

#1 Count elements in a list

how_many = 0
for element in ONE_MILLION_ELEMENTS:
    how_many += 1
print how_many
                        

26.5 ms

print len(ONE_MILLION_ELEMENTS)
                        

96.7 ns

274 000 times faster

#3 Permissions or forgiveness ?

class Foo(object):
    hello = 'world'
foo = Foo()
                        

if hasattr(foo, 'hello'):
    foo.hello
                        

149 ns

try:
    foo.hello
except AttributeError:
    pass
                        

43.1 ns 3.5 times faster

#3 Permissions or forgiveness ?

if (hasattr(foo, 'foo') and hasattr(foo, 'bar')
    and hasattr(foo, 'baz')):
    foo.foo
    foo.bar
    foo.baz
                        

401 ns

try:
    foo.foo
    foo.bar
    foo.baz
except AttributeError:
    pass
                        

110 ns 3.64 times faster

#3 Permissions or forgiveness ?

class Bar(object):
    pass
bar = Bar()
                        

if hasattr(bar, 'hello'):
    bar.hello
                        

428 ns

try:
    bar.hello
except AttributeError:
    pass
                        

536 ns 25% slower

#4 Membership testing

def check_number(number):
    for item in MILLION_NUMBERS:
        if item == number:
            return True
    return False
                        

%timeit check_number(500000)
                        

18 ms

500000 in MILLION_NUMBERS
                            

#4 Membership testing

100 in MILLION_NUMBERS
                        

1.55 µs

999999 in MILLION_NUMBERS
                        

15.7 ms

#4 Membership testing

MILLION_SET = set(MILLION_NUMBERS)
%timeit 100 in MILLION_SET
                        

46.3 ns 33 times faster (vs list)

%timeit 999999 in MILLION_SET
                        

63.3 ns 248 000 times faster (vs list)

%timeit set(MILLION_NUMBERS)
                            

#8 Checking for True

if variable == True:
                        

35.8 ns

if variable is True:
                        

28.7 ns 24% faster

if variable:
                        

20.6 ns 73% faster

#8 Checking for False

if variable == False:
                        

35.1 ns

if variable is False:
                        

26.9 ns 30% faster

if not variable:
                        

19.8 ns 77% faster

#8 Checking for empty list

if len(a_list) == 0:
                        

91.7 ns

if a_list == []:
                        

56.3 ns 60% faster

if not a_list:
                        

32.4 ns 280% faster

#9 Def vs lambda

def greet(name):
    return 'Hello {}!'.format(name)
                        

329 ns

greet = lambda name: 'Hello {}!'.format(name)
                        

332 ns

#9 Def vs lambda

>>> dis.dis(greet)
 0 LOAD_CONST    1 ('Hello {}!')
 3 LOAD_ATTR     0 (format)
 6 LOAD_FAST     0 (name)
 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
12 RETURN_VALUE
                        

#10 list() or []

list()
                        

104 ns

[]
                        

22.5 ns

4.6 times faster

#10 dict() or {}

dict()
                        

161 ns

{}
                        

93 ns

1.7 times faster