(De-)Serializing complex objects using Python – Complex Objects – JSON to the Rescue



(De-)Serializing complex objects using Python – Complex Objects – JSON to the Rescue

0 0


pycologne-deserializing-using-python

A Talk about how to (de-)serialize complex python objects using JSON

On Github ReneVolution / pycologne-deserializing-using-python

Image: Kevin Dooley

(De-)Serializing complex objects using Python

Created by René Calles / @renecalles

__init__()

  • About Me
  • What is (de-)serializing
  • But why?
  • Complex Objects
  • Extending JSON
  • Showtime
  • Caveats
  • Time for Questions

About Me

class ReneCalles(Programmer, MediaEnthusiast):

    birthday = datetime.fromtimestamp(443739925)

    hobbies = ['Programming', 'Fishing', 'Gaming', 'Music',
               'Meet Friends']
    buzzwords = ['WebApps', 'Cloud', 'Streaming', 'Open Source']

    programming_languages = {'Python': 2012,
                             'C/C++': 'want to do more',
                             'C#': 'seen'}

    def on_free_time(self):
        return random.choice(self.hobbies)

					

(De-)serializing

“ ...is the process of translating data structures or object state into a format that can be stored (for example, in a file or memory buffer, or transmitted across a network connection link) and reconstructed later in the same or another computer environment.”

Wikipedia

But why?

  • Saving an objects current state
  • Talking to JSON Rest-API's

Complex Objects

Bookshelf

class BookShelf(object):
    def __init__(self):
        self.books = []

    def add(self, book):
        self.books.append(book)

    def remove(self, title):
        try:
            idx = next(idx for idx, book in enumerate(self.books) if
                       book.title == title)
        except StopIteration:
            raise BookNotFound('Could not find book: %s' % title)

        del self.books[idx]

    def count(self):
        return len(self.books)

Book

class Book(object):

    def __init__(self, title):
        self.created_at = datetime.now()
        self.title = title
        self.author = Person()
        self.description = 'No Description available'

Person

class Person(object):

    def __init__(self):
        self.first_name = "John"
        self.last_name = "Doe"
        self.title = "Dead Manager"

    @property
    def name(self):
        return '{0} {1}'.format(self.first_name, self.last_name)

JSON to the Rescue

Encoder

class ObjectEncoder(JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return {'__type__': 'datetime.datetime',
                    'iso_format': obj.isoformat()}

        if isinstance(obj, COMPLEX_OBJECTS):
            obj_data = obj.__dict__.copy()
            obj_data['__type__'] = type(obj).__name__
            return obj_data

        if type(obj).__name__ == 'Book':
            obj_data = obj.__dict__.copy()
            obj_data['__type__'] = type(obj).__name__
            return obj_data

        super().default(obj)
						

Decoder

def object_decoder(obj):

    if '__type__' in obj:
        cls_name = obj.pop('__type__')

        if cls_name == 'datetime.datetime':
            format_str = '%Y-%m-%dT%H:%M:%S.%f'
            return datetime.strptime(obj['iso_format'], format_str)

        elif cls_name == 'BookShelf':
            new_instance = BookShelf()
            new_instance.__dict__.update(obj)

            return new_instance

        elif cls_name == 'Book':
            module = __import__('book', globals(), locals(), [cls_name])
            klass = getattr(module, cls_name)
            book = klass(obj.pop('title'))
            book.__dict__.update(obj)

            return book

        elif cls_name == 'Person':
            new_instance = Person()
            new_instance.__dict__.update(obj)

            return new_instance

    return obj
						

Test me if you can

Test me if you can

class TestSerialization(unittest.TestCase):

    def test_serializing_roundtrip(self):
        shelf = BookShelf()
        for i in range(42):
            shelf.add(Book('Python Magic'))

        with open('../bookshelf.json', 'w') as json_file:
            json.dump(shelf, json_file, cls=ObjectEncoder, indent=4)

        with open('../bookshelf.json', 'r') as json_file:
            shelf = json.load(json_file, object_hook=object_decoder)

        self.assertIsInstance(shelf, BookShelf)

        self.assertEqual(shelf.count(), 42)

        for book in shelf.books:
            self.assertIsInstance(book, Book)
            self.assertIsInstance(book.author, Person)

Caveats

  • Object comparison may fail after deserialization
  • Ensure that unneeded attributes will not be serialized
  • Requires work for every new class added
(De-)Serializing complex objects using Python Created by René Calles / @renecalles