Prediction.io – Bigdata and machine learning

View Github Repository Open presentation in a new window

duyetdev

See all presentation from duyetdev

Prediction.io – Bigdata and machine learning

1 1

prediction.io-introduction

On Github duyetdev / prediction.io-introduction

Prediction.io

Introduction Prediction.io - DASE architecture

Van-Duyet Le / me@duyetdev.com

Bigdata and machine learning

Building machine learning an application from scratch is hard.

Available Tools

Processing Framework

  • e.g. Apache Spark, Apache Hadoop

Algorithms Libraries

  • e.g. MLib, Mahout

Data Storage

  • e.g. HBase, Cassandra

Integrate everything together nicely and move from prototyping to production?

  • collect data
  • train your algorithm
  • build a layer to serve the prediction results
  • manage the different algorithms
  • evaluations
  • deploy your application in production

Build and Deploy Machine Intelligence

in a fraction of the time.

PredictionIO

  • PredictionIO is a machine learning server for building and deploying predictive engines on production in a fraction of the time
  • Built on Apache Spark, MLlib and HBase.
Open-source Machine Learning server. Core - engine deployment platform built on top of Apache Spark. Event Server built on top of Apache HBase. DASE architecture of engine is the "MVC for Machine Learning". Swap and evaluate algorithms as you wish. Zero downtime training and deployment. Engine template. 
$ pio template get
$ cd MyEngine
$ pio build; pio train; pio deploy

Start with an Engine Template

Data Source                - readTraining()
Data Preparator            - prepare()
Algorithm:                 - train() 
                           - predict()
Serving                    - serve()
Evaluator                  - evaluate()

Customize Code with Software Design Pattern

  • DASE - the "MVC" for Machine Learning
  • Built-in support to Spark MLlib, or create your own algorithm
cli = predictionio.EventClient("")
cli.record_user_action_on_item("view","John","HackerNews")
# predict top preferences
eng = predictionio.EngineClient("")
rec = eng.send_query({"uid":"John","n":5})

Connect to your App with SDKs

DASE

  • [D] Data Source and Data Preparator
  • [A] Algorithm
  • [S] Serving
  • [E] Evaluation Metrics

PredictionIO helps you modularize these components so you can build

Training a Model - The DASE View

Respond to Prediction Query - The DASE View

Query (Input) :

$ curl -H "Content-Type: application/json" -d
'{ "user": "1", "num": 4 }'
http://localhost:8000/queries.json

Predicted Result (Output): 

{“itemScores”:[{"item":"22","score":4.072304374729956},
{"item":"62","score":4.058482414005789},
{"item":"75","score":4.046063009943821}]}

Install

$ bash -c "$(curl -s https://install.prediction.io/install.sh)"

Running

Start the Event Server 

$ pio eventserver

Deploy an Engine 

$ pio build; pio train; pio deploy

Update Engine Model with New Data 

$ pio train; pio deploy

THE END

Q&A

Prediction.io Introduction Prediction.io - DASE architecture Van-Duyet Le / me@duyetdev.com