Google Glass: Introduction – GDG DevFest Fresno 2014

View Github Repository Open presentation in a new window

justinribeiro

See all presentation from justinribeiro

Google Glass: Introduction – GDG DevFest Fresno 2014

0 0

google-glass-development-gdg-devfest-2014

Talk for GDG DevFest 2014.

On Github justinribeiro / google-glass-development-gdg-devfest-2014

Google Glass: Introduction

GDG DevFest Fresno 2014

Justin Ribeiro justin@stickmanventures.com

@justinribeiro +Justin Ribeiro justinribeiro

Slides: https://goo.gl/NG6bDU

What are we talking about today

Choices: Design, Patterns and what nots Dive into Mirror API Making Mirror do more with App Engine The GDK!

Glass: not your average screen

You should be designing for Glass Direct ports = not a great choice

Don't get in the way

Supplement the users life, don't take it over

Stay contextual and relevent

Info at the right place, right time

Avoid unexpected actions

Be aware otherwise your user will turn you off

Build for people

The robots are okay, they're not interested in Glass

Design has never been more important

More Reading: Design Principles

We need some building blocks

  • UI elements: static cards, live cards, immersion
  • Invocation: "OK Glass", voice menu, touch menu, contextual voice

The actual "doing" part

  • Focus on the action, not the agent
  • Cut down the time between intent and action
  • Easy to say

Understanding the patterns

Periodic Ongoing task Immersion

Periodic

  • Insert timeline cards into the timeline without invocation
  • Good case for notifications from an remote service
  • Most common case for Mirror API

From Google's Glass Design Docs - Periodic

Ongoing task

  • Long running live cards
  • User can leave/return as needed
  • Only the GDK

From Google's Glass Design Docs - Ongoing

Immersion

  • Take over the timeline experience
  • Users leave by swiping down
  • Only the GDK (Android activities)

From Google's Glass Design Docs - Ongoing

Tools that rock: Glassware Flow Designer

Easiser way to design your app flow

What if my app doesn't fit the pattern?

  • Consider your end goal for your application
  • GDK: real-time, offline, access to the hardware
  • Mirror API: platform indepent, common infratructure, use built-in functionality
  • Hybrid approaches: Mirror card adds menu item that invokes intent to start

There is no one answer

  • OH: "Java is the only way to write for Glass".
  • OH: "Mirror API isn't powerful".

Both statements above = NOT TRUE

Decide for yourself, don't let others decide for you.

Only you know your goal and spec.

Jumping in to the Mirror API

  • You're platform independent! You can choose your language and platform.
  • You can make raw requests or for simplier access utilize the Google APIs Client Libs for your language

Mirror API Setup

Why Google+ API?

OAuth

  • Create your Client ID for you app under APIs & auth > Credentials
  • You can set the Redirect URI's to wherever your project will be dev/deployed

Great! Let's fire something up!

  • Doing the sign in - Google+ Sign In button
  • Button implements JavaScript hooks in client side that call our server
connectServer: function() {
    console.log(this.authResult.code);
    $.ajax({
      type: 'POST',
      url: "//" + window.location.host + '/connect?state=' + this.state,
      contentType: 'application/octet-stream; charset=utf-8',
      success: function(result) {
        console.log(result);
      },
      processData: false,
      data: this.authResult.code
    });
  }

Server handles the tokens

try:
    user_credentials = _oauth_flow().step2_exchange(request.data)
except FlowExchangeError:
    response = make_response(json.dumps('Failed to upgrade the authorization code.'), 401)
    response.headers['Content-Type'] = 'application/json'
    return response

# Set our session
session['credentials'] = user_credentials
session['user_id'] = user_credentials.id_token['sub']

Once we have tokens we get into Mirror API

  • Timeline
  • Menu
  • Subscriptions

I thought the Mirror API had more?

Yes, it does. But let's dive into the basics first.

The timeline and you: Insert a card

  • Timeline consists of cards or various types and styles
  • Inserting a welcome card with a user auth'ed Mirror service
mirror_service = _authorized_mirror_service(user_credentials)

mirror_service.timeline().insert(body={
    'notification': {'level': 'DEFAULT'},
    'text': 'Welcome to Glass! This is my very first timeline card insert.'
  ).execute()

Timeline cards have lots of options

mirror_service.timeline().insert(body={
    'notification': {'level': 'DEFAULT'},
    'html': '<article><section><p class="text-auto-size">Welcome to
      <em class="yellow">Glass!</em> This is my very first timeline card insert.</p>
      </section></article>'
  ).execute()

Timeline cards: more options!

  • Location
mirror_service.timeline().insert(body={
    'notification': {'level': 'DEFAULT'},
    'location':  {
      'latitude': 37.7692,
      'longitude': 120.8569,
      "displayName": "Season 6 Dreams Meetup",
      "address": "Group Study Room F, Greendale Community College"
    }
  ).execute()

Timeline cards: more options!

  • Payload
mirror_service.timeline().insert(body={
    'notification': {'level': 'DEFAULT'},
    'text': 'Video time!',
    'payload': 'http://mymystical.url/video.mp4',
    'menuItems': [{'action': 'PLAY_VIDEO'}]
  ).execute()

Was that a Menu Item in that last example?

  • Menu options give uses the ability to act on a timeline card
  • Lots of built in options: REPLY, DELETE, READ_ALOUD, NAVIGATE, OPEN_URI and many more
  • Can set custom menu options with CUSTOM that callback to a subscription

A basic MenuItem example

mirror_service.timeline().insert(body={
    'notification': {'level': 'DEFAULT'},
    'text': 'Timeline card with a couple menu options.',
    'menuItems': [
        {'action': 'CUSTOM', 'id': 'my-id', 'values': [ { 'displayName': "My Menu Item" } ] },
        {'action': 'DELETE'}]
    }).execute()

From Menu Actions to Subscriptions

  • When a user acts on a menu item, we can listen for those action via a Subscription
  • We set a callbackUrl that allows our application to receive a POST notification from the Mirror API.

A sample subscription for timeline

mirror_service.subscriptions().insert(body={
   'collection': 'timeline',
   'userToken': user_id,
   'verifyToken': 'something_only_you_know',
   'callbackUrl': 'https://my-project-id.appspot.com/myGlassCallback',
   'operation': ['UPDATE']
  }).execute()

Breaking it down

  • collection: timeline or location
  • userToken: something to id your user
  • verifyToken: something only you know to verify notification
  • operation: UPDATE, INSERT, DELETE

Then there's callbackUrl

  • In dev, you can use the SSL proxy
  • In prod, you have to use an SSL endpoint
  • The problem: no one wants to deploy remote even with a SSL proxy
  • Solution: ngrok

ngork for the win

  • Allows you to create a reverse tunnel
  • Gives you an nice web panel to see incoming requests
  • Gives you an SSL endpoint
 ~ $ ngrok -authtoken YOURTOKEN TARGETPORT
 ~ $ ngrok 8080

Ngrok in action

Ngrok viewing Mirror callback data

The code change is tiny

mirror_service.subscriptions().insert(body={
   'collection': 'timeline',
   'userToken': user_id,
   'verifyToken': 'something_only_you_know',
   'callbackUrl': 'https://RANDOM.ngrok.com/myGlassCallback',
   'operation': ['UPDATE']
  }).execute()

That's the crash course

I must do more!

Making Mirror shine

  • Mirror has a lot of power, we can send all kinds of sticky, relevent data
  • But how do we do that?
  • To the samples!

Request: How do you send a card daily?

  • Daily sort of jobs generally require cron
  • If we're on our own boxes, we could schedule a script in crontab
  • If we're on App Engine, we can use App Engines Cron

A simple App Engine Cron job

@glassdailycard.route('/dailyjob')
def dailyjob():
    today = datetime.date.today()
    query_get_todays_card = CronCards.query(CronCards.date == today)
    result_get_todays_card = query_get_todays_card.get()

    # Do we have a card to send today?
    if result_get_todays_card is not None:
        timelinecard_body = {
            'notification': {'level': 'DEFAULT'},
            'text': result_get_todays_card.card,
            'menuItems': [{'action': 'DELETE'}]
        }

        query_get_users = UserProperties.query()
        for user in query_get_users.fetch():
            user_credentials = _credentials_for_user(user.key.id()).get()
            _authorized_mirror_service(user_credentials).timeline().insert(body=timelinecard_body).execute()

    response = make_response("{}", 200)
    response.headers['Content-Type'] = 'application/json'
    return response

glass-daily-task

Request: How do you send a card random in future?

  • Generally requires a work queue of some type
  • If we're on our own boxes, we could use beanstalkd
  • If we're on App Engine, we can use App Engines Push Task Queue

A simple App Engine Task Queue

@glasstaskfuture.route('/glassCallback', methods=['POST'])
def glassCallback():
    """Our callback when the use selects out CUSTOM menu option on the start card."""
    notification = request.data

    if random.choice([True, False]):
        # we're going to send it along to the task queue
        google.appengine.api.taskqueue.add(
          url='/taskHandler',
          payload=notification,
          countdown=60)
    else:
        now = datetime.datetime.now()
        future = now + datetime.timedelta(0,60)
        google.appengine.api.taskqueue.add(
          url='/taskHandler',
          payload=notification,
          eta=future)

    # It's cool Mirror API
    #
    # Remember, we must return 200 as quickly as we can!
    #
    return('OK')

glass-task-future

  • Simple starter project that shows basics of using App Engine Task Queue with a subscription callback
  • Uses Google+ Sign-In
  • Repo: justinribeiro/glass-task-future

What about the GDK?

  • Like developing for Android
  • For when you need more access to Glass hardware

UPDATE: Nice things that happened this week

  • Whoo hooo, notification sync with WearableExtender!
  • Includes voice reply, pages, stacks...the works
  • I'll talk more about this during Wear talk

The Live card

  • When you need that long running context
  • "OK Glass"...voice trigger
  • Low-Freq updates to Android views
  • High-Freq updates can use graphics framework (OpenGL)

Example: low frequency mqtt live card

Because we need to talk to Arduino people.

Of course that code is on Github

Push it: Glass to the web

  • Lots of ways to do this
  • I heart the MQTT - let's use a broker!
  • Glass > Broker > WebSocket > Your Browser

glass-gdk-timer-mqtt

A Glass like View

  • UI Widgets: CardBuilder and CardScrollAdapter
  • CardBuilder.Layout allows us to create Glass-like cards
  • We can convert to an Android View or RemoteViews
  • CardScrollAdapter gives us Glass-like timeline

Sensors oh my!

  • Host of sensors for use: TYPE_ACCELEROMETER, TYPE_LIGHT, TYPE_ROTATION_VECTOR
  • Touch Gesture > D-pad key events
  • Location can be tricky: don't use LocationManager.getBestProvider()

Speaking of location...

LocationManager locationManager; // initialized elsewhere

// This example requests fine accuracy and requires altitude, but
// these criteria could be whatever you want.
Criteria criteria = new Criteria();
criteria.setAccuracy(Criteria.ACCURACY_FINE);
criteria.setAltitudeRequired(true);

List<string> providers = locationManager.getProviders(
        criteria, true /* enabledOnly */);

for (String provider : providers) {
    locationManager.requestLocationUpdates(provider, minTime,
            minDistance, listener);
}

</string>

With great GDK power, comes overheating

  • If you push the hardware too hard you will get overheating messages
  • Offload if you can, your battery is limited
  • Just because you can, doesn't mean you should

Further reading / resources

The End

Thank you!

@justinribeiro +Justin Ribeiro justinribeiro