Paris AI, Robotics & IoT Meetup #2

Clodéric Mars

03/12/2015

In this presentation, I'll introduce what we believe to be the main challenge for the next generation of IoT services and show how craft ai has been designed to help developers overcome it. Press space to move to the next slide.

Smart Objects services now...

Dashboards

The first big trend is: dashboards to see data coming from objects and send commands. Let's confuse end users with aircraft pilots and give them a bunch of graphs, metrics and actuators to play with.

This example from Withings features sleeping patterns, percentages, numbers... how exciting!

This is the result of a zealous user showing everything she can see in the SmartThings dashboards. Pie charts, flags, bar graphs, science! I don't believe that this information is very useful or actionable for the user.

Just one more example, in a Smart Home context from Zipato. This time the user is not only given metrics, but also direct control over an array of actuators... which he has to manipulate one by one.

Notifications

Another big trend is notifications. In practice, users don't go to their dashboard to pull informations, hence this other strategy: pushing notifications in real-time, as events occur.

Sometimes it's not so bad, you get qualified, hopefully interesting, information.

But this can quickly get out of hand. Here 3 simultaneous notifications are sent from a single SmartThings service. This is not unusual and it is a consequence of the lack of reasoning on incoming events: in this case, Baby Cakes and rose@securitygem.com probably arrived together through the Garage.

Delivering only the raw event is rarely useful: "Do the watch actually tells me my arm just moved?" I hope it's not what it means, but in any case this alert is not useful.

The poor quality of these smart notifications leads to only one outcome, they pile up...

Are these the best we can do?

Smart Services are...

Services can be actually smart.

The Nest Thermostat is one of our references. It provides a useful service by having a smart and simple UX and by seamlessly learning from the users input as well as reacting to its context, the presence of someone in the house.

Another example of a smart service around a connected device is Prizm, a virtual DJ that chooses the best music to play based what it perceives about the context and what it has learned about the users tastes.

Outside of IoT, several virtual assistants are emerging. They provide services that are perceived as intelligent and useful by users. Apple Siri is able to answer queries in context, for example they take into account which movie is currently playing, and based on information it has on the user.

Google Now relies on its user's context to proactively push useful information.

Facebook M is the latest addition to this family. It provides a full-fledged concierge, even relying on humans to deliver smartness.

Smart needs

Awareness

Learning

Two aspects are common to these 5 examples of smart services:

• Awareness, i.e. gathering and leveraging data from the context through sensors or connection to existing services (messaging, social, ...);
• Learning, i.e. progressively understanding how each individual user interacts with the service, its tastes and preferences.

AI!

AI techniques can be used to enable awareness and learning.

AI?

I'm sure most of you associate AI to Big Data & Statistical Machine Learning, maybe you're convinced you need to hire a Data Scientist right now! But there is much more to AI.

But!

The thing is, this kind of AI is not that straightforward.

• Algorithms and data structures are pretty hard to implement,
• Even if you use existing libraries or services, it's still very hard to control to achieve your business objectives,
• Even if you have a data scientist in house, business experts still need to understand how this automated decision-making impact their service.

Maybe, there's another way to do AI!

We need to take a step back and think again. With our need for awareness and learning, we can find better AI techniques and better ways to apply them.

But first, some background

To understand where our ideas on AI come form, let me give you a little bit of background on our experience working on game AI technologies for several years.

Game AI

Autonomous characters

Entertainment | Art | Pedagogy | Simulation

Game AI covers a variety of techniques designed to create autonomous characters in several industries: Video Games (duh), Movies, Training Simulation & Serious Games, or any simulation of systems where humans are involved.

This is a simulation project I've worked on at Golaem for the French railways company SNCF.

It involved autonomous navigation algorithms as well as the modeling of passenger behaviors from statistical and qualitative survey.

Again at Golaem, the same human behavior modeling stack is used by Golaem Crowd a product dedicated to the cinema industry.

What's crucial is to enable artists to direct the virtual extras to achieve their vision. AI is at the service of this vision.

Most of the craft ai team, including myself, worked at MASA Group on LIFE, an AI middleware for Games & Simulation.

Our main problematic was to enable designers to create autonomous characters that exhibit the behavior they needed. We achieved that using Behavior Trees inside a carefully designed tool.

Use cases of LIFE include games, such as this "The Sims"-like games where characters interact with their environement autonomously based on their need and desires.

For this kind of AI, control and explainability are crucial to balance the gameplay.

The main use case for LIFE is military training simulation. In this example, the characters are created to follow specific military procedures.

Reaching its training goals requires:

• a controlled environment to make the trainee work on specific skills,
• an autonomous environment that can react properly to whatever the trainee decides to do.

Lessons Learned

I've worked for 6 years in this field and, on top of being able to show pretty videos, I learned some valuable lessons that are at the core of craft ai's vision.

Intelligent = credible as intelligent

Perception | Usefulness | != Algorithm

Creating "intelligent" interactions / experiences / services is about convincing the end-user they are intelligent and meet her needs.

More importantly, you don't need fancy algorithms to make something intelligent.

AI = Content production

Control | Predictability | Explainability

Designers know the ideal behavior their autonomous characters should have, so given unlimited time & resources they could create content accounting for all possible cases. AI techniques enables the creation of such content within realistic constraints.

That's why these techniques need to allow the same Quality Assurance process any content creation would. Hence my emphasis on control, predictability and explainability.

Developers != AI experts

Combinatorial complexity | StatefulnessAsynchronicity

The last takeaway, and in fact the drive behind the creation of AI authoring tools, is the realization that most developers are not comfortable with decision-making AI techniques.

Furthermore, only some big companies have the resources to house teams of specialists.

craft ai was born from the realization that we can leverage what we learned in Game AI to enable every developer to create aware and learning services.

Enable developers to create AI that is

doing what they want

aware of their users' context

learning from their users

Directed Adaptable Agents

The approach behind craft ai values the control of the AI by the developer. This is key to make the solution efficient and easy to use. We focus our effort on enabling the creation of useful services for the end-user.

We name this approach directed adaptable agents: the developers direct agents that are able to adapt their behavior to the context and history.

In practice, craft ai consists of two modules:

• The workbench web app to define and inspect how the agents behave;
• The cloud-based engine runs the behavors in instances, it is accessible with a REST API, it is used in two ways:
• Integrate craft ai with the developed application;
• Integrate craft ai with services and sensors through what we call actions.

This is craft ai browser-based workbench. It's basically an IDE for Behavior Trees (BTs): the visual language craft ai uses to create AI by implementing the behavior of agents.

This explicit grammar helps a lot with the combinatorial explosion thanks to its control flow nodes (in colors) and the asynchronicity and interruptibility of external calls thanks to its action formalism (reresented by !).

For further information check our user documentation.

Once defined, BTs can be run by craft ai's cloud-based engine and controlled through its REST API.

An app using craft ai will create, generally, an instance per end-user, each consisting of one or more agents executing BTs. During the instance's lifetime each agent will store information about the end-user in its knowledge. This will be used to make more personalized decisions, to learn from the user.

As mentionned earlier, the Nest Thermostat is a very good example of a smart IoT service. That's why we implemented one using craft ai.

Thanks to our explicit agent-based approach, we've been able to combine the learning of the temperature planning, differencitate it explicitely between working and non-working day and add some heater monitoring.

The demo is accessible and explained in more details here.

10 experts working on it since 03/2015

Released on 10/2015

We've been working on the product since March 2015 and released a first version in October 2015. We're very lucky to already work with a range of early adopters in different industries. From personal assistants to enterprise processes, craft ai is not just for IoT.

Hackathon IoT

27- 29 Nov 2015

Before wrapping up, let's take a look at actual projects that were developped using craft ai during a Hackathon we sponsored in late November.

10 teams worked for 2 days to create connected devices and their services. Further information on the hackathon can be found here (in French)

AI?

Make sense of sensors

Model physical systems

Push relevant notifications

Personalize assistants

Most projects in the hackathon used, or planned to use, some AI to implement their service. The last two points are perfect use cases for craft ai.

SNCF Now

Signal equipment issues with the push of a button

The SNCF Now team, winners of the mobility category, developed a physical button, that can be placed on existing equipments in train stations, to let passengers report breakdowns and other issues.

The project includes a maintenance platform, powered by craft ai, that routes alerts to the right technician in the station.

Smart Mirror

A beauty assistant in your bathroom

The Smart Mirror, winner of the beauty & wellness category, is a bathroom mirror with a camera and screen that is able to provide personalized beauty advices based on scans from the camera, user habits and external information such as the events in her day.

craft ai is used to implement this personal assistant leveraging these different data sources.

Take Away

Fun!

Commit

Manage Expectationsobject-centric, no time for servicesmarketer/developer ratio

Being a tech partner for this hackathon was really fun, we sere able to explain craft ai and get feedback from a group of motivated users. This was also a big commitment, between the workshop we organized a week earlier and the presence during the week-end this was an important effort for the team.

Finally, we had to manage our expectations, most of the teams were interested by what craft ai allowed them to do but most didn't had the time to actually play with it. The hackathon was very object-centric and there was only a few developers.

Good IoT services are Aware & Learn

Usable tools are Whiteboxes

As a wrap-up for this presentation, I'd like to reemphasize my main point:

• Awareness and learning are key to the creation of good services, especially in the IoT field,
• The concept of whitebox, the understandable and controllable behavior of services, can really help make AI accessible to every developer.

Sign up now!

@clodericmars | @craft_ai

craft ai is already available to help you with that, sign up now at workbench.craft.ai!

View sources on github...

The content of this presentation is published under the Attribution-NonCommercial 4.0 International