Travelling Through Pipes and Flying Through the Air; – the mobile network and the need for Offline



Travelling Through Pipes and Flying Through the Air; – the mobile network and the need for Offline

0 0


lwsslides

Slides for London Web Standards 10/02/14

On Github nrooney / lwsslides

Travelling Through Pipes and Flying Through the Air;

the mobile network and the need for Offline

Natasha Rooney - @thisNatasha

Hi! I’m Natasha Rooney Today I’ll be talking about two main topics - The mobile network - dealing with this with new web techniques, including offline.

Why do we care?

So why do we, as developers and technologists, care about the mobile network or running web apps offline?!
The way we design our web apps in the past meant we didn’t need to care about where the user was. Most likely they would be at home with a reliable connection. Nowadays and with a mobile device in hand they are able to access the web whilst travelling around. Naturally they will want experience the web in the same way when on wifi that’s why we care, we want to ensure that at every stage the user still has a good user experience with his favourite web apps. This means we need an understanding of...

WiFi

wifi

Mobile Networks

mobile networks

And all of these:

HTTP, TCP, HTTPS, TLS, XHR Reqests, HTTP2, UDP...

and all these others!

Mobile Networks

&

Dealing with cruddy connetions

(old APIs, new APIs, super cool ultra new APIs)

But today let’s talk about mobile connections and dealing with the connection from ultra 4g to sod all.

3G

4G

Data Throughput Up to 3.1Mbps with an average speed range between 0.5 to 1.5 Mbps Practically speaking, 2 to 12 Mbps (Telstra in Australia claims up to 40 Mbps) but potential estimated at a range of 100 to 300 Mbps. Peak Upload Rate 5 Mbps 500 Mbps Peak Download Rate 100 Mbps 1 Gbps Switching Technique packet switching packet switching, message switching Network Architecture Wide Area Cell Based Integration of wireless LAN and Wide area. Services And Applications CDMA 2000, UMTS, EDGE etc Wimax2 and LTE-Advance Forward error correction (FEC) 3G uses Turbo codes for error correction. Concatenated codes are used for error corrections in 4G. Frequency Band 1.8 – 2.5 GHz 2 – 8 GHz

source: http://www.diffen.com/difference/3G_vs_4G

- most of you here will have a device capable of up to 3g or 4g speeds - many people get mixed up with LTE and 4G well, - LTE will involve a complete rollout of new hardware and software into the radio and core networks of mobile operators - 4G, doesn’t necessity mean this...

4G Requirements

  • All-IP packet switched network
  • Peak data rates: <100 Mbit/s (high mobility)
  • Peak data rates: <1 Gbit/s (low mobility)
  • Dynamically share network resources
  • Scalable channel bandwidths: 5–20 MHz
  • Peak link spectral efficiency: 15 bit/s/Hz (downlink), 6.75 bit/s/Hz (uplink)
  • System spectral efficiency: <3 bit/s/Hz/cell (downlink), 2.25 bit/s/Hz/(indoor usage)
  • Smooth heterogeneous networks handovers
  • Support next generation multimedia.
- the names ‘3g’ and ‘4g’ are both a set of requirements for your device and network connection to meet - it so happens that 4G requirements line up with the goals of LTE, but some 3G evolutions have also managed to hit these requirements, allowing marketing teams to advertise ‘4G’ without an LTE rollout. LTE rollouts will be deployed in parallel and on a separate spectrum from 3G. But even though 4G is being rolled out we still have two problems:

1. I’m not always on 4G

True, - luckily each device holds radios for 2G and 3G (so this is what you see when your 4G signal drops), which is helpful and...

2. 3G is still the most popular by a long way

And even though 4G is rolling out globally, HSPA+ (3G tech) will be the most dominant of this decade)

LTE in the UK?

93,211 connections

3G in the UK?

49,013,306 connections

source: GSMA Intelligence

so...

Answer: Support Everything!

3G, LTE, 2G, Pigeon (maybe not pigeon)...

- LTE and HSPA+ are growing fast, but you still need to support everything - and this isn’t the same as supporting wifi. So, understanding the network and it's latencies will help us achieve this.

The Network

Well, this is a network, the basic version. RAN: towers and tracking areas Core Network: connects RAN to the net External Network: The internet!

Radio Resource Controller

Inside the RAN (and located at different points depending on the ‘G’) lives the Radio Resource Controller. - The RCC controls all the connections between the device and the network - it allocates bandwidth, power state of the radio, scheduling (who connected when) etc. - power state because the radio eats battery! - therefore it impacts latency

Radio Resource Controller

Wifi doesn’t use this method as there are smaller number of users is a much more confined area. You’ll notice how wifi suffers as soon as there are too many users, this is due to high collision rates. But what happened when we make a request? [TEXT FROM BOOK] Wifi is different “can transmit at any point, but due to the use of the shared radio medium can also lead to high collision rates and unpredictable performance if there are many active users. Further, because any WiFi peer could start transmitting at any time, all others must also be ready to receive. The radio is always on, which consumes a lot of power.” 3G and 4G: The Radio Resource Controller (RRC) mediates all connection management between the device in use. The RRC has direct impact on latency, throughput, and battery life of the device in use. As the name implies, the Radio Resource Controller assumes full responsibility over scheduling of who talks when, allocated bandwidth, the signal power used, the power state of each device, and a dozen other variables. Simply put, the RRC is the brains of the radio access network. Want to send controlled by the network

Making a Request

Hold on this may take a while! STSPE[i]PSM SYNC / RRC - Lets imagine the phone is in an idle state, the radio will first synchronise with the nearest radio tower, and and request a new context to be established. - LTE - requires several round trips, up to 100 ms - OLDER - RRC on the serving gateway inside the core network, so this could take seconds!

Making a Request

TRANSMIT / - We now have our resource allocation/scheudling and we’re able to transmit data (yay!) between device and tower is “user-plane one way latency” 5ms 4G, so see how RRC slows things down! (must stay in high powered state though)

Making a Request

CORE NETWORK/ SGW PDNGW - Then go go through the Core Network, through the serving gateway to the packet data network gateway and out to the internet! SGW point of interconnect between the radio-side and the EPC. Routes the incoming and outgoing IP packets PDNGW point of interconnect between the EPC and the external IP networks. Routes packets to and from the PDNs. Performs various functions such as IP address / IP prefix allocation or policy control and charging latency varies from carrier to carries

Making a Request

INTERNET

Making a Request

Making a Request

CORE NETWORK / PDNGW SGW: We then go back to the PDNGW and then to the SGW Interesting fact time! All connections are terminated at the PDNGW. What does that mean? Well, a persistent TCP connection could exist between the PDNGW and the external newtork, even if your radio is off!

Making a Request

MME - The SGW queries the MME. The MME helps the SGW working out which tower to send the data. This is important as if the user has moved to a different location, the tower which established the connection might not be the right one to send the data back to. Towers are stored in “Tracking Areas”. This is a collection of towers. So, if the device is idle the MME will need to find it so it pages the towers which are in the tracking area, the towers send a broadcast, the device wakes up for a short while, understands that it needs to reconnect and connects going through the same RRC negotiation. TOWER - the tower that found the user notifies the MME

Making a Request

SGW - which tells the serving gateway, then going back to the

Making a Request

TOWER - then to the DEVICE If the device is not idle this whole paging stuff is skipped. So first hop is the greatest when it comes to latency! Good to know. ----------- RRC: 10 to 100 milliseconds. (4G) HSPA+ state transitions are competitive with LTE. (3G) HSPA and CDMA state transitions can take several seconds. Every network transfer, no matter the size, incurs an energy tail. http://www.3gpp.org/technologies/keywords-acronyms/100-the-evolved-packet-core

Latencies

So say the user has their content, radio has been down for a bit so the RRC move into a state where the network context is still established, but no network resources are currently allocated (can be long or short, DRX). If the user sends a new request they hit up a whole load of new latencies: RRC needs to change to a different state to allow the user to transmit again - (Control-plane latency: Fixed) up to 100 ms for idle to active, up to 50 ms for dormant to active. Latency for each packet to reach the radio tower - or the device in the other direction! - (User-plane latency: Fixed) up to 5 ms. Then whatever happens between the radio tower and the external network through the core network - (Core network latency: Carrier dependent) in practice, 30–100 ms. Then from the PDN GW to the Internet - (Internet routing latency: Variable latency)

Some Guidelines

  • send larger data, fewer times
  • send data when radio is on
  • avoid polling (more later!)
  • refrain from keep-alives.
  • design for the worst connection
  • anticipate latency
- should maximize the amount of transferred data while the radio is on and then seek to minimize the number of additional data transfers. - interestingly this isn’t the same for wifi, transferring little bites is much better here, but big bursts is good for radio. So, what HTML5 APIs can we use to help us here. There has been a lot of work on this lately, let's go through these here...

Can HTML5 APIs help?

navigator.onLine

var x = "Is the browser online? " + navigator.onLine;
document.getElementById("demo").innerHTML = x;

source: http://www.w3schools.com/jsref/prop_nav_online.asp

around 6 years ago navigator.onLine notifications was specced
this told the app when a user was online! Fabby!

navigator.onLine

But how do we know when we’re online?

navigator.onLine

A user could have a connection to their router, but the router could be disconnected from the ISP, or the ISP could be disconnected from the gamefaqs site.

navigator.onLine

False positives with this API were far far too common (also happens with virtual connections). Say we get around this by querying the server we’re requesting data from, isn’t this just the same than a normal http request? To learn more, see the HTML5 Rocks article, Working Off the Grid.

Network Information API

Originally: "Exposes the current connection type. The value returned is one of the following strings, case-sensitively: unknown, ethernet, wifi, 2g, 3g, 4g, none.""

source: http://www.w3.org/TR/2011/WD-netinfo-api-20110607/

So, yeah, earlier we have asked developers to pay attention to the network, so we made this funky net info spec! What does it do? Well first you could ask it to return the network the user was on, ‘3G’, ‘Wifi” etc.

LTE 50-70 Mbps

(comparable to some WiFi networks)

But this proved less than useful as 4G speeds are about as good at some wifi.

Network Information API v2

//onchange event
						navigator.connection.addEventListener('change', show, false);

						//get bandwidth
						if (navigator.connection.bandwidth > 2) {
							i.src = "http://example.com/pony_hd.png";
						} else {
							i.src = "http://example.com/pony_ld.png";
						}

source: http://www.w3.org/TR/netinfo-api/

So, the network info api evolved to allow the developer to find out what the bandwidth of the user was. Hey awesome!
Well, not really. Say we start off at the office, we start working with a web app and it tests our connection bandwidth to see what it can send. Then we move, we go down the street, and onto the tube. All the while our app expects some super fast connection and then fails. Err… crappy experience. On top moving between different coverage areas of the radio tower, congestion from nearby users, ambient interference etc also makes a difference!

"change is the only constant in mobile networks"

so, I like what Ilya says about this topic “change is the only constant in mobile networks” So this is why the guidelines say to design web apps for the cruddiest connection “as even on 4G, you may measure your throughput as just a few hundred Kbit/s, and then move your radio a few inches and get Mbit/s+ performance!” So bandwidth and latency estimation is a tricky business, and maybe even a uncessary evil (the worst kind of evil!). TAKE WHAT YOU KNOW ABOUT MOBILE NEWTORKS AND APPLY IT TO YOUR WEB APPS.
Ok so there are some problems with the net info api, but there does seem to be a good use case for it still.
in the ‘closing the gap with native’ focus that is driving some of the web standards right now, we noticed a massive gap between native and web when it came to ‘downloading over a wifi network vs a mobile network’ for content. Yes, as 4G gets rolled out the difference in performance will be negligable. But, your mobile network may charge you for data, and your mobile network isn’t as reliable at holding a consistenly high connection, so you might not even want something like a movie downloading over this medium. On top of all of this the radio on your mobile drains battery only second to the screen. You may not want such a high bettery consumption activity happening on your phone.
spec work is now happening on this, which will hopefully make a much more robust solution based on much stronger use cases.

Minimise Polling

But I can't use push...

so our guidelines also said we should minimise polling which is a shame because polling is a great way to get around the push notification issue for web (or lack of one!). But even tiny requests are a big drain on a device’s battery, as every request, however small, will push the device into a connected state if it isn’t already in one. So the guidelines say no polling and pull all your data into fewer, larger requests, and non-critical ones should just wait.

Server-Sent Events

Push API

source: http://www.w3.org/Mobile/mobile-web-app-state/

With new APIs however we can get around the push API issue with an actual push API! Two HTML5 APIs exist which deal with push messaging: push and server sent events. you can see here the implementation is a little different right now.

Server-Sent Events

// Connect to server, listen for messages, manage data:
						var source = new EventSource('/stats');
						source.onopen = function () {
						  connectionOpen(true);
						};

						source.onerror = function () {
						  connectionOpen(false);
						};

						source.addEventListener('connections', updateConnections,false);
						source.addEventListener('requests', updateRequests, false);
						source.addEventListener('uptime', updateUptime, false);

						source.onmessage = function (event) {
						  // a message without a type was fired
						};

source: http://html5doctor.com/server-sent-events/

The Server-Sent Events API allows triggering DOM events based on push notifications (via HTTP and other protocols.). This is great for news feeds or a twitter streaming app. The main points of interest: new EventSource(url) — immediately starts listening for events on the given URL. readyState — tells us if we’re connecting (0), open (1), or closed (2). onopen, onmessage — two events that we can listen for on the new EventSource object. By default, the message event will fire when new messages are received. addEventListener — we can also listen for custom messages. event.data — the contents of the message (string) close — closes the connection from the client side.

Push API

(W3C Working Draft)

//register
						function registerPush() {
						  navigator.push.register().then(
						    function(value) {
						      pushRegistrations.push(value);
						      registerWithAppServer(value);
						    },
						    function(reason) {alert("error: "+ reason);}  
						  );
						}
						//handler
						navigator.setMessageHandler("push", gotPush);
						//manage the message
						function gotPush(message) {
						  //do stuff
						}

source: http://www.w3.org/TR/push-api/

The Push API was a little further behind development as it was locked behind a patent row! The Push API would allow Web applications to receive server-sent messages whether or not the said Web app is active in a browser window. It’s only in working draft right now.

Offline

Native App Win?

- Even after all these guidelines, hints and tips, native apps still have some performance gains in times of intermittent or no connection (as the shell of their app is cached as well as the last pulled down content) - Some people find that as mobile networks get faster (LTE) that maybe offline isn’t as necessary, but the issues with switching between base station tracking areas and going off-net and on-net (stupid mobile network term) means the user experience isn’t always consistent.

App Cache

//register
						CACHE MANIFEST
						# 2013-10-22:v1

						# Explicitly cached 
						CACHE:   
						/offlineanime/anime.json   
						/offlineanime/app/css/app.css
						/offlineanime/app/img/404.png   
						/offlineanime/app/lib/angular/angular.js   
						/offlineanime/app/lib/angular/angular-route.js       
						/offlineanime/app/js/controllers.js    
						/offlineanime/app/js/app.js
						/offlineanime/app/partials/anime-list.html
						/offlineanime/app/partials/anime-detail.html 

						# Resources that require the user to be online. 
						NETWORK:
						# Don't actually need other URLs because * covers them       
						# Needed * for ajax requests
						*

						FALLBACK:
						http://static.hummingbird.me /offlineanime/app/img/404.png #doesn't work!
So in an effort to solve the offline issue the App cache was created. Working with a few other offline-capable APIs a developer could create a shiney offline-capable app. And it was easy! just a nice declarative app.appcahe file. Alas… - issues - Jake "comes with a manual" Trying not to be too swayed by what I had read I actually tried to build an offline app with app cache too. My app was built on angular, using XHR requests, using images from others domains, storing content via json in localstorage (to string). I built the app, it worked. then I had to make the app cache. I dreaded it. I was sick to my stomach but I knew I had to do it. So i wrote an app cache with a few urls and then these were my nightmares:

App Cache NIGHTMARES

  • AppCache wasn't loading(Solved by editing http.conf)
  • EVERYTHING BROKE!(Solved by declaring everything)
  • XHR requests stopped working(Solved by adding * to Network in Appcache file for some weird reason)
  • Nasty image placeholders when offline(Solved by nasty image error hack)
  • Debugging became an difficult(Solved only by more beer.)
but there were some things i liked, it was delcarative, worked great with localstorage (or localstaorage was great). But these issues just couldn’t be ignored. so do we just kill this whole offline thing? or look for a better way?

Service Worker!

- apps rather than pages - Power to the developer, but also puts responsibility on developers on what/how/when stuff gets cached. [I LIKE THIS]

Service Worker Can:

  • manage caches
  • listen for network events (e.g. resource requests)
  • decide what content to display
- service worker will: manage caches listen for events eg resource requests decide what content to display you can have...

Multiple Caches

Cache Info

(What is cached? How hold are they? Is an update available?)

Multiple Fallbacks

(Based on Promises)

ServiceWorker: Caching

this.addEventListener("install", function(e) {
						//Create a cache of resources. Begins the process of fetching them.
						  var shellResources = new Cache(
						  "/app.html",
						  "/assets/v1/base.css",
						  "/assets/v1/app.js",
						  "/assets/v1/logo.png",
						  "/assets/v1/intro_video.webm",
						  );

						  //Add Cache to the global so it can be used later during onfetch
						  caches.set("shell-v1", shellResources);

						  //The coast is only clear when all the resources are ready.
						  e.waitUntil(shellResources.ready());
						});

						this.addEventListener("fetch", function(e) {
						  //No "onfetch" events are dispatched to the ServiceWorker 
						  //until it successfully installs.

						  //All operations on caches are async, including matching URLs, so we use
						  //Promises heavily. e.respondWith() even takes Promises to enable this:
						  e.respondWith(caches.match(e.request.url, "shell-v1"));
						});

source: https://github.com/slightlyoff/ServiceWorker/blob/master/explainer.md

Go read the explainer document! Here i want to jump straight over SW’s registration and into the caching (my fav part)! caching.js Each ServiceWorker has a global caches Map which holds instances of Cache. CACHE: Promises which represent responses which may or may not yet be available from the network. CACHE MAP: contains list of desired cached urls, and these all now have promises attached new caches will need to be loaded 100% before they take over

ServiceWorker: Fallbacks

this.addEventListener("fetch", function(e) {
						  var url = e.request.url;
						  if (url == inventory) {
						    e.respondWith(
						      fetch(url).then(
						        null,
						        function() { return caches.match(fallbackInventory); }
						      )
						    );
						  }
						});

source: https://github.com/slightlyoff/ServiceWorker/blob/master/explainer.md

- get a promise from fetch - If it resolves successfully (that is, gets what it wanted from the network), that's the value passed to the page. Simple. - If not, the .then() function adds a callback to handle the error which, in this case, returns some default content. We don't need a callback for the success case here, so that's just specified as null. - You can have many fallbacks!

ServiceWorker: Rules

(just a few)

new service worker only happens when old one dies and all tabs are closed fallback to normal resource loading in all instances iframes not ok multiple tabs ok wildcards are ok - longest prefix match too last reg wins iframe belongs to its own serviceworker (CORs) SW don’t hold state

So, Mobile Network?

  • understand the mobile network
  • latencies
  • some guidelines
  • design for the worst connection
  • use push instead of polling
  • bigger bursts
  • HTML5 APIs including offline
Ah yes that’s what we were talking about. So, let’s go through what we learned:

Thank-you

Especially: Ilya Grigorik (@igrigorik), Jake Archibald (@jaffathecake), Alex Russell (@slightlylate),GSMA Intelligence, W3C WebApps, SysApps, DeviceApps WGs, W3C Webmob IG Read Ilya's Book! "High Performance Browser Networking"

0