The Importance of import and export

View Github Repository Open presentation in a new window

benjamn

See all presentation from benjamn

The Importance of import and export

1 9

empirenode-2015

Slides for my talk at EmpireNode 2015

On Github benjamn / empirenode-2015

The Importance of import and export 

{ github,
  twitter,
  instagram,
  facebook
}.com/benjamn

export * from "http://benjamn.github.io/empirenode-2015"

Let's talk about eval

The mere presence of eval in a function thwarts almost any kind of optimization or static analysis.

Most JavaScript programmers I know consider eval more harmful than goto.

Heaven forbid a snippet of user input should end up in one of those strings!

There's no good way for transpilers to support eval, so they basically don't even try.

As a community we have decided it is never acceptable to use eval when there is any other way to solve the problem.

And yet we use eval all the time.

Of course it hides behind many different abstractions:

  • <script src="http://..."></script>
  • <script>...</script>
  • new Function("...")()
  • require("vm").runInThisContext("...")
  • <a href="javascript:...">Click me!</a>
  • <input type="button" value="No, me!" onclick="..." />
  • setTimeout("...", 1000)

Nothing happens

unless first an eval.

Carl Sandburg

There is an eval in everything.

That's how the code gets in.

Leonard Cohen

There's no escaping eval

But it can be tamed, and Node does this better than any other JavaScript platform, thanks to CommonJS.

CommonJS is great

  • All your code runs in a specific module scope.
  • Any given module runs at most once, the first time you require it.
  • Module source code is loaded from static files, according to simple rules.
  • Module load order emerges naturally.
  • Module exports remain distinct.
  • Global scope stays clean.

Once your code starts running, you get to make all the decisions about what additional code is allowed to run, and when.

Has CommonJS won?

From the perspective of Node and NPM,it certainly seems so.

But what about code running in browsers?

Is "winning" really all we care about?

Can we do better?

What's wrong with CommonJS?

Problem 1: How code is loaded

Node has it easy. Just ask the file system!

Not so simple or efficient to do hundreds of synchronous HTTP requests over the network.

Instead, there has to be a way to deliver bundles of code to the client.*

* Bundling woes deserve a talk of their own.Listen to Trek Glowacki.

Problem 2: Dependency cycles

// wrap.js var log = require("./log").log; exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} );
 

Problem 2: Dependency cycles

// wrap.js var log = require("./log").log; exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

What's wrong here?

Problem 2: Dependency cycles

// wrap.js var log = require("./log").log; exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Can we fix it?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Can we fix it?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Can we fix it?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Can we fix it?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Can we fix it?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Why does this work?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Will it always work?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Will it always work?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Will it always work?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Will it always work?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var log = require("./log"); var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Will it always work?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var log = require("./log"); log("STARTING SERVER"); var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

Will it always work?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var log = require("./log"); log("STARTING SERVER"); var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

What about now?

Problem 2: Dependency cycles

// wrap.js exports.deferred = function (fn) { return function wrapper() { setTimeout(fn, 0); }; }; var log = require("./log").log; exports.logged = function (fn) { return function wrapper(...args) { log("calling the function"); return fn.apply(this, args); }; }; // log.js var wrap = require("./wrap"); exports.log = wrap.deferred( function (...messages) {...} ); // main.js var log = require("./log"); log("STARTING SERVER"); var wrap = require("./wrap"); require("http").createServer( wrap.logged((req, res) => {...}) ).listen(8080);
 

What happens when the first request comes in?

Problem 2: Dependency cycles

  • Not forbidden!
  • Relatively simple, deterministic resolution policy.
  • Highly sensitive to load order, which is beyond the control of the modules participating in the cycle.
  • Example from the React codebase where we gave up on modularity in the name of preventing cycles.
  • Cycles can be useful.
  • Cycles should work.

Problem 3: exports vs. module.exports

Immediately returning a partially populated exports object in case of circular dependencies would work so much better if that exports object was guaranteed to become complete eventually.

But modules can change the very identity of exports by reassigning the module.exports property, rendering that partial exports object totally irrelevant.

Different modules that require the same module can end up with totally unrelated results!

Problem 4: Multiple exports

It's definitely nice that exports objects can have multiple properties.

But it's nearly impossible to determine whether a certain property is actually used.

Not a huge problem on the server, but client bundlers like Browserify and Webpack end up including tons of dead code.

Self-discipline?

  • Never assign to module.exports, unless you are certain your module has no (circular) dependencies.
  • Store references to imported exports objects: var a = require("a"); // Mostly safe. var foo = require("a").foo; // Dangerous! exports.good = function (arg) { return a.foo("good", arg); // Uses the latest value of a.foo. }; exports.bad = function (arg) { return foo("bad", arg); // Uses a stale value. };

There's nothing about CommonJS that helps you (or your teammates) keep this discipline.

CommonJS may be the most popular module system we have

But it's a shame that we're even talking about one module system winning a popularity contest.

Almost every other language avoids this contest entirely, by providing a built-in module system, and most of them do it with a special syntax.

A native module system...

  • eases cooperation between authors and consumers of libraries,
  • eliminates the need for each library to provide its own module-loading system,
  • regularizes the structure of applications, and
  • allows the developer community to stop debating the merits of different code sharing mechanisms.
  • <blink> We have more interesting problems to solve!</blink>

A language without a native module system is a language in which no one is ever quite sure how to share their code with the widest possible audience.

Betting on the wrong best practice (CommonJS? AMD? UMD?) is a recipe for obscurity, while reverting to the simplest common denominator (global variables) feels like giving up.

And that's why the new ECMAScript 2015 import and export statements are so vitally important.

A native module system would be a huge relief even if it was more restrictive than CommonJS!

The Go module system simply forbids circular dependencies.

That's one solution.

If you don't like it, don't use Go.

But in fact the ES2015 module system was designed with all the strengths and weaknesses of CommonJS in mind

Thanks to the hard work and cleverness of people like Dave Herman, ES2015 modules solve or at least mitigate all four of the problems I mentioned earlier.

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

let {readFile, writeFile} = require("fs");

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

let {readFile, writeFile} = require("fs"); import {readFile, writeFile} from "fs";

What's the difference?

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

let {readFile, writeFile} = require("fs"); import {readFile, writeFile} from "fs"; export function ensureTrailingNewline(path, callback) { readFile(path, "utf8", (err, text) => { if (err) callback(err); else writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

Consider this usage example.

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

let {readFile, writeFile} = require("fs"); export function ensureTrailingNewline(path, callback) { readFile(path, "utf8", (err, text) => { if (err) callback(err); else writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

What happens with the destructuring version?

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

const _fs = require("fs"); let readFile = _fs.readFile, writeFile = _fs.writeFile; export function ensureTrailingNewline(path, callback) { readFile(path, "utf8", (err, text) => { if (err) callback(err); else writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

Imported properties are simply stored in variables.

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

const _fs = require("fs"); let readFile = _fs.readFile, writeFile = _fs.writeFile; export function ensureTrailingNewline(path, callback) { readFile(path, "utf8", (err, text) => { if (err) callback(err); else writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

This has all the problems as our earlier example!

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

// Remember this hazard? let foo = require("a").foo; export function ensureTrailingNewline(path, callback) { readFile(path, "utf8", (err, text) => { if (err) callback(err); else writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

This has all the problems as our earlier example!

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

import {readFile, writeFile} from "fs"; export function ensureTrailingNewline(path, callback) { readFile(path, "utf8", (err, text) => { if (err) callback(err); else writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

How about the ES2015 version?

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

const _fs = require("fs"); export function ensureTrailingNewline(path, callback) { readFile(path, "utf8", (err, text) => { if (err) callback(err); else writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

Behind the scenes, a reference to the module is imported.

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

const _fs = require("fs"); export function ensureTrailingNewline(path, callback) { _fs.readFile(path, "utf8", (err, text) => { if (err) callback(err); else _fs.writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

References to the imports then get "rewritten" as if they were member expressions.

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

const _fs = require("fs"); export function ensureTrailingNewline(path, callback) { _fs.readFile(path, "utf8", (err, text) => { if (err) callback(err); else _fs.writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

With ES2015, ensureTrailingNewline always uses the latest version of readFile and writeFile.

So how does it work?

At first glance, the ES2015 import statement looks pretty similar to a destructuring variable declaration:

const _fs = require("fs"); export function ensureTrailingNewline(path, callback) { _fs.readFile(path, "utf8", (err, text) => { if (err) callback(err); else _fs.writeFile(path, text.replace(/\n*$/, "\n"), callback); }); }

Popular package where this matters: graceful-fs

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function a() {...}

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function a() {...} // In d.js: import a from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} // In d.js: import a from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} // In d.js: import a from "./abc"; import a1 from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} export function b() {...} // In d.js: import a from "./abc"; import a1 from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} export function b() {...} export const c = 299792458; // In d.js: import a from "./abc"; import a1 from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} export function b() {...} export const c = 299792458; // In d.js: import a from "./abc"; import a1 from "./abc"; import {b, c} from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} export function b() {...} export const c = 299792458; // In d.js: import a from "./abc"; import a1 from "./abc"; import {b, c} from "./abc"; import {b as bee, c as lightSpeed} from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} export function b() {...} export const c = 299792458; // In d.js: import a from "./abc"; import a1 from "./abc"; import {b, c} from "./abc"; import {b as bee, c as lightSpeed} from "./abc"; import {default as a2} from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} export function b() {...} export const c = 299792458; // In d.js: import a from "./abc"; import a1 from "./abc"; import {b, c} from "./abc"; import {b as bee, c as lightSpeed} from "./abc"; import {default as a2} from "./abc"; import {default as a3, b, c} from "./abc";

What about module.exports?

ES2015 modules can define a default exported value:

// In abc.js: export default function () {...} export function b() {...} export const c = 299792458; // In d.js: import a from "./abc"; import a1 from "./abc"; import {b, c} from "./abc"; import {b as bee, c as lightSpeed} from "./abc"; import {default as a2} from "./abc"; import {default as a3, b, c} from "./abc"; import a4, {b, c} from "./abc";

Default and named exports, together at last!

You might even say the ES2015 module system enforces the discipline I outlined earlier... which means it doesn't have to be self-discipline.

And that makes all the difference.

Multiple exports?

ES2015 export statements must appear only at the top level of a module, and must have one of the following forms—all of which have names:

  • export var a = ...;
  • export let b = ...;
  • export const c = ...;
  • export function d() {...}
  • export function* e() {...}
  • export class F {...}
  • export default expression;
  • export {a, b, c, d, e as genFn, F};

Multiple exports?

ES2015 import statements provide no way to obtain the exports object itself.

In fact, native implementations of ES2015 modules need not use objects to represent exports at all!

That's just an implementation detail that happens to be convenient if you're compiling for an environment that supports CommonJS.

This makes it possible to determine, statically, for any form of import statement, exactly which exports it does and doesn't care about.

Detailed explanation by Axel Rauschmayer of every variation of import and export syntax.

Rollup (Rich Harris)

A tool from the future.

But enough about the future.

How can you use ES2015 modules today?

I would love to tell you, "Just use Meteor!" And that will soon be the case.

Until Meteor 1.3 is released, I've put together a skeleton NPM package that you can clone and modify, or just use as inspiration:

git clone https://github.com/benjamn/jsnext-skeleton.git cd jsnext-skeleton npm install npm test

Try it, break it, republish it as your own, submit issues!

How can you use ES2015 modules today?

Some stuff you can do with this skeleton package:

  • write ES2015 in the src/ directory,
  • transpile from src/ into lib/ using an npm prepublish command,
  • run mocha tests against code from both src/ and lib/, ensuring identical output, and
  • publish both lib/ and src/ code to NPM, so everything Just Works™ but also ES2015-aware tools like Rollup can work their magic.

Soon you'll be publishing both ES2015 and CommonJS to NPM, but what use are either of those formats to folks who really just want a .js file they can load with a <script> tag on a web page?

To bundle, or not to bundle?

Some say: bundles need to be created at publish time, so that library authors can deal with bundling errors, instead of burdening the consumer.

Others say: if library authors bundle in their dependencies, consumers of multiple libraries may end up with conflicting copies of those dependencies.

Still others: if we could all just use the same language for writing modules, then our bundling tools would have a much easier job.

The part of the talk where I sing the praises of Meteor

Meteor's build system takes care of pretty much every aspect of bundling for you. Only binary dependencies need to be published for multiple architectures.

Meteor uses an optimizing constraint solver (compiled from C++ to JS using Emscripten) to ensure compatible package versions.

Meteor will support ES2015 modules in version 1.3, if I have anything to do with it. And I have everything to do with it. No, seriously, you know who to blame if modules don't make it into the 1.3 release.

My pleas:

  • Start writing ES2015 modules sooner rather than later, assuming I have persuaded you that they are awesome.
  • No need for funny file extensions. ES2015 is .js.
  • Publish your source files to NPM, in addition to your CommonJS files, so that sophisticated tools can begin making use of them.
  • Add a jsnext:main property to your package.json, so the tools know where to look.

Better yet, if you want to help me submit automated pull requests to every Node project on GitHub, find me after this talk! 

     { github,
       twitter,
       instagram,
       facebook
     }.com/benjamn

ben@{benjamn,meteor}.com

Thanks!

The Importance of import and export Ben Newman (Meteor) EmpireNode 2015 { github, twitter, instagram, facebook }.com/benjamn