Marionette JS

Advanced Usage, Tips & Tricks

Created by Ghislain 'Aus' Lacroix / @aus

Running tests on device

Is my device supported?

Aries or Flame?

Yes!

Do I have a µUSB cable?

Yes!

Are you using a USB Hub?

Is it powered?

Yes!

Preparing your local environment

Install node.js v4.0 or greater.

Ensure you have npm v2.* and only 2.*

Mac or Linux are the only supported platforms

Virtual Machines are OK but we recommend VMWare Fusion, Player, or Workstation.

[Linux] On some distros, you will need to install clang

Preparing your device

If you care about what's on your device, back it up!

Full flash w/Engineering Build! (Aries Full Images)

[Optional] Shallow flash of Gecko if testing a custom build

Push Gaia test profile to device

tl;dr

            
$ cd ~/Downloads/b2g-distro
$ ./flash.sh
$ cd ~/Projects/B2G
$ ./flash gecko
$ cd ~/Projects/gaia
$ make really-clean
$ make reset-gaia
$ BUILDAPP=device REPORTER=spec make test-integration
# Run it again, without building the profile again
$ BUILDAPP=device REPORTER=spec make test-integration-test
# Run it again, but just a specific app's tests
$ APP=clock \
    BUILDAPP=device \
    REPORTER=spec \
    make test-integration-test
            
          

Watch it run!

My device refuses to run tests!

Perform a full re-flash of the device from a full image.

Enable all the logs!

Host logs (this is where your test runs)

              
$ BUILDAPP=device \
  HOST_LOG=gecko_host.log \
  VERBOSE=1 \
    make test-integration
              
            

OMGWTFBBQ?!? It is still not running.

Update marionette-js-runner so it doesn't swallow python output.

marionette-js-runner/host/host.js

              
// Set me to true!
var _FORCE_DEBUG = false;
              
            

After you've done this, you need to update the module so that node will use it.

              $ npm run refresh
            

Troubleshooting local issues

Did you read all the steps for setting up your environment?

No, really, read them again. Don't skip any of the steps!

Run node -v and npm -v -- are they the correct versions?

Did the node modules install properly? Try reinstalling them.

rm -rf node_modules && npm install

Did you see my note about Linux and clang earlier? Did you actually install clang? This will show up as an install error during npm install if it's not installed.

Troubleshooting issues on CI

Where is my task definition? taskgraph.json

Where is the task definition for Gij? tests/taskcluster/tasks/marionette_js_tests.yml

Wait, but it calls this magic thing in 'ci'? tests/ci/marionette_js/

How do I fix intermittent test failures?

Run it again?

:(

Use the docker image we use on taskcluster.

First, you'll need docker installed.

docker run -ti taskcluster/gaia-taskenv:2.0.4

Edit how we call make test-integration here

              
# Maybe we're debugging a single test file
TEST_FILES=/home/worker/git_checkout/path/to/test/file.js \
    make test-integration
# Or a specific app only.
APP=myapp make test-integration
              
            

You can even replace that entire script! I would absolutely recommend doing this.

              
var spawnSync = require('child_process').spawnSync;
var env = {
  TEST_FILES: "/home/worker/git_checkout/path-to-test-file.js"
};
var runsToPerform = 30;
var status = 0;

for(var i = 0; i < runsToPerform; i++) {
  var childProc = spawnSync('make',
                            ['test-integration'],
                            { maxBuffer: 4096 * 1024,
                              env: env });
  if (childProc.status !== 0) {
    console.info(childProc.stdout);
    console.error(childProc.stderr);
    process.exit(99);
  }
}