mirror of
https://github.com/ascribe/onion.git
synced 2024-12-23 01:39:36 +01:00
184 lines
7.7 KiB
Markdown
184 lines
7.7 KiB
Markdown
Introduction
|
|
============
|
|
|
|
When in doubt, see [Gemini](https://github.com/gemini-testing/gemini) and [their
|
|
docs](https://github.com/gemini-testing/gemini/tree/master/doc) for more information as well as configuration options.
|
|
|
|
Contents
|
|
========
|
|
|
|
1. [Installation](#installation)
|
|
1. [Running Tests](#running-tests)
|
|
1. [Gemini Usage and Writing Tests](#gemini-usage-and-writing-tests)
|
|
1. [PhantomJS](#phantomjs)
|
|
1. [TODO](#todo)
|
|
|
|
|
|
Installation
|
|
============
|
|
|
|
First make sure that you're using NodeJS 5.0+ as the tests are written using ES6 syntax.
|
|
|
|
Then, install [PhantomJS2](https://www.npmjs.com/package/phantomjs2):
|
|
|
|
```bash
|
|
# Until phantomjs2 is updated for the new 2.1 version of PhantomJS, use the following (go to https://bitbucket.org/ariya/phantomjs/downloads to find a build for your OS)
|
|
npm install -g phantomjs2 --phantomjs_downloadurl=https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-macosx.zip
|
|
|
|
# If using OSX, you may have to install upx and decompress the binary downloaded by npm manually:
|
|
brew install upx
|
|
|
|
# Navigate to the binary, ie. /Users/Brett/.nvm/versions/node/v5.4.0/lib/node_modules/phantomjs2/lib/phantom/bin/phantomjs
|
|
upx -d phantomjs
|
|
|
|
```
|
|
|
|
Finally, [install Gemini globally and locally with npm](https://github.com/gemini-testing/gemini/blob/master/README.md#installation).
|
|
|
|
|
|
Running Tests
|
|
=============
|
|
|
|
Run PhantomJS:
|
|
|
|
```bash
|
|
phantomjs --webdriver=4444
|
|
```
|
|
|
|
And then run Gemini tests:
|
|
|
|
```bash
|
|
# In root onion/
|
|
gemini test gemini/* --report html
|
|
```
|
|
|
|
If you've made changes and want them to be the new baseline (ie. it's a correct change--**make sure** to test there are no regressions first!), use
|
|
|
|
```bash
|
|
# In root onion/
|
|
gemini update gemini/*
|
|
```
|
|
|
|
|
|
Gemini Usage and Writing Tests
|
|
==============================
|
|
|
|
While Gemini itself is easy to use on simple, static pages, there are some nice to knows when dealing with a single page
|
|
app like ours (where much of it is behind an authentication barrier as well).
|
|
|
|
Command Line Interface
|
|
----------------------
|
|
|
|
See [the docs](https://github.com/gemini-testing/gemini/blob/master/doc/commands.md) on the commands that are available.
|
|
`npm run vi-*` is set up with some of these commands, but you may want to build your own or learn about some of the
|
|
other functions.
|
|
|
|
Authentication
|
|
--------------
|
|
|
|
Authentication presents a tricky problem with Gemini, since we can't inject any cookies or even run a start up script
|
|
through the browser before letting Gemini hook in. The solution is to script the log in process through Gemini, and put
|
|
waits for the log in to succeed, before testing parts of the app that require the authentication.
|
|
|
|
Browser Session States
|
|
----------------------
|
|
|
|
Gemini will start a new instance of the browser for each browser configuration defined in the .gemini.yml file when
|
|
Gemini's launched (ie. `gemini update`, `gemini test`, etc).
|
|
|
|
Although each new suite will cause the testing browser to be refreshed, the above means that cookies and other
|
|
persistent state will be kept across suites for a browser across all runs, even if the suites are from different files.
|
|
|
|
**What this comes down to is**: once you've logged in, you'll stay logged in until you decide to log out or the running
|
|
instance of Gemini ends. In general practice, it's a good idea to clear the state of the app at the end of each suite of
|
|
tests by logging out.
|
|
|
|
(**Note**: Persistent storage, such as local storage, has not been explicitly tested as to whether they are kept, but as
|
|
the cookies are cleared each time, this seems unlikely)
|
|
|
|
Test Reporting
|
|
--------------
|
|
|
|
Using the `--reporter html` flag with Gemini will produce a webpage with the test's results in `onion/gemini-report`
|
|
that will show the old, new, and diff images. Using this is highly recommended (and fun!) and is used by default in `npm
|
|
run vi-test`.
|
|
|
|
Writing Tests
|
|
-------------
|
|
|
|
See [the docs](https://github.com/gemini-testing/gemini/blob/master/doc/tests.md), and the [section on the available
|
|
actions](https://github.com/gemini-testing/gemini/blob/master/doc/tests.md#available-actions) for what scripted actions
|
|
are available.
|
|
|
|
Our tests are located in `onion/gemini/`.
|
|
|
|
Some useful tips:
|
|
* The `find()` method in the callbacks is equivalent to `document.querySelector`; it will only return the first
|
|
element found that matches the selector. Use pseudo classes like `nth-of-type()`, `nth-child()`, and etc. to select
|
|
later elements.
|
|
* Nested suites inherit from their parent suites' configurations, but will **override** their inherited configuration
|
|
if another is specified. For example, if `parentSuite` had a `.before()` method, all children of `parentSuite` would
|
|
run its `.before()`, but if any of the children specified their own `.before()`, those children would **not** run
|
|
`parentSuite`'s `.before()`.
|
|
* Gemini takes a screenshot of the minimum bounding rect for all specified selectors, so this means you can't take a
|
|
screenshot of two items far away from each other without the rest being considered (ie. trying to get the header and
|
|
footer)
|
|
* Unfortunately, `setCaptureElements` and `ignoreElements` will only apply for the first element found matching those
|
|
selectors.
|
|
|
|
PhantomJS
|
|
=========
|
|
|
|
[PhantomJS](http://phantomjs.org/) is a headless browser that allows us to run tests and take screenshots without
|
|
needing a browser.
|
|
|
|
Its second version (PhantomJS2) uses a much more recent version of Webkit, and is a big reason why Gemini (as opposed to
|
|
other utilities, ie. PhantomCSS) was chosen. Due to the large number of breaking changes introduced between PhantomJS
|
|
1.9 to 2.0, a large number of tools (ie. CasperJS) are, at the time of writing, lacking support for 2.0.
|
|
|
|
While you don't need to know too much about PhantomJS to use and write Gemini tests, there are still a number of useful
|
|
things to know about.
|
|
|
|
Useful features
|
|
---------------
|
|
|
|
You can find the full list of CLI commands in the [documentation](http://phantomjs.org/api/command-line.html).
|
|
|
|
Flags that are of particular interest to us:
|
|
* `--webdriver=4444`: sets the webdriver port to be 4444, the default webdriver port that Gemini expects.
|
|
* `--ignore-ssl-errors=true`: ignores any SSL errors that may occur. Particular useful when hooking up the tests to
|
|
staging, as the certificate we use is self-signed.
|
|
* `--ssl-protocol=any`: allows any ssl protocol to be used. May be useful when `--ignore-ssl-errors=true` doesn't work.
|
|
* '--remote-debugger-port`: allows for remote debugging the running PhantomJS instance. More on this later.
|
|
|
|
Troubleshooting and Debugging
|
|
-----------------------------
|
|
|
|
Remote debugging is possible with PhantomJS using the `--remote-debugger-port` option. See the [troubleshooting
|
|
docs](http://phantomjs.org/troubleshooting.html).
|
|
|
|
To begin using it, add `debugger;` statements to the file being run by `phantomjs`, and access the port number specified
|
|
after `--remote-debugger-port` on localhost:
|
|
|
|
```bash
|
|
phantomjs --remote-debugger-port=9000 debug.js
|
|
```
|
|
|
|
PhantomJS will start and then immediately breakpoint. Go to http://localhost:9000/webkit/inspector/inspector.html?page=1
|
|
and then to its console tab. Go to your first breakpoint (the first `debugger;` statement executed) by running `__run()`
|
|
in the console tab. Subsequent breakpoints can be reached by successively running `__run()` in that same console tab.
|
|
|
|
At each breakpoint, you can to http://localhost:9000 on a new browser tab and click on one of the links to go to the
|
|
current execution state of that breakpoint on the page you're on.
|
|
|
|
---
|
|
|
|
To simplify triaging simple issues and test if everything is working, I've added a short test script that can be run
|
|
with PhantomJS to check if it can access the web app and log in. You can edit the `lauch_app_and_login.js` file to
|
|
change the environment to run against.
|
|
|
|
```bash
|
|
# In root /onion folder
|
|
phantomjs phantomjs/launch_app_and_login.js
|
|
```
|