I had a Rails consultancy (Makandra) recently work on a JS-heavy application that I happen to own, and they got Selenium singing on it, which had been beyond my capabilities for years. One of their tricks, which you can inspect the implementation of in their (public) utilities library [+], is using basically a vendored Firefox per project and VNCing into that Firefox to drive things around. It is thus off-screen and out of the way when you're using it, but apparently is more true-to-reality than headless.
The test suite they wrote has about ~600 tests and while they're slower than I'd like (2~3 minutes) they've been bulletproof since we got my dev environment configured properly. It includes some fairly complicated interactions, most relevantly around our calendar interface.
I had been using Firefox driver in Xvfb but wasn't happy with the performance/stability. So I built a Selenium driver out of Java only (using JavaFX's embedded WebKit) and used a headless JRE windowing toolkit (Monocle). My project is still a pre-release but the headless capability, Java-only system requirement, and its ajax handling might make it useful to some people currently: https://github.com/MachinePublishers/jBrowserDriver
1. Not quite sure. I've only used PhantomJS via Selenium Ghost Driver. From that usage they're similar. The main difference is that my driver uses only Java so under the hood the JRE is launching WebKit through JNI and everything runs in the same JRE process.
2. Current WebKit version depends on the JRE used. Oracle Java 1.8.0_45 has WebKit version 537.44.
3. Java maintainers will update WebKit periodically, including within a major version. E.g., here they update WebKit for the 1.8.0_60 JRE: http://openjdk.java.net/jeps/239 ... Other than that I'm not sure.
Honestly, if it was useful, I would probably use this my side project which is commercial, but in no way competes with what you're doing (load testing). I might make changes or improvements, and I would generally contribute those back. Affero doesn't "mix well" so it is pretty much a non-starter for me.
I've found this is true for a lot of projects and it seems like restrictive licenses prevent projects from going mainstream.
I have trouble with Selenium's failure rate too. End up writing a test engine in Javascript. It handles async js func call with js function callback when finished to get rid of all the sleep-wait-retry type logic in Selenium.
Works very well. I can run the 100+ test cases in all IE/FF/Chrome/Safari, ios/android browser without change one line of JS/Test code. Runs fine with desktop with wire connect to cellphone browser on cell connection.
It tests out all the app backend db logic also. The time/pass/fail info are submitted back to the test backend db.
Can you give an example of how it works? Say navigate to a page, fill in a form, click submit and verify that some text is present after submitting the form
The test suite they wrote has about ~600 tests and while they're slower than I'd like (2~3 minutes) they've been bulletproof since we got my dev environment configured properly. It includes some fairly complicated interactions, most relevantly around our calendar interface.
[+] https://github.com/makandra/geordi