Click here to monitor SSC

Test Engineer at Red Gate Software Ltd.

Switching from Selenium 1.x to WebDriver/Selenium 2 and HtmlUnit

Published 26 August 2010 7:34 am

Recently I became aware of the work that is being done to merge the WebDriver and Selenium codebases. The result is Selenium 2, a project that aims to offer the best bits from both. For more details about the merge, see here.

Having only used Selenium and not WebDriver before, there were a couple of things about WebDriver that piqued my interest. The first is the very clean object-based API that it provides, allowing you to make calls such as:

browser.FindElement(By.Id("q")).SendKeys("Simple Talk");

to type into a text field (in this case the search box on the Google home page). I like the clarity that this syntactical style provides, making it easy to understand existing tests and write new ones.

The second thing was WebDriver’s support for HtmlUnit. HtmlUnit is a Java project that implements a UI-less web browser entirely in memory. This makes the process of browser automation much more efficient, as the overhead of creating a new browser instance is greatly reduced. Unfortunately, since its a Java project, taking advantage of HtmlUnit from .NET hasn’t been a simple prospect up until now (although Steve Sanderson wrote a great blog post about how to do it using IKVM, see here). With the advent of Selenium 2, this process gets a whole lot easier!

The Selenium 2 alpha builds can be found here. I fired up the standalone server in a virtual machine, and create a new WebDriver against that machine as follows:

new RemoteWebDriver(new Uri("http://testVM:4444/wd/hub"), DesiredCapabilities.HtmlUnit());

Note the URL – to create a DefaultSelenium against this instance of the server, the URL would be http://testVM:4444/, whereas to create a WebDriver it is necessary to tack the /wd/hub onto the end.

To compare the performance of using HtmlUnit against Firefox (as I was previously) I copied one of my test fixtures into a new project and migrated the 12 tests over to use a RemoteWebDriver instead of a DefaultSelenium. Since I last wrote about using Selenium Grid, the number of tests has grown from 372 to 2736. To deal with this growth I increased the number of machines in the Grid to 8, the net result of which was that a fixture containing 12 tests running in at most 8 concurrent threads would take about 1 minute 30 seconds to execute. By comparison to this, 12 WebDriver/HtmlUnit-flavour tests running serially (since the Selenium Grid project doesn’t support WebDriver) took only 24 seconds! that’s almost 4x quicker (3.75x, to be exact), despite the lack of parallelism.

This convinced me that it was worth switching all 2736 tests over to use WebDriver, in spite of Selenium Grid’s lack of support for it. So after a couple of hours work migrating my tests and helper methods over to the new API, I was ready to do a full test run. Against the Grid (and its 8 machines), this took almost exactly 3 hours. Using HtmlUnit, the test run completed in 45 minutes for a 4x speedup :)

So even in these early stages, the Selenium 2 project is shaping up to be pretty exciting for .NET web testing. I look forward to getting my hands on an early build of Selenium Grid that supports WebDriver, so I can start taking advantage of HtmlUnit in multiple tests concurrently :D

3 Responses to “Switching from Selenium 1.x to WebDriver/Selenium 2 and HtmlUnit”

  1. RossPatterson says:

    I’m curious – do you attribute the 75% reduction in execution time primarily to Webdriver or HTMLUnit? My expectation is the latter, but I don’t know for sure.

  2. Ben Adderson says:

    @Ross – definitely the latter. The vast majority of the performance gain comes from how much faster it is to start up and drive an HtmlUnit instance than a Firefox instance. Even if WebDriver were less efficient than Selenium (which I doubt) the loss due to that is massively outweighed by the gain due to using HtmlUnit.

  3. bitknower says:

    Hi there,

    My company is seriously considering using Selenium to test our webapp, but we’re trying to figure out if we should start with using SE1 and later migrate to SE2 or if we should just start out using SE2. It’s very important to us that we can test in IE, FF, Chrome and Safari.

    1.) How easy do you think it would be to migrate tests from SE1 to SE2? Your article mentioned that you migrated a couple thousand tests in just a few hours…does this mean that it’s a fairly trivial task? If so, we’ll probably start off using SE1 and move to SE2 later. If not, then we might use SE2 right away.

    2.) I noticed that SE2 does not currently have a driver for Safari (perhaps one is in the works…since there appears to be safari code in their repository?). Should we just use SE1 since it supports all of the above browsers and then migrate to SE2 when it’s more mature?

    3.) We would like to use Grid to run tests in parallel but SE2 does not yet support it (though it appears that it may in the near future?). I know you stated that HtmlDriver is faster, but we would be using the browser specific drivers so we wouldn’t have this speed benefit. Do you know when Grid will work for SE2?

    Thanks for your thoughts!

Leave a Reply