Apr 17, 2007 / Selenium Usage in Real Life ASP.NET 2.0 Application

4 comments

We are writing Selenium tests about 2 months at TargetProcess and at the moment we have created 34 tests with 190 assertions. It is not as much as we have NUnit tests (750), but coverage increases and we expect to have good coverage within half a year.

TargetProcess based on ASP.NET 2.0 and AJAX, quite hard combination for automated testing. In general we've looked into testing framework integrated into Visual Studio, but it does not support AJAX (support is possible, but demands too much effort). Then we've tried Selenium and it worked. In general it is very easy to create tests using Recorder, however there are some problems that seriously affect maintainability.

ASP.NET IDs Workaround

The first ad obvious problem is complex ASP.NET ids. As you know, ids are auto-generated, and you have something like ctl00_mainArea_bugDetails_BugDetails_btnUpdate in the code. When you record test you will have something like that:

<tr>
 <td>click</td>
 <td>ctl00_mainArea_bugDetails_BugDetails_btnUpdate</td>
 <td></td>
</tr>

What happens if you change mainArea id to something else? Right, all your tests will break. Global replace may solve the problem, but still it is error prone. We are using XPath to find required button and code looks like that:

<tr>
 <td>clickAndWait</td>
 <td>//input[contains(@value,'Save')]</td>
 <td></td>
</tr>

Locator //input[contains(@value,'Save')] will find element INPUT with value 'Save'. It is less now if you change mainArea id all tests still work.

That is the reason we write tests manually. Recorder is not as clever as we want it to be.

AJAX Support

Selenium supports AJAX without problems, but you should know one hint. Selenium does not know when AJAX action is completed, so if you need to wait till completion ClickAndWait will not work. Instead you should use pause. Expand All action is asynchronous and the following code will not work correctly

<tr>
 <td>clickAndWait</td>
 <td>//a[contains(@title, 'Expand All')]</td>
 <td></td>
</tr>
<tr>
 <td>verifyTextPresent</td>
 <td>Completed Story</td>
 <td></td>
</tr>

Correct code shown below. ClickAndWait action replaced with click and pause actions.

<tr>
 <td>click</td>
 <td>//a[contains(@title, 'Expand All')]</td>
 <td></td>
</tr>
<tr>
 <td>pause</td>
 <td>1000</td>
 <td></td>
</tr>
<tr>
 <td>verifyTextPresent</td>
 <td>Completed Story</td>
 <td></td>
</tr>

There is a risk that pause will be too short and request will not be received, so there is one more way with waitForCondition action.

<tr>
 <td>waitForCondition</td>
 <td>var value = selenium.getText("//div[@id='lstAction']"); value.match(/User Story saved/);
</td>
 <td></td>
</tr>

However we did not see problems with pause and it is simpler for us.

Custom Actions

We had one problem (it is quite common problem for web applications). In lists we have different actions (view, edit, delete) and in test you may need to execute the action for specific item. For example, delete specific user story from the list. It is hard to find required link with XPath since there are many looks-alike links. As a solution we've created clickInTable custom action.

<tr>
 <td>clickInTable</td>
 <td>//table[contains(@id, 'projectsListControl')]</td>
 <td>Test Finish Project|delete</td>
</tr>

ClickInTable action finds required table by id, then finds row by item name and finally finds required action. It took several hours to create such method, but it worth the effort.

In the last part of the Selenium posts I will share some thoughts about Selenium-TargetProcess integration.

posted by Michael Dubakov @ 1:04 PM

Apr 12, 2007 / Acceptance and Regression Testing With Selenium: Part II - Selenium Specifics

0 comments

Why Selenium is special? There are just two key factors: it is easy to use, it may be adapted to your goals without much problems and effort. Simplicity and extensibility are key success factors in many applications and Selenium is not the exception. Why it is simple? What simplicity means for automated tests framework? As a tester I want easy tests creation, easy to understand format to tests maintainability and in the best case no programming at all. In ideal case I want to generate all automated tests right from my test cases. It means if I write test case it can be generated to something executable in minutes with single key push. While Selenium is not that powerful, it is closest tester-dream-tool I know about.

Recorder

Many tools have recorders, Selenium is not an exception. You may start web browser, run Selenium IDE and record any test you want. Funny thing is that we almost not use recorder to create tests for TargetProcess. Why? We find it more fast and easy to create test manually. Do you believe in that? Selenium tests are so easy to create that you even don't need The Recorder!

Format

Selenium test is just an old HTML file with various directives. Here is an example of simplest test

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>add_user</title>
</head>
<body>
<table>
<tr>
 <td>open</td>
 <td>http://localhost/tp2/Default.aspx </td>
 <td></td>
</tr> 
<tr>
 <td>verifyTextPresent</td>
 <td>ToDo</td>
 <td></td>
</tr>
</table>
</body>
</html>

Each test is selenium represented by set of actions or steps. And each step is a row in table. The example above has two steps and two rows (<tr> tag). First step opens url and the second step verifies that text ToDo presents on the page.

Learning Curve

It is small. Within an hour you will be able to create simple test and run it. Within a day you will figure out some best practices and will be able to create quite complex automated tests for your application. Within two days you will be able to create some extensions that optimize your work and make tests more robust. It is one of the simplest testing tools I had work with for sure. How much time you will need to learn WinRunner?

Here is the list of things you should know to create selenium tests:

• Dozen of commands like open, click, verifyTextPresent
• XPath basics (actually you need half an hour to learn them)
• DOM basics
• (optional) JavaScript, for extensions creation

Extensibility

Extensions mechanism as you expect is simple as well (I am tired of the word "simple", but such as Selenium!). All you need to do is add javascript function into user-extensions.js. There is one complexity however - you have to know JavaScript and DOM.

Labels: acceptance testing, regression testing, selenium

posted by Michael Dubakov @ 12:29 PM

Apr 9, 2007 / Acceptance and Regression Testing With Selenium: Intro

0 comments

Quality Assurance is a very important part of any project (I think nobody will argue, maybe only some self-assured developers who “always write bug-free code”). In many companies testing is no-fun activity, since tests automation is out of there. Typical tedious process looks like that:

1. QA persons read huge specification (or very small and brief spec which is even worth in such cases)
2. Then they create test plan and set of test cases. This is the most interesting part of testers work
3. Then developers push Release Candidate build into QA team and QA engineers execute all test cases manually
4. After weeks of testing they create report that shows actual product quality

I should say that manual testing is the most boring part of QA work (unless it is an Exploratory Testing). Click here, check system message, fill that fields, click Save, verify results. It’s a very, very boring stuff. The worst thing is that in some cases the same test case should be re-run several times. Do the same things again and again? No one in IT industry likes that, we have computers to handle such things after all! So why manual testing widespread? There are several reasons:

• Not perfect tooling support (especially for web development)
• Usually (this tendency is changing) testers have a little or no programming experience to create automated tests
• Managers in many companies think that effort required to create automated tests is “huge”, “too expensive”, “not worth the results”

Fortunately, during last several years something changed in the industry. First, agile development practices like TDD and Acceptance Testing adopted and became more popular. Second, simply genius tools appeared. In this article I will share our experience with Selenium — fantastic tool for acceptance testing in web-based projects.

Labels: acceptance testing, regression testing, selenium

posted by Michael Dubakov @ 6:23 AM