The Automated Tester

Selenium Tutorial - User Extensions

I have had a number of requests lately asking me if I could give advice on how to create user extensions for their tests. I have replied to those who emailed me but thought that it would be good to explain the basic structure of User Extensions. I have put a copy of an old user extension that has become normal Selenium Functionality as an example of a user extension.

The tutorial is aimed at users who understand JavaScript and have fairly good understanding of Selenium. It will also show you how to add the user-extension.js in Selenium IDE, Selenium Core and Selenium RC so that you would be able to reuse it fairly easily.

Tester 2.0 - Definition of an Agile Tester

I have been to a number of SIGiST Conferences over the last year and have always been impressed by the calibre of the speakers there. There has been a theme lately talking about Agile Development and Agile Testing but the thing that always seems to missing is someone giving their interpretation on Agile Testing. To actually describe what an Agile Tester is! And what they should be doing. Below I am going give my opinion of what an Agile Tester by describing skill set needed.We are after all only as good as our skill set! I am going to rate them from 1 to 10 with 10 being needed and 1 being desirable.

Skill	Score	Reason
Creativity	10	A good tester needs to be extremely creative when trying to test applications. Developers like to think that they are clever and get all the bugs before its released. Unfortunately this is not true when a tester starts being creative in the way they test the application and breaks it very quickly!
Innovation	10	A good tester needs to be able to innovative with the tools that they have. An example would be in SaaS(Software as a Service) companies who constantly striving to be the next big thing and competing against the likes of Google, Microsoft, Salesforce, etc. This drives innovation in development which means that it drives innovation in testing!
Test Automation	9	Every Agile tester needs to be able to do some form of automated testing. I am not saying that the tester must know how to use every tool that is out there but they need to understand how to create the automated tests.
Exploratory Testing	7	Agile Testing tends to mean Exploratory testing. I know that some people will disagree with this but because testing tends to happen on the fly and appears ad hoc but it isn't. Exploratory testing does have a set of rules that need to be adhered and will mostly bring a number of bugs to the surface just as well as normal scripted testing.
Communication	7	One of the main pillars of Agile Development is communication. This is very important in my opinion because a good tester must be able to communicate bugs to developers and requirements to end users. A good tester will also be able to challenge technical architects on their ideas. I know the last sentence can be very hard to achieve in large organisations but it doesn't mean that you can't send them an email to make suggestions.
Development	5	An Agile Tester should not be afraid to look at a developers code and if need be, hopefully in extreme cases, go in and correct it. I am not suggesting for one minute that they must be able to code entire applications but must have the confidence to look at a bug, spot the error in the code and either write a unit test to break it or point out the line where they think the issue is. An example of this is where doing Paired Programming with one developer we shaved of nearly 70% off the time it took to run a unit test test suite just by doing a one line change to their tests which got them to make one other small change.
Ability to work in pairs	3	I am a fan of being able to knock ideas off each other. I wouldn't do paired programming all the time but I like the concept and how it seems to get things done properly with little fuss.

This is not a comprehensive list but these for me are the most important things. Tester 2.0 is something that a number of testers need to strive for if they do not want to get left behind. There is a definite blurring of lines between development and testing. Gone are the days of "I have finished my part, over the wall it goes!". There is also a move to more "Grey Box" testing because of this blurring.

Selenium Tutorial - Selenium Remote Control Tutorial.

I have created a brand new tutorial teaching people how to start using Selenium Remote Control. This tutorial builds on all the other tutorials that you have worked through on the site. It starts by showing you how to "install" Selenium Remote control and how to use the scripts that you created in Selenium IDE.

I hope that you find this useful!

Interpretations of the V-model

I have recently been doing interviews and one question that I like to ask interviewees is "Where do you feel testers should sit within the V-Model?"

The V-Model (as pictured below) is a representation of how most "development" tasks have an equal "testing" task. I like to think of it as testings equivalent to Newton 3rd law.

This should be a good thing because it shows that there are different aspects of testing that need to be done during the software development lifecycle.

All the interviewee's seemed to give a different answer to the question above. Being the interviewer I was hoping for one of 2 answers.

The first answer would be one that made me rethink my thoughts on the V-Model. To be honest as an interviewer I am always looking forward to someone making me rethink a topic.

The other answer is that testing should encompass the entire V-Model. From the beginning of the Business Requirements stage to the end of the Acceptance Testing stage. The reason why I think this is because Quality Assurance should be in every aspect of the software development lifecycle.

I know that this is something from the Waterfall methodology but its a good way to see what needs to be done in the lifecycle and can easily be transformed into an Agile development.

Selenium Tutorial - Record and Replay Tutorial Update.

I have updated the Record and replay tutorial in preparation for the first Selenium Remote Control tutorial for this site. The update shows you how to create a test suite to go with all your test cases.

A test suite is a really good way to keep your test cases grouped together by behavioural areas.

ISEB/ISTQB - Is it really worth it?

I have been reading James Bach's blog for a while and in the latest post he has commented on Alan Richardson's commentary on ISEB/ISTQB.

Finally I have someone that agrees with me that ISEB/ISTQB while being good in theory is pretty useless in real life. I know that most people reading this will be saying "That is a very bold statement!". I know that it is and below I will explain why I agree with Alan.

Background

Testers have been fighting for years to remove the stigma that testing software is easy and that any person can do it. This, I feel, is because software testers used to be glorified data inputters with a little technical background. No wonder developers saw a secondment to testing as a demotion. Then Kent Beck came along and wrote Test-Driven Development: By Example. A developer who came along and made testing fun for developers! Something that testers could never get right! So lets have a look at the pros and cons of having a ISEB/ISTQB certificate.

Pros

Lets start why I think ISEB/ISTQB is good. I just said that testers have been fighting for years to remove a stigma that testing software is easy. ISEB came along and people who didn't work in the industry suddenly thought that testing was a little harder because you needed a certificate.
It also sets out standard names for things so that if one tester speaks to another they are speaking in the same tongue. Its also the latest buzzword so HR people will more than likely to give you an interview when you apply for a job.

Cons

The ISEB/ISTQB foundation course teaches the exact same things that I was taught during my first year of university. So not only do you have to pay your way through university but then have to pay from £140 to £700 to get the certificate.
I met a ISTQB examiner in my last job and he did not seem to understand testing in the real world. So are the testers that are getting these certificates also getting a working understanding of testing? I honestly don't think so!
The other thing, and this is a big thing for me, is that the foundation course book mentions Agile Development 3 times. Those three entries are no more than a quick mention. Is it worth getting a certificate that doesn't give a chapter what is becoming the most common development methodology.

Results

ISEB/ISTQB is unfortunately going the way of the the MSCE certificate of the late 90s. Its becoming no more than a buzz word and I feel is lowering the standard of testing. I know that I am not the only person who thinks this. I read a blog entry not so long ago where testers in India are seen as the poor cousins. Something that we've been working against!

This has to stop and we as testers are the ones who can stop this! Come on testers, lets start a revolt and get certificates to mean something!

Selenium Grid - Why is this the way of the future?

I have been working with Selenium Grid since Philippe Hanrigou and others at ThoughtWorks released it at the end of last year. I knew then that it was going to become the standard way for automated GUI web application testing.

As an application it has great potential since it is has all the best bits of Selenium but it does multiple browsers in parallel. Suddenly your testing that used to take n units of time to run are taking 1/n to run. As a practitioner you can get things done quicker, as a manager you can get things done for cheaper! Everyone is a winner.

The other thing that makes the grid work well, and this is a thing that makes Selenium Remote Control work brilliantly, is the ability to use any language to control the test. People use Java, Ruby, .Net, Python and many more to create their test scripts and run it with the appropriate client. If you have the appropriate client application like TestNG for Java or DeepTest for Ruby or PNUnit for .Net you can start running tests with one script in parallel. The one limitation toof using these test tools is that you only really have one browser application, like Firefox, being tested at a time. This is because you only have one instance of the Selenium object in your code.

The way that I have got around this is to have many instances of the Selenium object in my code that each control a certain type. This type could be Internet Explorer 7 on Vista or Firefox 2 on XP.

An example of this would be (Note: I use c# to create my tests and then run them with NUnit)


        [Setup]
        public void setup(){
        //Vista IE7 Instance
        VistaIE7selenium = new DefaultSelenium("hub", 4444, "IE7 on Vista", "http://testserver");

        //XP Firefox2 Instance
        XPFirefox2selenium = new DefaultSelenium("hub",4444 , "Firefox2 on XP", "http://testserver");
        XPFirefox2selenium.Start();
        }


        [Test]
        public void GridTest(){
        VistaIE7selenium.Open("/index.html");
        XPFirefox2selenium.Open("/index.html");
        }

As you can see it has the ability to create multiple instances of Selenium and then run them. This then leaves the handling of the Selenium Remote Control up to the Grid. As Philippe says, "It allows you to gridify your tests".

The negatives of Selenium Grid is that its still a work in progress. I know, thats hardly a reason to call it a negative. There are also a number of bugs but its growing section of OpenQA.org so give it a chance! I have 7 Selenium Remote Control instances running against my Grid (all virtualised with MS Virtual Server and WMWare) and I haven't had many issues. I have hooked Selenium Grid into our CruiseControl.NET server to test all the different browsers. I still have a lot of work to do with this but I know that it will be worth it in the end!

If you want to automate web testing and you want to do it efficiently use Selenium Grid! It will make your life a lot easier!

Selenium Tutorial - waitFor Tutorial

I have finally created the 5th tutorial for Selenium. This tutorial deals with getting test scripts to wait for elements to load onto the page or disappear off a page. This becomes very useful when testing AJAX web applications. The tutorial is the last in the Selenium IDE and Selenium Core tutorials. I will start concentrating on developing Selenium Remote Control and Selenium Grid Tutorials as these will be of more value to testers in the future. If people would like me to create any other future tutorials send me an email. The email address can be found at the bottom of all the main pages. By the end of this tutorial everybody should have a very strong knowledge of Selenium's commands and how they can benefit you when testing web application with AJAX.

Site Update

I apologize that I haven't updated my site in a while. I have been busy sorting my domain out trying to get "The Automated Tester" brand going. I am sure some of you have noticed that I have populated the footers on all the major pages. I have also updated a number of typos and bugs with scripts. I have also completed a waitFor tutorial but I just need it to be proofed and then I will be release it. It should be released in the next week. I will then start creating tutorials on how to use Selenium Remote Control and Selenium Grid. In other news I have been given an invite to Google App Engine. So over the next few months I will be creating an entire web application for people to practise creating Selenium scripts against.

Blog - What makes a good automated test?

I have been asked to try automate the testing of a wider range of for the group. This is going to be quite challenging since I have not automated desktop applications for about 15 months and the software we make is quite unique. I will also hopefully be teaching the other testers how to maintain the automated tests and eventually create their own. This challenge has made me think back on my automated testing experience and ask "What makes a good automated test?". A good automated test has all the hallmarks of a good manual test except that its comparatively quick and cheap to run. This means that an automated test is successful when it finds bugs and repeatable where ever the software is installed. In terms of web testing it would be a test script that can be created and then stored in a central place and be run from the central place. Ideally this would be also run on the continuous integration server when doing builds. The thing that makes a good automated test great is scalability. I recently watched a YouTube video of the Selenium Users Meet-up. There were Google staff there saying that their testing farm handles 51000 tests and most Friday's all of these tests run. They then went on to complain that Selenium struggled to scale to that size. To be honest at that size I am not surprised! P.S. To those who are waiting for the next Selenium Tutorial I am hoping to have it complete in the next week for everyone to use.

Blog - What would people like for the next Selenium Tutorial?

Thanks to everyone who has been using my site and all those who are subscribed to my RSS Feed. I haven't released another tutorial for couple of weeks and have started thinking of what would people like next for a Selenium tutorial. I am struggling to think what people would really like. I would appreciate it if people would take the time to let me know, via the comments section of this blog, what tutorial they would like next. Please don't think that your questions are too novice. I would like to help as many people out as possible! I look forward to your replies!