SWET2 - A great way to spend a weekend

Back home again, tired and inspired, after a weekend of test talk outside Göteborg. The second installment of SWET, SWET2, with 15 testers. This was my first peer conference and I hope there'll be more to come. The format was great and we had some very inspired, and inspiring, discussions. I come back with a bunch of new thoughts and ideas that I will try to mold into something I can use, when the opportunity arises.

Thanks to Rikard, Martin and Henrik for arranging a great gathering.

The delegates were: Christin Wiedemann, Torbjörn Ryber, Azin Bergman, Fredrik Scheja, Henrik Andersson, Johan Jonasson, Ola Hyltén, Sigge Birgisson, Simon Morley, Rikard Edgren, Henrik Emilsson, Martin Jansson, Steve Öberg, Robert Bergqvist, Saam Koororian. All contributed to a great weekend!

The tweets from SWET2 might tell you something about the discussions that went on

Jam session / Test session

I try to relate all sorts of different experiences and events that take place in everyday life to testing. It helps me remember and it lights up new perspectives on testing. Maybe most importantly I think it's a fun exercise to do. Keeps me thinking of testing, and other things, in new ways and that helps me stay creative and passionate about it. That's my opinion and it works for me.

One thing that comes up every now and then is jam sessions and performance or rehearsal of music. There are several things that relate to testing. The common way to stand/sit in a jam session is in i circle so that everyone can see each other. Body language, eye contact and, depending on if anyone cranked up his/her amplifier to 11, some spoken or shouted word. The most common use of the spoken word in communication during a jam session is usually changing key or moving to a new harmony progression. In these cases it's very short. "F" may be all that is said and when the next bar is reached everyone, hopefully, changes to F. You can change in the middle of a bar of course but changing at the start of the next bar is an example. Rhythm and speed may vary and usually starts with one or two persons picking up on each others playing and creating something new. new progressions may be created this way to. In a good jam there is evolution and learning happening. Is this starting to sound familiar? I think it was Miles Davies who said that he had only gotten better by always playing with musicians that were better than he was. Not extremely better but better. That way he pushed himself, learned more, and became better.

To me jam sessions has always been great opportunities to learn and develop my own skills as a musician. How does this relate to testing? I think a lot is pretty obvious to you by now. Working in a team will help testers to learn more, faster. Do pair testing to help less experienced tester learn from a more experienced tester. Like a good jam exploratory testing will make the tester learn and hone his/her skills during a testing session at the same time that he/she learns more about the software being tested. Change key or move to a new harmonic progression is the charter and it will give ideas for a charter for the next jam, the next session. Changes during a jam is exploring musically in the same way as we explore when we test. We see something that catches our attention or we apply a heuristic that we dig up that we believe is useful here and now. It's like coming up with a riff during a jam that may or may not work in the current context. I guess all musicians carry around an ever growing set of riffs and lick that have been picked up and developed over the years. It's the same thing we do as testers. We develop our tools every time we use them. We can do that if we are aware and think. We don't always have the luxury of being able to pair up with another tester but if you get the chance take it. Go for it! Grab the opportunity. You'll learn something and the other tester will learn from you.

Managing a jam session usually comes down to providing a place to be, maybe set up some amps, drums, keyboards and a PA. Making coffee, provide a sofa and some chairs are good things too. It's pretty much very basic facilitation that we're talking about. It's not about controlling every move of the participants, maybe throw in a first idea of a place to start. More experienced participants will come up with something on there own and agree to start off there. To me this relates very much to testing sessions and the test managers role. If you have the opportunity be a hands on guy and team up with one of your testers and learn something new and help your tester learn as well. Facilitate. Make sure the testers have what they need. Access to oracles identified before starting the session. Coffee! Make sure they get left to do there job and that they are not interrupted. As the test manager be a blocker and let the testers test.

The debriefing at a jam session is either non existent or it's very informal. Usually there will be a few words exchanged when a jam ends, at least if we've found something good and it has rocked in some way. The less successful attempts usually are ignored and you just sit down and let someone else play. That luxury is rarely available in a work situation with deadlines but sometimes it's a good idea to take an early lunch, just take a walk outside and think of something else or, one of my favorites from Brian Eno, "Do nothing for as long as possible." If you try it you'll find how hard it is to do absolutely nothing. It's a fun exercise and not as easy as it sounds.

The inspiration for this came from many discussions and talks at EuroSTAR 2010. Thanks to Lynn McKee, Michael Bolton, Rob Lambert, John Stevenson, Markus Gärtner and many more.

This is the first of my posts that were inspired at EuroSTAR 2010. There were many interesting discussions and a lot of ideas came out of various talks. I will be writing more soon.

I failed!

I failed! Yes I did and I'm not ashamed about it either. I learned an important lesson and I thought I’d share it.

As I have mentioned before on this blog I am involved in both testing and in developing a PDM system with a large number of users and it was in this system I failed to find a pretty nasty bug.

The bug had been in production for some time and no one had reported any problems. Now we were putting in new functionality that opened this Pandora’s Box. The bug was particularly nasty because as a user you would get no feedback from the system at all even if you fell victim to it. This is an extremely nasty class of bugs. The bugs that will mess up data and you have no idea.

So what was it? What did this nasty bug do? It changed data in the background, underneath an edit form, causing you to believe you were changing metadata for one object when in fact you could be changing the same metadata for a different object. The metadata not being displayed in the standard presentation of the objects you could do this without seeing it. The only way you’d know something was wrong was if you looked for the change you thought you made and discovered it wasn’t there. You might guess that you closed the editing form without committing the change or that possibly the system failed to commit the data. I don’t think that a lot of people, even testers, would think that perhaps they changed the data for some other object. If your testing a brand new system you would look for things of that nature but in a system that has been live in production for years and has been evolving over those years you would, at least I would and did, assume that there’s no way a bug of that kind could have slipped through. That was one of my mistakes! Don’t make assumptions without testing them . I know this, and I’m sure you do to, but the human mind being what it is, we assume things. We take things for granted and we are biased. Might as well admit it. It’s true.

We found it, or a colleague of mine found it, and there was much rejoicing. Well, that’s not the counting the poor guy who had to fix it quickly since we were in the middle of putting a new release in place in the production environment. He did pull of a good enough fix.

All is well that ends well and the good enough fix will be made better and eventually launched into the production environment in June. I learned a few things and that is always good. Don’t make assumptions without testing them, test all around and question yourself if you don’t find any bugs. Test all around is my key to remember to test clicking on stuff outside the form/dialog/div that I’m currently in and also to looking for interactions between the surrounding context of a form/dialog/div that might not be obvious or even readily visible.

Think for yourselves. I try to!

Tester Tested!: Checkmate heuristic :: A security testing attack

I just read Pradeep's blog Tester Tested!: Checkmate heuristic :: A security testing attack and I just loved it. Thinking is the key here, as it always should be. If you want to test something you can't do it without thinking. Following a classic scripted test case means you'll have to shut down your natural curiosity and your natural thinking. You need to be in machine-mode and just perfom. I'm sorry, but you'll never find anything that way. You'll only find what the developers already should have found and fixed.

In one of the projects I'm working on currently we do have something that we can call a script, if we want to and if we, for political reasons, need to. It's really just a map of the system written out in the english language instead of as a drawing. It shows the whole system on two levels. First it's the five, or six, major functionallity areas of the system and secondly it's the major functions within each area.

This is really all we need and since the people who do test the system, and the new releases, are thinking human beings, it works great. We tend to find issues and bugs in every release and the most interesting are the ones that have been in the system probably for years without us finding them or anyone reporting them. They may or may not be critical. Some are and some are not. Anyway we are all excited when we find a bug! If it's an old one we tend to get even more excited. Finding bugs in the new, or changed, functionallity is easy. It's finding the old ones that is hard, and therefore more fun.

I'll keep posting new stuff as often as I can, and if I find something worth sharing.

I found a bug!

Yes, I found a bug and it made me so happy! Last Saturday we deployed a new release of a PDM system at a large company and everything was going fine. We had all new components in place and were testing to make sure we hadn't missed anything when I decided to test something we hadn't changed in this release. Why? Well, it was a case of "Why not?" combined with "The wrong place". I need to come up with a good name for this combination of heuristics/triggers that I find useful. "Why not?" is, in my book, simply applying a punk attitude and question the system profoundly blended with a bit of reckless abandon, also very much punk. In this case it led me to "The wrong place". The wrong place is any part that is not affected by the changes deployed. Or any place of the system where I believe that the changes I test have any impact at all. So I decided to test input that produced a problem in one part of the system, where we had made changes, in a completely different place. This place also being one that handles input in a totally different way for different purposes and in this case a place that was not affected by the changes we deployed.

I did find a bug! The tester in me was excited and happy and knowing the system well, having been a developer on this system for several years now, I also realized that this is an easy bug to exterminate. Since we were deploying a new release in the production environment we didn’t do anything about the bug other than make a note of it.

Had I not been thinking for myself and used my testing tools the bug would still be there. Undetected, unreported and unsolved. The bug wasn’t critical but it still was a bug that caused loss of information, loss of meta data in this case, unreported and difficult to detect since you would have to know that there was supposed to be more information than you see. It’s never easy to spot what isn’t there! "What’s missing?"is a trigger I use when testing. It demands creativity and an open mind and is something I constantly have to remind myself of using. Is something missing here? Could there be something missing here? In most cases I find that if there is a time you ever needed an oracle this is it! Go talk to the developer, if available, or RTFM. Anything, or anyone, that can provide you with more information.

I'm still learning and I get excited when the tools I've learnt to use works so well.

In the trenches and slowly moving ahead

I heard a rumour while I was attending Öredev in Malmö in november about a teamleader at my company. It was an inspirational rumour. The main thing was that he wanted to scipt more testing of the product he is working with!
Is this inspirational, you may ask, and I have to say YES. It should be inspirational to anyone who believes in the possibillities of an exploratory testing approach. There's a good debate in the making when someone wants to go in that direction.

We planned a meeting to talk about the impressions from the conference with a focus on testing. I went into the meeting with a guerilla attitude but no such tactics were needed. It was a good meeting. We quickly discovered that the they do exploratory testing, but in a very freestyling way and only rarely. The test lead on that team does it when completely new services and new major features are getting ready for testing and have reached a level where it can be done.

I should tell you that the product we're dealing with here does not come equipped with a UI. That part is provided to our customers by other companies. This makes testing a bit more challenging, and interesting. All ideas about testing under these circumstances are welcome. I want to learn more and I try to listen and learn from all sorts of sources.

Anyway, we didn't really arrive at any decisions about ET but we decided to keep talking about integrating it in the processes. The development team is working hard on implementing development processes that are intended to help raise the quality of their output and instead of trying to change to much at the same time testing, and test approaches, where put on hold. Have you seen this before? It's not all that bad, there is alot of positive vibes and an open mindedness for change that will lead to improvement in the delivery and development so I have hope for the future. I just got to remind them that there are ways to get better at testing to!

A blog about testing!

Last week I attended the Öredev conference in Malmö and spent the first three days learning about Rapid Software Testing from James Bach. The last to days of the conference I spent listening to a diverse group of people including Karen N. Johnson giving a talk about story telling techniques and how to use them in our context, Torbjörn Ryber, who shared some interesting experiences, Pradeep Soundararajan shared experienceses from India. There was alot happening and one of the other speakers I listened to was Mattias Göranssson, from Sony Ericsson in Lund, talking about introducing Heuristic Based Testing at SEMC. It was very interesting and since he could not share much detail and the fact that they have been at it for less than a year, it lacked a little in examples. It would be interesting to learn more about how they go about introducing exploratiry testing, since that is what he is trying to do, and what experiences that will produce. I'm looking forward to hearing more as they move forward.

The experience of the conference was over all good. The food was bland but the talks were good. Pradeep told me he had been to many conferences in India were the food was great, and that was what people came for, but the speakers were bland. I prefer it the way it was i Malmö allthough I would really like good food AND good speakers. We'll see when that happens.

James Bach did a good job in the Rapid Software Testing course. I think we all felt challenged and alot of us did our best to challenge James which did lead to some very interesting discussions and we all came out of that with new knowledge and new experiences. The two things that are reoccurring in James's course , I think, are two major themes: Focus and Defocus and most importantly Think For Yourself!
I will make a t-shirt for myself that says: I Think For Myself! That is just a small, but not insignificant, thing I will bring with me from the course and conference in Malmö last week.

I will try to share as much, and as often, as I can, my experiences in testing on this blog. I will use it to try and get reflections from other testers. I will try to use it to learn from you! Please leave a comment about anything if you come accross something you think is good, not that important to comment but appreciated, something that you think is so off the wall and so completly wrong you just have to speak up, I love a good debate and a good discussion.

Don't hesitate! Plunge in!