Learning experiences

So, the man who claims to be thinking for himself woke up and remembered he had a blog. I haven't written anything here in a long time but I will start over and try to establish some kind of flow.

I got an idea for a series of posts today. It was my own idea but I didn't see it until Pradeep Soundararajan kicked it back to me. He suggested I write a series of posts about what makes my "eyes and brains tiered". That was a challenge back at me based on a comment I made on his blog. So i will!

As the title suggests I will keep going along this track for a while. It will be about what I use to learn new stuff and maybe even how I use my new knowledge in my work. That may be related or relatable to testing. That is entirely up to you. Read it. Comment it. Use it. Don't use it. If you do find something useful please use it and remember where you found it. I give no guarantees that anything here will be useful for anyone other than me.

Experience #1

The other day my brother asked me if I wanted to join him look at a house. He and his wife are moving down here from Stockholm and I thought it might be fun and maybe even helpful to him. It was one of those evenings when it was just me and my 3 and a half year old daughter home so I put her in her seat next to mine in the car and we were off. On the way I we talked and I pointed out that soon we'll pass the garage where they fixed my car about a month ago. She started talking about that air comes out, at two places on my car. I was completely lost there for about 30 seconds until she said it was at the back of my car and that that was what they fixed. Then it clicked. She was talking about the exhaust, of course. It was air to her, exhaust to me. Two places, was really one. At least in my world.

The problem with the car was that after sliding, not very fast, of the road I got stuck in the snow and had to get pulled out of the snow by a very big and friendly tractor. The catalytic converter got beat up and my car got a bad case of asthma and the exhaust fumes couldn't get through so it leaked out and ended up in my air intake when I hit the accelerator. So it came out in two places!

I was so happy when I understood what she meant. She got all the parts right, using her own words. I had to use my tired brain to decode her language, she speaks very well and has a large vocabulary for her age, so it's usually easy to understand what she's talking about. In this case it took some thinking. I quickly realized, and actually thought this as we were driving along and I was praising her for remembering the details, that this is what I encounter everyday when dealing with clients and, not always to a lesser extent, my colleagues.

Today this helped me understand a clients problem late in the afternoon. His words were not the ones I would use but I remembered that drive and the talk I had with my daughter and I do believe I has a pretty good idea about how to find and fix the problem tomorrow when I go to see him.

Sometimes this is obvious. The fact that we call things by different names that hold different meaning and has different values for us. In those cases it's easy to see through it and understand the person we're talking to. A few questions and we're usually on the right track. The tricky part is when we assume that the person we're talking to is coming from the same place as we are. That's when we think that we understand each other but later we find out that not only were we not on the same page but not even in the same book! That's when it can get very expensive!

In testing we may be fooled by something that looks familiar and we think we understand what we're supposed to do or think we understand what is going on. If we don't question familiar looking stuff we can miss a lot. It's about communication with the system and testing it! I might use my "Why not?" heuristic in that case. It's about communication with oracles and here we really need to think in order to make sure we understand the oracle. It's not a very useful oracle if we don't understand it. To me it is all about remembering and recognizing the current context. I'm sure there are more ways were remembering this is important. I just can't think of them right now.

The point to me was that I learnt something in a context that was not one where I was expecting to learn. I didn't dismiss my daughter because I didn't understand her at first, or because she's only 3 and a half years old, and that fact alone made me proud of myself as a father, and I did learn an important lesson. Listen and think and keep listening while you think. It will help!

I failed!

I failed! Yes I did and I'm not ashamed about it either. I learned an important lesson and I thought I’d share it.

As I have mentioned before on this blog I am involved in both testing and in developing a PDM system with a large number of users and it was in this system I failed to find a pretty nasty bug.

The bug had been in production for some time and no one had reported any problems. Now we were putting in new functionality that opened this Pandora’s Box. The bug was particularly nasty because as a user you would get no feedback from the system at all even if you fell victim to it. This is an extremely nasty class of bugs. The bugs that will mess up data and you have no idea.

So what was it? What did this nasty bug do? It changed data in the background, underneath an edit form, causing you to believe you were changing metadata for one object when in fact you could be changing the same metadata for a different object. The metadata not being displayed in the standard presentation of the objects you could do this without seeing it. The only way you’d know something was wrong was if you looked for the change you thought you made and discovered it wasn’t there. You might guess that you closed the editing form without committing the change or that possibly the system failed to commit the data. I don’t think that a lot of people, even testers, would think that perhaps they changed the data for some other object. If your testing a brand new system you would look for things of that nature but in a system that has been live in production for years and has been evolving over those years you would, at least I would and did, assume that there’s no way a bug of that kind could have slipped through. That was one of my mistakes! Don’t make assumptions without testing them . I know this, and I’m sure you do to, but the human mind being what it is, we assume things. We take things for granted and we are biased. Might as well admit it. It’s true.

We found it, or a colleague of mine found it, and there was much rejoicing. Well, that’s not the counting the poor guy who had to fix it quickly since we were in the middle of putting a new release in place in the production environment. He did pull of a good enough fix.

All is well that ends well and the good enough fix will be made better and eventually launched into the production environment in June. I learned a few things and that is always good. Don’t make assumptions without testing them, test all around and question yourself if you don’t find any bugs. Test all around is my key to remember to test clicking on stuff outside the form/dialog/div that I’m currently in and also to looking for interactions between the surrounding context of a form/dialog/div that might not be obvious or even readily visible.

Think for yourselves. I try to!

Tester Tested!: Checkmate heuristic :: A security testing attack

I just read Pradeep's blog Tester Tested!: Checkmate heuristic :: A security testing attack and I just loved it. Thinking is the key here, as it always should be. If you want to test something you can't do it without thinking. Following a classic scripted test case means you'll have to shut down your natural curiosity and your natural thinking. You need to be in machine-mode and just perfom. I'm sorry, but you'll never find anything that way. You'll only find what the developers already should have found and fixed.

In one of the projects I'm working on currently we do have something that we can call a script, if we want to and if we, for political reasons, need to. It's really just a map of the system written out in the english language instead of as a drawing. It shows the whole system on two levels. First it's the five, or six, major functionallity areas of the system and secondly it's the major functions within each area.

This is really all we need and since the people who do test the system, and the new releases, are thinking human beings, it works great. We tend to find issues and bugs in every release and the most interesting are the ones that have been in the system probably for years without us finding them or anyone reporting them. They may or may not be critical. Some are and some are not. Anyway we are all excited when we find a bug! If it's an old one we tend to get even more excited. Finding bugs in the new, or changed, functionallity is easy. It's finding the old ones that is hard, and therefore more fun.

I'll keep posting new stuff as often as I can, and if I find something worth sharing.

I found a bug!

Yes, I found a bug and it made me so happy! Last Saturday we deployed a new release of a PDM system at a large company and everything was going fine. We had all new components in place and were testing to make sure we hadn't missed anything when I decided to test something we hadn't changed in this release. Why? Well, it was a case of "Why not?" combined with "The wrong place". I need to come up with a good name for this combination of heuristics/triggers that I find useful. "Why not?" is, in my book, simply applying a punk attitude and question the system profoundly blended with a bit of reckless abandon, also very much punk. In this case it led me to "The wrong place". The wrong place is any part that is not affected by the changes deployed. Or any place of the system where I believe that the changes I test have any impact at all. So I decided to test input that produced a problem in one part of the system, where we had made changes, in a completely different place. This place also being one that handles input in a totally different way for different purposes and in this case a place that was not affected by the changes we deployed.

I did find a bug! The tester in me was excited and happy and knowing the system well, having been a developer on this system for several years now, I also realized that this is an easy bug to exterminate. Since we were deploying a new release in the production environment we didn’t do anything about the bug other than make a note of it.

Had I not been thinking for myself and used my testing tools the bug would still be there. Undetected, unreported and unsolved. The bug wasn’t critical but it still was a bug that caused loss of information, loss of meta data in this case, unreported and difficult to detect since you would have to know that there was supposed to be more information than you see. It’s never easy to spot what isn’t there! "What’s missing?"is a trigger I use when testing. It demands creativity and an open mind and is something I constantly have to remind myself of using. Is something missing here? Could there be something missing here? In most cases I find that if there is a time you ever needed an oracle this is it! Go talk to the developer, if available, or RTFM. Anything, or anyone, that can provide you with more information.

I'm still learning and I get excited when the tools I've learnt to use works so well.

In the trenches and slowly moving ahead

I heard a rumour while I was attending Öredev in Malmö in november about a teamleader at my company. It was an inspirational rumour. The main thing was that he wanted to scipt more testing of the product he is working with!
Is this inspirational, you may ask, and I have to say YES. It should be inspirational to anyone who believes in the possibillities of an exploratory testing approach. There's a good debate in the making when someone wants to go in that direction.

We planned a meeting to talk about the impressions from the conference with a focus on testing. I went into the meeting with a guerilla attitude but no such tactics were needed. It was a good meeting. We quickly discovered that the they do exploratory testing, but in a very freestyling way and only rarely. The test lead on that team does it when completely new services and new major features are getting ready for testing and have reached a level where it can be done.

I should tell you that the product we're dealing with here does not come equipped with a UI. That part is provided to our customers by other companies. This makes testing a bit more challenging, and interesting. All ideas about testing under these circumstances are welcome. I want to learn more and I try to listen and learn from all sorts of sources.

Anyway, we didn't really arrive at any decisions about ET but we decided to keep talking about integrating it in the processes. The development team is working hard on implementing development processes that are intended to help raise the quality of their output and instead of trying to change to much at the same time testing, and test approaches, where put on hold. Have you seen this before? It's not all that bad, there is alot of positive vibes and an open mindedness for change that will lead to improvement in the delivery and development so I have hope for the future. I just got to remind them that there are ways to get better at testing to!

A blog about testing!

Last week I attended the Öredev conference in Malmö and spent the first three days learning about Rapid Software Testing from James Bach. The last to days of the conference I spent listening to a diverse group of people including Karen N. Johnson giving a talk about story telling techniques and how to use them in our context, Torbjörn Ryber, who shared some interesting experiences, Pradeep Soundararajan shared experienceses from India. There was alot happening and one of the other speakers I listened to was Mattias Göranssson, from Sony Ericsson in Lund, talking about introducing Heuristic Based Testing at SEMC. It was very interesting and since he could not share much detail and the fact that they have been at it for less than a year, it lacked a little in examples. It would be interesting to learn more about how they go about introducing exploratiry testing, since that is what he is trying to do, and what experiences that will produce. I'm looking forward to hearing more as they move forward.

The experience of the conference was over all good. The food was bland but the talks were good. Pradeep told me he had been to many conferences in India were the food was great, and that was what people came for, but the speakers were bland. I prefer it the way it was i Malmö allthough I would really like good food AND good speakers. We'll see when that happens.

James Bach did a good job in the Rapid Software Testing course. I think we all felt challenged and alot of us did our best to challenge James which did lead to some very interesting discussions and we all came out of that with new knowledge and new experiences. The two things that are reoccurring in James's course , I think, are two major themes: Focus and Defocus and most importantly Think For Yourself!
I will make a t-shirt for myself that says: I Think For Myself! That is just a small, but not insignificant, thing I will bring with me from the course and conference in Malmö last week.

I will try to share as much, and as often, as I can, my experiences in testing on this blog. I will use it to try and get reflections from other testers. I will try to use it to learn from you! Please leave a comment about anything if you come accross something you think is good, not that important to comment but appreciated, something that you think is so off the wall and so completly wrong you just have to speak up, I love a good debate and a good discussion.

Don't hesitate! Plunge in!