SkillsPosted by Martin Jansson Wed, March 18, 2009 08:45:12A few years ago I switched company from a software company to a hardware focused company. I understood from start that this would be a difficult task in getting to learn the new technologies and platforms, but I did not see what this would do for testing.
When you test software that you are totally new, but where you have experience with the environment such as operating system as well as technologies used, you are still able to dig deep into testing and will probably find lots of bugs.
If you instead test something where you are both new to the software/hardware/system and to its environment you will have it a lot harder to find bugs. You will stumble upon things like… oh this went wrong, then learning ah I did not configure it correctly or oh it is supposed to work like that. You are unsure what to expect and what is actually wrong.
Think of yourself as an explorer where you have no knowledge about the gear you are using, where you have landed, what this green thing you stand is and so on. Do you dare explore? That hot red thing over there... let me just dip a toe. Oh it was lava.
So, when you find yourself in such a situation you should do some research, read lots of books and get to know what territory you are in. Having knowledge about the environment and platform you stand on will enable you to do a better job as a tester.
DocumentationPosted by Martin Jansson Tue, March 17, 2009 19:18:29”Trac is an enhanced wiki and issue tracking system for software development projects. Trac uses a minimalistic approach to web-based software project management. Our mission is to help developers write great software while staying out of the way. Trac should impose as little as possible on a team's established development process and policies.”, see http://trac.edgewall.org/
This is a very nice light-weight tool. Some of the features:
* Got a wiki
* Possible to connect to SVN to see changes
* Easy to link check-ins with tasks, bugs, etc
* Setup milestones for planning
* See timeline for project management
I can highly recommend this tool if you want to do project management to some level while at the same time utilizing the benefits from SVN.
PeoplePosted by Martin Jansson Tue, March 17, 2009 19:09:31From what I have seen there are few recruiters that know what testers and test leads do. Considering that we had a EuroSTAR focusing on defining the profession can be seen as that testing as a profession is quite new and can therefore be considered immature.
Many recruiters look for a certain set of skills, education, certificates and knowledge in tools when seeking testers and test leads. The emphasis on tools and certificates seem to be great. They are unsure what keywords to use when searching for the aces in testing.
There are so many parameters to think about when recruiting testers and test leads, which makes this a hard task. Consider that each company is different, each tester might work differently, each object that needs testing is different, and there are a multitude of technology areas that need different skills and so on.
We assume that persons with different background, gender, education, skill sets and so on will generate different ideas on how to test and will probably come to different results. Transferring this fact to recruiting will make it hard for a recruiter.
I have no magic formula for what key words to look for when seeking testers other than that they should be good at testing, which I’d figure is hard to filter out in a job application.
Cem Kaner has written a good article about things to consider when recruiting. This is not only for test related jobs, but for recruiting as a whole, which you can find here.
Johanna Rothman has also written a good article on recruiting testers, which you can find here.
PeoplePosted by Martin Jansson Tue, March 17, 2009 15:12:38The indispensible worker is the one who always saves the day by touching the program by giving some magic input in order for it to work. He will quickly get the work done and make the customer happy. Upper management will see him as the perfect employee, but many co-workers will think differently.
In the indispensible workers back-water you will notice half-finished tasks, programs that need the magic input in order to work, documentation that usually are not the latest version or just not there, hidden tasks that was performed once a week manually to avoid the weekly disaster and so on.
In my experience, if someone has become indispensible, where the company cannot work without them being there, it will often lead to disaster. This can easily be seen if someone is sick, on vacation or if they leave the company, when certain things start to break down or that the team stands still waiting for the magic touch.
As a co-worker you avoid these traps by requiring documentation enough for someone else to perform the task or that you have at least a backup for the critical tasks. The hardest thing is getting upper management to see that it is not often good to focus on the short term solutions where you save a small bit of time in the moment, but where you have to pay for it tenfold in the future.
PeoplePosted by Martin Jansson Wed, November 26, 2008 20:16:43This article is about combining the role of the tester with other roles. These are observations based on my own experiences. I’ve not studied any books or other articles about this.
I assume that each role has its own mind set, focus and goals. I will try to identify the agendas and possible mind sets that I had at each time and then try to form an opinion on how a certain combination worked together with the tester role. I will cover the roles of test lead, test manager, project manager and documentation specialist.
A few objectives/goals of the tester are (taken from materials from Cem Kaner):
* Find defects that matter
* Find important bugs, to get them fixed
* Maximize bug count
* Block premature product releases
* Help managers make ship / no-ship decisions
* Check interoperability with other products
* Minimize technical support costs
* Assess conformance to specification
* Conform to regulations
* Minimize safety-related lawsuit risk
* Find safe scenarios for use of the product
* Assess quality
* Elucidate the design and prevent errors
The tester and test lead have similar, if not almost the same objectives. One observation I can share is that as a test lead or team lead for testing you sometimes dwell a lot on resource planning, keeping everyone occupied at testing. This stole a lot of focus from thinking of new test ideas.
The tester and test manager is a bit different. Depending on which stage you are in organization you will have different agendas as a manager. If you are focused on test processes, propaganda around testing, selling the test concept to the organization or whatever it is, you will behave differently. What I experienced was taking a step away from the actual testing and putting more focus on the processes. I even considered letting certain issues pass, thus not dwelling on them so that the organization can learn from the mistakes. I thought that going in to do this temporary fix would be good in the short run, but it is better for it to fail so that we can have good discussions and perhaps doing something in the long run. Many of these thoughts were contradictory to what I usually thought as a tester. So, in order for this to work I needed to be very clear when I was the tester or test lead in the project and not acting the test manager.
Another experience from being test manager combined with being a tester is the first months of my employment as a test manager. I entered as a tester in a project while at the same time was supposed to build the test organization at the company with everything from introducing tools to having presentations about what testing was. In the project there was a test lead that had no experience from being test lead and little experience from testing. We clashed several times since it was very hard to go from being a tester to being a test manager, often in the same discussion. As a test manager I wanted us to use a bug tracking system and a test management system in a certain way, while he saw no use for these. In any case, it was very hard to do a good job either as a tester or as test manager having different agendas at the same time.
In another situation I was project manager, test lead and tester. I was also supposed to be documentation specialist, but I was able to duck that in the last seconds since I was incapable of taking on a whole new focus in my work. Actually, I almost broke down completely since as a documentation specialist I needed to have full focus on the documentation and manuals. I would not have been able to maintain the meta-perspective as a test lead or project manager at that stage. Combining the project manager and test lead was not that hard, in some cases I made project management decisions with a touch of test lead focus. Still, it was easy to notice that the dominant role was project management having the focus on deliverables, time, cost and resources. In those situations quality was pushed down on the list automatically. When I took on the test role I had to be very clear on that I was tester and that I was allowed to focus on that. Combining these roles was very hard and sad to say, quality and testing did not get the attention that they should have seeing that project management became dominant.
As a summary, if possible do not combine the tester with any other role than test lead. It is too easy to push away the strengths of the tester’s mind-set and objectives. Seeing that project management becomes dominant might also mean that it can be hard to keep the focus on so many things as a project manager. If you think about all the things that testers and test leads do, it will just become too much. In projects where you do not have someone who have the role of tester and test lead you will miss so much. In projects where you have someone acting tester and test lead in combination with other roles, do know that you will not be able to use their full strength and expertise.
SkillsPosted by Martin Jansson Wed, November 26, 2008 18:56:00I've just gone through a three day course in Rapid Software Testing (RST) run by James Bach. This is the first time I've seen him in person. I've previously seen his responses and discussion from the yahoo group software-testing. I’ve seen the presentation on RST by Michael Bolton and I think either of them would do this well.
I thought he would be a lot more aggressive in person, but he was just inquisitive and straight-forward. It was very nice to hear him talk about ideas and war-stories that are similar to my own experience, but things that I might have not been able to think through. It is obvious that he has a vast experience in testing from many different situations. He talked many times about his integrity in testing (though phrasing it differently), that he has quit projects or jobs when he could not accept the “evil bureaucracy” or managerial demands that was outrageous.
The course itself focused heavily around how to do rapid testing and the concepts around exploratory testing. He had many exercises that were all of them thought through and challenging. I got lots of new ideas for testing, test managing, test theorizing and test planning. As always you were able to get a look at new tools that can be used in testing as well as books that you must add to the bookshelf.
I can recommend doing this course if you want to dwell deeper into the concepts of exploratory testing and rapid testing. I would have enjoyed working with him seeing that many of his ideas are great and along my own ideals.
IdeasPosted by Martin Jansson Sun, July 06, 2008 11:03:54Test patterns, quality patterns, Q-patterns or whatever we wish to call it. I am referring to test ideas that can be reused in similar contexts. This could for instance be File Handling, Data Types, Installation, Upgrades etc. There is nearly an infinite list of areas that could test patterns could be available for.
When you start a new project with test areas that you are not all familiar with... you wish to have some test ideas or test patterns that you might be able to apply easily. Experienced testers usually have this documented or in their heads. The test patterns created for one company is usually proprietary, so you are not allowed to bring it with you.
So when you start from scratch at a company there is a lot to wish for. You try to use what you have and need to adopt to possibly new systems. Eventually you will notice that you have almost started from scratch. You start redoing the same things that you have done before at previous companies.
I have asked some of the major test consultancy companies about how they do with test patterns and test data. So far all of them has looked at my strangely and said that it is not possible to reuse the test cases they produce. Also, that these test cases are property of the customer.
Imagine that consultant companies with a few thousand testers start assisting each other with test data, test patterns and experiences. Generally, it seems like they are not doing this at the moment. The smaller companies would have a hard time to compete if this was the case.
If there was a public website where these kinds of data could be stored it would make things easier for all testers. Perhaps it is a dystopia, but I see no reason why we need to rethink our test ideas each time we get to a new company. I can understand that these test patterns in themselves hold great value to the company and that they would like it to be theirs alone. Still, I think it should be possible to have the general areas more public.
If anyone have thought of this before and know of a place where to find these so called public test patterns and test data please assist me in pointing me in the right direction. If not, is this something we should take the initiative in starting?
IdeasPosted by Martin Jansson Wed, May 21, 2008 16:11:35A project manager, that had no knowledge about testing whatsoever, asked me how he should go about testing? The question was vague since he did not know where to start. I wrote him a quick email listing a few things to consider:
* Create a list of all use cases. For each use case consider possible errors.
Identify how these use cases and the errors from these could be tested and
verified.
* Create a list of techniques used. Identify common errors for the specific
techniques. Identify issues that affect these techniques in one way or another.
These issues and common errors can be used as a base for testing.
* Create a list of the major areas of functionality that need to be tested.
Create a matrix and categorise the areas in two columns (column one: risk
anything goes wrong, column two: cost if anything goes wrong) from 1 to 5, where
1 is most costly or highest risk. Sort the categories.
* Identify important milestones when there are deliverables for testers.
* Identify what deliverables you expect from testing. This can be test reports,
bugs. Consider when and how often you want these. Be reminded that all
administration will take time from regular testing.
* Which test resources are available and to what percentage. Investigate when
they are away and if they have other tasks that they work with.
* What testing has been done so far? What is documented? What did they find?
* Is there a backlog of bugs? Are they well tended and classified? Can they be
reused and are they still valid?
* Decide on how you want to work with bugs and enhancements
* Have the testers received all possible documentation needed to start working?
This can be support documents, help-files, design/functional specifications and
so on. These documents can be used for test ideas.
* What will the test environment look like? Will it be a virtual one or a full
customer-like environment with hardware and software?
* Have you considered how many prototypes that should be available for the
testers? Consider that many might break. Testers without something to test is a
no-no.
* Is there any other configurations that we should consider that is outside the
test environment that these builds/prototypes need to run in?
If your testers are new to testing consider this:
* Use the requirements to create user scenarios that in turn can create more
test ideas.
* Some tests might need to be planned long before they are executed. Example of
this can be EMC testing or Safety testing, that might be performed externally in
other test labs.
* System testing should be done by testers, thus not a developer or product
manager.
* Do not test everything. Focus on costs and risks first. All areas that go
untested should be a calculated risk done by stakeholders or decision makers.
* Test everything. All major functionality should have at least one test. Important areas should have deeper testing while less important can have shallow
testing.
* Consider long term investments in testing. Will there be more versions of the
product that need even more testing. It might be good to invest in test
automation if it gives a ROI. Should test cases and test result be saved for the
future, in most cases you would like that.
All of this information is usually structured in the various documents that expert testers are used to such as test plans, test strategies and so on. But for someone that did not know anything about testing I just gave him this.
I got no response from the project manager on this though, so either it was total crap or I scared him away.
