listeningTo: The Trauma Model by King 810
inRealLife: My whole company is remote spread out all over the world, but we get together once a year in NYC for a two-day team meeting, and it’s this week! I’m excited to see everyone, but I’m more importantly excited for tomorrow night’s team excursion to see To Kill a Mockingbird on Broadway! (Ed Harris’s first day on the job!) TKAMB is my absolute favorite novel of all time for many reasons – one of my doggie’s name is Scout and I have a large-ish forearm tattoo inspired by a few of the classic book covers. I’ve been in a rut, and I think this will be the thing to pull me out.
whatIReadThisWeek: I’m still feeling less like myself, so no personal reading, which in itself is making me feel down, but, also, Maggie Stiefvater’s new book comes out tomorrow and I think that will excite me enough to get back on track.
While these articles aren’t super recent, I have had them saved for weeks in anticipation of this post and I did read most of them in the last week or so. If you’d like to read along:
- Simon Tomes + Elizabeth Zagroba via The Club on MoT: Exploratory Testing Power Hour
- James Bach + Michael Bolton via Satisfice.com: Exploratory Testing 3.0
- Undeveloped Bruce: Our first exploratory testing session
- That’s the Buffet Table: Pathway Exploratory Testing
- TestBuddy (video): Start Exploratory Testing Today – Risks & Questions
whatILearnedThisWeek: In part 1 the other week I focused mostly on what the heck a mind map was, and this part will be more about what the heck exploratory testing is, and how the two can be used together. As a quick update on the last post: I have an official Mind Map skeleton!
I’m really excited to continue the project. I decided to map the full application, first, but am still thinking it will be more useful to create a map on the feature-level where I can add way more details and not be so overwhelmed. This is really just the basics of the site and it’s already feeling like a lot to look at. I need to pretty it up!
Reading more about exploratory testing the last few weeks have also strengthened my hunch that the two processes can be used together in a really nice way. Questions posed by That’s the Buffet Table drove the point home for me that, yeah, this is something I should be doing regularly; “Are you really performing only the specified steps and checking the specified expected results or are you doing more?” Uh, always more, I get so distracted… “How do you tell people what you saw when you moved away from the script?” I don’t, I try to just remember to come back to it later, which often ends up not happening since by the time my hands are in the application and I’m working through my manual regression test plan, I’m already so stretched that there is usually not enough time to address non-release related things. “Are your cases/steps answering questions about risks?” No, I don’t think they are. I would say 99% of my test cases are searching for bugs, which are risky, sure, maybe I’m not understanding what is mean by “risks”, but, the way I’m thinking about it is not necessarily finding bugs, but finding what makes the application vulnerable, or what provides a stupid user experience which could risk the business. Is there a better term for this?
But, anyway, what is exploratory testing? I’ve mashed up my own definition based on the limited reading/watching:
Exploratory testing is poking around your application for a specified amount of time with no real purpose other than to see where you are taken and uncover any underlying bugs or weirdness not commonly caught with structured test plans to come up with documentation to share with your team of areas of risk, bugs to fix, processes to rethink.
I’ve seen a few sources discussing formal exploratory testing sessions where time is specified, charters are written, and plans are in place long before the session begins, and I’ve seen some more loosey-goosey “hey let’s click around and see what breaks”. I think I would prefer the later, since everything I do is generally loosey-goosey, but keeping in mind that I have this obsession with using my mind map to guide my exploratory testing sessions, I think having a formal session and guide will make the event more successful.
TestBuddy’s video proposes 5 steps to successful exploratory testing:
- Understand the product’s value.
- Come up with possible risks.
- Turn risks into questions.
- Run a test session to answer our questions.
- Share your findings.
Performing the actual test session isn’t until step 4, but I struggle to think of answers to steps 2 and 3 before exploratory testing. I think once I have more of an idea of where the risks lie, I can better form questions that I want answered in subsequent test sessions, but for the first one, I want to go in blind.
Undeveloped Bruce writes about how her team uses charters and a specific Six Thinking Hats method, which I’m very intrigued about, but, sadly, since it’s still just me here and though I feel like a different person depending on the day, I don’t think I can personify 6 different kinds of people in one session all by my lonesome. I’ll be interested in researching the method more if I ever get the luxury of working on my personas project…. but, anyway, there is a nice example of their testing charter in the article, though, that I am hoping to use as a guideline as to what I should be pre-planning before jumping in which includes things like “target or scope”, time, and who will be focused on what. If my first session alone goes well, I may try and recruit some other people on my team for a little experiment. I have to be careful, though, since we all have big grand plans for the site and I don’t want to be misleading about being able to “fix” any of the weird stuff we uncover in a timely manner – we still have a massively ambitious road map for the next 6 months that I’m not at all anxious about (I say through a forced smile and terrified eyes) (send help).
whatIAmThinkingAbout: If I make more targeted mind maps at the feature level as I keep musing about, I think I can make the two of these strategies really work well together. My immediate thoughts are about new feature planning, but when considering testing I think focusing on one large feature or a group of related features will be worth the time. We encourage members to leave reviews for books they get access to on the site, so a smart experiment could be working through the get-a-book-and-leave-feedback process, which is slightly more complex than it sounds, at least there are a ton of moving parts that can be explored…
I don’t always (ever?) have the luxury of working on big projects outside of my “normal” job responsibilities. There has been a recent conversation with my supervisor, though, that it’s probably time to start acting managerial (despite “manager” being in my title for 4 years) which will involve passing off some of my normal day-to-day stuff to the actual project manager. I’m hoping this frees me up to do things like make my mind map, schedule regular exploratory testing sessions, finally get to this personas project that has been haunting me for 2 years, and, like, actually be able to maintain my test scripts, not to mention some time for personal development.
With the fantastical free space coming up, I think a smart project to jump on first will be implementing exploratory testing sessions, even if it’s just an hour or so at the beginning of a release cycle or some time that makes sense. I struggle to determine what of my work I want to track in Jira like a real dev, or what work I just do quietly by myself and bring up out of the blue when it seems relevant and listen to everyone be like “wait what when did this happen?” I’m leaning towards scheduling a 1 pointer in Jira every now and then just to 1. honor my specified amount of time dedicated to this, 2. have a place to post my documentation, and 3. let anyone who cares know it’s going on and see if they want to join in for a session or 2.
recommendationsAndTakeAways: The biggest takeaway for me came from the exploratory testing Power Hour question about taking and sharing notes. Simon Tomes responded, “I enjoy using the PQIP approach: I document Problems, ask Questions, share Ideas and give Praise for stuff I discover that I think is cool.” I think this format of note taking is fantastic for any setting, but I can see it making exploratory testing sessions even more digestible for the rest of the team if I can present these four categories to them and ask for them to add their own thoughts.