Playing in the sandbox: the flawed and imperfect process of testing new tools
By guest contributor Kiran Khullar
Are all the new digital tools available a little overwhelming to you? Not sure which one(s) to use when? Me too.
I needed to know the answers to those questions to do my job and to stay sane. So I created a “sandbox” — a contained space to test and try different things. In this case, the sandbox I created was to test applications that I might need to use to work more efficiently, and to do my job better from a distance.
Here’s the real catch, I need to know which applications work seamlessly for me, for people in other departments and for people outside the Government of Canada. Sound familiar? In order to do this testing, I needed people in all of these places. I hoped for people with different ways of thinking, and different technology abilities and confidence.
In order to do testing with people in all of these places, I asked for help. I posted on a few slack spaces, and on Twitter asking if people would like to help me test and try to break tools, applications and platforms that were relevant to working at a distance from others (e.g. web conferencing, collaboration boards, etc.). I was keen to know the limits of a tool in addition to its strengths (strengths are why we learn about a new tool; constraints are why we get frustrated). It’s this combination of strengths and constraints that makes a tool great for one activity and less ideal for another.
The good news is people said yes!!!
When I asked for help, I was afraid that I wouldn’t have volunteers join me. I had support from people in the Free Agent community, from public servants and people that I didn’t know from various slack spaces and Twitter. There was an immediacy to my request and people responded — perhaps because the challenge I was trying to solve was one so many of us were (and are) experiencing! I have much gratitude for all these people!
In addition to their generosity, people showed me that grown-ups like sandboxes too. We all like to try things in a way that’s not overwhelming. Through this process, I learned about way more application features than I would have found on my own. For example, did you know that MS Teams has a closed captioning option, or that Jisti has speaker stats? I didn’t and I found out nuggets like this by working with others. You can check out more about what we learned in the results of our sessions (session 1, session two, session three).
Along the way, I made some mis-steps. My background is in facilitation and public participation, so I should have remembered that people like knowing who is on the call with them. Ooops. In session two, the participants themselves did this (yay!), and in session three I finally remembered to leave time for introductions. Again, I have so much gratitude for those that joined and even more for those that stayed, and came more than once.
I’ve wondered why people came, and I think it’s because participants had enough personal incentive to join (i.e. they got something out of it), and they enjoyed playing in the sandbox. We made it safe to play — and to make mistakes — by creating a time-limited, purpose-driven community where it was ok to experiment and admit you were learning. I know it helped me understand how easy (or not) it was to onboard onto a tool. Together, we could see others having the same or different issues, and observe the group helping each other solve challenges.
This wasn’t an easy process though. Along the way, I had to pivot. In my mind, my objective was to find out quickly about these tools. But participants shared that the objective was not clear to them, and asked me questions, gave me feedback and took time for conversations (many thanks to Seamus, Dave, Ted and Eric) that helped me crystallize my thoughts. Now, the objective is to see which tools are easy to access and use, and, once we know that, to take a deeper dive into the tool. Other pivots included frequency changes to once every two weeks to allow space for write-ups to happen and a time extension from 30–60 min which ideally allows more time for introductions and play. By listening, responding, and adjusting in response to their requests, I think I helped to reinforce our time as a safe space.
Here’s what I know for sure. The Government of Canada Digital Standard’s call to collaborate widely, means that I get to invite people into my work. And that has paid dividends for me.
I’d like to extend my gratitude to the people playing in this sandbox. The list includes Lily Spek, Dave Sampson, Ted Reinhardt, Marcela Popowhich, Seamus Brugh, Susan Lehmann, Kate McKerlie, Rebecca Verral, James Kuffner, Eric Shoesmith, Alexandra Haggerty, and Abe Greenspoon. The best part is that people want to share and try other tools that I had not even thought of and that excites me! Check-out our list here. There are many tools listed, and it is far from exhaustive. It definitely doesn’t include tools we don’t know about, so if you know of a tool that needs testing, add it! I only recently learned about Twitch and BigBlueButton.
I hope you’re inspired to make a sandbox and run your own test. Based on my experience, define and share your objective, ask others, try it and listen to their feedback. If that feels too far right now, then please join me and others in our bi-weekly tests. You can reach me @cdnKiran. Most importantly, don’t let the tools and applications out there constrain you. Know the constraints of the tools and let’s help each other unleash our collective creativity.
