Penelope Rance

How did you get started in UX?

So I actually got involved in the digital space in 2004. I’d come back from India where I’d been a journalist, travelling around and doing yoga type things and when I came home to Jo’burg there were no jobs for me as a journalist.

So I started teaching yoga, but then a friend of mine popped up. He was running something like the Amazon of Vedic books from Kathmandu. He ran it on a system called osCommerce, which is an open source ecommerce system. At that point in history I’d had very little to do with the internet, I remember him coming round with his laptop and he opened it up and said ‘this is what the back end of the shop looks like’. And I was like ‘Oh wow that’s amazing!’

So he taught me all about the ecommerce back end and how to run orders. People kept emailing and asking where their books were, and I thought if I could just automate a message to them to say we’ve received your order and it is being shipped from Delhi, they’d stop emailing me. So eventually, he said to me after several of these types of requests, ‘here’s a budget. Anytime you have a request just do it with the developers in Kathmandu, and if it’s over a certain amount let’s talk about it’.

So I redesigned the whole back end of the system, basically automating my job. Looking back, it was operations for the ecommerce business.

Then I moved to England and got a job with a vegan shoe company who were using the same system and rebuilt their back office, which then got me into working with one of the biggest yoga teaching institutions, where I redesigned their back office.

And in the midst of all this I met some clever people and they said ‘you’re a content strategist’. This was around 2006 when content strategy was just starting to become a thing, thanks to Kristina Halvorson, and I was like ‘Oh, that’s a thing?’

I’d been a writer and now I was doing these operations-type things, and I ended up working with them under this new title of content strategy, which is how I met Leisa Reichelt.

She had read some of my posts on the work I’d been doing and hired me to work with her at the University of Surrey. And there she realised that not only was I a content strategist, but I could also get shit done. So when she went to work with Government Digital Service (GDS) she hired me and said ‘I’ve got 40 researchers, who are all producing research reports, which is content and you’re a content strategist, please can you figure out how we organise that content so it doesn’t just disappear?’

So that’s when I started to learn about research and ended up building labs and research panels and a bunch of just get-shit-done type things. I actually never got round to building the research library. And that’s how I sort of realised there was this world of research and operations and got into this whole space.

What are you doing now?

Now I work for Atlassian as a Research Ops Manager, and I manage a team of eight. We didn’t exist when I first arrived three years ago, so we’ve built it up from scratch. We deliver services and tools and all kinds of support and feedback to people who do research at Atlassian. I call them AWDRs - Atlassian’s Who Do Research.

We look after about 80 researchers in Atlassian under Leisa and various leaders, then we have about 500 to 600 Atlassian’s Who Do Research.

They are designers, content designers, product managers, we even have some engineers and security engineers who have got no one to do research for them so they’re picking it up.

We offer self-service tooling and support for qualitative moderated research, and we offer both research coaching and nine training courses. Most of the courses are optional, but we do have one 20 minutes course on ethics and data handling, which you have to do to access our tools and services. We’ve trained several hundred people in the last year. Those who’ve gone through the training have a much greater appreciation of what it takes to do research - aside from just helping them do better quality research it also helps them appreciate that research is really quite specialist, so slowly but surely there’s a greater appreciation for the value of having an actual researcher in the team, which is really important.

The next step for us is how do we build up the research? How do we build up capability so that people don’t have to do their own research? So they’re not even compelled to do research because there’s either a real deal researcher in their team because we’ve helped to prove the value of research and helped them hire, or because we’ve got an awesome library in place so that before you even think about doing research you go and check out what we already know and maybe we don’t have to do that research (unusual, rarely ever happens!).

Or they’re going to observe live research that’s already happening, so they’re kind of queued into a bigger network of research that they can experience firsthand and feel less of the need to reach out to customers just to have a chat or because they’re unaware that someone else has asked the same questions. So that’s the next level of maturity that we’re heading towards, but it takes time, years even, to get to that level.

How would you define research ops?

So research ops isn’t that different from traditional operations, and the idea of giving the organisation the power to act, or to allow them to execute excellently.

Research operations is about making researchers’ lives easier, but what unfortunately happens is that there is an expectation that you are going to throw in some full service research recruitment and your researchers will never have to recruit another participant ever again. How wonderful would that be? But that’s not the reality at all. That’s not operations, that’s a research assistant, which is a completely different thing and impossible to scale (unless you have endless headcount, and who has that?).

The learning for me has been that operations is about really thinking through the operating model. When I’m trying to look after 500 people I’m not running a Michelin star restaurant where we can come out and ask you what your specific needs are - don’t worry of course we can leave the cheese off. No, this is like mass catering, airline food, or that snazzy burger place where you have to go up to the counter to order and the food gets brought to you, maybe.

So there are various levels of how you operate, and operations is about understanding where the research organisation wants to go. What are their goals? And then being able to translate those desires into models for how you deliver those things at the scale you’re currently at. Oh only five people, that’s easy, I can deliver full service to all of you, sit down, I’ll bring the food out for you, don’t worry.

But when you’re talking about 500 people, yeah dude you’re gonna have to learn how to work the cola machine, there’s a coffee machine in the corner, figure out your own sugar, but we’ll put some nice instructions out and point you in the right direction.

So that’s what operations is about. How do we enable the strategy? How do we make the strategy of research come to life? And what kind of models are we going to need to enable that over years? So I’ll look at something and think, that works fine now, but if you add another five researchers it breaks, that’s not going to be scalable if you add another 100. What I’m doing is enabling the organisation to continue to work even while it’s growing.

This is a very different definition to the one I would have given you three years ago. As a consultant you fly in somewhere and deliver this great panel or lab or whatever, and thankfully, they have all worked! But then you step away and you’ve never thought about the strategy and the organisation scaling and how it all fits together and how you maintain these things and the people you’ll need in the team to keep that going. None of that has crossed your mind - truly.

But when you’re having to run and build a team over three years in a fast-paced, fast-growing organisation you really have to start thinking about what it actually means to be operationalising research.

If someone is interested in Research Ops, where should they start?

A good place to start is the Harvard Business Review. Yes!

Traditional operations has an enormous amount to give to research operations. We think we’re so new and pioneering and special, and in the context of research we kinda are. I mean Microsoft has had a research operations team for decades, Google had for, I’m guessing, a decade, Facebook have, Airbnb, Salesforce too. It’s not like research ops never existed, it just wasn’t on the map. All I did was kinda draw it on the map and now it’s gotten significantly bigger.

But if you go back to the fundamentals of manufacturing into service operations, it’s all there for us and we’ve got an enormous amount to piggyback off of.

Harvard Business Review is the one place I’ve found that you’ve got really well-written content about value chains, supply chains. When we talk about impact in research we’re just talking about value chains.

There’s so much writing by incredible professors at Harvard University and other places, so that’s a great place to start. You go on to Harvard Business Review, pay a couple of 100 bucks a year and start diving through this incredible resource.

Then you’ve got the Research Ops community, which I no longer run. I stepped away in 2019, I’d just moved country and was trying to set stuff up at Atlassian, and I knew I couldn’t do it justice. It’s being run by Bridgette Metzler now, who’s kept it going and done a great job of growing it.

Then if you are a full time research ops person, you can get in touch with me to join a small invite-only club called the Cha Cha Club. We have just over 120 people now. Cha Cha stands for cheerleaders and chums because when you are working in this space most people don’t have the luxury of an eight person team, they're a team of one and they’re quite new in the space and they’re just trying to figure it all out. We’re there to say ‘Dude I know the feeling, I’ve been there before.’ We share and we champion. But this is only for full time ops people, not kind of dabbling or 40%. It helps keep the conversation really focused.

And lastly there’s Medium. A lot of people are starting to write about research ops and share what they are learning. And, of course, I’m writing a book, which will come out with Rosenfeld Media in mid to late 2022.

So once you’ve read all this information and you know what you want to do, how do you convince the business to invest in research ops?

I was lucky enough to work at GDS in a golden era, in this incredible moment in time and there were some really amazing principles that have held true throughout my career and one of them is ‘show don’t tell’. I don’t tend to make this beautiful pitch and argue for the money, I’m not great at that sort of thing. It’s not the way I’ve ever worked.

For me, even as a team of one, you need to carve out some time for yourself in your schedule to be able to experiment with something or get something off the ground.

You can then show the value of doing something, even at a low level. You’re looking for something that is low hanging fruit, something you’ve got some kind of in with. So if I think informed consent - maybe you’ve noticed we’ve got no informed consent. I’m going to have to find some decent time to work on this and make it happen. I’m going to need to make friends with legal and ask them to make time for me. A huge amount of the work of operations is having enough specialist knowledge that you can work efficiently and respectfully with other deep specialists like law or knowledge management or librarianship or the people team and be able to bring them all together to be able to work on a thing.

So with the informed consent example you need to get it on the legal roadmap and start doing research with your researchers to figure out what kinds of consent forms they need. And maybe they come back and you realise we need five different consent forms! One that looks after people with accessibility needs, one that’s for people who already signed a privacy policy statement and one for CEO’s who are going to be annoyed at signing a consent form again.

This is one of those scenarios where what you need is the airline meal of consent forms, what’s the most common one that you need? You can at least get that one out there and then start to look at the others later.

And if you can deliver on that and can show some forward movement or impact of doing that kind of work then you can start to argue more easily. Look what I managed to do with 20% of my time. Show the metrics around your successes and then you can say look if we had someone full time working on this here are the most immediate projects we could benefit from and why.

It’s difficult to argue for things off the back of nothing.

What’s the one thing you think everyone needs to know about research ops?

That it’s wildly underestimated.

So you can break things up into the eight pillars or whatever number of things you want and you can make a nice framework as I and many others have done, and it all looks great. But it’s like a map. It’s wonderful for understanding the lay of the land before you set off on your journey and even while you're on your journey, but it’s not the experience of being on the street. So you might look at the map and drive to the place and get out the car and you’re like oh my god it smells! I didn’t see there’s a sewer right here - it wasn’t on the map. Or you get crazy hay fever, because the map doesn’t show that it’s spring and there’s tonnes of pollen around.

The map doesn’t deliver that experience and that’s the same with any framework.

So where researchers are wildly underestimating is you can have all these tiny little elements and tidy columns but really it’s a venn diagram. If we go back to that informed consent example you’re suddenly looking at how people recruit, who they’re recruiting, where in the recruitment process it sits, where in the whole research process it sits and what tools you use. Maybe you pay some money for that tool, so how much does that cost? Now you’re looking at procurement, you’re working with the legal team and you’re looking at the experience of the participants and how many people have signed the consent form. So that brings in knowledge management. Are you managing personal data? Because that brings in the legal team again. And are you using a tool for this? That’s money which brings in procurement again and has to go through the IT department and how many people have access means people need support. And so on and so forth.

So anytime you pick up on one element, you’re actually working across all the elements, you can never separate them out. If people would just realise that, then they would stop underestimating the work of research ops; so much of the work is just invisible.

So then when someone says they have a great idea - lets go to Rome to do research on how letters arrive and let’s do that once a quarter, easy.

They would understand the work we’ve now got to do.

What are the language barriers in that country? Do I need to educate you in basic Roman etiquette? I’ve got to work with the travel agency to make sure your visa is sorted out and flight bookings done correctly. I need to be aware of Italian privacy law and maybe change the consent forms and actually maybe the consent forms need to be in Italian too, and so on and so forth.

And so I work through every single one of the elements of operations to understand that one little aspiration which is a shit tonne of work for research ops so the research goes well and is executed excellently. I mean you could just fly off to Rome and arrive with english consent forms or no consent forms but that’s not what we’re working for.

So you said your ideas have changed over the past three years. Where do you see the future of research ops going?

I think there is a future where instead of it being about research ops, design ops, dev ops, all those ops become one, that there’s just one flow.

So you’re doing some research and there’s some design happening and there’s some development work happening all using different ops teams, but this should all really be one type of operations.

How is something released to the audience? While development might build it, it’s also part of the experience and that’s where research comes back in, and design too. So really we are working on one customer experience and I would like to see at some point that we just have operations for delivering an end to end product to customers instead of everybody being siloed in their little spots.

What’s been your biggest challenge?

The biggest challenge, and actually reward, has been learning how to manage a team.

As a contractor you manage teams of people that you bring together, but it’s not the same as managing a team in a corporate environment or any kind of official organisation.

The learnings for me have been, sometimes you don’t know what you know until you get there. For me it has been the intricacies of managing a team of people and being in care of their professional welfare - and just their welfare sometimes. Looking at their career paths, your career path, the path of your team. How you manage up while trying to manage down and get all the stuff done on the ground as well. That’s been a really big learning curve for me.

It’s very rewarding to know how to manage a team of people and keep them all enthused and on track. You get to know everyone’s personalities - this person tends to have their head in the clouds and you have to get them on the ground delivering and that person tends to be delivering without thinking too much about the vision. And so that’s been intriguing and rewarding and challenging.

What are you most proud of?

Actually I would say the team I’ve managed to put together.

The challenge with research ops is that although there’s so much you can learn from traditional ops there aren’t many people trained in it, and it’s a growing field. There are a lot more people now that are sort of gathering experience than there were before, but I’m always fishing for some unique character who I’ve never hired before.

Like at the moment I’m hiring a librarian. I’m not a librarian, a content strategist does not a librarian make, although we have a lot of similar skills, these people come from four years of studying information sciences and having practice with hundreds of data points!

And so it’s been about going out and finding particular characters who have knowledge that I don’t, and bringing in sometimes very junior people and working with them to build up their confidence to help them learn what they need to know to do the job. It’s been really rewarding to do that and to end up with a very very switched on team who are known as a very high performing team in the organisation. That’s been great.

What advice do you wish someone had given you when you were starting out?

I wish I had understood the need to really get down to a clearer long term vision for myself of what we’re actually trying to do and deliver, so that my team then had their clear vision not for today but in five years time.

Some of that’s the maturing of an organisation, when it’s young it’s difficult to have that kind of five year vision. But we’re at a point now where it’s easier to have that five year vision.

If I had to do this all over again I would demand an even clearer idea of what the strategy is from an operational point of view.

I’ll give you an analogy here. So what we’ve done is create a bar and then all the beer taps. And you have your IPA and draught lager or whatever, we’ve been putting everything through those beer taps. And now I’m starting to look at this and think it’s pretty standardised we could put anything in these beer taps. Are we only going to put usability testing through them? What do we actually want to start utilising these pathways for?

So it’s fine not to have a strategy for your first couple of years, while you’re building and standardising, but as soon as you start to get into specialising then you need to start to think what is our strategy now? What are we doing?

Would a governing body to help with training, standards and ethics help or hinder UX professionals?

I think there’s a place in this space for a professional association and I think research ops are a good crowd of people to deliver on this where you have training and templates and guidance.

People have been reaching out and asking ‘where do I go for training on legalities?’ Not a couple of blog posts but a week on GDPR. So yes I think there’s space for it. I don’t think we’re at the place where we need to be offering certification and policing like they might in the information security space, but I think there’s a space for some kind of shaping up around that.

I’m actually hoping that the Cha Cha club at some point will start charging for membership. I’d like to use that membership fee to pay professionals to do these kinds of things. We’re not there yet but as a future plan.

So we can have conferences that are funded, people would still pay to join, but the organisation of it is funded and we’ve got the finances to pay for that type of professional level so we can have a beautiful website full of templates and things and we can run courses. But because nothing is done on a volunteer basis, and I’m a big fan of volunteers, we can do it to a different level and get it done within a certain amount of time.

So that’s the future plan for the Cha Cha club if I ever get there. It’ll take years and I think by the time I get there is when the industry will be starting to look at things like that.

Thank you to Kate for taking the time to speak with me. Kate’s new book Research at Scale: The Research Operations Handbook is out later next year published by Rosenfeld Media. I can’t wait for it to come out.

If you know someone you think I should speak to for this series, do let me know.

---

If you enjoyed this interview and would like to read more like this, check out The UX Life Chose Me newsletter. Subscribe to get interviews like this directly to your inbox once a month.