Playbook
GTM Playbooks
How GitGuardian Leverages Data and GitHub to Fuel Its Growth
Read more
Devin Rader is an expert at hiring Developer Relations professionals. During his 11-year tenure at Twilio, where he led the global Developer Evangelist team, he estimates he’s hired 40 to 50 “DevAngels”. That team, known for sporting red track jackets, is one of the most technically proficient and passionate in the industry about serving developer communities. The recruiting process they perfected over the years is really worth studying if, like Twilio, you want to set a very high bar for your DevRel talent.
Twilio’s Developer Evangelists ultimately work to raise awareness of Twilio and its products within the greater developer community. They lead top-of-funnel activities such as:
“Our Developer Evangelists not only shared information about the existence of Twilio but also helped Developers understand that sending SMS and building SMS- or Voice-related applications is achievable.”
The Twilio team had a very defined set of qualities they look for in a Developer Evangelist. Hiring was very deliberate. Evangelists do much more than organize meetups and buy pizza for developer events– they have a lot of technical knowledge and confidence to have sophisticated conversations educating potential users.
Ultimately, Twilio hired Developer Evangelists based on four different criteria.
1. Technical confidence & adaptability
Technical confidence, or a developer's confidence in their own ability as a developer, sat at the foundation of the Evangelist hiring rubric because, unlike a skill like writing or public speaking, it is a trait that the team didn’t think they couldn’t teach to someone. Demonstrating this type of confidence meant showing that even if a candidate didn’t feel like they knew the precise answer to a specific technical question, they felt confident that they could find the answer.
Adaptability meant seeing that a candidate understood that they needed to meet their audience (the interviewer) at their own level of knowledge. It wasn’t uncommon for the interviewers' primary programming language to be different from the candidates and we wanted to see how the candidate adapted their answers based on this.
Both of these assessments were important because the team felt they reflected in spirit the environments that they knew from experience Twilio Evangelists often operated in. Twilio Evangelists frequently found themselves asked to help a developer who was using a programming language or framework that the Evangelist wasn’t familiar with. Evangelists often found themselves in situations where they needed to be able to help developers with widely varying degrees of development experience.
Additionally, focusing on these soft skills rather than assuming the team needed candidates with the most technical experience created a larger and more diverse pool of potential evangelist candidates. The team hired Evangelists who were seasoned developers and Evangelists who came right out of school. Evangelists from traditional computer science backgrounds and Evangelists who had transitioned into tech from another career. Evangelists from startups and Evangelists from enterprises.
Twilio looked for candidates who demonstrated empathy and humility. Evangelists were asked to help all kinds of developers who came from widely diverse backgrounds and experience levels. To be effective at serving those developers, they needed to be able to put themselves in their shoes. Developers make choices for many different reasons and Twilio didn’t have a monopoly on knowledge. For every person who wanted to use code to build, Twilio Evangelists hoped to help make them feel like a hero.
“We wanted to hire people who would be a smart person in the room, but didn’t think of themselves as the smartest person in the room.”
Despite not being Devin’s favorite term because of its potential to be misinterpreted by some as an expectation to “work harder”, hustle was the best term the team came up with as a shorthand to describe a person's ability to identify and take advantage of opportunities. “We always felt there were infinite opportunities available to us when operating in the developer community and we wanted to find individuals who could recognize which were most valuable to Twilio,” said Devin.
“Resilience is about being able to roll with the punches,” said Devin. “When you’re a Developer Evangelist, you often end up in environments you do not control– it can be like presenting on stage with someone else’s AV equipment or having your flight to a conference get delayed.”
When it comes to hiring for leadership roles, Devin didn’t use the same scorecard. “We assessed leaders differently than ICs– we were much more interested in hearing about their experience coaching individual contributors,” he said.
When the Twilio team hired Developer Evangelists, they used a rubric that included not only positives, but also negative and neutral signals. The neutral ones were particularly interesting to Devin because counterintuitively one of the signals that emerged as most “neutral” was the prior experience in Developer Relations.
“Former experience in Developer Relations and notoriety within the developer community ended up not being a reliable predictor of success or failure as an evangelist..”
It turned out that prior experience in a Developer Relations role didn’t matter, as most of the hires did not have this background. Social influence was also a neutral signal– most people hired were not famous or well-known in the developer community.
Before starting the assessment process, Devin insured that candidates had a good understanding of the role. Many developers were intrigued by the idea of being an evangelist but didn't have a lot of exposure to the role, so they had lots of questions about what the expectations were. Moreover, the specifics of a Developer Evangelist/Advocacy role can vary widely from company to company so Devin often started by explaining what evangelism at Twilio meant.
Then, the recruiting and assessment processes followed the following steps:
This initial step served 2 purposes:
First, it baselined someone's knowledge.
The team built a bank of questions for various common programming languages, covering what they felt was the minimal knowledge of that programming language. They asked a candidate what programming language they were most comfortable with and used that set of questions. They also always used the same questions to help compare candidates.
Second, it assessed the candidate's own self-confidence.
While they were definitely looking for correct-ish answers to the screen questions, the confidence in which candidates would approach answering them was a big factor. Sometimes that meant admitting that their knowledge of a topic wasn't super deep. If they knew the main points and paired that with being willing to admit when they'd hit the limit of their knowledge that might be good enough.
This was designed to assess the four main elements of the rubric described above.
To validate humility, for instance, Devin and his team asked candidates to talk about their own experiences. In situations that may have required the candidate to rely on humility. For example, they asked candidates to: “Tell me about a time you had to reject a pull request. What happened? “ They frequently paired this specific question with “Have you had a pull request rejected? What happened". These types of questions let them evaluate how the person demonstrated humility when on both sides of rejection - the rejector and rejected.
To assess hustle, the team asked candidates to talk about their own experiences identifying and capitalizing on opportunities in their own careers. While these questions were heavily based on the candidate's own CV, they frequently centered around a candidate's participation in a developer community or times they had opportunities to grow their own knowledge or experience.
For resilience, Devin and his team asked candidates to tell them about their own stressful situations and how they responded to them.
If a candidate passed the initial rounds, the Twilio team asked them to give a short presentation.
The assignment consisted of presenting a short 15-20 minute talk on a technical topic the candidate was passionate about.
“The only constraints we gave them were that the presentation needed to be a maximum of 20 minutes, include live coding, and couldn’t be about Twilio.”
The presentation was useful to assess how detail-oriented a candidate was. Did they read the criteria, follow the guidelines and come in prepared and rehearsed? From there, they could assess where a candidate was with public speaking and storytelling skills. It wasn’t a requirement that a candidate has expert-level skills but the team wanted to get a sense of how they could best set a candidate up for success. Would they need a lot of public speaking coaching up front? Would they need to grow in their storytelling ability? The presentation gave a way to think about what tools they had that could help that candidate grow those skills.
The best candidates followed the instructions, staying within the time, remembering to include live coding, and telling stories with a beginning, middle, and end.
To make the final call on a candidate, the team would gather and discuss whether they wanted to move forward. “In the early days, we would get together and essentially use a ‘thumbs up, thumbs down’ technique,” said Devin. “As we scaled and got more sophisticated, we developed a more proper rubric and rating system to automate the process more.”
In the early days of hiring Developer Evangelists, the team found new hires mostly by tapping into their own personal networks. Asking those they knew and mining personal channels - Twitter, StackOverflow, GitHub, Meetups, etc. - were most fruitful at the start.
As the team grew, they worked with recruiting partners that sometimes came from engineering recruiting backgrounds. The team had to spend a fair amount of time with these partners to help them understand how to look for those unique signals that identify a developer who may make a great evangelist. After all, it can be difficult to see whether developers have organized meetups or participated in community activities just by looking at LinkedIn. Because of this, Devin eventually asked to have his own access to LinkedIn Recruiter so that he could participate directly in sourcing himself.
As the Twilio brand became better known, it was easier to build a pipeline of candidates for a role. Simply posting a role publicly could generate a fair number of inbound leads. However, finding the best candidates with that unique skill set and traits remained extremely challenging.
When it came time to make the hire, Devin wanted candidates to understand the role well.
Three areas in particular often were concerns for candidates.
First, the team had to help candidates understand how Twilio thought about success. Specifically that the Twilio team was not measured on sales or revenue metrics, but on building brand awareness and credibility with developers by being visible and helpful in their communities. The goal was to inspire and equip developers to build, hopefully with Twilio, but they weren’t responsible for selling the tool directly.
Second, the team frequently addressed concerns about developer skill atrophy.
Candidates often came from engineering-oriented roles where they were writing code on a regular basis, which allowed them to keep their marketable developer skills fresh. Evangelists at Twilio were not directly contributing code to the product but were definitely spending a significant amount of time writing code. Whether learning a new technology to craft a presentation or write content, creating or contributing to open source projects or writing internal team tools, the team offered many opportunities to continue writing code.
Third, the team showed candidates how much Twilio values Developers and DevRel.
Not only do these roles exist at Twilio, but Twilio itself has three engineer founders who understand the importance of this type of role. They highlighted that these roles were important to Twilio from the very beginning and that they had been and always will be a priority.
There are a lot of early-stage companies in this situation– rather than having a large number of Evangelists working together to build awareness in the community, they have one Developer Evangelist reporting to a small marketing team, the CTO, or even the founder.
Devin’s advice? If you still need to validate that a developer-first GTM motion is viable for your product, hire a Developer Evangelist and have them build a short (5 minutes) product demo that tells the product's story. What can your product do for developers, how does it work? Inspire them to want to give it a try and equip them with the basic information they need to feel like they have the skills to do it.
Tell your Evangelist to go to as many different local developer meetups as possible as often as they can for six months. “The goal of this is to immerse themselves in the community,” said Devin. “Offer to be the meetup sponsor for that night. Offer to buy the food for the meetup in exchange for being able to give a 5-minute demonstration of the product.”
This process allows your Developer Evangelist to iterate quickly on the core story you tell Developers about your product and get immediate feedback on what resonates.
“Ultimately, there is no substitute for putting your product in front of developers and getting their reaction. Do they understand the story you’re telling them? Do you see the wheels turning when you show them your demo?”
Armed with these insights, a Developer Evangelist can then help the entire company craft a better product for Developers and especially a better story to get them interested.
Thanks so much, Devin, for sharing all these great insights, and congratulations on building such a world-class DevRel team at Twilio. By the way, some of them are in transition as of February 2023 and Twilio’s second round of layoffs. If you can work with one of them, don’t hesitate. Despite wearing red track jackets, they are gold!
.png)
