The three pillars of engineering team productivity with Dan Lines and Ori Keren of Linear B

Transcript

Ledge: Hey, Ori, Dan. Really cool to have you on. Thanks for joining me today.

Ori: Thanks for having us.

Ledge: Would you guys mind giving just a little background story of yourself and your work, so the audience can get to know you a little?

Ori: Yeah, we’ll try to be brief. This is Ori. I’ll start with telling you a little bit about my background and then we’ll hand it off to Dan.

In my history, I have the history of leading big R&D groups, building them from scratch and preparing them to join big companies. I did it twice with Interwise. It was acquired by AT&T in 2007. And I did it again, alongside with Dan, in CloudLock which was a very successful acquisition by Cisco – that was a cyber-security company in 2016.

I’m a developer. I love to develop. I lead teams. I lead big groups. And having said that, I think I’ll hand it over to Dan to expand a little.

Dan: I come from a software background, and Ori and I formally worked together at a company called CloudLock, where I was a second employee there.

I had many roles in the software organization. Eventually I was our VP of engineering – and Ori by the way was the VP of engineering before me, he handed it off to me. I grew the team from about 30 to 75 people globally. We had people in the US, Ukraine, Israel, and UK.

My passion area is around people and productivity.

Ledge: I know you guys jumped into the new, or newish, company, Linear B, you guys are working together on.

Talk a little bit about that. I know it’s focused on engineering team productivity – a huge topic, everybody wants to figure out how to do that better.

So, how are you addressing that problem?

Ori: We can dive into like what’s even engineering productivity, but maybe just a little bit of a story of how we started Linear B.

After we worked a while in Cisco, at least for me and I know that for Dan too, we had a nice time to go and reflect on what just happened. We went through like five years of, I call it like you sprint but you run a marathon, and then we reflected back and said, “Okay, what happened to us in situations where we wanted to understand if the team is performing well? We want to communicate about the productivity that we were getting out of the team with our peers, VP of sales, VP of marketing, with our CEO, with our board even, and of course with our people?”

And we discovered that in a lot of those scenarios, we did a lot of ad-hoc solutions or one-off solutions to pull data and to understand where we stood.

Then we engaged in a conversation on how it can be done better, or what kind of tool or what kind of metrics and what kind of measurements we would like to have to ourselves if we do it again. And that was kind of the seed to start Linear B.

I’m sure Dan has things to add to that story.

Dan: Yeah. And I think those stories are good. Let me start with this.

I think being a software leader is one of the most challenging positions. For me, when I was growing the team like 30 to 40 to 75 people, one of the hardest things I’ve done in my lifetime from a work perspective.

At one point I was reporting in to Ori, at another point we were peers. And when we’re trying to make a decision about, what should we do with the team? How should we organize our team? Should we invest into quality? What do we do to improve our productivity?

We’re putting together literally spreadsheets together for hours, collecting data about different aspects of the organization. Looking back on it, that’s all we had. And it’s a little bit insane that there’s not a solution out there that does that for you.

After the CloudLock acquisition, Ori and I got together and just said, “We need to bring this to the engineering community. The community needs this. Let’s build it and see who would be interested.” And it turns out there’s a lot of people interested.

Ledge: That doesn’t surprise me. How do you begin to crack that nut? Everybody wants to be able, from the engineering seat, to report to all their functional stakeholders and leadership. I mean it makes a ton of sense. And it’s very difficult to translate the languages between those two worlds.

Marketing maybe doesn’t face such a thing because just the language of the data is the same, but I think the language of engineering needs a total translation to the business layer.

So, walk through some of those steps.

Ori: It’s a really good question because, yes, I think in some aspects it’s much easier to measure organizations, like the sales organization the marketing and all of that, because it may be closer sort of business.

But on the other hand, if you think about it, it’s kind of ironic that the R&D organization is the one that has the ability to measure better than everybody because it’s engineers. They’re obsessed about data. They know how to approach data.

There’s no silver line or one answer on how to do that but what we did in order to start cracking this reel on how to do it right, we just started talking to our network. We started talking to people at different levels. So developers that worked with us and for us that we appreciated, a lot of partners, a lot of people that now will do pilot with us, leaders of engineering organizations, CEOs who are also a key figure here, and other executives.

We started collecting something that we can dive into soon on, what do we think is important to measure, and exactly how to bridge that gap between, okay, this is engineering language but this is like the business aspect or the business language.

I think we came up with something which is a good start on how to answer that question.

Ledge: Yeah, please don’t keep it a secret. What are those measures?

In my work I’ve seen throughput, I’ve seen velocity, a lot of that agile language. But none of that means anything to business users because size of ticket, and how do you break things down, and themes, epic stories – I mean it just gets ridiculously confusing. It’s like speaking a different language.

So, how have you connected the dots there?

Dan: Let me try to dive in here a little bit. There’s not one metric that’s going to tell you what your productivity is. That’s probably obvious, but I should state that in the beginning.

We came up with a three-part methodology to start with. The three categories of our methodology are as follows. The first one is around quality and value. Why? Everything you do has to have high quality, in our opinion. If you’re just producing a bunch of garbage, for lack of a better word, that’s not productivity. So quality and customer value is number one.

The second category for us is activity and throughput. Ledge, I think you just said throughput – a activity and throughput. At the same time, if you were producing nothing, zero things, of course that’s not going to lead to productivity. So you do have to measure your output.

You do have to look at, well, how many stories are we creating? How many pull requests are going up? What’s the impact of the code going into the code base?

As well as, on the throughput side, for example, how long does it take us to release to production? How long does it take us to review a pull request? Those things matter. That’s the second category.

The third category – because something that I think is probably different from maybe 20 years ago, software product creation is really a team sport now. So, our third category is around social and teamwork aspects.

We feel, if you put together quality, activity and throughput, social and teamwork, now we have a framework that we can observe from a data perspective that captures the three pillars.

Of course we can get into more and more detail, dive into each one of those, but we feel that’s a good start.

Like Ori was saying, we know a lot of VPs of engineering, we know a lot of developers which is great. We ran that three-part methodology by them and we’re getting, “Yeah, that make sense. Let’s start there and move forward.”

Ledge: I do want to dive into the subcomponents of each one, because I think you’re hitting on a lot of the factors that I hear come up in these leadership discussions, VPS, CTOs.

One thing that strikes me that I’m interested to know, where does it live to know that you’re building the right things? You can build high throughput, high quality incorrect stuff that’s not aligned with product, marketing and sales. Does that live in the third function there?

Dan: So you’re asking what if I have high throughput, high quality, but I’m not delivering the thing that my marketing…

Ledge: … and marketing sales and the product actually wants. You can have a major breakdown there.

Ori: That’s a really good question. We went back and forth, but the way we’re looking on those three pillars that Dan just spoke about, somebody can think that, “Okay, so activity and throughput is just for maybe the code elements that I’m producing. And then maybe quality just stand for quality of the code that I produce. Then the teamwork also.”

But the way we’re looking at it, these are not phases in the life cycle. You can find quality, for example, in the exact questions that you just asked, right? If we can measure the match between what we deliver through the happiness of our customers, that’s another quality measurement.

So when we say quality it’s not just about the code side of the house. And the way we’re looking at it, we started with these three pillars and we’re going to continue to improve. We want to reach those areas too where you can actually measure the things that you’re talking about.

Ledge: It strikes me like a complex system approach, which is really what you find in every organization or discipline is, you can try to segment off your own little piece of data but ultimately you’re measuring the effectiveness and the efficiency of the entire org, because everything is interconnected. So you’ve just taken a different view.

I think of it as, you’re choosing to get on one of the horses on the carousel but the reality is it’s the carousel that matters.

So, yeah, please walk through the areas there and just talk about the actual data sources that you need to measure to even capture these things. I imagine there’s quite an ETL challenge in there to make apples to apples out of all that source data. What are those things that make the pillars?

Ori: If we want to talk about the data sources, we’re looking at it as, as much as data sources you will add you’ll get a better result or better match to what you’re trying to achieve.

I’ll try to explain. We mapped all the sensors, and this is how we call them, that participate in the process of building software, planning the software, shipping the software, and monitoring it on production – we put them on a list and went and talked to our partners or in the pilot … and said, “Okay, if we need to prioritize – and of course it’s not just one, but if we need to start thinking on what do we need to connect to, which one will be your first one?”

The first sensors that we were connecting are the sensors that manage the code. And today, if you think about it, if you look at the Git providers, they’re not just storing the code. You can find a lot of things about the process of how the code was created there. What was the interaction inside the team in order to achieve this snapshot of the code?

So there’s a lot of information there, a lot of interactions between the team. If you look on pull requests – which is a mechanism for people to say, okay, I’m ready to get my code merged – we’ve learned from our experience that there’s a lot of interesting interactions there that you can find out insights about how close your team works.

We decided to start with this family of sensors, and we’re now adding more and more. We’re looking on things like Slack. We’re hearing from our customers that they want us to connect with those things too. We’re looking into future stuff to look into more into the CICD side of the house and to connect to Jenkins and this family of products. To understand how the code is being shipped and built.

So, to be honest, we’re just going to add sensors and the level of insight we grow as we add them. And we can find each one of those three pillars that Dan spoke about in each one of them.

Ledge: Right. I’m thinking about comments and dialogue around merge requests. Or, why did you do it this way? Why did you do it that way? Obviously, a lot of that is going to live in the Slack or collaboration system.

Then you’ve got your Trellos or JIRAs or anything that. You’re really capturing everywhere that there’s interaction.

Any designs on doing some kind of sentiment analysis or something like that to measure teamwork through those qualitative interactions?

Ori: Yeah, actually… Go ahead, Dan. Take it.

Dan: All right, yeah. Going back to the last question ,just to summarize it for a second, the code base – GitHub, GitLab, Bitbucket – that’s the home base to me. And there’s a lot of social things happening there with pull requests and stuff.

Then from our customers we’ve been asked, “Hey, will you connect a Slack?” That’s where you start getting into the sentiment analysis stuff, but also on my side I’ve been connecting with a lot of agile consulting experts.

They’re saying, “Well, how is the team interacting?” It is the engineering team getting information from support, for example. They should know what’s going on with the customer. Is there a communication link there?

And then on the project management side, Ledge, I think you mentioned maybe JIRA, Trello – that’s where the quality of your stories are created, how long is it taking us to go from story creation to completion? That type of stuff.

And then, just one thing before we go to the sentiment, in case Ori wants to jump in here. We’re building our system open. So, API-first approach is one of the things that we learned from our previous company that we wish we would have done even better.

We’re doing it from the beginning now because, for example, we’ll run into a customer that says, “Hey, I invested in a security tool. It cost me a lot of money. Hey Dan, are you able to plug that security tool into Linear B? I want to know, for each team in each developer, are we struggling on security with this team? Should I get us some training? Or not. You might be doing great.

And so we allow you to plug that tool in because there’s so many different sensors to connect to.

On the sentiment analysis side I’ll pass it back to Ori, but that’s definitely something that we’re looking into, sentiment and engagement.

Ori: Just to quickly answer that, we’re investing into that. You can find a lot of interesting things in comments about pull requests.

I like to tell stories about people that worked with me. I saw some very talented people that wanted to give comments that help, but they didn’t have the soft skills on how to deliver those comments. Sometimes they were harsh – not necessarily in the right language. And these people were a miss for the organization.

We’re investing into giving feedback to those developers on how you can better pass your feedback, and maybe you can improve the team more.

So definitely investing into sentiment analysis. Each developer in each team, we can say the level of interaction, and how easy it is to accept criticism or even complements from those teams.

And the other side of the house, we had… Not exactly the other side of the house but even the fact that you’re identifying a developer that maybe is not the best individual contributor but his reviews are a pleasure because he reads all the code.

I’ve been a developer and I’m still a developer. We like to write code, not necessarily at the same passion that we like to read code. So if you find a person that reads the code and gives comments that built and contribute, sometimes this team there increases everybody’s level two times.

It’s very important to identify these people and empower them and give them strong feedback and positive…

Ledge: Yeah, absolutely. I have to imagine that pillar number three is really just maybe even outsized. I don’t even know if those things are equally weighted.

In all the conversations I have, soft skills, ability to do excellent teamwork. Google has done a bunch of research about just high performing teams, psychological safety is the overwhelming factor. If you can start to actually materially measure that, I believe that’s probably just a huge contribution to the field,

How do you consume research and so you’d understand what’s been discovered by the big dogs and how you might even measure it?

Ori: We started, first of all, a very basic modest approach to read all those research papers. I’m familiar with the Google one. It’s very, very interesting.

To be honest, I think the value in this industry is to start providing some insights. It’s still at a very more simple level. You can even learn things…

I have an experience with deploying large scale data science and machine learning projects, but sometimes the best insights are with simple things. So, we’re starting with simple things – even just to measure the relation between, how many PRs are left open and how much time it takes them to close and how many are getting attention?

You can start with the very simple things to understand about aspects like psychological safety that you just mentioned. We plan to invest on that a lot more later on because we think that’s where the future is.

I totally agree that this area of teamwork, and social and soft skills is a very interesting aspect of the teamwork. We plan to invest a lot more.

Ledge: Very interesting. This is very, very cool. Dan, why don’t you give us a finish up shot here? We’re running low on time so would love to get you the last word here.

Dan: I think I’ll just add one thing. At Linear B we’re starting as a global team. Remote work is becoming more and more important. I’m in the US, but I think there’s amazing developers worldwide

And every team is different. Every team is culturally different. The way that each team works is different. And so one thing that’s really important for us on the productivity side is to make sure that our software has levers. That you can kind of push and pull if quality is a little more important to you or if the social side is a little more important to you.

We recognize that and we’re building it in from the beginning. I guess to close it out, we’re all about continuous improvement. Using data to make teams better. And we’re just excited to…

Ledge: So, it’s linear.io. Who’s an interesting user to you and who do you want to take a look at out there? Please give a shout out for the company. I want to make sure that the promotion is useful to you.

Dan: You can check us out at linearb.io. We have a trial that you can sign up for. We’re useful for CEOs, VPs of engineering, software leaders and developers.

Ledge: We know a few of those. Dan, Ori, thanks for joining us today. Super interesting stuff. Good luck with the continued development.

Dan: Thanks a lot.

Ori: Thank you very much for having us.