Season 3, Ep. 13 – ChatGPT for hiring, with Grey Garner, VP of Product at Gun.io

Transcript

(THE FRONTIER THEME PLAYS)

Faith (00:06):

Hi, Grey <laugh>. What’s for lunch? Popcorn, nice. I’m assuming somebody made that who’s better than me at making popcorn, because, as you know, 100% of the time, I burn popcorn.

Grey (00:22):

Popcorn instructions are completely unreliable.

Faith (00:27):

I don’t understand it, because I’ve never read a popcorn instruction that is like, at all different from any other popcorn instruction. Like, it’s always the same. It’s like, set your microwave for two minutes, check it after 90 seconds, and as soon as the pops get more than like, a few seconds apart, it’s done. Take it out. Right? Like, that’s a universal popcorn rule. I have kind of fond memories of being in the office and being like, “I’m gonna make popcorn for everyone. This is gonna be awesome. Team building!” and burning it, and everyone having to leave, (Grey: Yeah.) so…

Grey (00:59):

Yeah. Burnt popcorn smell, it goes from amazing to I have to get outta here.

Faith (01:06):

Like, there’s a chemical spill. (Grey: Yeah <laugh>.) Yeah <laugh>.

Grey (01:11):

The problem with the microwave is that like, I don’t wanna monitor the microwave. If I wanted to monitor the thing, then I do like, the old school like, Jiffy Pop thing. At least that’s kind of fun, (Faith: Right.) Like, I wanna set it on a thing and have it work. Like, I don’t wanna monitor it for like, pop frequency <laugh>, like…

Faith (01:31):

Yeah. And I feel like I’m a child of the era when we still kind of thought that microwaves were gonna give you brain cancer. (Grey: Mm-Hmm <affirmative>.) So it’s like, don’t stand in front of the microwave. (Grey: Mm-Hmm <affirmative>.) So, yeah. It doesn’t make sense. In college, we were like, just queens of the dollar bin, and nobody ever wanted the big jugs of popcorn kernels, so we’d buy those and just put ’em in a big pot, and that would be like, our dinner. We’d like, put hot sauce on it and be like, “Yeah, it’s totally real food,” (Grey: Yeah.) and that was definitely better. Maybe I just need to get back into that life.

Grey (02:02):

Well, now that you’re a TikToker, (Faith: <Laugh>.) just, you can, if you get to like, PopTok, then you’re, (Faith: PopTok! <Laugh>.) <laugh> go back to…

Faith (02:12):

Listen, before PopTok, it’s gotta be BawkTok. I’ve been dying to make BawkTok a thing, which is just chicken TikTok. (Grey: Mm-Hmm <affirmative>.) You know? (Grey: Yeah.) There’s so much good chicken content out there.

Grey (02:23):

Yeah. I mean, it’ll find you <laugh>.

Faith (02:27):

<Laugh>. Anyway, Grey, thanks for joining us again on the Frontier Podcast. Really appreciate it. This is a “Faith seeking advice from Grey” disguised as a podcast episode <laugh>, so full disclosure there. We’re gonna talk, just like everybody else is doing these days, we’re gonna talk about ChatGPT, LLMs in general, why people are running around screaming about them. And you are, whether you like it or not, our resident expert.

Grey (03:00):

That’s cool. Although, (Faith: <Laugh>.) I am not an expert, and, which is really interesting, like, I’ve been following it fairly closely, but there are people that are just taking this thing and running with it at light speed. So it’s, yeah. I’m really hanging on for dear life. (Faith: <Laugh>.) That’s my role, I think, is to just like, yell back at whoever’s like, listening like, “Here’s what’s happening now.”

Faith (03:30):

I’m imagining like, the cover of Harry Potter and the Chamber of Secrets, when the phoenix is carrying everybody who’s just like, holding onto each other, and whoever’s at the end, you know, they’re getting like, that max thrash.

Grey (03:42):

Mm-Hmm. Mm-hmm <affirmative>. Thrill-seekers.

Faith (03:44):

Yes. So maybe that’s me <laugh> at the very end of (Grey: Yeah.) this phoenix train. Well, thank you for that context, but still, I think on the team, you are the person who I trust the most to answer these questions for me. So, (Grey: Fair enough.) the point of this episode is, there is a really cool use case, from what I understand of ChatGPT and other LLMs that are sure to come, when it comes to hiring. So both, you know, hiring yourself, hiring for a role that’s open, and also being a job seeker. So we’re gonna get into those use cases and get kind of your take on some cool ways that people can tap into that, now, to make their lives easier. But to start, I would like for you to explain ChatGPT to me like I am five-years-old. No offense taken. Be as like, really dumbed down as you can <laugh>.

Grey (04:40):

Yeah. ChatGPT, at its core, in terms of functionality, is a new keyboard. You know, it’s a way to interact with a database or a technology in a novel, new way. And it’s just with your voice, versus inputs into a keyboard. And so the, you know, the cool part about it is that it’s got enough smarts baked in to understand nuance, and context, and syntax, and all of the things that go into processing language. So it’s a really efficient and smart keyboard that kind of types for you. You know, just like any other piece of technology, it relies on what it knows about, which is, you know, a database, but it can tap that database in a variety of different ways to come up with what feels like an interaction more than just, say, a search result. So it’s like 3D. It’s a 3D search tool at its core, but that’s certainly not all that it is, and we’ll probably get into that.

Faith (05:48):

Okay. 3D search tool. My mind can go in a million different directions with that. (Grey: Yeah.) So let’s drill down a little bit deeper. If I were to open a ChatGPT window and input a query, (Grey: Mm-hmm <affirmative>.) would I expect to get back from that query, something like a Google search page result where there’s various pages that I can go to and read myself? Or what would the response be from this, essentially, a chat bot?

Grey (06:19):

Yeah, that’s the cool part about ChatGPT is that it doesn’t just provide a set of links where you can go do the work. It answers the question. So it’s short circuits, and the, so the interaction is about what you’re looking for, and then, I think, this sort of starts tapping into how we need to think about chat interfaces differently than traditional search interfaces. And because you are probably, and most people are probably pretty good at creating queries in Google that get the type of result that you want, tricks of the trade. We’ve done that; we’ve kind of grown up with that, and we know how to do that. But what we expect are search results. And so the difference in Chat is that you just go straight to the answer. You don’t want a result that then you can go do your own research. You want the answer, and so that’s the efficiency that this interface brings. And so, and then that gives you the opportunity, because it’s interactive, and it is sort of a formatted, like a conversation, you can instantly drill into the result if it’s not what you like, if it’s not where you want to go. It can understand natural language, and you can interact with it to just get the answer you’re looking for, versus get a bunch of raw fodder for going to find the answer.

Faith (07:42):

Hmm. And it’s, the language learning model is consolidate…it’s filtering information that’s been input into it by what, I assume, is the worldwide web, correct? Or is there…

Grey (07:58):

Yes.

Faith (07:58):

…is there something else to it?

Grey (07:59):

Well, yes, you can think of it like having layers and the raw data. What it’s, you know, the learning part of it, it’s been taught stuff, and that stuff is from, not only what exists online, but also offline sources that have been digitized and injected into the raw data source. But the real power is how it knows how to make decisions about which source is better, how to combine two sources and aggregate that up into something that combines those two things, how to pivot based on user inputs. You know, your interaction where it can sort of bail on the first results and go get something else that’s baked into what it knows about, There are limits to what it knows about, and this sort of, there is sort of a line that is a super legitimate line to think about, which is sort of the dark side of all of these models when you get into, because it’s a limited dataset, and because someone had to make choice, some human made choices about what to put there, you know. It is subject to being wrong. (Faith: Mm-Hmm <affirmative>.)

Grey (09:11):

It’s subject to bias, right? It’s subject to being a little bit of an echo chamber, because what if you then have ultimate trust in the result, and the result is wrong, and then now you’re perpetuating something that is wrong? (Faith: Mmm <affirmative>.) So it can be, there are limitations to it, but if we’re talking about just the benefit side of it right now, like, what are they aspiring to do? Yeah, it’s a giant blob of immense amount, 175 billion-something bits of data…not bits, but like, pieces of information that the technology has the smarts enough to sort of manipulate and to make choices about.

Faith (09:56):

I wanna get into prompt writing best practices in a second and talk a little bit about, you know, you mentioned being able to drill down into responses and the quality of response that you get back is really impressive, but I wanna spend another beat on what you just were talking about, which is, you know, the dark side of LLMs. And I’m curious if you have a sense of like, what’s the sentiment among folks in the know, right now, around essentially having an information source that’s sourceless? Like, when I was in school, we learned research best practices, which were like, okay, here’s your research question. You have a wealth of information in front of you that you can choose from, whether it’s a book…in my education, we were still going to the library, but, you know, or the Internet. And so we were taught, here’s how you vet a source for bias, for accuracy. What’s the sentiment about that with ChatGPT?

Grey (11:04):

I mean, all of those things exist, and they are there, and you can ask for them, and it will provide that like, so…

Faith (11:09):

You’re saying you can ask for a source.

Grey (11:10):

Yeah, you can totally do that, and as you work more with ChatGPT, the interesting thing is you sometimes need to challenge it like that. You sometimes need to say like, “That sounds like bullshit.” <Laugh>. (Faith: Mm-hmm <affirmative>.) Like, “Where did you get that information?” And it will tell you, which is, you know…the personification of the technology, it’s richer than you would think, you know? And so, because it has, because it’s following a model, it has sort of some human-like qualities baked in. Like, it has a way of delivering information. It has a notion of how concise to be, how aggregated it should be, how abstracted it should be, how it should communicate with you, so you can challenge a lot of these dimensions that you wouldn’t think of. That’s why I kind of framed it as like 3D; like, it’s very personified. It’s almost like the communication has multiple dimensions to it that yes, you can challenge it for a raw source and it’ll return that for you, or you could have it sort of rephrase something and rephrase like, you know, explain it to me like I’m a five-year-old. You can ask it to interact with you, and you can customize those interactions to kind of get at, if you have sort of questions about, you know, what path you’re going down here and need some proof, it can provide that.

Faith (12:39):

Or you can ask it to ingest something, and learn, and then recreate based on your prompt, right? So like, as a marketer, I’m thinking like, here’s a bit of copy. Let me ask ChatGPT to ingest it, right? And spit it back out that’s in a style similar to this example of writing.

Grey (13:01):

A hundred percent. You know, there’s a double-edged sword to all of this. Like, if any one person sat down and wrote a job description, they’d bring their own style to it, and that’s great, but is that the best style? Is that the only variation? Of course, no. And so having a variety of ways to, of lenses to look at something, or a variety of output stylistically, or rephrasing things, or changing the sophistication level of something, they’re all really, really powerful. And it’s, I mean, and the efficiency of it is well beyond what an individual could probably do, and what you wouldn’t recruit a team of people to do. (Faith: Right.) So it occupies a really interesting niche in terms of value creation.

Faith (13:46):

Yeah. The other day I was working in Notion, who also just released their kind of in-page AI tool, and I was making a template for something, and so usually when I do that, if I’m creating like, a blank, bulleted list, if you hit Enter twice after a bullet, of course, it clears the bullet. So usually, I just do two spaces and then Enter, and now that’s the hot key for AI, “Please finish this.” (Grey: Nice.) And I didn’t realize that, and it was like, a Voice of Customer template. (Grey: Yeah.) I was gonna send to the team and ask them to fill out, and there were several prompts in there which were like, you know, “What are customers’ pain points?” Right? Like, “What are they not finding elsewhere that they come to Gun.io to solve?”

Faith (14:33):

And the Notion AI answered it. Like, I looked up, and I was like, “Is somebody already in this editing it?” And it was (Grey: <Laugh>.) Notion <laugh>. So, it’s interesting, because it was the same thing where I could read the sentiment and be like, “I know that,” like, “I’ve read that review in G2,” or “I’ve seen that on Glassdoor.” And so it’s clear that like, the information has a source, and it’s not just kind of making it up as it goes. (Grey: Yeah.) Like you said, the implications, also, for hiring are going to be fascinating, not just because it’s going to affect job descriptions and kind of the nature of work that folks are asked to do, but also, you know, how we <laugh> get past this log of usual hiring tasks which includes writing job descriptions and cover letters.

Grey (15:23):

I mean, job descriptions, and cover letters, and things are all not great, (Faith: <Laugh>.) because they’re not accurate (Faith: Mm-Hmm <affirmative>.) <laugh>. Right? Like, a job description is just not what the day in the life is like, you know, it just, it won’t be. It’s never gonna be that rich, and it doesn’t say much about the person who’s gonna thrive in that role. (Faith: Mm-Hmm <affirmative>.) Even with the best bullet points you can come up with, it’s still probably not right, and in the same way that a resume doesn’t capture who a person is. Right? (Faith: Right.) Like, a resume is a far cry from a person. So when you combine a resume with a job description, you’ve got like, not a whole lot. (Faith: Right.) And so the idea that you could have the job, the early stages of the job process, be more interactive, where the AI is sort of leading a conversation, and you can get sort of real-time reactions and how someone chooses to phrase and answer, or the examples that they choose to cite, or how quickly or how verbose they are in the way that they answer a question, can all be async, first of all. (Faith: Mm-Hmm <affirmative>.)

Grey (16:35):

It could be done without a ton of people in the loop. And then after that, it could be sort of summarized and bubbled up in a way that a keyword match on, you know, a job post and a resume is just never gonna do. So like, (Faith: Right.) the idea that we could process so much really, really rich data with robots is really interesting in terms of what we could actually get out of the early stages of whether someone’s a good fit for a job or not.

Faith (17:05):

Yeah. It’s so interesting, because my pea-brain was like, “So cool! I could just like, feed, you know, we write cover letters for developers all day long, and what if we could just feed their profile data, and then the job description, and that would create a cover letter to then be ingested by a human?” But it sounds like what you’re saying is like, “No, let’s, instead of using AI to create something for then a human to go and analyze, let’s use the AI to analyze humans,” right? And on more, kind of, axes than we’re currently able to do through, like you said, a keyword match in a talent management system. Right?

Grey (17:50):

Yeah, a hundred percent. I feel like we could do more quick, more quicker. (Faith: Yeah.) Part of the beauty of chat, as an interface, is that it’s more like the way that we communicate already as human to human. (Faith: Mm-Hmm <affirmative>.) That sort of form factor, for lack of a better description, is more intuitive for us. And so when you think about trying like, the same thing we talked about with a job description or a resume, when you try and take who I am as a person and put it on a piece of paper that’s easily, you know, consumable. It really doesn’t seem like me. But if I’m having a conversation with someone, it’s really easy, in that sense, to get a feel for the person and like, (Faith: Mm-hmm <affirmative>.) we use these things, you know, like these mysterious things like they had a, you know, a good vibe or their feel was good; they answered that question really well. Well, why is that? It’s like, it’s the actual words, or is there something in between that, is the nuance of that interaction that we sense when we’re using, sort of, an interface that’s very much like the way that we do it naturally as humans. So I think that’s the space, you know, (Faith: Yeah.) and it’s both sides of the equation. I think it’s, mutually, it has a value proposition on both sides. (Faith: Right.) Better for job seekers. It’s better, you know, for hiring companies.

Faith (19:11):

I mean, there’s just so much opportunity, because today, the two…if you’re doing traditional hiring and not hiring developers through Gun.io, which you should be doing if you’re hiring developers, but if you’re doing traditional hiring for any other role, there’s really two options. And the first is, you filter all applications and resumes by hand. (Grey: Mm-Hmm <affirmative>.) Which is what I do when I hire, because we don’t use a talent management system for marketing roles or sales roles. And in order to do that, in order to have kind of that initial screen actually work and not just be like a subconscious assessment of like, whose resume design I like, I have to ask for writing samples, and I have to do so in a way that pulls out their personality, gives me a sense of how they communicate what’s important to them, what their, kind of like, personal brand is like, and that’s a lot to ask of candidates. And it’s a lot to ask of a hiring manager to, I mean, if I leave a job up for more than a day, I’ve got over 200 applicants in 24 hours. So that’s a lot of processing. But it’s interesting, because that style of hiring, which is what I do, what we do here for non-engineering roles, requires completely the opposite input as a talent management…or a talent…candidate tracking? Candidate management?

Grey (20:38):

Yeah. Like an ATS thing, yeah.

Faith (20:39):

Yes, yes. Applicant tracking system. ATS. Because when you’re applying for a job that’s gonna be like… your application materials will be ingested by an ATS and filtered for keywords. It’s like SEO, right? They just want something that’s keyword stuffed and written for a machine. But if I see that in an application, I’m like, “No, this person, they don’t really want this job. They just, they could have had ChatGPT write this, in fact.” So anyway, the space in between those two really not good solutions is massive. And what you’re describing as a use case of ChatGPT, or other LLMs, is a really compelling solution there.

Grey (21:27):

Yeah, I agree. And I think one of the things, maybe just to, for posterity, to get it out there, one of the capabilities of ChatGPT that I think is super powerful is the ability to role play. (Faith: Mmm <affirmative>.) Part of prompt writing and getting ChatGPT in on the right angle for what you want is to position it appropriately. So on its face, you can ask it a question, and it will return a fairly generic answer. But then if you, but then you can say, you know, “Hey, I want you to play the role of a technical recruiter,” then it can tap a whole different set of data about best practices for technical recruiting and simulate an interview with a technical recruiter. That is a very different interaction than if you just said, “Hey, how should I get a job in a tech company?”

Grey (22:24):

You know? So the same tool, right? But it’s a role shift in the tool that it’s capable of doing. That can be helpful as a counterpoint. Again, like, if you think of this thing as sort of being a counterpoint to you, like, what are you asking, and what are you asking of the counterpoint, in order to get to the answer that you want? And so (Faith: Mmm <affirmative>.) that dimension, that access, is really important when you think about how do I get the most out of a tool? It’s not like Google where everything is sort of flat and linear. It’s malleable in the sense that it can be a persona, and it understands what that persona is, and it has constraints around that.

Faith (23:09):

I hadn’t thought about that. Right? Like, asking it to ingest a job description on your resume and say, you know, “Now coach me through an interview,” or “Play the role of a technical interviewer,” (Grey: Right.) And then asking for feedback like, “Hey, was there anything on my resume or in my experience that I should have noted in that question?”

Grey (23:29):

Yeah, exactly.

Faith (23:30):

That’s really cool. I mean, I think the, you mentioned this up top, but it’s probably worth digging into one last time, while we’re on the subject of using ChatGPT in hiring cases. Obviously, it’s massively important when, in the work that we do with hiring, to do everything we can to reduce bias. And we also know that AI models are really prone to absorbing human bias and perpetuating that. So I’m curious about your take, you know, whether you’re a hire or you’re a job seeker, and you wanna be cognizant of those shortcomings, how do you think we can work around those, kind of in these early days of AI chat models and using them to help us in hiring?

Grey (24:20):

Yeah, I think that’s a tough one. I think you have to be intentional about it, just like you would if you are doing what you described earlier with a completely manual process, you know, (Faith: Mmm <affirmative>.) if you are diligent about your own assessment, right? Like, am I bringing something to the table unnecessarily? You know, the level of sort of professionalism that you would bring, doing it manually, you have to, you could, there’s a way to do that like, with this type of interface. So I think a lot of that is not gonna be necessarily dictated by the technology; I think the technology is somewhat agnostic in that sense. I think it’s about user expertise, honestly. And I think the better we get at understanding our tools, the more, hopefully, the feedback cycle will be, “Why wasn’t I successful using this tool? Well, maybe I brought a bias that I wasn’t aware of, or maybe I brought, maybe I didn’t ask the right questions. Maybe I was insensitive about the way that, you know, framed the question,” or whatever it might be, right? (Faith: Hmm. Yeah.) I think it’s ultimately on us to be better at using the technology, versus the technology to be self-regulating or to bring that to our attention. Although, there’s probably a role to play on both sides.

Faith (25:40):

It’s just a entirely new way of interacting with technology, because, to date, you know, my experience using technology is I consume what is there, and I sometimes create, because I do content creation in my job. We all are kind of content creators to some degree, but mostly what we’re consuming on the Internet has nothing to do with inputs, right? (Grey: Mm-Hmm <affirmative>.) Or our own inputs, it’s somebody else’s. (Grey: Mm-hmm <affirmative>.) So you’re right. It’s a totally new way to interact, and it requires a level of self-reflection that we haven’t had to have previously.

Grey (26:19):

Right. That’s it. It’s totally right, and I think that’s kind of the exciting thing about it. I mean, all of this has some really interesting, sort of, “glass half empty, glass half full” sides to it. And I think the really interesting part is exactly what you described, which is there’s a way to, as users, the technology can enable us to think differently about something, think differently about the world or how we interact, because we’re selfish, and we want the result that we want. (Faith: Mm-hmm <affirmative>.) So in a lot of ways, like, this technology is gonna enable us to advance the way that we interact with technology in a way that we’ve not even contemplated yet. At the same time, every time there’s an abstraction in technology, we become further away from the core data set. (Faith: Right.) Like, we don’t know how to do things that used to be common knowledge, you know, 20, 30, 50 years ago. We don’t have any idea how to do them now, but we can use a device (Faith: Yeah.) that can tell us how to do it. So there’s like, this give and take of where we’re going, directionally, but I do think there is a big opportunity for us to think about technology differently by virtue of just using the tools, you know?

Faith (27:42):

Mm-Hmm <affirmative>. Right.

Grey (27:43):

And I know just from personal experience, doing my own testing and interacting with the, you know, one of the things, you see all the guitars and stuff, so like, one of the things that is easy for me to wrap my head around, in terms of like, an output that I can sort of gauge whether or not it’s good or not, is can I get ChatGPT to write a decent song? That’s like, an ongoing project of mine. (Faith: Mm-hmm <affirmative>.) And the difference between where I started and now, which is, I can have it output something that wouldn’t probably be embarrassing to play <laugh>, right? (Faith: Yeah.) Versus in the beginning, which is like, that is worse than anything that I’ve ever heard. (Faith: <Laugh>.) It’s not the technology, it’s me, right? It’s like, I’m getting better at it, and (Faith: Yeah.) I’m getting better at understanding the nuance of how to write a prompt, or I’m getting better at being, you know, very specific about the output that I want or some other dimension of it. And I think my own learning is what’s actually the thing that’s different in this whole world. (Faith: Right.) And so if you think about the model, itself, improving with all these millions of inputs, and the user’s improving, then Murphy’s Law is gonna totally apply to this, and it’s, (Faith: Mm-hmm <affirmative>.) whatever the…

Faith (29:05):

Murphy’s Law (Grey: Murphy’s Law.) might apply to this. Maybe? (Grey: <Laugh> <unintelligible>.) <Laugh>. The other hockey stick for technological advancement.

Grey (29:14):

That one, that law that I can’t remember. Yeah. That law, Bill’s probably in the background screaming right now, like…

Faith (29:22):

He is…the number of times that someone on this podcast has said, “Yeah, that law, the technological advancement one.” We have got to find answers here, people.

Grey (29:31):

<Laugh>. Right. Let me ask ChatGPT.

Faith (29:33):

(THE FRONTIER THEME FADES IN) <Laugh>. Oh, good call.

Grey (29:36):

So that law, which is embarrassing that I forgot what it is now. I’m totally lost.

Faith (29:40):

You and me both, and I’ve had weeks to look it up (Grey: <Laugh>.) <laugh>.

Grey (29:46):

That’s happening, and that will happen, and it’s exciting.

Faith (29:48):

Yes. Well, this conversation has inspired me to use Notion’s AI tool not on accident. Now that I know what the hot key is. (Grey: <Laugh>.) So we’ll give it a go, maybe right now <laugh>. 

Faith (30:02):

Thanks for listening to the Frontier podcast, powered by Gun.io. We drop two episodes per week, so if you like this episode, be sure to subscribe on your platform of choice, and come hang out with us again next week, and bring all your internet friends. If you have questions or recommendations, just shoot us a Twitter DM @theFrontierPod, and we’ll see you next week. (MUSIC STOPS)