AI Workflows in Business: A Practical Guide

Michael Krigsman: Today on Episode #808 of CXOTalk, we're discussing workflows and AI with Mike Knoop. He is the co-founder of Zapier and the head of Zapier AI.

What is the Zapier Automation Platform?

Mike Knoop: Zapier is a workflow automation platform. You can use Zapier to connect together and stitch together the different tools you use. 

We support over 6,000 different integrations now. Probably any software you use, Zapier can integrate with. Primarily used by nontechnical folks. 

Michael, you were even telling me yourself that you're a really savvy Zapier user and you're not technical. That allows you to do some really cool things. You love the ease of use, and that's something that resonates, I think, with most of our customers and users.

What is Zapier AI?

Michael Krigsman: You had an executive role at the company. You're the cofounder of Zapier. And now you're the head of AI, of Zapier AI. What's going on with that?

Mike Knoop: I'd been paying attention to AI and LM trends. I gave a presentation to the company about GPT-3 when it came out. 

But there was a paper that came out last January (in 2022), this chain of thought paper, which was sort of an AI research paper. It had this key insight, which is, by prompting LMs, by asking them to think step-by-step, you dramatically increase the performance of a lot of the evals or the test suites that researchers run these models through from somewhere between 25-30% accuracy up to like 80-90% accuracy for some of the evals. 

This blew my mind because the capabilities of the models had been in the market for 12+ months at that point. This was literally just a new way to talk to the model, and you get these dramatic improvements in the accuracy scores for what people want to use them for. 

That was really, really surprising to me. And I was like, "I have to figure out, are we on a critical path to human-level general intelligence and AGI? What does that mean for our business?"

I gave up that executive role you were talking about and just went all-in. That's been my full life for the last 12 months working on AI stuff.

What are AI Automations?

Michael Krigsman: What have you discovered, and can you now also intersect that with automations?

Mike Knoop: The first six months, I was just running code. I was running code along with my cofounder Brian and CTO.

I think the first thing we realized was these language models, these AI models, once they're trained, they're frozen in time. They don't have access to fresh data. The model weights are sort of frozen, and they're limited in terms of what they can interact with. They can't interact with the real world. 

In order for these things to be really useful, I think most people, most users, want them to integrate or interact with tools, software, and applications that they use and care about. That was our first theory was, "Okay. Zapier has 6,000 integrations on it. We have a lot of tools that maybe we could give access to these models to be able to use."

We prototyped for about six months and ultimately came up with, I thought, a pretty cool way to expose a lot of the actions on Zapier like creating contacts in HubSpot, sending Slack messages, creating draft emails in Gmail, retrieving deals in HubSpot – things like that in a format that an LM could use.

We got introduced to Greg over at OpenAL along the way, and he invited us to be one of the launch partners for the ChatGPT plugins that launched back in March. That was really our first public foray, I think, into AI products whereas Zapier is one of those first launch partners, and the story goes from there.

I think the main feedback I got from that moment was I think it really accelerated Zapier's brand awareness around AI. I heard Twitter commentary where folks were like, "Oh, is Zapier now the second most important AI company out there?" 

To be clear, I don't believe that. I don't think we've earned that, but that was the acceleration we got where there was a lot of AI—I call it—curiosity coming our way. The curiosity was often in the form of, like, "What's real? Y'all seem like you're ahead. What are you seeing? What's working for y'all? What's hype?" There's a bit hype thing, "What's hype and what's real?"

That's where I spent most of the last six months of my life this summer was spending time with customers, spending time with our internal teams to go figure out, okay, what is real here? What are the real use cases? What's actually going on?

Michael Krigsman: When we talk about workflows and automations, just briefly give us some context about that. Then let's talk about what happens when you integrate these AI aspects into it. 

Mike Knoop: The concepts of AI and automation are mode collapsing into the same thing. I think, for a lot of business users, they just don't even think about the difference there. 

The promise of both is the same, which is, "Hey, this is software that does work for you while you sleep." That's what they want this stuff to be able to do for them. 

Sure, maybe Zapier hasn't historically been thought of as AI, but what the product lets you do, workflow automation is software that does work for you while you sleep. And so, I think there is renewed interest coming into Zapier and automation just through this AI curiosity lens.

Use Cases for Automating Business Processes with AI

Michael Krigsman: What are some of the use cases for integrating AI tools with other software and business processes?

Mike Knoop: Let's start with a marketing use case. I think this is probably one of the highest ROI use cases in Zapier that I was able to find, and I think it starts to hint around general patterns that other folks can maybe apply this stuff in their own companies.

I'll set this up by describing, on Zapier, we have these things called Zap templates. Zap templates are templates of things you can use Zapier for. 

They're blueprints, if you want to think of them like that, about how to connect multiple tools together. And they usually have a use case attached to them, some sort of inspirational use case to try and get the user interested in thinking about why they'd want to connect these two, three apps.

An example of one might be, whenever I get a new lead in my Typeform, I want to automatically create a contact inside HubSpot. Okay, that's kind of the blueprint. 

Then the use case for that might be, and the description of the use case might be something like, you want to shortcut one of your sales reps' time to respond to leads to minutes by automatically getting new leads into HubSpot. Now you're getting a sense of, "Oh, okay, I see why I might be interested in using that use case," if I'm just checking out Zapier.

These Zap templates are really, really important for us. They drive inbound acquisition. They drive SEO for us. They're also really important for activation. They're a really important thing going from a user who is signing up to helping them get started with the product for this, like, top of the funnel.

The challenge with them is they're all human-created, and they're very hard to create, actually. We have humans who are building these things. 

They can build about ten a day in really good quality. At least that was up until this year. We had this big hole of a million Zap templates that we had not yet created on the website that we knew we wanted to create. 

The reason for that is we launched two new integrations, two new apps every single day. There's this sort of N-squared, combinatorial problem where every new app we want, we want to have Zap templates for all of the other 6,000 apps. 

The space of how many Zap templates we want grows way faster than our ability to create them. This is a workflow where we started plugging a language model in the summer, and we moved a human from a creation moment into a review moment. 

We are using ChatGPT plugged into the middle of a workflow that automatically creates these Zap templates, the ones we don't have. It suggests use cases, and it dumps them out into a big spreadsheet. 

Then we have that human who was doing the creation before. They're moved into a review mode now where they're just going down the spreadsheet and saying, "Okay, that's a good one. That's a good one. Here's a bad one. Tweak this and this one, now it's a good one." 

The change in rate here is we went from one human being able to build ten a day to now they're reviewing 1,000 a day. Literally a 100x rate in labor enhancement by plugging language models, and ChatGPT specifically, into the middle of one of these workflows. 

I think that's probably one of the most impressive and really cool use cases internally, and it really leans into what the technology is good at with LLMs today, which is turning squishy ideas into concrete blueprints, as well as being creative in terms of trying to think through what the use cases could be.

Michael Krigsman: Please subscribe to our newsletter and subscribe to our YouTube channel so we can notify you about upcoming CXOTalk shows.

We have an interesting comment from Twitter. Arsalan Khan, who is a regular listener who always asks very thoughtful questions, says, "AI seems like it's business process reengineering on steroids when it comes to automation, so it's rethinking the nature of the automation itself and how it functions." What do you think about that?

Mike Knoop: The guidance that I give to teams internally (and I have all year) is the thing you should reach for these tools first, or the thing you should do first with them, is go back and revisit every impossible problem that your company or team has faced that you weren't able to wrap your arms around. Revisit that problem with language models in hand now. 

I can give you an example of this that we're using internally. Error messages on Zapier are one of these really, really high perplexity, high problem, high perplexity space problems where the variation, the variability of the error messages that we get back from all of the 6,000 integrations on Zapier is very, very hard to deal with. There are no patterns to it. 

Error messages are often written with developers in mind, not non-technical business users in mind. So, when we expose one of these error messages directly to a user, it creates a really bad user experience. 

We've known about this problem since 2012, and we tried to solve this in the early days with trying to relabel error messages, coming up with heuristics, even some data training models to try and identify and tag error messages to make them more friendly and easier. The problem was just too big. It was too hard. The error message space was too great for us to really make a dent in that problem.

Now, that is a problem that we have revisited with language models because one of the things language models are really good at is reading very, very high perplexity space, raw, unstructured text like error messages, and translating those things into things that are more easy to use and understandable. We're using language models now to translate those errors messages and, in some cases, even suggest resolution steps that are like, "Oh, what you did is you forgot to map this required field in this Zap, and the app didn't pass a piece of data through. What you need to do is go out of filter and make sure it always has it. Now you'll be fixed." 

It's like, "Whoa! That's really, really cool." So, I tend to agree with them. I think every single business process that you might have tried to solve in the past that maybe doesn't work as well as you want it to, it's back on the table, I think, to revisit with LLMs. 

How to Automate ChatGPT with Zapier

Michael Krigsman: Can you give us an example, say, using ChatGPT, because, as I think about this, it's not clear to me at all (if I'm trying to write something or rewrite something) how I could automate this? What kind of interface can you have to ChatGPT for something like that? 

Mike Knoop: When we think of ChatGPT, we think of a user-facing website we go to where you can have a conversation with a bot. One of the cool things is OpenAI has API access to ChatGPT. 

This exists on Zapier. So, if you say you're building a Zap, you can plug ChatGPT into the middle of the workflow, and you can do it without needing to have a human go to ChatGPT and send a message to the bot in order to get a response back. 

Let me give you an example of another use case we use internally, and I'll walk through the more concrete steps in it and talk through where ChatGPT comes in. Another really great use case we use in our sales org. We're looking at, I think, an extra $100,000 AR a month right now due to language models. 

We use HubSpot to track all of our inbound deals. We use Gong to record all of our sales calls. Prior to this year, one of the things we were doing is, every new sales rep, whenever they get off the call with a lead, they have to take notes.

They have to transcribe notes into HubSpot. They need to log things like: 

• Were there any action items? 
• What are the next steps for this lead? 
• What's their propensity to buy? 
• What are the blockers to this deal? 
• What do we need to do for them?

It takes about 10 or 15 minutes. It's the sale rep's least favorite part of their job is just data entry. Because of that, it ends up being very inconsistent from rep to rep in terms of the quality of the data that you get in there. 

This is a use case where we started deploying LMs against it. We get the Gong transcriptions of the call, so plain text. Then we have ChatGPT in the middle step, so we trigger a Zap from those Gong transcriptions. We get the full text.

We have ChatGPT then read that text of that sales transcription, and we ask it questions. We ask it things like: 

• Hey, what are the action items? 
• Hey, what's the sentiment of this lead? 
• What are the next steps? 
• What are the blockers? 
• What's the propensity to buy? 

We get that data out in a very structured way inside of a Zap. Then we type that into HubSpot. If the lead exists, we'll update the lead with that new, structured—and augment it with that—data. If it doesn't, we'll go and create the lead and just add the data in. 

This is an indent workflow. You can imagine it in three steps where the input, the trigger is, "Oh, there's a new Gong transcription that's done." The middle step is a ChatGPT step inside Zapier where we ask it questions and we extract out these features. Then the last step is now we're going to go create that stuff in HubSpot.

This is a great one, I think, because it's literally giving reps time back to spend more time with opportunities and leads. It's getting rid of the least favorite part of their job. Then when our sales managers were waking up in the morning and looking over all of our HubSpot opportunities and leads, they were trying to figure out, "Who do I spend time with today or this week?" the data quality is significantly higher because you're getting consistent output from the language models each time.

Michael Krigsman: You're automating the summary of transcriptions from the sales calls based on a standard – Is this correct? – set of questions that you're asking the LLM.

Mike Knoop: That's right, and these are questions that we asked our sales reps to do before, which is, okay:

• Log whatever the actions are. 
• Log the sentiment. 
• Log the propensity to buy. 
• Log the deal size.

All that kind of stuff is something language models can do. I think the general takeaway for this is that one of the best things that language models are able to work with is unstructured text.

I think a lot of businesses have structured text, databases, tables, and things like that. You can use language models to do some cool stuff there, and we can talk about some of those use cases. 

But by far and away, I think the most general use case that we use language models for in Zapier right now is operating over unstructured customer data that's coming in, whether that's coming in through a sales call transcription or whether that's coming in through a product survey that we're running where we're asking users to give us raw, unstructured feedback, or whether that's coming in through Zendesk and our issue and ticketing platform. 

All of these have a high degree of raw, unstructured text that normally you might have a human operate over and pull sentiment features out of. But it's something that LMs do a really, really good job of. 

Michael Krigsman: What's the quality? The quality is obviously sufficient for you to get a sense of what's going on in the call. But overall, what's the quality, the level of accuracy, and also the ability to capture nuance that a human might include if they were thoughtfully writing the notes? 

Mike Knoop: We are legitimately booking more revenue per month as a result of this stuff. 

I think there are very few use cases where we will put a language model into a workflow or an automation where a human is not in the review loop, though, somewhere. 

When I was talking about those Zap template use cases before, we're not automatically publishing those to the website coming out of a language model. We have a human sitting there looking at what the output is and using their judgment of saying, "This is good. This is bad. This is good. This is bad."

I would expect, over the course of technology here, we're going to get to a spot where you could remove that human, and you have enough confidence in the workflow. But for most of the ones we're doing, we're not there yet. 

You want to probably look at first deploying this technology into use cases where there's labor enhancement opportunity but where there still is going to be a human looking at what that is and being able to use their judgment and being able to say, "Hey, that doesn't look right or smell right. I'm going to go back to the source of truth and audit that." 

Michael Krigsman: For the internal use of that data where you're really just trying to get a sense of what's going on, the LLM output is directly sufficient. But if you're publishing something publicly, then it becomes the first step. Then you need somebody to look at it, review it, edit it, and so forth. Is that correct?

Mike Knoop: We do the marketing side. We do some personalized email use cases now, stuff where we will have LLMs automatically draft follow-up emails to some of our leads. 

There's one step in our lead pipeline where we want to go back to leads, and we want to offer them use cases, very personalized use cases for what they could use Zapier for. Use case discovery is one of our big challenges with any new user is helping them figure out what can the product do for them. Of all the workflows, which ones should you care about?

This is another thing that's very human labor intensive in order to look at all of the data about this lead, all of the tools they use, and think creatively about, like, "Ooh, what are the use cases that we know about that might be useful for them and for their business?"

This is another spot where we're now using ChatGPT as a middle step in a workflow to automatically create and craft personalized use case emails back to that lead. Now at this point, we still have humans that are reviewing all of those before they get sent. 

We've heard some really cool feedback, too. Literally, we have replies to some of these LLM-crafted emails that say, "This is the best sales email I've ever received." [Laughter] So, there are legitimately good feedback coming in from this. 

But even that one, we still haven't deployed it in an automated, touchless way. We still have humans reviewing those.

How to Drive AI Adoption in the Enterprise

Michael Krigsman: We have another interesting question from Twitter. Again, this is from Arsalan Khan. He's coming back, and he says, "How do you encourage people to use AI? At the same time, how do you prevent them when that's appropriate? Do you have enterprise governance, and how does the governance relate to the overall company?"

Mike Knoop: This is something we lucked out. When I was talking about that hackathon we did, there was a lot of internal enablement we did where we went through the landscape of AI tools. 

We were trying to figure out, okay, which ones can we let everyone use? Which ones have good training policies? Can we put PII and customer data into these things?

I think the reality is there's just a lot of fear, uncertainty, and doubt, unfortunately, around what companies, what products, and what APIs don't train and are safe to put customer PII through. At this point, we are ChatGPT Enterprise buyers as well, and that sort of unlocks a lot of things where we're able to now use ChatGPT in a very sort of safe way that's guaranteed, by default, we can put customer data and PII into this, and that's unlocked a lot of cool internal use cases for us. 

To this person's question, it's something I heard a lot about as I spend time with founders and execs. Everyone is trying to answer this question, and every legal team in the world is trying to wrestle with this. 

I can share our experience of where we settled on that, which is, assuming that the tool or the API does not train by default and there's no chance of customer data or PII leaking out through a side channel into another user somewhere, like downstream, that's good enough. That meets our bar that we'd want to see in order to unlock usage.

But I think every org has different legal tolerances. Different fields are going to have different regulations around this stuff. And everyone is trying to figure it out, but that's how we treat it today.

How AI Automation Simplifies Enterprise Workflows

Michael Krigsman: We have another interesting question from Twitter, from Lisbeth Shaw, who says, "How does AI and workflow automation help simplify enterprise workflows?"

Mike Knoop: I think one of the coolest things that I've seen is having a language model generate them. [Laughter] 

I think one of the hardest parts of using and building workflow automation is the word I just said. You have to build it. You have to understand. You have to have somewhat of a process-oriented mindset. 

Sure, you don't have to be an engineer or technical to be able to use Zapier, but you do have to be a little sort of software savvy. I think this is one of the things that really does limit the adoption of things like Zapier. 

I'm not sure that the paradigm of how we traditionally think about building Zaps where you have to go into a canvas and pick all these apps and get the field mapping exactly right and deal with the API error messages and all that. I'm just not sure that that paradigm scales to hundreds of millions of users. Language models are the first technology I've seen, I think probably ever in Zapier's existence, that have a chance to literally make a step function increase in the amount of people and users that can build, create, and get value out of workflow automation. 

One of the things we launched this year was this thing called our AI Concierge. It's in Zapier's onboarding flow when you sign up where you can describe what is your goal, what are you trying to accomplish, in just natural language. 

We use an LM to pattern match against the tens of millions of use cases that we've seen over the last decade and recommend Zaps where we'll prefill in all the fields for you, and we'll prefill in all the triggers and actions, so you don't have to build. You don't have to get all that stuff done. We're seeing a really, really positive reception to that stuff.

I think, in terms of simplifying, I think that's probably the biggest. That's the big opportunity here. These tools let more people in the world translate their business problems and their integration and workflow problems into software that can do work for them.

Details on Automating Prompts and Results with ChatGPT

Michael Krigsman: Mike, the use case that you were describing earlier where you are automatically taking transcripts, summarizing those transcripts, pulling out the key points, and then populating HubSpot, can you describe conceptually how do you go from the transcript to the summary automatically? The other parts are conventional, right? That's just the standard workflow automation. But it's the interface with ChatGPT. 

Mike Knoop: When you're building a Zap in Zapier, you can imagine them as steps in a workflow. Step one, there's a block. You would pick Gong. 

You would say, "Hey, I want to trigger this workflow whenever a new Gong transcription is ready." After a call wraps up, they do the transcription on their side. They give you that full, plain text transcription.

Step two is you would choose ChatGPT inside Zapier. There's an app on Zapier, on our platform, called ChatGPT, where you can have the action to send a message. In order to send the message—this is kind of a technical art term—you create a prompt, effectively. 

Think of the prompt as the same thing that a user would type into ChatGPT. Except, with the prompt inside Zapier, you can put way more context into the prompt. 

You could say in the prompt, "Hey, ChatGPT. Here is a conversation with a Zapier lead:" and then you paste, you map in the text, the transcript from the first step into that prompt. Then you have a couple more new lines and say, "Now what I'd like you to give me back is 1) a summary that fits into two paragraphs. 2) From zero to ten, what's the propensity for this lead to buy? 3) Call out any action items in bullet point form that you want us to take action on." That's kind of the rough structure.

Then when that step runs, Zapier will send the prompts to ChatGPT offline, in the background. The user is not online or having to do anything here. 

We get the response back, we get those answers back, and we parse them and extract them out. Then you can use each of those bits of information—the two-paragraph summary, the bullet point action items, the propensity to buy—as pieces in follow-up steps. Then in step three, create a HubSpot lead, you've got each of those pieces of data that you can map into the appropriate HubSpot field. 

Michael Krigsman: Zapier integration then includes the ability to create one prompt or multiple prompts. Then you might have a section that says, "Give me a short paragraph or three sentences summarizing this," and then that becomes an element, a unit, so to speak. Then you might have another prompt that says, "Now give me bullets summarizing this." Is my understanding correct?

Mike Knoop: You described it even better than I did. I was simplifying. But yeah, typically, for a use case where you'd want to ask ChatGPT multiple questions, you would have one step per question. That's usually the way it works. 

Michael Krigsman: Then what about the user interface for doing this? Prompt writing is really hard. On one level, prompt writing is really easy because you can put whatever you want. 

Mike Knoop: Yes. 

Michael Krigsman: But as you said much earlier in our conversation, the better the prompt, the better the output. And so, what about that UI aspect to help guide the user into creating prompts that will be most effective?

Mike Knoop: In the traditional version of Zapier when you're not thinking about an LLM or ChatGPT in one of these steps, there's an exact right answer of how to build a zap. There's an exact right answer of how to construct the API call to HubSpot, you know, which fields go where. [Laughter]

If you get it wrong, it just doesn't work. It errors out. It breaks. The zap just doesn't work.

I think one of the really cool things that I've observed from users using AI tools and ChatGPT is that they never break. They never don't work. They always give you something back. 

Now, they might not give you exactly what you want, but it always works. It always gives you something.

The loop that you get into is very similar to the loop that engineers and developers have, which is, they start from zero. They build their first prompt. They test it, which Zapier has a test button inside of our zap editor.

After you write your prompt, you can click "Test." Send it. You get the response back, and you read it. 

You're like, "Oh, cool. Okay, now I saw how ChatGPT responded to this prompt that I wrote. Now, hmm... I didn't want three bullet points. I only wanted two bullet points." Okay. I'm going to go add a rule to my prompt and say, "Only give me two bullet points." 

You get into this loop where you're improving your prompt by looking and just testing it over and over again. I think this really, really important insight into why AI tools can literally be 10x easier than using traditional software, building traditional workflows is because they always work. They never break. 

ChatGPT is never going to give you an error back and be like, "Oh, sorry. I couldn't do it." [Laughter] Or give you some 500 esoteric developer errors.

It's going to do its best. It's going to try. And I think that's the most important thing for building any workflow, building any automation, building any sort of thing on top of an LM system is you have to get into that loop where you try it, see the output, and make changes in it from there. 

Top Challenges When Creating AI-Based Workflows and Automations

Michael Krigsman: We have another great question from Twitter. What are the top challenges with implementing or creating AI-based workflows?

Mike Knoop: The biggest challenge that most people in Zapier experienced is figuring out what are the types of use cases and workflows where it meets the technology really well. 

There are a lot of things that people do for their jobs. Language models are not going to be effective for all of the tasks that they care about. 

I think often it's like trying to get a sense of where do AI and language models work really well and where do they not work really well. Then narrowing in on use cases where they work really well and avoiding the ones where they don't.

An example, we talked about accuracy before. There are use cases where you might need 100% accuracy in order for that workflow to be acceptable, in order to be able to use it in production. If that's the case, language models are probably not the right tool for the job (at this point in time with the technology).

Now, if your accuracy acceptance rate is more like 90% or 95%, or you're willing to put a human in the review loop there to get that guaranteed 100% accuracy rate, now you're opening up the use cases that you're acceptable. 

I can give you an example here from when we were building the Zapier ChatGPT to really highlight this. One of the things you can use Zapier for (and the ChatGPT plugin) is to manually take actions in tools you use. 

Now we're talking about the actual, literal ChatGPT product and website. If you go to it, and you enable the Zapier plugin, you can tell ChatGPT to take actions in apps on your behalf. 

A simple fun one is you tell the bot, "Send a Slack message to my testing channel saying hello." ChatGPT will call Zapier, call our Slack action, and map all those fields, the testing channel, pick the right one, and send it. 

When we were building and testing this, one of the things we saw was we had two different testing channels. We had an internal testing channel, and then we had another channel that just happened to have the word "testing" in it that was shared with one of our partners. And it picked the wrong one.

These are examples where it's like, oh, okay. If I need that, if that is a type of catastrophic failure for you as an organization where you might have tons of customer channels that are really, really sensitive, and if the language model picked the wrong channel to send a message to, if that would be a very catastrophic failure for you as a business, okay, you're probably going to not use that use case. 

It's probably not reliable enough to be able to say, "Okay, I'm very certain this is going to get it right every time, and I'm going to restrict it." Maybe you limit the language model. You hard-code the channel name instead of letting it guess and figure out which one you want.

Now the exact same use case for a different user, though, might be totally safe. Maybe their Slack account doesn't have any sort of external customers in it. Maybe there are no partner channels. Maybe the way they treat their Slack is everything is internally public, and it doesn't really matter if one of these AI systems or bots sends it to the wrong place. They're fine accepting that tradeoff, that sort of accuracy tradeoff. 

That's what I mean by saying every use case, even the same way to connect the apps together, the use case acceptableness is going to vary even by users and by companies. I think that's the biggest challenge is just trying to understand that and map it into where it's okay and where it's not.

Michael Krigsman: Really, what you're talking about is going back to that governance issue that we spoke about earlier. Then identifying the user cases, therefore, becomes a function both of the efficacy of the use case – Will it solve the problem? – but, at the same time, will it fit into our governance framework? Then there's also going to be privacy implications at the same time – things like that. 

Mike Knoop: My general advice for most folks looking to deploy this stuff for the first time or looking for those use cases are the technology is really good at being creative. It's really good at translating unstructured text into structure. 

Those two things, those are strengths of the technology right now. If you can map use cases that you have internally for processes into those two problems where you might have humans doing a lot of creative generation or you have humans doing that translation from unstructured data into structured, those are places that I'd look first to deploy this stuff. 

Michael Krigsman: We have another question, again from Arsalan Khan. He comes back for the third time, and he says, "Within your organization," but I'll extend this to your customers as well, "are you aware of organizations that are using ChatGPT to pull information from multiple internal systems to provide an answer or a dashboard? For example, AI pulling lessons learned from completed projects, and doing this on an automated basis."

Mike Knoop: Everybody wants this. It does not exist yet. I saw Sam tweet about that this is coming. 

Every company, every customer I've talked to has the exact same want and wish, which is, "I want to be able to customize these models with my own enterprise data. I want to be able to plug in my third-party tools into these things. I have my internal systems where I want data exposed to these things."

To a certain degree, you can do some of that with Zapier today. But in the context of this specific question around using ChatGPT to do it, it's something everyone is asking for and it doesn't exist yet, but I think it's coming.

Michael Krigsman: We have a project going on with Harvard Business School looking at the historical transcripts of CXOTalk in order to do exactly like this. It turns out it's really, really hard to do this. [Laughter]

Mike Knoop: What are you trying to extract over the transcripts? What are the interesting features you want to try to get out?

Michael Krigsman: Our transcripts, we've done this now it's ten years, and we have many hundreds, almost a thousand transcripts. And so, we're interested in how has the sentiment of senior executives, CIOs and so forth, changed over time? What are they looking at? What kind of problems are they looking at now? And then any other kind of patterns that might emerge, looking across these multiple perspectives, examining this body of issues that we talk about.

Mike Knoop: Have you tried using the new data analysis plugin that ChatGPT offers on their Pro plan? 

Michael Krigsman: We've actually tried a number of different techniques. This is maybe a little bit geeky for everybody, but there is a data scientist at Harvard Business School, and he tried a bunch of analytic methods on his own. Then we took the material, and we put it into the data processing part of ChatGPT. It suggested many of the same methods that he had been trying.

Mike Knoop: Hmm... 

Michael Krigsman: It came up with very similar results, and they weren't good enough. 

Mike Knoop: Got it. Interesting. 

Michael Krigsman: They weren't close enough to what we need. 

Mike Knoop: It's good insight. If the human couldn't figure it out, it would be... It's going to be... Those are probably not—

Coming back to, like, what is it good at, what is it not good at? If the human can figure it out, I'm pretty confident a language model can figure it out. If a human can't figure it out, I'm pretty confident a language model is not going to be able to figure it out. 

I think the places you were looking to deploy, especially in the workflow automation context, I think you'll looking for spots where you already have humans successfully doing some sort of task where they're already reviewing those transcripts, they're already pulling this data out successfully, and that's a spot where I think you can look to labor enhancement to use LMs.

Evolution of Low-Code, No-Code Software Tools

Michael Krigsman: Can we talk about low code and no code tools? That's a very important topic for CIOs. Any thoughts about that: where we are in the lifecycle of these tools, where it's going? Obviously, you're right in the middle of that.

Mike Knoop: One, adoption is real. We see this in our user base. We reached ten million users over the last ten years. There are a lot of people out there who want this stuff.

Eighty percent of Zapier users do not self-identify as technical or developers. These are folks that are business users. 

They're in operational roles, sales ops, marketing ops, support ops that unblock themselves by using no-code tools when they can't get engineering time or attention. Or when they are able to get engineering time and attention, maybe it takes nine months. Then what they get back is not what they wanted. Everybody is frustrated, right? 

I think, especially for growth and mid-market orgs and larger orgs, that's sort of the promise of this stuff is it makes everyone in your org an engineer (in some ways). 

I think the reality is, as these tools get more engrained inside of organizations, a business user starts with one of these tools. They use it in a prototyping sense. 

They prove out, oh, there's some massive, good ROI for this particular workflow automation that I just built.

Then there becomes this really important moment where it actually becomes somewhat of a production use case for them. It's not just playing around. It's not just prototyping. It's not just the side-channel thing. It actually becomes a very business-critical process. 

That's what we're seeing more and more is a lot of the use cases on Zapier evolving and graduating from these more prototyping use cases into, like, "Man, this is now super critical for my business, and how do I manage that production use case in a way that's going to give my business the business continuity? How is it going to give my engineering and IT teams the visibility into it for them to be confident in being able to make changes?"

Honestly, there are a lot of concepts from engineering that we are now building and shaping into our own Zapier toolset for these production use cases around things like having staging environments or how to have change logs and draft modes, just building a lot more of the classic engineering workflows into no-code tools. I think, as you look ahead, that's going to be more and more the case is these systems that business users build that start off in prototyping mode and now become business critical. You're going to see a lot of, I think, no code tools (Zapier included) just offer more and more ability for those things to be safe, reliable, and durable in business-critical use cases as well.

Position of Zapier vs. Competing Workflow Automation Solutions

Michael Krigsman: We have a couple of other really good questions from Twitter. This first one is from @jonebgood54. He says this: "How do you see Zapier with respect to the other top enterprise automation players (MS Power Automate, UIPath, and so forth)? Having used and trained thousands of developers on those tools," he says, "I think your UI is similar, but the UX is more intuitive and the genAI is super powerful!" 

Mike Knoop: There are two common themes that I frequently hear from users about why they love Zapier. I think one of them is that it has the biggest and broadest integration ecosystem out there. So, pretty much any software that you use, Zapier has an integration for it – over 6,000 integrations.

We launch two new integrations every single day at this point as well. So, that's just something that we're the best at. I think that's one thing I hear a lot.

The other one is ease of use. Again, for non-technical folks, more process-oriented, software-savvy folks, Zapier is easier than a lot of the alternatives like having to write code. [Laughter]

I think that the biggest reason, ultimately, users love Zapier is it allows them to go zero to one. Certainly, automation and workflow have these time-saving aspects to it where you can use automation and get minutes back or hours back every week, and that's certainly true. But the folks who really are the largest advocates of things like Zapier, I think, are the people who the alternatives are not like, "Oh, I chose Zapier over this other tool."

It's like, actually, my alternative was like, "I just couldn't do it at all. I'm not technical. I couldn't have wrote the code. I had no idea how to do this, and I found and discovered Zapier. This unlocks something really important for me personally, for my team, or for my business."

I think that's the power and the promise of a lot of this no-code stuff. Ultimately, talking about AI, the promise of AI is I think we live in this sort of duality where our users, our best users, tell us Zapier is easy. I think that is true.

When I look at the broader knowledge worker market, building knowledge workers, I think the reality is Zapier is still too hard for everybody else. I think that's where AI comes in. 

Using AI, using LMs to make Zapier even 10x easier than it is already today, that's what we're trying to accomplish. That's the path we're on. I think, if we accomplish that, that really unlocks the door for hundreds of millions of users to be able to use stuff like Zapier.

Generative AI and the Enterprise Workflow Automation Ecosystem

Michael Krigsman: Our last question is from Lisbeth Shaw who says, "How will generative AI disrupt the enterprise workflow automation ecosystem?"

Mike Knoop: I think if we meet the promise of what this stuff can do, 10x more people, 100x more users can use it. I think that's the upside. [Laughter] And I think that's a really empowering world to live in, too. 

You've got to play the second-order effects of what does it mean when 100 million business users are building critical business systems in things like Zapier. But I know those users are going to be really happy because the alternatives are much worse.

Michael Krigsman: I will just mention that if I were a CIO today, I'd be starting now to think about the implications when you begin to have real proliferation of users being able to weave together these different kinds of tools. It's happening now. But if Mike's prediction is correct, this is going to have a big impact on you CIOs. I'm talking to you. 

With that, a huge thank you to Mike Knoop of Zapier for taking the time to be here. Mike, thanks so much. It was a really interesting conversation.

Mike Knoop: Yeah. Fun. Hopefully, this was useful for folks.

Michael Krigsman: A huge thank you to our audience. You guys are an awesome audience, and your questions are so, so excellent.

Now, before you go, please subscribe to our newsletter, and subscribe to our YouTube channel so we can notify you about upcoming CXOTalk shows. We have incredible shows that are coming up.

Everybody, thank you so much for watching, and we'll see you again next time.