What People Are Getting Wrong About AI in Content Marketing

I write words with the ultimate goal of promoting and selling products and services. 

LinkedIn’s algorithm knows this is what I do. So when I log in, I’m immediately fed copywriting and content writing posts. 

Lately, the topic of AI seems to be what all those posts have in common. No surprises there. 

They’re littered with all manner of opinions. A full spectrum of hot and spicy takes on AI. If I disagree with a post, all I have to do is scroll past it and I’ll see its complete opposite waiting right below it.

I don’t have a particularly strong opinion on AI one way or the other. It’s a tool. I believe that, like any tool, what matters is what you do with it. 

But when it comes to AI as it relates to writing words for marketing, i.e., ad copy, website copy, and content, I think a lot of people are missing the point. Focusing on philosophical questions, like the threat AI poses to human creativity, when we should simply be asking ourselves if AI is helping us market better or not. 

So here’s my hot take: At the end of the day, whether or not you use AI for writing doesn’t matter. 

What does matter is that you’re doing the best possible marketing you can, with the tools available to you, and doing so ethically. If you use AI in everything you do, and your marketing is hitting its goals, great. If you’ve tried AI but found your human-written headlines perform better, that’s great too.

Marketing has always been about generating value for the organizations it’s promoting, not about how pure the process looks behind the scenes. To me, that’s the part of this conversation that keeps getting lost.

This piece isn’t an argument for or against AI. It’s not a manifesto. And it’s definitely not another piece about whether robots are stealing our jobs or our souls.

It’s a practical look at AI through the only lens marketing actually cares about: results.

Marketing Has Always Been Results 

If the message lands, the process becomes irrelevant.

Today, we often talk about marketing in terms of secondary goals, things like brand awareness, differentiation, engagement, and email sign-ups. While they may be masquerading as such, those aren’t the end goal. They’re steps along the way. You can’t buy from a brand you’ve never heard of. You won’t choose Brand A over Brand B unless you understand the difference. And you can’t run an email campaign without an email list.

Strip it all down, and the common thread is obvious: every one of those goals exists to increase the odds of selling more. Marketing serves many functions, but its raison d’être has always been to drive revenue and create value. That’s why, as marketers, the only thing that truly matters is whether our efforts help drive revenue. 

That brings us to the topic of AI. If using AI (and using it ethically) can ultimately help you boost sales, why wouldn’t you? The question then is not whether or not you should use AI in digital marketing, but rather, whether or not using AI delivers better results, and if so, where? 

Let’s Discuss Responsible
AI Use Before Moving Forward

Be deliberate, not defensive, of your AI use.

My central argument here is that choosing to use AI in your marketing isn’t a decision that should be made from positing some kind of moral high ground or pushing any kind of AI ideology. It needs to be a performance-driven decision based on data. 

Now, we need to address a caveat of using AI ethically. We’ve already discussed this topic in another post, but here’s the TL;DR of it:

Using AI ethically means designing and applying AI in ways that respect people, protect their rights, and minimize harm. This includes being fair and unbiased, transparent about how AI is used, protecting privacy and data, ensuring accountability for outcomes, and keeping humans in control of important decisions.

Related: The Hidden Environmental Effects Of AI In Digital Marketing

With that out of the way, and for the purposes of our discussion here, from here on out in discussing AI, I’ll assume that it is being used ethically and responsibly. Sound good? Excellent. We’re on the same page. Now let’s get back to the topic at hand.

So far, we’ve established that:

• As a marketer, your number one job is promoting your brand and creating value.
• Deciding to use AI should be solely based on creating said value.  

So now’s the time to put aside how we feel about AI. Let’s take a look at some numbers to answer the following question: 

Can using AI in your digital marketing help drive better results than not using it? 

To answer that question, you need to be ready to lead with curiosity and test things out for yourself. We love our A/B tests in marketing because they let us know what works better in situations that may surprise us. 

The A/B Showdown of the Decade:
Human vs AI

Regardless of how I, or anyone, feels about AI, you shouldn’t rely on opinions. Opinions do not move the needle; data does! 

Instead of debating whether marketing should be human-led, AI-generated, or somewhere in between, test it. Test everywhere and everything you can to make concrete decisions based on data. Pit human-written ad copy up against AI-assisted copy. Put emails and content into the ring. See who comes on top in making UX decisions and making consumer predictions. Measure what actually performs with whatever metrics matter most to you and prioritize whatever works best. Because… why wouldn’t you?

That’s it. No ideology required.

Objectivity matters here. Set your preferences aside and let the results lead. Sometimes that means human insight wins. Sometimes AI-driven marketing automation does. Most often, it’s a combination of the two: knowing when to create, when to generate; when to think, and when to prompt.

If something feels off, ethically, creatively, or culturally, that’s a valid constraint. Go human; trust your gut. Just don’t confuse discomfort with a performance metric. 

The key is continuous testing as tools and platforms change. Your competitors are already experimenting, so the only way to stay ahead is to keep measuring, adjusting, and improving. Do that, and you won’t need hot takes or think pieces to decide where AI belongs. You’ll have your own answers.

Fortunately for you, studies have already been done comparing AI with humans, so there are some data you can take a look at before you start testing things for yourself.

Related: A/B Testing: Where Science Meets Marketing

A/B Testing AI: What does the data tell us?

Let’s get into the weeds and take a look at some studies that have already been done.

Hop Skip Media: Human-Written Ad Copy Outperforms AI

Hop Skip Media ran a controlled experiment comparing human-written and AI-written ad copy. The target audience was business owners and marketing managers seeking PPC advertising support.

Each copywriter (human and AI) produced one responsive search ad (RSA), with each RSA containing 15 headlines and four descriptions, aligned with Google Ads best practices. The ads ran on Google Search for eight weeks with a total budget of $500, allowing for a direct, side-by-side performance comparison of the two approaches.

The results: “The human-written ads achieved 45.41% more impressions and 60% more clicks, resulting in a significantly higher CTR of 1.33%.”

The author suggests that human copywriters outperform AI because, essentially, they understand people better. That shows up in multiple ways, like reading emotional cues, navigating cultural nuance, crafting creative and emotionally resonant messages, and adapting language to context and platform. These are distinct skills, but they all stem from lived human experience.

The remaining factor is more technical: AI’s effectiveness depends heavily on training data quality and model maturity. At the time of the study, limitations in available data and tools (including the absence of platform-native models like Google’s Gemini) likely constrained AI performance.

Source: Search Engine Journal

Semrush: Humans Prefer AI-Generated Copy

As part of its Think Big With AI report, Semrush surveyed 700 U.S. consumers on their preference for human- or AI-written content. Respondents were presented with two versions of different content formats and asked which one resonated with them more, without knowing which was generated by AI or written by a human. 

The content formats were a blog post intro, a short social media ad, a blog post paragraph, a social media post, a social media ad, and a product description. 

The results: The AI-generated content won in every single format. 

As the authors note, this was a survey, and results in real-life scenarios could differ. But there are relevant insights to be drawn, for example, the AI content was more direct and used simpler language. The human content, meanwhile, used “colorful imagery” and “emotional language.”

The main lesson from this survey is that AI can be a useful tool for content creation with a human in the loop. A human needs to provide the prompting, as well as review and edit all outputs, but AI can play a powerful role in creating first drafts and helping make content more readable. 

Study: The Impact of Visual Generative AI on Advertising Effectiveness

In this study, researchers looked at where visual generative AI actually delivers value in advertising. They compared three approaches: ads created entirely by human experts, ads created by genAI and then tweaked by humans, and ads generated end-to-end by genAI.  

The results: Ads created fully by genAI consistently outperformed the other approaches, driving click-through rates up by as much as 19%. In contrast, using genAI just to enhance human-made ads didn’t move the needle. The data suggests AI works best when it has room to create holistically, without tight constraints. But there’s a catch: disclosing AI involvement reduced ad effectiveness by up to 31.5%. 

In short, genAI can drive real results, but how you use it and how transparently you frame it matter just as much as the tool itself. 

Study: Large Language Model in Creative Work: The Role of Collaboration Modality and User Expertise 

Study Design: Researchers conducted an experiment where expert and non-expert users wrote advertising copy with and without LLM assistance, comparing two collaboration modes: LLMs as “ghostwriters” (taking the main role in content generation) versus “sounding boards” (providing feedback on human-created content). Quality was measured by the number of clicks ads generated on social media platforms.

The results: This study suggests that how you use AI matters more than whether you use it at all. For non-experts, AI worked best as a sounding board, giving feedback that improved clarity and execution and helped novices perform more like experts. For experts, on the other hand, AI didn’t add much value; their content was already good, and the model didn’t add much on top of that. 

Interesting to note is that letting GPT-4 draft ads led experienced marketers to perform worse than if they’d not used AI at all, largely due to anchoring: people stuck too closely to the AI’s first draft, made fewer meaningful changes, and ended up with more generic, less creative ads. Across the board, AI also inflated confidence (probably because it can be all too eager to please humans) without reliably improving results, and it didn’t help make users more creative.

Taken together, I think these studies make one thing clear: AI isn’t inherently good or bad at marketing. It really all depends on what you’re asking it to do, how you’re asking it, and how you define success. In some contexts, AI seems to outperform humans; in others, vice versa. Really, the only mistake you can make is assume AI is better or worse for your marketing without testing it. 

How to Test AI in Your Marketing

If there’s one takeaway from all of this, it’s that you need a testing mindset about AI before you develop an opinion about it. The fastest way to decide where AI belongs in your marketing stack is to treat it like any other variable and measure it against the outcomes that matter.

So here’s a simple framework you can use to start testing AI:

• Pick one channel: Google Search RSA, Meta static, email campaign, landing page hero, blog content, etc.
• Create 3 conditions: Human-only, AI-only, Human+AI.
• Measure the right metric:
  • Search/paid social: CTR, CVR, CPA, incremental lift (not just CTR)
  • Email: CTR + downstream conversion (not just opens)
  • Landing pages: CVR + revenue per session

You won’t be able to learn anything meaningful if results are swinging from day to day, so you’ll want to run your test until you start seeing stable results. Stability alone, however, isn’t enough. The length of time you run a test also depends on how much traffic or volume your test receives. Lower-traffic channels need more time to reach a statistically significant threshold, while higher-volume campaigns can often reach reliable conclusions much faster. 

The key is to run your test long enough to gather sufficient data for confidence in the outcome (the exact timeline will depend on the channel you’re testing and your average conversion volume). From there, you can start to decide for yourself where you stand on using AI in your marketing. 

When it comes to the future of AI in digital marketing, though, there might be a ceiling to how much AI can help improve results. If everyone is using similar models in similar ways, the inevitable next question is…

What kind of digital marketing
wins in an AI-saturated market?

AI has the potential to help you achieve better results; it could also backfire. That’s okay. The point is that, as marketers, deciding to use a tool needs to be grounded in achieving better results. 

If AI helps you write better ads, automate routine tasks, or free up time for higher-value strategic work, you should use it. If it doesn’t, you shouldn’t. Deciding where it belongs in your marketing stack should always come back to one question: Does this drive better outcomes?

On the topic of how AI is changing digital marketing, all of this testing raises a bigger question: if AI becomes ubiquitous, what actually creates an advantage?

If more and more marketers are using AI-driven marketing tools, it’s not hard to imagine where it all leads. A world where most things on the internet are written by, or with the help of, AI. More content created using the same systems, trained on the same data. The headlines you see start to sound familiar, blog posts blur together, and ads follow predictable patterns.

In that world, it’s very likely that AI won’t produce the same results it used to. AI will lead to average, and average will become invisible. 

So what actually wins? 

Differentiation. In this context, differentiation is exactly where human perspective matters most.

Human thinking is shaped by inputs AI doesn’t have: lived experience, genetics, emotions, values, judgment, context, upbringing, and consciousness that give rise to unique thoughts in any given moment. Every strategic decision you make, from what to test to what to say, and what to prioritize, is filtered through those inputs. 

Large language models don’t have those inputs. They generate outputs based on patterns in training data, not lived experience. Their training data is the internet and not life. They can accelerate execution, but they can’t independently judge relevance, nuance, or impact.

That distinction is critical if your goal is results. 

AI doesn’t know what it’s like to be your customer. It hasn’t felt their frustrations, motivations, or trade-offs. It can approximate empathy, but it can’t experience it, and without that experience, it can’t reliably decide what will resonate, what will feel authentic, or what will convert.

In an AI-saturated market, the advantage goes to whoever uses AI most intentionally. Test where it helps, reject where it doesn’t, and rely on human judgement to guide those decisions. 

AI can help you move faster. Human insight determines whether you’re moving in the right direction.

When results are the goal, that difference is everything.

Let Results Lead the Way

AI can be incredibly useful when it improves outcomes, when it helps teams move faster, test more ideas, or focus their energy where it matters most. When it doesn’t, it shouldn’t be used. That decision doesn’t require belief or fear or ideology; it simply requires measuring outcomes.

But as AI becomes more common, the playing field levels, and what remains is perspective. Human judgment shaped by lived experience, values, and context. That’s where differentiation comes from, and that’s what cuts through sameness.

The marketers who win won’t be the ones who use AI the loudest or avoid it entirely. They’ll be the ones who test thoughtfully, use it intentionally, and know when human insight matters more than automation.

So remember this, dear reader: 

Results are what matter. Everything else is just noise. 

Now go forth and test things for yourself!

What blog post would be complete with a CTA to keep you on our site, amIright? So here it is: if you’re interested in doing more testing yourself, consider our 10 AI prompts to strengthen your marketing strategy. 

Works cited:

Lee, Hyesoo and Todri, Vilma and Adamopoulos, Panagiotis and Ghose, Anindya, The Impact of Visual Generative AI on Advertising Effectiveness (October 21, 2025). Available at SSRN: https://ssrn.com/abstract=5638311 or http://dx.doi.org/10.2139/ssrn.5638311

Chen, Zenan and Chan, Jason, Large Language Model in Creative Work: The Role of Collaboration Modality and User Expertise (September 19, 2023). Management Science, volume 70, issue 12, 2024[10.1287/mnsc.2023.03014], Available at SSRN: https://ssrn.com/abstract=4575598 or http://dx.doi.org/10.1287/mnsc.2023.03014