Keyword research has been the foundation of content creation for search for around 2 decades. Users researched keywords with tools, entered seed terms, arranged by volume, chose targets, and formed content based on them. Though that method is still the same, it is no longer the only method used to create content for search. The reason you have witnessed a shift in traffic sources over the last year is due to the same reasons.
More and more of your audience members don’t browse search results by entering short keyword phrases. Users are opening one of the many AI tools like ChatGPT, Perplexity, or Google’s AI research mode, and “talking” to them. The nature of search is changing to the point where keyword research and planning is suddenly obsolete. Because of this, an auxiliary discipline has developed, Prompt Research.
The two research disciplines do not have to work against each other. The two research disciplines satisfy overlapping and distinct demands while creating and planning search content. This is why it is important to know how each of the two research disciplines works, and how they differ. Once you understand this, it is clear how the two can work together.
What is Keyword Research?
[IMAGE_1]
Keyword Research was a method used to find the short phrases and keywords searchers input into search engines. Keyword Research uses tools that analyze keyword competition and difficulty and related search terms, such as Google Keyword Planner, Ahrefs, or Semrush. Keyword Research is the precursor to the planning and creation of search content.
The whole model depends on aggregation. If you search “best running shoes for knee pain” in Google, there are likely millions of people searching the same term. A keyword research tool can provide insights into the search volume for that term, how competitive that search term is and synonyms that are closely related to that term. This was largely the framework for most of SEO strategy built over the last twenty years and is useful to understand demand at scale and to optimize pages for the classic, organic blue link search results.
What is prompt research?
[IMAGE_2]
Prompt research is the process of understanding the full, natural language questions people ask ChatGPT and convert them into similar formats to understand the questions being asked in other AI systems such as Perplexity or Google’s AI mode. Instead of “best running shoes knee pain,” the prompt may be, “I have mild knee pain from running, which shoes should I look at, and do I need custom orthotics too?”
The same underlying need with a totally new structure. Prompts tend to be longer, more specific and loaded with context, user constraints, their level of understanding, and even their budget or location. That additional information is actually useful because it shows that the person is way more advanced in the process and is not just browsing.
Due to the lack of volumetric data for the individual prompts that are asked a handful of times in slightly different forms, the type of AI research dealing with prompts works at a more abstract level. Rather than tracking the exact phrasing, this type of research looks at recurring questions and how AI systems answer those. This includes the sources cited, the brands suggested, and the content gaps that are leaving a brand completely out of those suggestions altogether.
Why is Prompt Research Increasing so Rapidly?
The most straightforward answer is that AI-assisted discovery is no longer a behavioral anomaly. The most recent industry data shows a considerable portion of consumers in the United States (some measure this as 35%+) are utilizing AI for product discovery compared to an even more dwindling segment of consumers using traditional search for product discovery. By the time a majority of buyers go to Google, they have already developed a shortlist elsewhere.
This fundamentally changes the shape of the funnel. If your content strategy relies solely on keywords, there is a high risk of optimizing for the latter half of the buyer's journey while completely ignoring the first half of the journey (the stage where AI tools are creating shortlists that your content needs to be a part of).
There’s a structural reason prompt research couldn’t grow from keyword research. Search volumes show what people ask, and content strategy has been built upon this foundation for the last twenty years. Now with AI search, users don’t input standardized keywords, they input prompts. Users input ad hoc prompts, and analyzing each prompt individually doesn’t scale. That’s why prompt research treats prompts as clusters and topics as opposed to isolated queries, just like how keyword tools treat search terms.
How Prompt Research Actually Works
A good prompt research process has some steps that are more or less the same, even though the tools may differ.
- Identify your target audiences and personas
Different people phrase requests with varying levels of competence, interest, and time. A beginner and an expert asking about the same thing will generate very different prompts, and you need to address both.
- Utilize keyword research as a starting input
This is the merging of the two disciplines as opposed to the competing of the two disciplines. Your keyword list is a valid starting point for prompt research. Take a keyword and come up with three or four natural language questions someone would ask an AI about that keyword.
- Focus on decision-stage prompts
Some prompts matter more than others. Most informational prompts are useful for capturing visibility at the top of the funnel, but if you want to move the business forward, you want to know how AI begins comparing options, creates a shortlist, and makes recommendations (the middle and bottom of the funnel). Most importantly, you should know the prompts that trigger comparisons and create a shortlist or direct recommendation.
- Examine how AI systems respond to those prompts
Go ahead and try the prompt yourself. See which sources get cited and which brands get named. See how the answer is framed. This will show you the winning answer format and how your potential answer content could fill the gap.
- Track presence, not position
[IMAGE_3]
Rank tracking doesn't work here like it usually does. AI generated answers are more volatile than search results. The same prompt run on separate days can yield answers that are significantly different, with some studies showing variation of over 40% for the same prompt from one week to the next. Rather than trying to track a particular position, continuously check for your presence across those checks over time.
Building One Workflow Instead of Two Separate Ones
The most prudent thing to do is not choosing between keyword research and prompt research. Rather, both should be done for every single potential topic. Traditional keyword research becomes most valuable for finding a topic of focus, confirming that the topic is of interest to searchers, and capturing the demand for that topic. Immediately following, you should rephrase those keywords as natural language questions that people might ask their AI assistants about the same subject.
To help search engines understand what your article is about, you should write your content to align with the answer structure. These types of structures incorporate a clear direct response as the first or second sentence of new sections. Once you have adapted your content to new structures, you should include your short-tail keywords throughout the content, but avoid keyword stuffing. This allows you to optimize traditional SEO strategies. Once you have formatted your content to be primed for search, check the AI Overview and chatbot responses for your topic. You should then be aware of your competition's structure and the level of detail they are including. A fully optimized AI response is able to answer all short-tail and long-tail variations of a user's query.
By incorporating this strategy, a singular piece of content can fulfill audience engagement for both traditional search and AI Answer behavior. This offsets the previously required tradeoff that forced users to choose between the two.
An Example
A digital marketing training institute creates blog content about topics like updates in Google Ads and SEO, career paths in digital marketing, and more. Their keyword research shows that the term ”digital marketing course Bangalore” has some volume. This is a high-intent keyword and should be placed on its own page.
However, if we quickly perform keyword research for the same topic, we find that instead people are asking what digital marketing courses in Bangalore actually help you get placed in a job, or if it is worth taking a digital marketing certification, even with an engineering degree. These keywords are more specific and longer in length and reach a higher level of intent (or decision) than the keyword alone.
Building one page that targets the traditional SEO keyword and placing sections of that page, or closely linked companion pages, to each of the prompts and including a clear, direct answer near the beginning of the section. One content creation approach has much more potential to appear in Google's ranking and be used as an answer by AI assistants, instead of being optimized for only one function.
Prioritizing AI Writing Prompts
With keyword research, determining the highest priority items to target is as simple as ordering search volume. It is not as straightforward with AI writing prompts. More often than not, other factors end up being more important.
Here are a few of the most important factors
- Decision proximity
- Strategic fit
- Current gap
- Repeatability
These will all require some judgement, but I've also found that it is less formulaic than keyword research. It is much easier to rank prompts and then go read through how the AI responds, and then work from there.
Understanding Success
Shifting to AI search requires a different way of thinking about measurement. Here are some alternative methods:
- Citation analysis: How many times does your domain or your brand get referenced when a target prompt is used repeatedly over several weeks?
- Source quality analysis: Do AI systems include your content as a source for your target prompts, or do they still cite your competitors?
- Traffic from AI systems: New tools in analytics separate traffic from ChatGPT, Perplexity, etc., and although the numbers are small, you should check this out too.
- Behavior on pages built for prompt clusters: Once an AI answer brings a user to your site, your page engages them in the same way as traditional design.
Your existing approaches to traditional SEO are not replaced by these new methods. Rather, these methods offer new ways to understand visibilities that traditional keyword-based reporting cannot capture.
Common Mistakes to Avoid
- Considering prompt research a substitute for keyword research rather than a complement.
- Focusing on prompt variability rather than prompts that elicit comparisons and recommendations.
- Anticipating reliable volume or position data for prompts, as you would for keywords.
- Writing content reflective of a keyword phrase that does not actually provide an answer to the more nuanced and complex question a prompt conveys.
- Believing that the AI visibility check is a one-time proposition. Volatility of AI dictates that tracking must be ongoing.
Tools Worth Knowing About
[IMAGE_4]
During the last year, many software tools dedicated to prompt research have been developed. These tools have macro functions that assist in prompt and topic discovery, and micro functions that assist in tracking your brand's presence on prompts. Some tools, like Semrush's Prompt Research Tool and Similarweb's AI visibility, utilize large proprietary AI databases. Other tools, like SISTRIX and seoClarity, are focused on breaking prompt clusters and content briefs creation. Most of the established SEO tools have recently added at least an elementary version of this functionality. Before investing in a new tool, you should check if your existing tool has added this functionality recently.
FAQs
Is prompt research replacing keyword research?
No. Keyword research helps you discover high demand search terms that people use. Prompt research shows the AI-driven format that people use when talking to the AI assistant. The best content strategies incorporate both keyword and prompt driven research.
Do I need special tools to do prompt research?
No, you can do prompt research on your own. You can take some of your top ranking keywords and rephrase them as questions to see how they answer in the assistant. Dedicated tools are more useful once you want to track your presence across many prompts at scale and to see changes over time.
How is 'ranking' measured for prompts if there's no position data?
In the absence of search results, prompt research does not measure ranking position. Instead, prompt research measures visibility based on the number of times a brand or article is referenced when a prompt is run repeatedly. Since answers to a prompt can vary dramatically, this needs to be checked frequently rather than just once.
Should small businesses waste their time on prompt research or is it better for large brands?
It is important for small businesses to do prompt research as well. Large brands can have more prompt resources as they have more content to cover informational areas. Often it is better for a smaller business to use prompt research to cover a narrow set of decision stage prompts that are directly related to their product offering rather than trying to cover informational areas.
What is the distinction between prompt research and GEO (generative engine optimization?
Prompt research is about the methodology used to find and prioritize the most worthwhile questions. On the other hand, GEO generally refers to the practice of tailoring content in such a way that generative systems are capable of recognizing, extracting, and citing it. In essence, prompt research is the foundation of a GEO strategy.
Will keyword volume data cease to be important?
It is unlikely that keyword volume data will become insignificant anytime in the future. Traditional search still exists, and as such, keyword volume reflects genuine, measurable demand in search. Traditional search has not been completely supplanted, so volume data is still one of the more trustworthy indicators so it is still worthwhile to use this data in prompt research, unlike disregarding it.
Final Thoughts
Keyword research continues to have its place and these ideas do not encourage the stopping of keyword research. What has changed is that keyword research is not as all encompassing as it once was. Prompt research covers the part of the funnel that keyword tools have no means of reaching - the questions people ask in a conversational context and in a manner that is rich in context before someone even thinks of going to a search engine. Brands that developed a workflow that integrates both are looking to maintain their presence in the results that are ranked, AI Overviews, and chatbot answers, as opposed to assuming AI-Driven search is the only way to discovery.